Trial By Error: The HRA’s Letter about the Investigation of Bristol Research

By David Tuller, DrPH

Last week, the National Health Service’s Health Research Authority published a long-awaited (by me) investigation of 11 studies from Bristol University. All 11 studies were spearheaded by Professor Esther Crawley, the University of Bristol’s high-profile pediatrician. I had flagged these studies as problematic because in every case the Bristol team exempted them from ethical review on the questionable grounds that they constituted “service evaluation” rather than “research.”

This “independent” investigation confirmed the obvious–the ethics statements were inadequate to describe the activities involved, as I had pointed out. To be clear, a shrewd sleuth initially sussed out these irregularities and alerted me to the issues; again, my work has piggy-backed on other people’s discoveries.

I have many thoughts about this investigation and the inadequacy of the recommended remedies. For now, however, I am posting the letter I received from the HRA at the same time the agency sent me the report itself. As the letter requests, I am posting the text in full rather than selected excerpts. (The formatting of the text in WordPress differs somewhat from the formatting of the original letter. Because of my lack of technological prowess, I’m not sure how to reproduce the image of the letter in the blog. The phrases below in bold replicate the use of boldface in the letter itself.)

**********

22 October 2019

Dear Dr Tuller,

Ref: 18.TP.01
Re: Independent Panel Review of 11 publications on chronic fatigue syndrome and myalgic encephalomyelitis
REC reference: 07/Q2006/48
Study title: What happens to children with CFS/ME? The study of a longitudinal cohort of children who access a paediatric CFS/ME service. Version 2.

We wrote to you on 31 January 2019 responding to a number of concerns you raised regarding Research Ethics Committee (REC) approved studies relating to Myalgic Encephalopathy (ME) and Chronic Fatigue Syndrome (CFS). One concern regarding the school absence study (07/Q2006/48), remained outstanding at the time of our previous letter. This is summarised below:

• Concern that the methodology followed in this study with regard to data collection procedures did not match those procedures as set out in the protocol and should not therefore have been classed as a service evaluation by the REC.

• Query why the journal accepted the claim that investigating a pilot intervention to identify new patients could be exempted from ethical review as a service evaluation.

• Query regarding whether journals should accept, at face value, the claims of authors, RECs and university committees or whether independent judgement should be exercised by journals in making a decision.

We were unable to respond to your concerns previously as an independent review of the publications relating to this study was being conducted. The independent review has now been completed.

Please find enclosed the report from the independent panel, jointly commissioned by the Health Research Authority (HRA) and the University of Bristol, to review eleven publications on chronic fatigue syndrome and myalgic encephalomyelitis which all cited the NHS Research Ethics Committee (REC) reference 07/Q2006/48. Please note personal identifiable information has been redacted from the report as required by the Data Protection Act and General Data Protection Regulation.

The panel’s purpose was to determine as follows:

a) Whether the collection and analysis of the data referred to in each publication was covered by an NHS Research Ethics Committee ethical review (‘ethical review’) and favourable opinion, including whether there was any departure from a reviewed project that was not covered by additional ethical review.

b) Whether the collection and analysis of the data referred to in each publication was part of service evaluation (either under HRA guidance or in line with National Outcomes Database practice), and therefore did not require ethical review;

c) If not, whether ethical review for the collection and analysis of the data (i) was required at the time of publication and (ii) would be required under current Governance Arrangements for Research Ethics Committees (GAfREC) if published today;

d) Whether further action should be considered in relation to any publication based on the outcome of this review.

The membership of the panel was jointly agreed by the HRA and the University of Bristol and was made up of two members of academic staff from the University of Bristol and an independent chair nominated by the HRA who was a clinical academic from another Russell Group university. The panel members had no prior connection to the research or the research team and were appointed based on their specific experience and expertise in relation to ethics of research into health-related matters.

The HRA is satisfied the panel was suitably independent with no material conflicts of interest, had the appropriate expertise to assess the publications and conducted its review in a fair, impartial and objective manner.

In light of the controversy surrounding this area of research, we agreed at the outset that the anonymity of the panel members would be preserved and that the actions arising from this investigation would form the basis for publication. The HRA will therefore put a statement on our website aligned to the content of this letter. We would request that extracts from this letter are not published. If it is published, it should be published in its entirety so that it reflects a full interpretation of the findings.

Teresa Allen and Professor Sir Jonathan Montgomery met with members of the University team responsible for clinical research on Monday 19 August to hear the outcome and recommendations from the panel’s review. Please be advised the HRA has accepted all of the panels’ recommendations as set out in the report and summarised below.

Before it looked at the publications, the panel considered all of the available communications between Professor Crawley and the REC which took place prior to the submission of her first application which had the REC reference number 07/Q2006/48, also known as the School Absence Study.

One critical point to note is that the correspondence documents held in the HRA HARP system made available to the panel were an incomplete record for this study. We only currently therefore have access to the research protocol, application and communications sent by the REC to applicants for this study. In 2006, most documents issued to the National Research Ethics Service were hard copy paper format and the policy at that time only required retention of documents for 3 years following a favourable opinion unless there was a reason to hold onto them.

A communication between the REC and Professor Crawley seeking three points of clarification was not therefore available for review and the panel had to make assumptions based on the response letter from the REC to Prof Crawley. We know that this communication took place but neither Professor Crawley nor the HRA has this original communication. The response letter was available within the document pack. It was clear from the response that Professor Crawley had been planning to undertake clinical assessment work which would lead to the establishment of a National Outcomes database for children with CFS/ME and she was seeking advice from the REC on how existing and new data points should be treated.

The REC, (formerly known as North Somerset and Bristol) had reviewed the study application and concluded that the work that she planned to undertake was in fact a Service Evaluation. The wording used in the letter of response by the REC was considered by the panel to be open to mis-interpretation. One interpretation being that all work undertaken using data from the National Outcomes Database was covered by this original REC decision. Further wording in the letter could also be taken to mean that the REC was giving permission for further work to be undertaken using this data without submitting new applications to the REC for a follow up opinion.

To this effect, the panel found that the REC communications were insufficient and the HRA has already made changes to the way that we produce REC minutes and written communications to applicants to ensure that this type of confusion will not arise in future. Our current processes also require all enquiries regarding studies to be submitted in writing to the Chair so that they can be considered together with submissions. We are also currently in the process of updating our archiving policy to ensure documents are held for an appropriate length of time.

The key findings of the review are listed below:

a) The REC documentation was incomplete and was in some respects lacking in clarity and open to interpretation;

b) No favourable ethical opinion was obtained in respect of any of the papers reviewed;

c) In all eleven papers, the REC reference, as cited in the ethics statement, was
either inappropriate or inadequate to cover the activity described;

d) However, no additional ethical review was required and the position could in all the cases considered be rectified by rewording and clarification of the ethics statement;

e) In two papers, additional minor textual changes in the main body of thepublication would improve clarity.

The panel then considered each of the 11 publications in turn against the original complaint that the publications were describing research and not service evaluation. This was assessed against the HRA published toolkit (2013) to determine whether published work is research or not and the Governance Assurance Framework for Research Ethics Committees (GAfREC 2001) which was valid at the time that each piece of work was undertaken together with more recent versions of this framework.

The view of the panel concurs with our view in the HRA that all 11 of the publications contain material which would classify them as research.

However, a critical point to note is that not all research requires the submission of an application to an NHS REC. Research which includes the use of anonymised patient data is excluded. It is for this reason that the panel have concluded points (b) and (d) above.

The panel have however concluded that the REC reference cited was either inappropriate or inadequate to cover the work described in the publications (c) and in the case of 2 papers that clarity could be achieved by changing some of the text.

The panel were not asked to comment on any of the research findings.

Actions to be taken by Professor Crawley:

Professor Crawley will now be asked to submit amended text to the publishers for each of the 11 publications to include some alternative wording replacing the service evaluation statements and points of clarity for two publications. She will also be asked to notify the funding bodies that these changes are being submitted and to notify us when these changes have been made.

We have also sought the assurance from the University and Professor Crawley that the service evaluation reference 07/2006/48 will no longer be cited in any future publications.
Having considered these findings, we are satisfied that the HRA has already implemented changes which will avoid a recurrence of any mis-interpretation of the REC opinion and our archive policy is being updated so that we will include scanned copies or uploaded correspondence from applicants in our archives.

We are satisfied that Professor Crawley took the correct course of action at the outset in terms of engaging with the Ethics Committee to seek clarity about the original work that she planned to undertake and that the ambiguous wording of the REC communication has generated a number of publications with inappropriate references with respect to the work which followed.

We are also satisfied that she either had, or did not need to seek, additional ethical opinions for the published work reviewed by the panel.

It is clear therefore that there is no case to be made for any disciplinary action against Professor Crawley with respect to this investigation. There are however a number of lessons which can be learned by both parties.

We are satisfied the first of your concerns is addressed above, namely:

• Concern that the methodology followed in this study with regard to data collection procedures did not match those procedures as set out in the protocol and should not therefore have been classed as a service evaluation by the REC.

With regard to your remaining concerns, as detailed below, we are unable to advise any further as these are directed to journal and journal editors and as such beyond the HRA’s role and remit.

• Query why the journal accepted the claim that investigating a pilot intervention to identify new patients could be exempted from ethical review as a service evaluation.

• Query regarding whether journals should accept, at face value, the claims of authors, RECs and university committees or whether independent judgement should be exercised by journals in making a decision.

We trust this addresses all of the concerns you have raised however if you remain dissatisfied you may wish to write to the Health Service Ombudsman.

The Parliamentary and Health Service Ombudsman Millbank Tower
Millbank
London SW1P 4QP
Tel: 0345 015 4033
Website: www.ombudsman.org.uk

{ 8 comments… add one }
  • Janice Johnson 28 October 2019, 8:41 am

    Did I miss it? I couldn’t find their Profuse thanks to you that you for persevering I. bringing these issues to light.

    Plus no mention of apologies to all the patients who have been harmed by these research papers

  • Lady Shambles 28 October 2019, 8:49 am

    Naughty Esther. Naughty REC who allowed naughty Esther to play fast and loose with her REC number (weirdly one which pertained to a paediatric study but was used for an adult research paper … go figure.. maybe children are now allowed to self identify as adults in Crawley Land??). Naughty activists requiring the investigation team to be anonymised (maybe that’s because Esther showed them the fake mock up of a death threat letter which she proudly claimed to a packed audience as being sent to her..but we know that was a lie too). Naughty, naughty, naughty. HRA can see that it’s all been rather naughty. But HRA isn’t going to do much about it. And Naughty HRA doesn’t have powers to demand naughty journals and naughty journal editors to correct their egregious mistakes.

    Never mind that there is a very sick population at the root of all this. Russel Group are happy because they self-policed themselves into a ‘nothing to see here, move along now’ ( where have we heard that before I wonder?). Bristol are happy ditto. HRA seem to have no power over anything at all and the REC probably wouldn’t understand the notion of ‘ethics’ if it hit them in the face. So other than Naughty Esther having to write a few lines and having her favourite REC reference confiscated (my word she loved that reference and boy did she get mileage out of it) .. nothing has changed. Except that every day a person, a child, lives with ME is another day in hell for them. But don’t let that get in the way of your job security, your reputations. Heaven forbid!

    But well done David Tuller and well done shrewd sleuth. I guess this isn’t the last we’ll hear of this 😉

  • jimells 28 October 2019, 9:03 am

    I learned decades ago that “professional ethics” means “we promise not to squeal on each other”. It’s good to know that UK health professionals are still dedicated to upholding that fine tradition.

    I do find it hard to believe that Dr Crawley does not have copies of her correspondence. I should think that she would want to have stuff to donate to the eventual Saint Esther Library, which will be built to honor her dedication to psychobabble and harmful treatments.

  • CT 28 October 2019, 9:42 am

    Does anyone else find it contradictory that the area of this investigation was deemed SO controversial that the identities of all those involved was kept secret, and yet the Russell Group opted for an investigation by the very institution that was involved in this controversy, rather than go for a completely separate process that they can apparently hold if they consider it appropriate – (https://russellgroup.ac.uk/policy/policy-documents/research-integrity-statement-of-cooperation/) ? To my mind there was a stark-staringly obvious conflict of interest in this case and so it was entirely inappropriate to involve Bristol University in the investigation.

  • Richard Vallee 28 October 2019, 11:18 am

    I think there would be value in asking the HRA to comment on whether violation of rules of conduct are now generally acceptable, or whether this is a special case. Because the report essentially recognizes multiple violations of rules and conduct as well as several ethics breaches, but essentially considers those violations to be of no concern.

    So are those violations acceptable in all cases in the future? No need for ethics approval? Requirements are now optional as long as… as long as what? What are the conditions to benefit from arbitrary exemptions of committing multiple violations yet being told to simply retroactively amend the record?

    Because we see the same with BMJ and their optional requirements to pre-register trials. It’s a firm requirement. Unless exempted from. Not clear what is needed to get exemptions from requirements. Is this for every medical researcher in the country? Any researcher taking a BPS approach? Any researcher affiliated with the Great Sir Sigmund Wessely? Any researcher named Esther? How do these exemptions from norms and retroactive clearing of ethical violations are supposed to be processed? Should researchers ask approval for future violations? Simply have to pinky-promise that those violations did not affect the outcome? Do they have to speak in code and make the right handshake? Who keeps the metric system down?

    Asking for millions of friends.

  • Alison Wallace 28 October 2019, 12:16 pm

    In all this, there does not seem to be any concern that there is ethical oversight of research to protect the interests of patients. If this was not a service evaluation but research then was it ethical research to carry out on sick children?

    It should not be left to patients to worry about the harm being done to children, that is a job done by professionals but where are they?

    The wording of the document may have been ambiguous which lets the researchers off the hook but still leaves dubious experiments being carried out on a vulnerable population. If that is not important why do ethical committees exist at all?

  • boolybooly 28 October 2019, 12:16 pm

    Well done 🙂

  • CT 28 October 2019, 4:26 pm

    Does this explain anything – (https://www.youtube.com/watch?v=A3-uKBng7kU) at 16.22 ?

Leave a Comment