Trial By Error: Another Letter to BMJ’s Dr Godlee

By David Tuller, DrPH

Last week, The Times ran an article about the open letter to The Lancet concerning the PACE trial, quickly followed by a piece in The BMJ. This morning, I e-mailed the following letter to Fiona Godlee, editor-in-chief of The BMJ and editorial director at BMJ. I cc’d officials at the Health Research Authority, some NICE stakeholders, and Carol Monaghan MP on the letter, since all have interests in the proper assessment of studies in this domain.

********

Dear Dr Godlee—

I want to thank you for featuring a news article in The BMJ about the recent open letter to The Lancet concerning the flaws of the PACE trial.

However, I was surprised that the open letter itself was not referenced in the footnotes. This omission makes it harder for readers who want to review the text of the open letter as well as the full list of the 114 experts, ten members of Parliament and 70 patient and advocacy organizations that signed it. I assume failure to include the citation for the open letter was an oversight. Perhaps it could still be added. [Disclosure: I was the main organizer of the open letter.]

In addition, the article includes an inaccurate explanation for why patients objected to PACE. The main reason was not because “they thought that it suggested that ME/CFS was all in the mind and could be cured by cognitive behavioural therapy and exercise,” as the article asserts. Rather, patients protested the PACE findings because they immediately recognized that the study was scientifically unsound and included, among other flaws, an analysis in which participants could meet key outcome thresholds at baseline—a self-evident absurdity.

Yet the medical and academic establishments dismissed and ridiculed these legitimate concerns and portrayed patients who raised them as irrational, anti-scientific and even dangerous. Unfortunately, the leading medical journals did not distinguish themselves in this regard. The experts who signed the open letter recognize that the patients were right.

If BMJ now wants to demonstrate that it is interested in reporting accurately on this field, one place to start would be to correct an error in the 2011 news report about the PACE trial results. This BMJ article inaccurately used the word “cured” in describing participants who were reported to have met so-called “normal range” thresholds. The Lancet paper, of course, did not address the question of “recovery” or “cure,” so the BMJ article’s use of the term helped disseminate misinformation about the findings. (An accompanying Lancet editorial claimed that participants who achieved these “normal range” thresholds met a “strict criterion” of recovery. Since participants could have met these thresholds at baseline, this claim was ridiculous.)

Beyond that, BMJ has remained silent on the two high-profile pediatric ME/CFS studies that I have repeatedly tried to bring to your attention in recent months: The school absence study published in BMJ Open in 2011, and the Lightning Process study published last year in Archives of Disease in Childhood. As I have noted, editors at both journals have abrogated their professional responsibilities in how they have dealt with these matters.

Let’s recap.

In the case of BMJ Open, the school absence study had exempted itself from ethical review on the grounds that it was service evaluation. Although this claim was demonstrably untrue, BMJ Open’s editor-in-chief insisted that the journal had reviewed the concerns and found them to be unsupported. When questions persisted, BMJ Open submitted a self-serving, inaccurate and opaque account of the dispute to the COPE Forum, then closed the case without responding to the forum’s sensible advice to seek clarification from the study authors.

Finally, the editor-in-chief invited Virology Blog to submit a letter for publication, with a promise to seek a detailed response from the study authors and their institution. We declined to submit such a letter because we believed BMJ Open already had an obligation to seek a detailed response from the study authors and their institution. We didn’t understand why the editor-in-chief needed a letter from us in order for the journal to pursue its normal editorial functions.

The invitation to submit a letter therefore did not strike us as a straightforward offer but rather as a sneaky way for BMJ Open to wriggle out of a tough spot. As we saw it, if we submitted a letter for publication and the editor-in-chief obtained the promised responses, the journal could then post an exchange of views and pose as a champion of open debate—without having to address the mess it created by publishing the school absence study in the first place. We saw no reason to enable such a strategy. In the end, the editor-in-chief’s transparent efforts to evade responsibility and accountability were disheartening and frankly disturbing–not to mention harmful to the scientific enterprise.

In the case of Archives of Disease in Childhood, the Lightning Process study violated BMJ’s strict policies on prospective trial registration, and outcome measures were switched after more than half the participants had provided data. The investigators omitted these key details from the published paper. The journal promised in January to investigate the situation. Reviewing the documentation should have taken no more than a few minutes, and yet the investigation continues to drag on after more than six months. The delay can only be described as incomprehensible. It allows Lightning Process practitioners to keep bragging that BMJ has pronounced their woo-woo intervention to be effective.

Dr Godlee, facts are facts. Your editorial team’s apparent inability or unwillingness to acknowledge the obvious and address these issues is arguably putting vulnerable kids at risk. When do you plan to do the right thing here?

(I have cc’d officials at the Health Research Authority, stakeholders in the NICE guidance development process, and Carol Monaghan MP, all of whom have an interest in the assessment and disposition of studies in this field of research.)

Best–David

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley

Comments on this entry are closed.

  • Graham McPhee 27 August 2018, 12:15 pm

    Back in 2012 the BMJ ran an article about admitting to making an error (https://www.bmj.com/content/344/bmj.e3085.full). Perhaps they may find that a useful reference.

  • Jereme 28 August 2018, 1:59 pm

    I just don’t understand how they have the nerve to peddle this nonsense in the face of so much overwhelming biomedical research and to argue with sufferers of ME who have lost their lives to this illness despite being subjected to CBT and graded exercise therapy … it is shocking and disgracedul. Thank you professor Tuller for giving a voice to the unheard often bedridden chronically ill and standing up for good science.

  • Nancy Blake 31 August 2018, 4:11 am

    The paychiatric lobby, lead by Professor Sir Simon Wessely is deeply embedded within the British ‘establishment’…this powerful network of political, social, educational, financial and media influence which continues to support the biopsychosocial model of ME/CFS. It derives its cultural power from deeply held convictions about the ‘hysterical’ tendencies of women, and its economic power from promotion of the false belief that treating ME/CFS with psychiatric treatments that make them worse, but allow insurers/the DWP to deny disability payouts will save money. Creating a population of unnecessarily disabled paupers does not save nearly as much money as would offering treatments which allowed us to get better and get back to our education/jobs/normal social roles.
    In rhe meantime, going to a doctor for ME/CFS remains the most dangerous thing a person with ME can do, and there is no sign that this is going to change…the BPS model and reliance on PACE-type evidence is solidly in place within the NICE Guideline Review process.

    http://www.positivehealth.com/article/cfs-me/what-can-we-expect-from-the-current-review-of-nice-guideline-cg53

    As for the Lightning Process…it would be interesting to learn whether there were any financial links between Esther Crawley, who is so enthusiastic, and Phil Parker, the holder of the trademark, who stands to become a pretty wealthy guy if LP is incorporated into the revised NICE Guideline, which is clearly what they are going for.

    Unfortunately, this ‘treatment’ isn’t ‘woo woo’ enough to be harmless. NLP offers powerful techniques to enhance motivation and influence beliefs, and is the most effective form of psychotherapy around. This makes it exceptionally harmful when misused to treat ME/CFS: it’s CBT/GET on speed. Crawley at least is honest enough to admit that most children didn’t want to have anything to do with it. And that it was very difficult, in many cases, to get feedback.

    Let’s look at why this may have been. Remember the ‘double-bind’ theory of schizophrenia? People can get a bit crazy if you 1) Ask them to do two opposite things at the same time 2) They can’t get out of the situation. 3) They can’t tell anyone they are in the situation.

    Parker, in his distortions of NLP, tells his patients that an important part of it is always to say you are fine, no matter how you actually feel, and that it won’t work if you tell anyone about what the treatment was.

    So you send these children home, and then pursue them with questionnaires and phone calls to report back on how much the treatment helped them.

    Now you’ve not only harmed them physically, you’ve endangered their mental health as well.

    http://www.positivehealth.com/article/cfs-me/me-cfs-nlp-and-the-lightning-process-in-the-looking-glass