By David Tuller, DrPH
I have sent the following letter to Fiona Godlee, editorial director of BMJ and editor-in-chief of The BMJ, on behalf of Professors Brian Hughes and Vincent Racaniello as well as me. We were responding to the recent editorial regarding the new draft of ME/CFS clinical guidelines from the National Institute for Health and Care Excellence–as others have already done through BMJ’s rapid response function.
**********
Dear Dr Godlee:
The recent editorial from Professors Turner-Stokes and Wade about the new draft of clinical guidelines for ME/CFS from the National Institute for Health and Care Excellence (NICE) is problematic on multiple fronts. [1]
For three decades, a small group of medical professionals have forcefully promoted the claim that two specific interventions — cognitive behaviour therapy (CBT) and graded exercise therapy (GET) — can lead not just to improvement but to recovery from what they prefer to call chronic fatigue syndrome. In 2011, The Lancet published the main results from the largest randomised trial of CBT and GET for the illness, the so-called PACE trial; at first glance, the study appeared to support these views. [2]
The PACE investigators referred to their trial as the €œdefinitive€ test of their approach to therapeutic intervention for this disabling illness. Lancet editor Richard Horton hailed the research that appeared in his journal as €œremarkable€ and €œutterly impartial€. [3] No less an eminence than Professor Sir Simon Wessely, who was involved with PACE but was not a co-author, proclaimed it €œa thing of beauty€. [4]
Professor Wade himself declared the following about PACE in a statement released on his behalf by the Science Media Centre, a communications agency with close ties to the trial investigators: €œRandomised controlled trials provide the best and only reliable evidence on safety and effectiveness of any intervention in any condition. The trial design in this study was very good, and means that the conclusions drawn can be drawn with confidence.€ [5]
The PACE trial, of course, has since been widely discredited for €œunacceptable methodological lapses€ [6], and Professor Wade has apparently changed his mind about its probative value. The editorial he has co-authored argues that randomised controlled trials (RCTs) should not be viewed as acceptable when assessing €œcomplex interventions for complex conditions.€ It is noteworthy that he is publicly espousing this position after the renowned trial he previously championed has come to be regarded as fatally flawed.
The editorial’s main argument is that non-pharmacological treatments with multiple elements cannot be appropriately evaluated using the standard GRADE system. Trials of these therapies, the authors declare, are inevitably hampered by built-in limitations, such as the difficulty of blinding interventions and the need to use subjective outcomes in the absence of physiological measures. Whether or not GRADE is adequate for examining these kinds of non-pharmacological interventions, challenging the process and the rules after-the-fact raises questions about whether such protests are primarily principled or self-interested.
Indeed, the arguments advanced in this effort to discard a long-standing approach to the assessment of clinical trial evidence are easily dismissed.
First, the main objections raised about PACE and related studies have not been about the lack of blinding on its own or subjective outcomes on their own. It is the combination of the two in a single study that makes much of this research hard to interpret. This study design inevitably leads to unknown amounts of bias. When blinding is impossible, subjective findings must be accompanied by some sort of objective data to confirm that the results are genuine and not solely a factor of the study design.
The absence of biomedical tests for ME/CFS has not prevented investigators from incorporating a range of other objective measures. In PACE, the investigators chose four, how participants performed on a six-minute walking test and on a step-test, whether or not they were employed, and whether or not they were receiving social benefits. None of these measures matched the subjective reports of improvement from CBT or GET. The PACE authors subsequently dismissed their own objective measures as irrelevant or not objective after all, and claimed success anyway.
As part of the guideline development process, NICE commissioned an independent review of the evidence base for ME/CFS treatments. The independent review assessed dozens of studies, including PACE, that collectively covered 236 treatment outcomes–findings that have been used to support claims that CBT or GET were effective. Of these 236 outcomes, the evidence for 205 was deemed to be of €œvery low quality€, and 31 were deemed to represent evidence of €œlow quality€. Not a single outcome was supported by evidence that exceeded this abysmal threshold.
During the independent review, the most common methodological problem identified in this evidence base was €œrisk of bias.€ This is not surprising, given the tendency of the PACE investigators and their like-minded colleagues to design studies relying on unblinded interventions and subjective outcomes.
Second, just because you believe that you cannot meet a high theoretical standard does not entitle you to demand that you be judged against a lower one. The fact that your research falls short does not make the theoretical standard less important. Nor does it magically infuse your efforts with robustness or diminish the need for reliable data in order to assess performance.
Professors Turner-Stokes and Wade argue that their €œcomplex interventions€ cannot easily be standardized and are therefore incompatible with RCTs. For them, appropriate therapy is €œthe equivalent of a well tailored [sic] suit, fitted to the individual patient.€ This sounds a lot like the special pleading employed by homeopaths, crystal therapists, and chiropractors to argue that their treatments, too, are difficult to research. In other words, the logic used is akin to pseudoscience.
By stating that the €œrigid structure€ of RCTs is inappropriate for testing the effectiveness of rehabilitative strategies involving CBT and GET, Professors Turner-Stokes and Wade are throwing the once-vaunted PACE trial and all related research under the bus. They are also demanding that the rest of us change our own positions about the importance of maintaining research standards. Beyond stating that some people with the illness get better, they provide no reliable evidence that their individualised rehabilitative approach has improved patients’ health more than, say, the passage of time on its own.
This lax attitude to evidentiary standards has implications beyond the domain of ME/CFS. In a perplexing passage, Professors Turner-Stokes and Wade credit €œappropriate support and rehabilitation€ for improvements seen among people experiencing prolonged symptoms after an acute bout of Covid-19. Their claim is unwarranted. That some or many long-Covid patients have improved and even recovered, for reasons that remain unclear, proves nothing more than that some or many long-Covid patients improve and recover. Any definitive statement about effective treatment will need to await properly conducted research.
In our view, the concerns expressed by Professors Turner-Stokes and Wade about the draft NICE guidelines for ME/CFS are misguided. Complex therapies for complex conditions should be judged against the best available standards, and these standards should not be compromised simply to make a preferred treatment look better. In this regard, NICE is to be commended for evaluating the relevant research with objectivity and thoroughness.
The new draft NICE guidelines for ME/CFS are extremely welcome and long overdue.
Sincerely–
Brian Hughes, PhD
Professor of Psychology
National University of Ireland, Galway
Galway, Ireland
Vincent Racaniello, PhD
Professor of Microbiology and Immunology
Columbia University
New York, New York, USA
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
School of Public Health
University of California, Berkeley
Berkeley, California USA
References
[1] https://www.bmj.com/content/371/bmj.m4774
[2] https://pubmed.ncbi.nlm.nih.gov/21334061/
[6] https://www.virology.ws/2018/08/13/trial-by-error-open-letter-to-the-lancet-version-3-0/
Comments are closed.