By David Tuller, DrPH
The Journal of Psychosomatic Research seems to be suffering from some sort of identity crisis. Earlier this year, the editor and his two immediate predecessors published an admirable editorial in which they noted the serious risk of bias in subjective outcomes in studies that are not rigorously blinded. Yet the journal’s editorial advisory board is studded with investigators who specialize in unblinded studies that rely solely on subjective outcomes—in particular, Professors Peter White and Michael Sharpe, two of the lead PACE investigators, and Professor Per Fink, their Danish colleague.
And the journal keeps publishing studies based on such flawed research, in the process undermining its own credibility and raising questions about whether its editorial team is actually in charge of editorial policy. How else to explain the recent paper from Professor White and colleagues of one-year follow-up results from GETSET, a study of self-guided graded exercise therapy (GET) for the illness they call chronic fatigue syndrome (CFS)? This follow-up managed to present null results at one year as a success by engaging in blatant outcome-switching.
How exactly did the authors accomplish this feat? They prioritized a “within-group” comparison for the GETSET arm as the most important result rather than the “between-group” comparison that is the outcome of interest in any clinical trial. In other words, they highlighted first and foremost the fact that the purported early benefits from GETSET were maintained at follow-up rather than the most salient detail–that in the end there were no demonstrable differences between the intervention and comparison arms.
I have been pressing the journal’s current editor to correct that paper, which appeared to have been written in order to influence the deliberations of a committee considering revisions to a draft of clinical guidelines for ME/CFS sponsored by the UK’s National Institute for Health and Care Excellence (NICE). So far, the findings stand as first reported. This delay in responding to an obvious flaw is troubling. If Professor White and his colleagues are unwilling or unable to come up with an acceptable fix for their own errors, the journal has an obligation to do so unilaterally.
Now JPR has compounded the problem by publishing more of Professor White’s “fake news”—yet another plank in his disinformation campaign involving GET. In other words, the journal continues to ignore its own editors’ concerns about unblinded studies relying on subjective outcomes.
More nonsense from Professor White
Professor White’s new paper, which he co-wrote with a colleague, is called “Adverse outcomes in trials of graded exercise therapy for adult patients with chronic fatigue syndrome.” The first sentence of the abstract notes that “graded exercise therapy (GET) is an effective treatment for chronic fatigue syndrome (CFS), but concerns have been raised about its safety.”
The problem with this statement is obvious. Professor White pretends that the effectiveness of GET is a settled issue and that only safety concerns remain in dispute. This notion is categorically false. Anyone who has followed this issue knows that this declarative statement about GET represents Professor White’s personal opinion based on the kind of research that the journal’s editors have explicitly highlighted as suffering from a high risk of bias. Since all the papers he cites rely on unblinded studies with subjective outcomes, the claim is inherently suspect. As a review of the current literature would reveal, the best possible case to be made for GET is that its effectiveness is highly contested. No journal of repute should allow Professor White to make such an unchallenged assertion as if it were a fact.
The introduction itself immediately undermines the claim. It notes that the draft of new ME/CFS clinical guidelines from NICE does not support this conclusion about GET’s effectiveness and rated all the evidence as being of “low” or “very low” quality. It also quotes the NICE draft’s stark warning: “Do not offer people with ME/CFS: any therapy based on physical activity or exercise as a treatment or cure for ME/CFS.” In other words, Professor White’s unequivocal first sentence about the effectiveness of GET is an indefensible interpretation of the available data.
In fact, the PACE trial—touted here by Professor White, one of the lead investigators, as “the largest ever published” in this field, as if largeness somehow automatically conveys quality—included four objective outcomes. None of them matched the subjective claims of improvement. While the GET group did post a statistically significant improvement in one of the measures—the six-minute walking test—the marginal reported benefits were clinically insignificant. At the end of PACE, GET participants still performed much more poorly on this measure than healthy women from 70 to 79 years old as well as patients with pacemakers, Class II heart failure, cystic fibrosis and other major health conditions. They were also no more likely to be working, no more likely to be off social benefits, and no more physically fit than before, as measured by a step-test. These poor objective results are unmentioned by Professor White in the new paper—a telling omission.
Moreover, Professor White has failed to cite Wilshire et al, a 2018 re-analysis of PACE data that definitively debunked the claims that the reported results demonstrated the effectiveness of either GET or cognitive behavior therapy—his other favorite intervention. (I was a co-author of Wilshire et al.) I understand that Professor White and other members of the GET/CBT ideological brigades disagree with the conclusions of Wilshire et al. But it is inappropriate for them to ignore evidence from the published literature just because they don’t like it or find it inconvenient. And it is unacceptable that the journal’s peer review process has enabled Professor White’s decision to reject findings that do not suit his needs. That is called spin, not science. Readers of JPR deserve better.
Professor White cites his own GETSET trial of self-guided GET as further evidence in support of his effectiveness claim. Yet the purported success of that trial was based on subjective outcomes assessed right after the end of the intervention, when such responses are most likely to be at the highest risk of bias. Professor White does not cite the null results from the 12-month GETSET follow-up—perhaps because he apparently believes, per the still-uncorrected paper in JPR, that these results also represented a success.
Professor White further cites the most current Cochrane exercise review, which included eight studies and was published two years ago. Yet this updated review remains so riven with flaws that its conclusions cannot be taken at face value. Among many problems, the reviewers failed to incorporate objective results from PACE and other research, even though these outcomes overall contradicted the evidence from the subjective outcomes. And Cochrane itself has backed away from making definitive statements about the evidence, issuing the following statement to accompany the 2019 revision:
“Today, Cochrane publishes an amended version of the Review, ‘Exercise therapy for. chronic fatigue syndrome.’ In the last nine months, this Cochrane Review has been modified by the review’s authors and evaluated by independent peer reviewers and editors. It now places more emphasis on the limited applicability of the evidence to definitions of myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) used in the included studies, the long-term effects of exercise on symptoms of fatigue, and acknowledges the limitations of the evidence about harms that may occur.
“Cochrane’s Editor-in-Chief, Dr Karla Soares-Weiser, commented on the publication of the review, ‘Cochrane recognizes the importance of providing the best available evidence on interventions for ME/CFS to enable patients and clinicians across the world to make well-informed decisions about treatment. This amended review is still based on a research question and a set of methods from 2002, and reflects evidence from studies that applied definitions of ME/CFS from the 1990s. Having heard different views expressed about the evidence base for this condition, we acknowledge that the publication of this amended review will not resolve all the ongoing questions about this globally important health topic.'”
That is why Cochrane is in the process of developing an entirely new exercise review with an entirely new group of authors. Yet Professor White appears to have resolved for himself the “ongoing question” about the effectiveness of GET. He claims it is effective. Period. And I guess JPR thinks Professor White’s opinion is good enough to be presented as fact.
A peer review process that allows Professor White to get away with this sleight-of-research argument in the first sentence of his abstract does not pass the smell test. Perhaps papers from members of JPR’s editorial board get only a cursory peer review, or perhaps they get to select all their own reviewers. That’s certainly one possibility, given the decision to publish these two recent papers from Professor White. Or maybe the journal’s peer review process stinks overall.
I have not focused here on Professor White’s arguments concerning the possible harms arising from GET. But it is not necessary to discuss those issues in order to note that the underlying premise of the paper—that GET is “effective”—is unwarranted. It is troubling that JSR is once again allowing him a platform to publish stuff that he wishes were true, but isn’t.