By David Tuller, DrPH
I have pressed the Journal of Psychosomatic Research to correct a recent paper, “Guided graded exercise self-help for chronic fatigue syndrome: Long term follow up and cost-effectiveness following the GETSET trial.€ The senior author is Professor Peter White. Now the journal has published a revised €œHighlights€ section of the paper that accurately presents the study’s null results for the intervention rather than claiming success on an illegitimate basis. But the fact that this laudable change has been made is not indicated.
The €œHighlights€ section lists four main points taken from the study. It appears in the web version on the ScienceDirect platform but not in the pdf file, so I guess its actual status as an integral part of the paper is ambiguous. Should changes made to €œHighlights€ be noted as corrections, as with other sections? I would say yes. But others might regard it more as adjunct material that can be altered without notice, the way news headlines often are. (Perhaps this matter has already been adjudicated by the people who adjudicate these things, or perhaps it hasn’t arisen much before.)
Unfortunately, the abstract’s conclusion remains the same, and it is still misleading. The Journal of Psychosomatic Research should take the next step and correct this section as well.
Before explaining the details, I want to thank the journal’s editor-in-chief, Professor Jess Fiedorowicz, for taking the concerns I expressed seriously. I appreciate that he and his editorial team considered the matter and did something to rectify it. Everyone who now accesses the paper from the ScienceDirect site will see an accurate account of the results, rather than false claims of success, in a prominent place on the screen. That represents a win for science over spin, however modest.
(By way of background, the Journal of Psychosomatic Research is the official journal of the European Association of Psychosomatic Medicine (EAPM). Professor Michael Sharpe is the current vice president of the association’s executive council. Professors Per Fink and Judith Rosmalen are council members, as is Professor Fiedorowicz.)
**********
More of the details€¦
In a clinical trial, the comparison of concern is between the study arms. In this study, despite some reported short-term benefits, none were apparent at the 12-month follow-up. This null finding is the main message of the study. Instead, Professor White and his colleagues presented the €œwithin-group€ findings as the most important metric, that is, they looked at whether those who received the intervention continued to report feeling better.
But that’s not what they set out to study. This was a clinical trial. To focus on the €œwithin-group€ comparison rather than the €œbetween-group€ comparison was a form of unauthorized and unacceptable outcome switching.
I wrote about this paper here, and posted my letter to Dr Fiedorowicz, the journal’s editor-in-chief, here. These paragraphs from the letter explain the problem:
In this clinical trial, the investigators were testing a self-help graded exercise program, which they reported had shown some short-term benefits. According to the new study, Clark et al, the intervention provided no benefits at one-year follow-up over specialist medical care (SMC). Yet here’s how the findings were described in the €œhighlights€ section: €œGuided graded exercise self-help (GES) can lead to sustained improvement in patients with chronic fatigue syndrome.€
Given the null results, this description is troubling. In the study abstract, the conclusion is marginally better but still unacceptable: €œThe short-term improvements after GES were maintained at long-term follow-up, with further improvement in the SMC group such that the groups no longer differed at long-term follow-up.€
In sum, the results were presented as if the trial had shown the intervention to be effective€“even though the one-year findings should have reasonably led to the opposite assessment. The fact that the non-intervention group scored the same at the end is framed as a matter of lesser significance€“ignored completely in the €œhighlights€ section and given second billing in the abstract’s conclusion.
At some point in recent weeks, the previous first sentence of the €œHighlights€ section has been removed and replaced with this more accurate account: €œThere were no differences between interventions in primary outcomes at long-term follow up.€ (The remaining three points in the €œHighlights€ section are the same as before.)
Unfortunately, the lack of a correction means that no one reading the paper now would know that Professor White and his colleagues presented a problematic version of events, and that the journal allowed that version to pass peer review and be published. Since Professor White has repeatedly engaged in this sort of misrepresentation, such details are valuable for the historical record. Now they have been officially erased.
Beyond the €œHighlights€ section, the abstract’s conclusion is still unacceptable. It continues to prioritize the within-group comparison over the between-group results. Rewriting the conclusion would indisputably require a formal correction. Perhaps one is still forthcoming.
**********
Professor White’s Other Problematic Study
Besides the issue of the GETSET follow-up study, I have also appealed to Professor Fiedorowicz about a second paper from Professor White. This one relates to the question of possible harms from GET. In a post earlier this month, I described how this paper misrepresented the state of the science on the intervention in the first sentence of the abstract.
After posting the blog, I sent the following letter. I have not yet received a response about this concern.
**********
Dear Professor Fiedorowicz–
As you know, I praised you and your colleagues earlier this year for noting that studies that are not rigorously blinded but that nonetheless rely on subjective outcomes are at high risk of bias. I also pointed out that the journal’s advisory board includes some well-known investigators who specialize in just that kind of research. Unfortunately, the journal keeps publishing such studies, which would appear to undermine its credibility and raises questions about whether the editorial team is actually in charge of editorial policy.
In particular, I noted in previous communications the recent paper from Professor White and colleagues of one-year follow-up results from GETSET, a study of self-guided graded exercise therapy (GET) for the illness they call chronic fatigue syndrome (CFS). The initial study, of course, was unblinded and relied on subjective outcomes, so its findings were already inherently suspect by your own standards.
Beyond that issue, the follow-up study in JPR managed to present null results at one year as a success by engaging in blatant outcome-switching. The authors prioritized the “within-group” results, noting first that the purported early benefits from GETSET were maintained at follow-up, rather than the “between-group” comparison that is the key result of any clinical trial. In this case, there were no demonstrable differences between the intervention and comparison arms.
The paper appeared to have been written in order to influence the deliberations of a committee considering revisions to a draft of clinical guidelines for ME/CFS sponsored by the UK’s National Institute for Health and Care Excellence (NICE). As I have noted, this paper warrants a significant correction.
Now JPR has compounded the issue by publishing yet another paper from Professor White on GET. The new paper, which he co-wrote with a colleague, is called €œAdverse outcomes in trials of graded exercise therapy for adult patients with chronic fatigue syndrome.€ The first sentence of the abstract states that €œgraded exercise therapy (GET) is an effective treatment for chronic fatigue syndrome (CFS), but concerns have been raised about its safety.€
The problem with this statement should have been obvious to knowledgeable and impartial peer reviewers. Professor White is presenting the effectiveness of GET as a settled issue and suggests that only safety concerns remain in dispute. This notion is categorically false. This assertion about GET represents Professor White’s personal opinion based on the kind of research that you and your colleagues have explicitly highlighted as suffering from a high risk of bias. Since all the papers he cites in support rely on unblinded studies with subjective outcomes, his apparent certainty is unfounded. As a review of the current literature would reveal, the best possible case to be made for GET is that its effectiveness is highly contested. Professor White should not be allowed to make such a claim as if it were an established and accepted fact.
The paper’s introduction itself immediately undermines the claim. It notes that the NICE draft of new ME/CFS clinical guidelines rated all the evidence for GET as being of €œlow€ or €œvery low€ quality. The introduction also quotes the NICE draft’s stark warning: €œDo not offer people with ME/CFS: any therapy based on physical activity or exercise as a treatment or cure for ME/CFS.€ In other words, Professor White’s unequivocal first sentence about the effectiveness of GET does not conform to the available data.
In fact, the PACE trial, touted here by Professor White, one of its lead investigators–included four objective outcomes. None of them matched the subjective outcomes. While the GET group did post a statistically significant improvement in one of the measures, the six-minute walking test, the marginal reported benefits were clinically insignificant. At the end of PACE, GET participants still performed much more poorly on this measure than healthy women from 70 to 79 years old as well as patients with pacemakers, Class II heart failure, cystic fibrosis and other major health conditions. The participants in the GET arm were also no more likely to be working, no more likely to be off social benefits, and no more physically fit than those in the comparison arms. These disastrous objective results from Professor White’s own research are unmentioned in the new JPR paper, a telling omission of salient information that indisputably contradicts his statement about GET’s effectiveness.
Moreover, Professor White has failed to cite Wilshire et al, a 2018 re-analysis of PACE data, which authoritatively debunked the claims that the reported results demonstrated the effectiveness of either GET or cognitive behavior therapy, his other favorite intervention. (I was a co-author of Wilshire et al.) I understand that Professor White disagrees with the conclusions of Wilshire et al. But it is inappropriate for him to ignore evidence from the published literature just because he doesn’t like it or finds it inconvenient. And it is unacceptable that the journal’s peer review process has served to enable Professor White’s decision to reject findings that do not suit his needs or his theoretical framework.
Professor White cites his own GETSET trial of self-guided GET as further evidence in support of his effectiveness claim. Yet the purported success of that unblinded trial was based on subjective outcomes assessed right after the end of the intervention, when such responses are most likely to be at the highest risk of bias. Professor White does not cite the null results from the 12-month GETSET follow-up, perhaps because he apparently believes, per the still-uncorrected paper in JPR, that these results also represented a success.
Professor White further cites the most current Cochrane exercise review, which included eight studies and was published two years ago. Yet this updated review remains so riven with flaws that its conclusions cannot be taken at face value. Among many problems, the reviewers failed to incorporate objective results from PACE and other research, even though these outcomes overall contradicted the evidence from the subjective outcomes. Cochrane itself has backed away from making definitive statements about the evidence, issuing the following advisory to accompany the 2019 revision:
€œToday, Cochrane publishes an amended version of the Review, ‘Exercise therapy for. chronic fatigue syndrome.’ In the last nine months, this Cochrane Review has been modified by the review’s authors and evaluated by independent peer reviewers and editors. It now places more emphasis on the limited applicability of the evidence to definitions of myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) used in the included studies, the long-term effects of exercise on symptoms of fatigue, and acknowledges the limitations of the evidence about harms that may occur.
€œCochrane’s Editor-in-Chief, Dr Karla Soares-Weiser, commented on the publication of the review, ‘Cochrane recognizes the importance of providing the best available evidence on interventions for ME/CFS to enable patients and clinicians across the world to make well-informed decisions about treatment. This amended review is still based on a research question and a set of methods from 2002, and reflects evidence from studies that applied definitions of ME/CFS from the 1990s. [These definitions did not require the presence of post-exertional malaise, the way later definitions do.] Having heard different views expressed about the evidence base for this condition, we acknowledge that the publication of this amended review will not resolve all the ongoing questions about this globally important health topic.’€
Because of these issues, Cochrane is in the process of developing an entirely new exercise review with an entirely new group of authors. Nonetheless, Professor White appears to have resolved for himself the €œongoing question€ about the effectiveness of GET. He says it is effective. Period. And JPR has apparently now endorsed this unwarranted declaration.
I have not focused here on Professor White’s arguments concerning the possible harms arising from GET. But it is not necessary to discuss those issues in order to note that the underlying premise of the paper, that GET is effective, cannot be justified based on the current research base. It is troubling that JSR is once again providing Professor White with a platform to present his personal views as fact. This statement needs to be corrected and appropriately qualified.
I have cc’d Professor Vincent Racaniello, a virologist at Columbia and host of Virology Blog, where I have posted a blog about this issue. I have also cc’d members of the NICE committee for the ME/CFS guidance, in case this problematic paper is raised during the deliberations over draft revisions. I have not cc’d Professor White, since I have been led to believe that he and his colleagues consider messages from me to be unwelcome.
Best, David
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley
**********
I followed up quickly with this note to clear up a type:
Dear Professor Fiedorowicz–
I have just noticed that in the second-to-last paragraph, I referred to the journal as JSR rather than JPR. I apologize for the error.–
Best–David
Comments are closed.