Trial By Error, Continued: My Questions for Lancet Editor Richard Horton

By David Tuller, DrPH

In January, I posted a list of the questions I still wanted to ask the PACE authors, who have repeatedly refused my requests to interview them about their ethically and methodologically challenged study. Richard Horton, editor of The Lancet, has similarly declined to talk with me, ignoring my e-mails seeking comment for the initial investigation, posted on Virology Blog last October, as well as for several follow-up articles. Now Dr. Horton has doubled-down on his efforts to keep a lid on the controversy by rejecting a letter that he personally solicited—a major breach of professional courtesy to the 43 well-regarded researchers and clinicians who signed it.

As Dr. Racaniello explained this week at Virology Blog, he submitted the letter on behalf of the group in March, in response to an express invitation from Dr. Horton. The invitation came right after Virology Blog posted an open letter, based on my investigation, that outlined the trial’s major missteps. Dr. Racaniello presumed from the wording of Dr. Horton’s invitation that the letter would, in fact, be published, as did the other signatories. On Monday, having been dissed by The Lancet, Dr. Racaniello finally posted the letter on PubMed Commons. He also called the PACE trial “a sham.” (I’ve called it “a piece of crap.” I might also have referred to it somewhere as “doggie-poo,” but I’m not sure.)

In rejecting the letter that he himself solicited, Dr. Horton certainly appeared to be trying to squelch the growing public controversy over PACE and its recommendations that graded exercise therapy and cognitive behavior therapy are effective treatments for chronic fatigue syndrome (or myalgic encephalomyelitis, CFS, ME, CFS/ME, or ME/CFS, or some other name). But The Lancet’s effort to shield PACE is doomed, not only because the study is so bad but because the emerging science presents a completely different portrait of the illness. On Monday, a paper in Proceedings of the National Academy of Sciences reported distinct patterns of metabolites in the plasma of ME/CFS patients—an important finding that, if confirmed, could finally lead to diagnostic tests. The PNAS paper and other recent research support the conclusion of reports last year from both the Institute of Medicine and National Institutes of Health: ME/CFS is a devastating physiological disease.

Back in January, Columbia statistics professor Andrew Gelman blogged about the harm Dr. Horton was already inflicting on his journal by not addressing the serious questions that serious critics were raising about PACE. The longstanding claim of the PACE authors, The Lancet and the trial’s other defenders—that the opponents were a small cabal of irrational, dangerous, and anti-psychiatry patients—has been exposed as false. The PACE authors, The Lancet and their colleagues wielded this narrative for years to discredit those challenging the trial. To their dismay, this tactic is no longer working.

The Lancet’s decision to reject the Virology Blog letter will only compound the journal’s growing reputational damage over the issue. It also seems deeply short-sighted, in light of last month’s powerful court decision ordering Queen Mary University of London, the professional home of principal PACE investigator Peter White, to release the raw trial data. That would allow others to determine whether the PACE investigators altered their outcome assessments strategies to produce results more likely to get published in The Lancet and other journals. (The answer should not surprise anyone except those in extreme stages of denial.)

The decision involved a freedom-of-information request filed two years ago by Alem Matthees, an Australian patient. Since the published results did not include the results per the assessment methods outlined in the PACE trial protocol, Matthees wanted the data necessary to calculate those results for the two primary outcomes of fatigue and physical function, as well as for the original definition of “recovery.” Last October, the Information Commissioner’s Office, an independent agency, found that QMUL had no grounds for refusing to provide the data. QMUL appealed that ruling to the First-Tier Tribunal, which issued the recent decision.

The U.K. medical-academic-media establishment has wholly endorsed the PACE trial’s unreliable findings and accepted the authors’ unsubstantiated claims that they have been subjected to a concerted campaign of threats and harassment. In contrast, the tribunal demonstrated a refreshing unwillingness to play along. In robust language, the tribunal smacked down the specious arguments raised by the university in its attempt to shield the data from public disclosure.

The chance that any participant could or would be identified from the anonymized data was “remote,” the tribunal found. The scenarios envisioned ed by QMUL’s data security expert, who sketched out far-fetched strategies that “activist” patients might pursue to re-identify and then harass trial participants, were “grossly exaggerated” and “a considerable amount of supposition and speculation,” wrote the tribunal. In fact, noted the tribunal, the only incident of “harassment” proven by QMUL’s experienced legal team was that someone somewhere once heckled Trudie Chalder, a principal PACE investigator who also testified at the tribunal hearing. (I also have some thoughts on Dr. Chalder’s testimony, but will hold those for another time.)

In contrast to the QMUL portrait of PACE opponents, the tribunal cited Virology Blog’s open letter to The Lancet as evidence of a robust scientific debate, noting that “the identity of those questioning the research…was impressive.” The tribunal also noted that QMUL’s decision to share data with friendly researchers but not with others had created the impression that it was acting out of self-interest, not principle. “There is a strong public interest in releasing the data given the continued academic interest so long after the research was published and the seeming reluctance for Queen Mary University to engage with other academics they thought were seeking to challenge their findings,” declared the tribunal in the decision.

The PACE authors, QMUL, Dr. Horton, and The Lancet are stonewalling the obvious, at the expense of millions of sick patients. Although Dr. Horton will never grant me an interview, I want to highlight some of the questions I have about his actions, claims and thoughts, in case someone else gets the chance to talk with him. This list of questions is certainly not exhaustive, but it’s a decent start.

So, Dr. Horton–Here’s what I’d like to ask you:

1) Do you agree that the invitation you sent to Dr. Racaniello certainly implied, even if it didn’t explicitly promise, that The Lancet would publish the letter? Since the letter submitted by Dr. Racaniello, on behalf of himself and 42 other experts, reflected the points made in the Virology Blog open letter that triggered your invitation, what changed your mind about whether it added something to the debate? Since you personally solicited the letter from Dr. Racaniello and his colleagues, do you feel you should have sent him a personal apology, rather than leaving your correspondence editor, Audrey Ceschia, to answer for your behavior?

2) In your invitation to Dr. Racaniello, you noted that the PACE authors would have a chance to respond, alongside the published letter. That was a fair plan. When did that plan of offering them a response morph into the plan of offering them a role in discussions about whether to publish the critical letter in the first place? What impact did their views have on your decision? Did the PACE authors argue, as they have in the past, that they have already answered all these criticisms?

This repeated claim that they have answered all questions is simply untrue. They have never explained, for example, how it is possible to be disabled and “within normal range” on an indicator simultaneously, and why 13 % of their participants were already “within normal range” on one or both primary outcome sat baseline. When anyone asks legitimate questions, they evade, ignore or misstate the issues—including in the correspondence following The Lancet’s 2011 paper. (This pattern of non-response is clear from their non-responsive responses to the charges raised in my Virology Blog investigation, and my rebuttal of their non-responses.)

3) What’s your reaction to the First-Tier Tribunal’s decision ordering the release of the PACE trial data? Do you agree with the tribunal’s observation, referring to Virology Blog’s February open letter to you and The Lancet, that the roster of scientists and researchers now publicly questioning the methodology and findings of PACE is “impressive”?

4) Do think QMUL should spend more public money to appeal the decision?

If QMUL decides to appeal, do you think this will fuel the already-widespread assumption that PACE had null findings per the original protocol methods of assessment?

5) The PACE interventions, as described in The Lancet, are based on the premise that deconditioning rather than any pathological process perpetuates the illness, and that increased activity and a new psychological mind-set will fix the problem. The descriptions of the interventions categorically exclude the possibility of a continuing organic disease as the cause. Do you think this portrait of the illness squares with the view emerging from this week’s study in PNAS and other recent research, including last year’s reports from the Institute of Medicine and the National Institutes of Health?

6) The IOM report identified “exertion intolerance”—the prolonged relapses patients often suffer after minimal activity–as the core symptom of the illness. Yet a key aspect of the PACE rehabilitative interventions, GET and CBT, is urging patients to increase their activity and to interpret a resurgence of symptoms as a transient event, not a sign of deterioration. Given the IOM’s focus on “exertion intolerance” as the central phenomenon, isn’t the PACE approach contraindicated?

7) Does it bother you that you published a paper in which 13% of the sample had already, at baseline, met the outcome thresholds for one or both primary measures? These outcome thresholds, which represented worse health than the entry criteria, were variously defined as being “within normal range” (the Lancet paper), “back to normal” (Dr. Chalder’s statement at the press conference for the Lancet paper), and “a strict criterion for recovery” (the Lancet commentary by colleagues of the PACE authors). Can you point me to any other studies published in The Lancet, or anywhere, in which positive outcome scores represented worse health than entry criteria?

8) Does it bother you personally that the PACE authors did not inform you or your editorial staff that a significant minority of patients were already “within normal range” on at least one primary outcome at baseline? (I presume they didn’t mention it to you because, well, it’s hard to imagine you would have published the paper if you or anyone there had been told about or noticed the inexplicable overlap in the entry criteria and the post-hoc “normal range” thresholds.)

9) During a 2011 Australian radio interview not long after The Lancet published the first PACE results, you said the following about the trial’s critics: “One sees a fairly small, but highly organised, very vocal and very damaging group of individuals who have I would say actually hijacked this agenda and distorted the debate so that it actually harms the overwhelming majority of patients.” Given that the First-Tier Tribunal expressed a different perspective on the stature and credibility of those criticizing PACE, do you still agree with your 2011 characterization of the trial’s opponents?

10) During the same interview, you stated that the PACE trial had undergone “endless rounds of peer review.” Yet the trial was also “fast-tracked” to publication, as indicated on the version of the article in the ScienceDirect database. Can you explain the mechanics of “fast-tracking” a paper to publication while simultaneously subjecting it to “endless rounds of peer review”? How long was the fast-track process for the PACE paper, and how many actual rounds of review did the paper undergo during that endless period?

11) Can you explain why the Lancet’s endless peer review process did not catch the most criticized aspect of the paper—the very obvious fact that participants could be simultaneously disabled enough for entry yet already “within normal range”/”back to normal”/”recovered” on the primary outcomes? Can you explain why the reviewers did not request the authors to provide either the original results promised in the protocol or else sensitivity analyses to assess the impact of the mid-trial changes they introduced?

12) Do you think it was appropriate for the PACE investigators to publish a mid-trial newsletter that promoted the therapies under study and included glowing testimonials from earlier participants about their excellent outcomes? Can you point to other published studies that featured such mid-trial dissemination of personal testimonials and explicit descriptions of outcomes? The PACE authors have stated that the newsletter testimonials did not identify participants’ trial arms and therefore could not have created any bias. Do you agree with this novel and creative argument that influencing all remaining participants in a trial in a positive direction is not a form of bias?

13) Did The Lancet’s peer review process include an evaluation of the PACE trial’s consent forms, given the authors’ explicit promise in the protocol to abide by the Declaration of Helsinki? The Declaration of Helsinki requires investigators to disclose “any possible conflicts of interest” not just to journals but to prospective participants. Yet the PACE consent forms did not disclose the authors’ close financial and consulting ties with the insurance industry. Do you agree this omission violates their protocol promise, and that given this violation the PACE authors failed to obtain legitimate informed consent from their participants? Without legitimate informed consent, did the PACE authors have the right to publish their findings in The Lancet and other journals? What should happen to the PACE papers already published, since the authors do not appear to have legitimate informed consent from participants?

14) Who do you think should be held responsible for the $8,000,000 in U.K. government funds wasted on the PACE trial? Who should be held responsible for the harm it has caused? What responsibility, if any, does The Lancet bear for the debacle?

Comments on this entry are closed.

  • Quel dommage… you get ’em, David!!!!

  • Auctis

    These are wonderful questions and of great public importance given PACE represents not just a scientific debate on say, plant steroid metabolism, but a debate which will have positive or negative results on Myalgic Encephalomyelitis patients access to:
    *care,
    *the type of care,
    *the type of research to be conducted in the future,
    * for the very sick-access to live saving welfare resources ,
    * correct treatment in hospitalizations, etc.
    * treatment which is supportative, or treatment which makes them worse.

    In short the whole panoply of social effects on real people, probably a million or more of them in the USA and UK.

    Hopefully other journalist, scientists, and public health officials, will also take up the mantle of these questions to Horton.

    It’s shameful the way PACE used the flawed Oxford definition and then claimed on high their results had global applicability to Myalgic Encephalomyelitis. It’s doubly shameful they won’t release anonymized data for a tax payer funded study. The trials ethics breaches make it triply shameful…I could go on but the above letter indeed covers all of these more eloquently!

  • Rebecca

    Fantastic, Dr. Tuller! Thank you!

  • Sandra

    What a straightforward, well-deserved, stinging rebuke of the PACE fiasco and all involved therewith. For many years I often thought we needed and prayed for a “David” to arrive on the scene to fight this “Goliath” of powerful interests blocking the truth about the debilitating physiological disease ME/CFS, thus preventing patients from getting the help, support, and respect they needed. I didn’t know if that would ever happen in my lifetime. But it has, thanks to you, David Tuller! You are that “David,” and I and the ME/CFS community are immensely grateful.

  • clark ellis

    You know, the problems with the paper are so bad I think there have to be doubts about whether it actually had any objective peer review at all. Maybe Horton invited the authors to do it themselves? That would be consistent with the power he gave them to veto the publication of your criticism. Compare also to the BMJ’s report of his role in Wakefield’s MMR scandal, which when brought to his attention, he let those authors conduct their own investigation into their own study. He’s consistent at least.

  • Loetta Vann

    I can fell them shifting in their seats from here. Thank you for pushing this forward and not stopping.

  • John Leslie Whiting

    Is it a conflict of interest for the editor of the Lancet, Dr Richard Horton, to be a strong endorsing party to the results of the PACE trial in a radio broadcast with Norman Swan in 2011?

    Was Dr Horton in Australia at the time and if so, what was he (and Michael Sharpe) doing there ?

    Insurance companies Independent Medical Examiners have guest speakers to keep them up to date with matters that assist such companies’ agendas. I can’t think of any other reason for such a visit by both doctors, if they did indeed visit Australia together.

  • Fleidman

    One might also ask: How did the PACE TRIAL, with all it’s ‘obvious’ errors, get past the reviewers? Who were the reviewers and what papers have they been involved in since?

  • Rosie Cox

    Love it! Slam dunk!

  • Tim Heatley

    Since this trial data is Government funded would it be appropriate to ask the National Audit Commission to investigate and comment on the data presentation and interpretation. Tim Heatley, M.E. Positive East Midlands

  • This isn’t the only rotten Lancet article. Nature publishes outrageous drivel as well.
    Nature published this article in May of 2016. “Apollo Lunar Astronauts Show Higher Cardiovascular Disease Mortality: PossibleDeep Space Radiation Effects on the Vascular Endothelium.” Delp, et al. This article is so bad if it were given to me as a class project, I wouldn’t even grade it. I would go back and apologize to my class, because I had failed my students on fundamental concepts.

    First, it’s obvious that with one group having a radically higher accidental death rate (49%) and the other less than 1/3rd (14%) this will skew cause of death from CVD. This is so glaringly obvious it whacks you over the head if you look at their data table.

    Second, the cancer stats are exactly as I would expect, [normal for non-smoker, healthy population, lower than total averages] and no significant deviation for lunar vs LEO astronauts. I cannot fathom an oxidation/stress mechanism that could have that strong an effect on CVD, but show nothing relative to cancer. In cancer treatment patients, to see such effects long-term requires very high dosing – orders of magnitude higher. These astronauts got a maximum (per this
    article) of 1.14 cGy which is 0.0114 Gy over 2 weeks. (In point of fact, I think they meant Sievert here.) But how on earth does that justify dosing their mice with 87 times the dose in gray the astronauts received?! And it’s an acute dose, delivered in seconds! The astronauts dose was distributed over weeks!

    Third, how did this study from supposed scientists use gray (Gy) instead of Sievert (Sv)? I saw that and initially thought, “Oh, alright. It’s wrong, but since their mouse work probably used x-rays and the RBE for that is 1, it’s ok.” But then I saw they bombarded these mice with Fe56 nuclei?!!

    The RBE conversion for Fe56 ions is 30-40 times Gy. Yes, it depends on the accelerator used, but Brookhaven isn’t going to spit out low velocity Fe56 ions. If these mice really got an acute dose of 30-40 Sv you should see them dying of radiation sickness if they were left to do so, and rather rapidly. Yes, at that dose, you could see some signs of cardiovascular inflammation.

    Fourth, it is known that high RBE irradiation tends to kill by damaging intestines more than bone marrow. So we should also expect to see damage to to vascular epithelium as well.

    Fifth, do the math and the real acute dose multiple for the mice versus the spread out dose for the astronauts was around 3,000X, not 87X, because the astronaut dose was in the ballpark for Sv for their trips to the moon, but the mouse dose was misrepresented.

    Conclusion – Either these scientists are utterly incompetent and not fit to hold a job anywhere, or they deliberately falsified their study in order to support a pre-ordained conclusion. That Nature published such garbage is a testament to how incompetent Nature’s publishing game is in the area of radiation effects. Nature’s 2012 publication titled, “The biological impacts of the Fukushima nuclear accident on the pale grass blue butterfly.” Hiyama, et al. is a similar level of incompetent rubbish.

  • Hmm. In the USA, a lawsuit can be filed for fraudulent use of government funds, and multiples of the grant money used can be forced payback to the government, with the plaintiffs getting 10% or so of the proceeds.

    Before rejecting such a suit out of hand, consider that a court case opens up the parties for discovery, and discovery can include things that aren’t subject to FOIA.

  • Julia Browell

    They were advising the Aust govt on health (cfs and MUPS/MUS … and biopsychosocial claptrap
    being among them) and pension matters. Along with Wesselley and White they have made a number of trips to Australia over the years and having long chats with Prime Ministers, Health Ministers, Human Services Ministers (inc pension and benefits) as well as senior public servants and influential doctor associations. My understanding is they have done similar in New Zealand as well.

  • clark ellis

    It’s a problem across science, and until we tackle it we will continue to get more junk like these studies. In the case of PACE it is junk that sets treatment and social policy for millions of vulnerable people, so its particularly worrying, and despite concerns over the safety of these treatments there is now a trial of the same treatments being done in children.

  • b. moore

    I am reading John Le Carre’s “The Constant Gardener,” written in 2001. I quote directly from p.240, which is itself a direct quote from Stuart Pocock’s “Clinical Trials” (a real book): “There is a tendency for students, and indeed many clinicians, to treat the medical literature with undue respect. Major journals such as the Lancet and the New England Journal of Medicine are presumed to present new medical facts which are not to be disputed. Such a naive faith in the ‘clinical gospels’ is perhaps encouraged by the dogmatic style that many authors adopt, so that uncertainties inherent in any research project often receive inadequate emphasis…” (The quote refers directly to the plot of the book which deals with the pharmaceutical companies using the African nations’ residents as human guinea pigs for testing their new drugs before taking them to the “expensive” markets. Fiction? Not likely.)

    Thank you, David Tuller, for these immeasurably valuable articles. Please keep pushing Lancet!

  • Umair Amin

    Visiting http://www.virology.ws/ for the first time. Nice Effort. 🙂

  • John Leslie Whiting

    Julia, this is incredible news.

    If this is so, then Horton, White et al. must have been given red carpet clearance by an eager and hungry Australian Government to allow or dare I say encourage these ‘scientists’ to significantly influence Australian health policies by suitable, albeit flawed information (science be damned) that these British readily provided to them. This ‘medical’ information was obviously welcomed as a means to meet a very desired Australian Government agenda, whose consequences we now experience as both unjust and harmful.

    I can see now why Dr Horton must keep his trap shut, or else, all hell will break lose.

  • John Leslie Whiting

    I would like to hear more. Who, in particular, did they meet?. Of course, the why is obvious, but the timing is particularly damning, as the Australians they met had agendas that they needed to implement urgently. The changes in the behaviour of members of Centrelink, under the auspices of the Department of Human Resources, particularly those ‘responsible’ for how patients are assessed, have been dramatic and draconian on pension holders. These changes were implemented in January 2012, and seem to be the direct result of the meetings these people had with them. I suspect that their travel arrangements were all funded by the Australian Government.

    How will management practices of patients change, now that the scam has been unveiled?(September 22, 2012)

  • GQ

    Thank you for your perseverance and determination in pursuing this with the Lancet.

    Richard Horton wrote this last year in 2015 and should be read by everyone.

    http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(15)60696-1/fulltext

    He is well aware of the problem. Horton is so sanctimonious in this article and is now in a position to act on his words when faced with the science scandal of the decade of the PACE trial but he does nothing and instead smears and ignores the critics. However he is at least consistent as he has displayed seriously bad judgement with Wakefield and the recent Machiarini scandals also on his watch and where he similarly buried his head in the sand.

  • Pingback: Warum hat trainieren nicht geholfen? | Posturales Tachykardie Syndrom()