virology blog

About viruses and viral disease

vexatious

Trial By Error, Continued: An Open Letter to the Board of the CFS/ME Research Collaborative

by

by David Tuller, DrPH

To Members of the Board of the CMRC:

Not long ago, at the annual conference of the British Renal Society, your deputy chair disseminated the false accusation that I had libeled her. As a corollary to that, she also disseminated the false accusation that Dr. Racaniello, the Columbia University microbiologist who hosts Virology Blog, had libeled her by publishing my work. I provided Dr. Crawley with a reasonable opportunity to offer either an explanation, evidence to support her serious charge, or an apology. Dr. Crawley has done none of these things.

I interpret that as Dr. Crawley’s admission that there is nothing in my blogs that needs correcting, and therefore nothing libelous. Yet her claim, which goes to the heart of my integrity and professional reputation, remains floating around out there in the ether. Slides that go viral live online forever. (Dr. Crawley’s lecture, of course, raises other concerns as well. For the moment, I am focusing on the one that involves me.)

I have no reason to believe this is the first time Dr. Crawley has made these reckless charges against Dr. Racaniello and me. She has probably stirred many audiences with her tales of courage in confronting “anti-science” zealots. However, I have no evidence of that, so I will pretend to believe this was an isolated episode. But I would be very, very unhappy to learn that Dr. Crawley has continued making such unsupported allegations. (Why was Dr. Crawley invited to talk to a group of kidney specialists, anyway? Can cognitive behavior therapy and graded exercise therapy now cure kidney disease just like they can cure ME/CFS?)

Although Dr. Crawley’s accusation is libelous and possibly actionable, I have no intention of suing her, as I have already indicated. But I do believe in moral closure. Dr. Crawley has so far failed her moral, ethical and professional obligation to rectify the situation. Given that she is the CMRC deputy chair, her unwillingness to clean up her mess also reflects poorly on the collaborative. Although I have conveyed my concern to you, I have heard nothing from the board. Perhaps I’ll hear something at some point in the future. But in not disavowing Dr. Crawley’s accusations in a timely manner, the CMRC is effectively endorsing and enabling her behavior, as well as allowing these falsehoods to continue to percolate among medical professionals. Your silence, like Dr. Crawley’s, is not acceptable to me. Nor is it acceptable to Dr. Racaniello.

On June 1st, I am honored to be giving the keynote presentation at the dinner before this year’s Invest in ME conference. I was planning to talk about PACE, and why it is–in my opinion—an incoherent pile of nonsense. I had not planned on talking about Dr. Crawley or the CMRC. Thanks to Dr. Crawley’s attention-getting performance, my plans have now obviously changed. I will of course still discuss PACE’s failings. But I’ve never been accused of libel before, so I’m eager to share the experience with colleagues.

Plus, Dr. Crawley herself has provided fantastic visuals: There she is on stage, and behind her looms an enormous slide that features, among other items, the phrase “libellous blogs” and a screen shot of one of my Virology Blog posts, along with my name. Slam dunk! Here I am, about to give a high-profile talk in front of top scientists and ME/CFS experts from around the world, and the cosmos gifts me with a slide that captures a researcher in the very act of slandering me! Even better, she happens to be a researcher very well-known to many if not all in the audience, by reputation if not in person! How cool is that? Small world!

In my talk, I will portray Dr. Crawley’s accusation as an example of the scare tactics employed by scientists unable to provide satisfactory and adequate explanations for their methodological choices. In Dr. Crawley’s case, as I reported in the purportedly libelous blog captured immortally on screen, these troubling choices include the routine conflation of “chronic fatigue” and “chronic fatigue syndrome.” That conflation dramatically inflates the reported prevalence of the illness and leads to all kinds of problems.

Dr. Crawley and other PACE cheerleaders, not to mention the PACE investigators themselves, are similarly unable to provide logical answers to key questions like: “How was it possible that 13% of the PACE participants were already ‘recovered’ on one or both primary outcomes at baseline, before any treatment at all? Why was this key information not included in the published papers? Why were these participants even in the trial in the first place?” It is easier to accuse me of libel than to answer these prickly questions.

Dr. Crawley also showed a slide of the patient petition campaign against the MEGA project. Perhaps she believes that the Wellcome reviewers were negatively influenced by this organized opposition when they rejected her preliminary application. That could easily be the case, although it is also possible, as I noted earlier, that the MEGA proposal was simply not up to Wellcome’s exacting standards. Unfortunately for Dr. Crawley going forward, this most recent incident and the viral photos of her “anti-science” lecture are likely to stick to her and her professional reputation like glue. Through an easy google search, prospective colleagues, collaborators, grant reviewers and major funders will likely learn that she has accused other researchers of libel without explaining herself, providing any evidence, or apologizing.

It is also troubling that Dr. Crawley has, once again, positioned the filing of requests under the U.K.’s freedom of information (FOI) law as part of the “anti-science” playbook. Her slides on “vexatious requests” and other FOI exemptions suggest she was schooling the nephrologists in the most effective ways to sidestep the law’s requirements and avoid legal obligations to provide documents and information. I wonder if the University of Bristol believes it is appropriate for Dr. Crawley to be disseminating such advice. I doubt very much she informed the nephrologists that the most high-profile decision on a FOI request related to PACE, from the First-Tier Tribunal last summer, demolished the argument that the request was “vexatious.”

In ordering the liberation of the raw, anonymized trial data, the First-Tier Tribunal also noted widespread concern in the scientific world about the PACE trial’s irregularities. In particular, the decision cited an open letter to The Lancet and its editor, Richard Horton, that I organized last year. That letter, posted on Virology Blog, was signed by dozens of leading scientists and clinicians, who wrote that the PACE trial’s flaws “have no place in published research.” An open letter to Psychological Medicine, posted in March and demanding retraction of the PACE “recovery” paper, was signed by more than 100 experts. In other words, any suggestion from PACE defenders that only “anti-science” patients are challenging this body of deficient research is demonstrably untrue.

The First-Tier Tribunal also found no evidence supporting the PACE investigators’ hyperbolic claims of being the victims of a campaign of harassment, although the ruling acknowledged that Trudie Chalder had been heckled somewhere. Let me reiterate this critical point: The expensive lawyers for Queen Mary University of London, representing the interests of the PACE team to the best of their august abilities, presented NO CONVINCING EVIDENCE that the investigators were subject to abuse and death threats. (Having said that, I have no doubt that Dr. Crawley and other researchers have received many distasteful, hostile e-mails. While that is deeply unfortunate, it is no excuse for the persistent refusal to acknowledge flaws in the research.)

Moreover, as a result of the court order demanding release of the data, everyone can now see what was obvious to patients long ago: The PACE investigators changed their endpoints mid-stream in ways that jacked-up their reported rates of “improvement” and “recovery.” Then, when patients wanted to see the results per the methodology promised in the protocol, the PACE investigators stonewalled and called those requesting the data “vexatious.”

At first, the “vexatious” meme worked for the PACE team. But not anymore. The First-Tier Tribunal saw right through it. The scientific case against PACE has now been documented in peer-reviewed journals, and more publications are on the way. Dr. Crawley might not have liked the First-Tier Tribunal decision. She can misrepresent the results of the reanalyses of the PACE trial data, as she did in an interview several months ago, and continue to hail PACE as a “great, great” trial. But invoking “vexatiousness” at this stage, after the tribunal ruling and publication of the reanalyses, is not a viable strategy.

Dr. Crawley has failed to notice that the situation has shifted dramatically in the past two years. Top scientists outside the bubble-think of CBT/GET adherents have scrutinized the PACE study, found themselves shocked at its deficiencies, and confirmed that the reported results are bogus. They have made their views public to demonstrate the depth of their convictions about the study’s flaws. Does Dr. Crawley consider all of them “vexatious”?

It is understandable the CMRC board would prefer to hunker down and ignore the growing public relations disaster that Dr. Crawley has created. But your group has already stumbled multiple times in its efforts to gain credibility with the patient community, not to mention with funding bodies like Wellcome. You don’t have many chances left, if any. Dr. Crawley’s viral cri de coeur against patients (and others) seeking the truth about PACE represents another black mark for the organization.

Ignoring Dr. Crawley’s behavior or letting the matter drag on for weeks and months is not a recommended strategy. With no explanation or apology for your deputy chair’s actions anywhere in sight, the CMRC’s reputation is sustaining serious ongoing damage. My best advice is to behave like responsible scientists and address the matter of Dr. Crawley’s unjustified public accusations against Dr. Racaniello and me. Enough is enough.

Best—David

David Tuller, DrPH
School of Public Health
Graduate School of Journalism
University of California, Berkeley

Trial By Error, Continued: A Few Words About “Harassment”

by

By David Tuller, DrPH

David Tuller is academic coordinator of the concurrent masters degree program in public health and journalism at the University of California, Berkeley.

 

Last week, a commentary in Nature about the debate over data-sharing in science made some excellent points. Unfortunately, the authors lumped “hard-line opponents” of research into chronic fatigue syndrome with those who question climate change and the health effects of tobacco, among others—accusing them of engaging in “endless information requests, complaints to researchers’ universities, online harassment, distortion of scientific findings and even threats of violence.”

Whatever the merits of the overall argument, this charge—clearly a reference to the angry response of patients and advocates to the indefensible claims made by the PACE trial–unleashed a wave of online commentary and protest on ME/CFS forums. Psychologist James Coyne posted a fierce response, linking the issue specifically to the PACE authors’ efforts to block access to their data and citing the pivotal role of the Science Media Centre in the battle.

The Nature commentary demonstrated the degree to which this narrative—that the PACE authors have been subjected to a wave of threats and unfair campaigning against their work and reputations—has been accepted as fact by the UK medical and academic establishment. Despite the study’s unacceptable methodological lapses and the lack of any corroborating public evidence from law enforcement about such threats, the authors have wielded these claims to great effect. Wrapping themselves in victimhood, they have even managed to extend their definition of harassment to include any questioning of their science and the filing of requests for data—a tactic that has shielded their work from legitimate and much-needed scrutiny.

Until recently, complaining about harassment worked remarkably well for the PACE team. Maybe that’s why they tried claiming victimhood again last October, when Virology Blog ran “Trial By Error,” my in-depth investigation of PACE. The series was the first major critique of the trial’s many indefensible flaws from outside the ME/CFS patient and advocacy community. Afterwards, the investigators complained that “misinformation” and “inaccuracies” in my stories had subjected them to “abuse” on social media and could cause them “a considerable amount of reputational damage.”

These claims were ridiculous—an attempt to deploy their standard strategy for dismissing valid criticisms. The PACE authors amplified this error in December, when they rejected Dr. Coyne’s request for data from a PACE paper published in PLoS One as “vexatious.” They had called previous requests from patients “vexatious” without attracting negative comment or attention—except from other patients. But applying the term to a respected researcher backfired, drawing howls from others in the scientific community with no knowledge of ME/CFS—the PACE team’s action was “unforgivable,” according to Columbia stats professor Andrew Gelman, and “absurd,” according to Retraction Watch.

(In fact, the PLoS One data, when ultimately released, will show that the paper’s main claim—that the PACE-endorsed treatments are cost-effective—is based on a false statement about sensitivity analyses, as I reported on Virology Blog.)

How did this theme of harassment and “vexatiousness” become part of the conversation in the first place? Starting in 2011, a few months after The Lancet published the first PACE results, top news organizations began reporting on an alarming phenomenon: Possibly dangerous chronic fatigue syndrome patients were threatening prominent psychiatrists and psychologists who were researching the illness. These reports appeared in, among other outlets, the BMJ, the Guardian, and The Sunday Times of London. The Times headline, a profile of Sir Simon Wessely, a longtime colleague of the PACE authors, was typical: “This man faced death threats and abuse. His crime? He suggested that ME was a mental illness.”

One patient had supposedly appeared at a PACE author’s lecture with a knife. Other CFS researchers had received death threats. Sir Wessely famously said that he felt safer in Afghanistan and Iraq than in the UK doing research into the disease—a preposterous statement that the press appeared to take at face value. News accounts compared the patients to radical animal terrorists.

According to the news reports, the patients objected to the involvement of these mental health experts because they were anti-psychiatry and resented being perceived as suffering from a psychological disorder. Editorials in medical journals and other publications followed the news accounts, all of them defending “science” against these unwarranted and frightening attacks.

In fact, the Science Media Centre orchestrated the story in the first place—not surprising, given its longtime association with the PACE team and its uncritical promotion of the various PACE papers. According to a 2013 SMC report reviewing the accomplishments of the first three years of its “mental health research function”: “Tom Feilden, science correspondent for BBC Radio 4’s Today programme, won the UK Press Gazette’s first ever specialist science writing award for breaking the story the SMC gave him about the harassment and intimidation of researchers working on CFS/ME. The SMC had nominated him for the award.”

It’s great that the SMC not only spoon-fed Feilden the story but was so pleased with the reporter’s hard work that it nominated him for a prestigious award. In a brochure prepared for SMC’s anniversary, Feilden himself thanked the centre for its help in organizing the scoop about the “vitriolic abuse” and the “campaign of intimidation.”

Of course, patients were attacking the PACE study not because they were anti-science or anti-psychiatry but because the study itself was so terrible, as I reported last October. Luckily, a growing number of scientists outside the field, like Dr. Coyne and the top researchers from Columbia, Stanford and Berkeley who signed an open letter to The Lancet demanding an independent review, have now recognized this. How are patients supposed to react when a study so completely ignores scientific norms, and no one else seems to notice or care, no matter how many times it is pointed out?

The PACE study’s missteps rendered the results meaningless. Let’s recap briefly. The investigators changed their primary outcomes in ways that made it easier to report success, included outcome measures for improvement that were lower than the entry criteria for disability, and published a newsletter in which they promoted the therapies under investigation. They rejected as irrelevant their own pre-selected objective outcomes when the results failed to uphold their claims, and used an overly broad definition for the illness that identified people without it. Finally, despite an explicit promise in their protocol to inform participants of “any possible conflicts of interest,” they did not tell them of their work advising disability insurers on how to handle claimants with ME/CFS.

Patients and advocates have raised these and other legitimate concerns, in every possible academic, scientific and popular forum. This effort has been framed by the investigators, The Lancet and the Science Media Centre as a vicious and anti-scientific “campaign” against PACE. The news reports adopted this viewpoint and utterly failed to examine the scientific mistakes at the root of patients’ complaints.

Moreover, the reports did not present any independent evidence of the purported threats, other than claims made by the researchers. There were no statements from law enforcement authorities confirming the claims. No mention of any arrests made or charges having been filed. And little information from actual patients, much less these extremist, dangerous patients who supposedly hated psychiatry [see correction below]. In short, these news reports failed to pass any reasonable test of independent judgment and editorial skepticism.

Despite their questionable scientific methods and unreliable results, the PACE authors have widespread support among the UK medical and academic establishment. So does the Science Media Centre. Media reports, including last week’s Nature commentary, have presented without question the PACE authors’ perspective on patient response to the study. The reality is that patients have been protesting a study they know to be deeply flawed. Sometimes they have protested very, very loudly. That’s what people do when they are desperate for help, and no one is listening. To call it harassment is disgraceful.

Update 2/3/16: After reading some of the comments, I thought it was important to make clear that I don’t doubt the PACE investigators and some of their colleagues might have received very raw and nasty e-mails or phone calls. Perhaps some of these felt threatening, and perhaps they called in the police. (I’ve worked as a reporter for many years and have also received many, many raw and nasty e-mails, so I know it’s not enjoyable—but pissing people off is also part of the job.) The news accounts, however, provided no independent verification of the investigators’ charges. And the point is that, whether or not they have been the recipient of some unpleasant communications, the investigators have repeatedly used these claims to justify blocking legitimate inquiry into the PACE trial.

Correction: I reviewed the three major articles I linked to, not every single article about the issue, so my description of the coverage applies to those three. I originally wrote that the articles contained “no” interviews with actual patients. However, the Sunday Times article did include a short interview with one ME/CFS patient–a convicted child-molester who blamed his crime on fall-out from his illness. I apologize for the mistake, although I leave it to readers to decide if interviewing this person represented a sincere effort on the reporter’s part to present patients’ legitimate concerns.

I also wrote that the articles included no statements from law enforcement confirming the claims of threats. The Guardian article contained this sentence: “According to the police, the militants are now considered to be as dangerous and uncompromising as animal rights extremists.” This statement is vague, anonymous and impossible to verify with anyone in particular, so I don’t view it as an authoritative statement from law enforcement.

Revisiting the PLoS One economics analysis of PACE

by

On October 23rd, virology blog published the third installment of David Tuller’s investigative report about the PACE study of treatments for ME/CFS. In the post, Dr. Tuller demonstrated that the key finding of an economic analysis of the PACE trial, published in PLoS One in 2012, was almost certainly false. The finding–that cognitive behavior therapy and graded exercise therapy were cost-effective treatments–relied on an inaccuracy in the paper about whether the results of sensitivity analyses were “robust.”

Since the publication of the virology blog series, the PACE study has come under sustained and blistering public criticism for its many flaws. The PLoS One paper is currently the center of attention as a result of the researchers’ insulting response to Dr. James Coyne, a well-known psychologist and PLoS blogger. Dr. Coyne requested data to verify the results from the PLoS One paper, and was told that his request was “vexatious.” The researchers have called patients “vexatious” for years, of course, but the effort to use this strategy against a respected researcher has caused an uproar. Several colleagues and I, including Dr. Tuller, cited this rejection recently in our own request for a different set of PACE-related data.

Because of the open data policies of the PLoS journals, requesting data on that basis was a smart move by Dr. Coyne, and he has done a brilliant job of rousing support for the larger issue of access to data in scientific research. The PACE authors must recognize by now that at some point they will have to release all of their data.

The PLoS One study reported that cognitive behavior therapy and graded exercise therapy, the two treatments long favored by the main investigators, were more cost-effective than other approaches. The investigators have routinely cited these findings in promoting use of the two treatments. The truth or falseness of these claims from the PLoS One study are at the heart of the current controversy

In fact, it is already clear that the claim is highly unlikely to withstand serious scrutiny, based on the public record. In the October 23rd post, Dr. Tuller demonstrated that subsequent public comments of the lead author contradicted a critical statement in the paper about the PLoS One study’s sensitivity analyses..

The relevant excerpt from virology blog is below:

In another finding, the PLoS One paper argued that the graded exercise and cognitive behavior therapies were the most cost-effective treatments from a societal perspective. In reaching this conclusion, the investigators valued so-called  “informal” care—unpaid care provided by family and friends–at the replacement cost of a homecare worker. The PACE statistical analysis plan (approved in 2010 but not published until 2013) had included two additional, lower-cost assumptions. The first valued informal care at minimum wage, the second at zero compensation.

The PLoS One paper itself did not provide these additional findings, noting only that “sensitivity analyses revealed that the results were robust for alternative assumptions.” Commenters on the PLoS One website, including Tom Kindlon, challenged the claim that the findings would be “robust” under the alternative assumptions for informal care. In fact, they pointed out, the lower-cost conditions would reduce or fully eliminate the reported societal cost-benefit advantages of the cognitive behavior and graded exercise therapies.

In a posted response, the paper’s lead author, Paul McCrone, conceded that the commenters were right about the impact that the lower-cost, alternative assumptions would have on the findings. However, McCrone did not explain or even mention the apparently erroneous sensitivity analyses he had cited in the paper, which had found the societal cost-benefit advantages for graded exercise therapy and cognitive behavior therapy to be “robust” under all assumptions. Instead, he argued that the two lower-cost approaches were unfair to caregivers because families deserved more economic consideration for their labor.

“In our opinion, the time spent by families caring for people with CFS/ME has a real value and so to give it a zero cost is controversial,” McCrone wrote. “Likewise, to assume it only has the value of the minimum wage is also very restrictive.”

In a subsequent comment, Kindlon chided McCrone, pointing out that he had still not explained the paper’s claim that the sensitivity analyses showed the findings were “robust” for all assumptions. Kindlon also noted that the alternative, lower-cost assumptions were included in PACE’s own statistical plan.

“Remember it was the investigators themselves that chose the alternative assumptions,” wrote Kindlon. “If it’s ‘controversial’ now to value informal care at zero value, it was similarly ‘controversial’ when they decided before the data was looked at, to analyse the data in this way. There is not much point in publishing a statistical plan if inconvenient results are not reported on and/or findings for them misrepresented.”