David Tuller

Trial By Error: Psychosomatics Journal Linked to PACE Authors Highlights Bias from Subjective Outcome

7 March 2021 by David Tuller 1 Comment

By David Tuller, DrPH

The Journal of Pyschosomatic Research, a high-profile publication from Elsevier, has recently published an article relevant to long-standing arguments about trials that are both unblinded and reliant on subjective outcomes–like, say, the PACE study and related research into psycho-behavioral treatments for ME/CFS. This specific question–how to assess research quality when subjective outcomes are involved–is at the core of ongoing debates over both draft guidelines for ME/CFS from the UK’s National Institute for Health and Care Excellence and Cochrane’s contested review of exercise therapies for chronic fatigue syndrome.

The authors of the article in the Journal of Psychosomatic Research are the current editor (Jess Fiedorowicz, of the University of Ottawa) and two former editors (James L. Levenson of Virginia Commonwealth University, and Albert Leentjens of Maastricht University Medical Centre). The two former editors currently sit on the journal’s advisory board. I assume that the article thus reflects the journal’s own perspective. And that perspective would seem to contradict claims made by some of the journal’s prominent advisory board members, including two of the lead PACE investigators.

The article, first available online in early March, is called “When is lack of scientific Integrity a reason for retracting a paper? A case study.” It discusses a 2004 paper published by the journal–a study investigating homeopathic treatment for what the investigators called chronic fatigue syndrome. Treatment allocation—to a homeopathic preparation or a placebo—was blinded to both participants and their providers. Not surprisingly, the findings did not support the notion that homeopathy was an effective treatment for chronic fatigue syndrome.

In 2019, the Journal of Psychosomatic Research received a request to retract the 2004 study. The request was based on the public admission by one of the homeopaths involved that she had engaged in actions meant to undermine the blinding of the treatment allocation. In the recent article, the three editors explain why they do not believe retraction is warranted in this case, despite what is a clear lapse in scientific integrity. The editors report that, after investigating the matter, they are convinced that the study findings remain robust and that any efforts to influence the outcomes did not work. They attribute this success to the rigor of the process of blinding everyone who was involved in the study.

So…the point of all this is not to report that homeopathy is not an effective approach for chronic fatigue syndrome (and presumably for the other clinical states grouped under that term, the name myalgic encephelomyelitis, and ME/CFS). It is to note that this team of top editors at a major psychosomatics journal places a great deal of value on the role of blinding in ensuring the integrity of research results. Their “case study” includes this significant passage:

“Reporting on the integrity of the blind has merit and is especially valuable when dealing with subjective outcomes for which there is a greater risk of bias due to any unblinding…. Un-blinded assessors of subjective binary outcomes may exaggerate odds ratios by an average of 36%. Subjective outcomes are frequently used in studies that fall within this journal’s scope, at the interface of psychology and medicine. We recommend assessing the integrity of the blind for any clinical trial, particularly those utilizing subjective outcomes akin to the primary outcomes of the…study in question.”

This passage does not directly address the problem of unblinded trials and subjective outcomes. But the editors certainly make no bones of the fact that subjective outcomes, absent blinding, are at “a greater risk of bias.” And they recommend that checking the “integrity of the blind” is of particular importance for trials “utilizing subjective outcomes”–like the various questionnaires chosen for the homeopathy study in question. It is understandable that these issues are important to these editors, given that, as they note, “subjective outcomes are frequently used in studies that fall within this journal’s scope.”

**********

Who is on the journal’s advisory board?

A corollary to the message of the cited paragraph is that trials “utilizing subjective outcomes” that are not blinded at all are at even greater risk of bias. It is therefore interesting that the Journal of Psychosomatic Research’s roster of advisory board members includes Professors Michael Sharpe and Peter White—two leading proponents of the view that subjective outcomes in unblinded trials are not a problem to interpretation, at least when it comes to their own research.

Colleagues of Professors Sharpe and White who hold similar views are also members of this journal’s advisory board: Professors Per Fink from Denmark, Judith Rosmalen from the Netherlands, and Jon Stone from the UK. These professionals are all experts in the field of so-called “medically unexplained symptoms,” or MUS–in which they place ME/CFS as well as irritable bowel syndrome and other entities of unknown etiology. Yet the body of MUS research is riddled with studies that are unblinded and rely on self-reported outcomes.

Despite the lack of objective support for many of their findings, investigators in this field routinely overlook the role that bias plays in research relying on subjective outcomes in unblinded studies. That role has now been highlighted in a very prominent manner by a journal that these same investigators serve as editorial advisory board members. Hm.

A leading example of this combination of being unblinded and relying on subjective outcomes is the PACE trial, with Professors Sharpe and White as two of the three lead investigators. PACE included four objective measures—a six-minute walking test, a step-test for fitness, whether people were employed, and whether they were on social benefits. All failed to meet the subjective reports of improvements on questionnaires about physical function and fatigue. Rather than acknowledging this fact forthrightly and discussing the implications of the poor results for the validity and reliability of their subjective measures, the PACE authors scattered these findings across multiple papers—a decision that helped minimize any attention to the collective failure of all of them.

The PACE investigators also questioned the objectivity of their own self-selected objective outcomes. They claimed they implemented the six-minute walking test differently than other investigators, so the disastrous results could not be compared to those from studies of people with a variety of conditions. They claimed that independent economic factors and changes during the period of the study meant that whether people got back to work or got off benefits would not be an objective measurement of the success or failure of treatment.

They had also dropped a key objective measure—wearing a movement monitor known as an actimeter for a week at the end of the trial. However, they retained it as a baseline measure, meaning participants wore it for a week at the start. Publicly, the investigators claimed they dropped it as an outcome because they believed that wearing it for a week would be too much of a burden for participants at the end of the trial. They did not explain why it would be more of a burden at the end of the trial than at the beginning.

When trial minutes were released after a freedom of information request, they revealed that the PACE team dropped the actimeters as an outcome after learning that Dutch colleagues had found their own results from this measure did not meet their positive subjective reports. In other words, retaining the measure did not appear to be a useful way to do what they seemed determined to do: “prove” their therapies worked. So they disappeared it.

Now the current and former editors of the Journal of Psychosomatic Research have made declarative statements about about blinding and subjective outcomes. So I’m curious: What is their opinion of the PACE trial? Do Fiedorowicz, Levenson and Leentjens know that this debacle was unblinded and relied on subjective outcomes, and that the investigators assessed their objective findings as irrelevant? Do they care about this contradiction between their own pronouncements on research integrity and the apparent belief by members of their advisory board that combining unblinded studies and subjective findings does not present a problem to proclaiming treatment success?

Trial By Error: A Q-and-A with Tracie White, author of The Puzzle Solver

3 March 2021 by David Tuller 2 Comments

By David Tuller, DrPH

Tracie White, a science writer at Stanford University, first stumbled across the story of Whitney Dafoe as an assignment from one of her editors. That initial encounter ultimately turned into The Puzzle Solver: A Scientist’s Desperate Quest to Cure the Illness that Stole His Son, an account that Kirkus called “a complex, well-related story of medical detective work.”

The story of Stanford geneticist Ron Davis and his son Whitney’s severe case of ME/CFS was certainly not unknown, given previous coverage–starting with Miriam Tucker’s excellent 2015 feature in The Washington Post. White’s is the first full-length account. During the reporting of the book, she grew close to the family, which included psychologist Janet Dafoe, Whitney’s mom, and his sister Ashley. I asked White a few questions about the process of writing the book.

Trial By Error: National Institutes of Health Director Francis Collins on Plans for Long COVID Research

24 February 2021 by David Tuller 12 Comments

By David Tuller, DrPH

The US government seems to be taking Long COVID seriously. In December, Congress allocated $1.15 billion over four years for research into the issue. This week, Francis Collins, director of the National Institutes of Health, announced the agency’s plans for that funding. (I’ve posted his announcement in full below.) In a post last month he highlighted the plight of the long-haulers and praised the most extensive report yet on their situation. That well-received research report was spearheaded and produced by a patient-led team from the Body Politic COVID-19 Support Group, an online community.

The relationship between what is generally being called ME/CFS and what is generally being called Long COVID is unclear. A number of high-profile news articles–including ones published by The New York Times, The Guardian, and Kaiser Health News–have noted the apparent overlaps in symptoms and in possible or hypothesized causes. These articles have taken at face value the notion that ME/CFS patients are suffering from a serious disease and have not presumed that psychotherapy and exercise are the optimal approaches to treatment.

Both ME/CFS and long COVID are complex phenomena–as is evident from confusion and disagreement over the appropriate nomenclature. ME/CFS is an unsatisfactory hybrid term used to refer to a range of described clinical entities. Long COVID is a convenient and easy-to-understand term but it conveys nothing about the condition’s expansive range of presentations. That variety is better expressed through the scientific name it has been given: Post-Acute Sequelae of SARS-CoV-2 infection (PASC). In other words, there are lots of different sequelae–not just one entity called Long COVID.

Ramped up funding for research into Long COVID could be beneficial for ME/CFS patients. My sense is that many of the latter are hopeful that these investigations could reveal biological mechanisms and pharmaceutical treatments that could be relevent for them as well–especially given apparent similarities in symptoms like post-exertional malaise and cognitive impairment. (I never expected to see the phrase “brain fog” in news headlines all around the world.)

At the same time, there is cause to be wary. This pandemic is now early in its second year, so so-called Long COVID is still a relatively short phenomenon–especially when compared to the decades of illness experienced by many with ME/CFS. Reports of persistent symptoms are known to be common after many viral infections. It is also known that these cases self-resolve most of the time–even if it can take a year or more in some cases.

If it is asserted prematurely or simplistically that Long COVID and ME/CFS are somehow the same, what happens if most of these legions of Long COVID patients get better in the next few months or over the next year? It could easily be presumed that the “multi-disciplinary rehabilitation”–or any number of helpful or non-helpful interventions–led to improvements, even if the recoveries would have happened in any event. In such a scenario, that advice could be presumed to be applicable to ME/CFS patients. Before declarative statements can be made, we need to see a lot more data.

In the meantime, it’s great that NIH has found more than $1 billion to investigate Long COVID. It certainly suggests that more money could have been found ten or twenty years ago to study ME/CFS than the pittance that has historically been allocated. While the amount has increased significantly in recent years, two or three times a pittance is still a relative pittance. (Jennie Spotila provides regular analyses of NIH funding at Occupy M.E., her blog)

Below is the announcement from NIH Director Francis Collins:

**********

NIH launches new initiative to study “Long COVID”

I write to announce a major new NIH initiative to identify the causes and ultimately the means of prevention and treatment of individuals who have been sickened by COVID-19, but don’t recover fully over a period of a few weeks. Large numbers of patients who have been infected with SARS-CoV-2 continue to experience a constellation of symptoms long past the time that they’ve recovered from the initial stages of COVID-19 illness. Often referred to as “Long COVID”, these symptoms, which can include fatigue, shortness of breath, “brain fog”, sleep disorders, fevers, gastrointestinal symptoms, anxiety, and depression, can persist for months and can range from mild to incapacitating. In some cases, new symptoms arise well after the time of infection or evolve over time. In December, NIH held a workshop to summarize what is known about these patients who do not fully recover and identify key gaps in our knowledge about the effects of COVID-19 after the initial stages of infection. In January, I shared the results from the largest global study of these emerging symptoms. While still being defined, these effects can be collectively referred to as Post-Acute Sequelae of SARS-CoV-2 infection (PASC). We do not know yet the magnitude of the problem, but given the number of individuals of all ages who have been or will be infected with SARS-CoV-2, the coronavirus that causes COVID-19, the public health impact could be profound.

In December, Congress provided $1.15 billion in funding over four years for NIH to support research into the prolonged health consequences of SARS-CoV-2 infection. A diverse team of experts from across the agency has worked diligently over the past few weeks to identify the most pressing research questions and the areas of greatest opportunity to address this emerging public health priority. Today we issued the first in a series of Research Opportunity Announcements (ROAs) for the newly formed NIH PASC Initiative. Through this initiative, we aim to learn more about how SARS-CoV-2 may lead to such widespread and lasting symptoms, and to develop ways to treat or prevent these conditions. We believe that the insight we gain from this research will also enhance our knowledge of the basic biology of how humans recover from infection, and improve our understanding of other chronic post-viral syndromes and autoimmune diseases, as well as other diseases with similar symptoms.

Some of the initial underlying questions that this initiative hopes to answer are:

What does the spectrum of recovery from SARS-CoV-2 infection look like across the population?
How many people continue to have symptoms of COVID-19, or even develop new symptoms, after acute SARS-CoV-2 infection?
What is the underlying biological cause of these prolonged symptoms?
What makes some people vulnerable to this but not others?
Does SARS-CoV-2 infection trigger changes in the body that increase the risk of other conditions, such as chronic heart or brain disorders?

These initial research opportunities will support a combination of ongoing and new research studies and the creation of core resources. We anticipate subsequent calls for other kinds of research, in particular opportunities focused on clinical trials to test strategies for treating long-term symptoms and promoting recovery from infection.

Research Studies: A SARS-CoV-2 Recovery Cohort—the central program of this initiative—will leverage ongoing COVID-19 studies, long-term cohort studies established well before the pandemic began, and new studies of people with Long COVID. These studies aim to characterize the long-term effects of infection in a diverse set of people and the trajectory of symptoms over time. The initiative will support a multidisciplinary consortium of investigators who collaborate and coordinate across studies. The initiative also will support two complementary studies: 1) large data studies from resources such as electronic health records and health systems databases that will be critical to understand how many people are affected and what factors contribute to recovery; 2) studies of biological specimens to understand injury to the brain and other organs.

Core Resources: A clinical science core, data resource core, and biorepository core will provide overall consortium coordination, clinical expertise in post-acute COVID symptoms, and facilitate the use of standardized data and biological specimens collected from the consortium studies by consented volunteers.

Our hearts go out to individuals and families who have not only gone through the difficult experience of acute COVID-19, but now find themselves still struggling with lingering and debilitating symptoms. Throughout this pandemic, we have witnessed the resilience of our patient, medical, and scientific communities as they have come together in extraordinary ways. NIH deeply appreciates the contributions of patients who have not fully recovered from SARS-CoV-2 infection and who have offered their experiences and insights to lead us to this point, including those with other post-viral infections. Through the PASC Initiative, we now ask the patient, medical, and scientific communities to come together to help us understand the long-term effects of SARS-CoV-2 infection, and how we may be able to prevent and treat these effects moving forward.

Francis S. Collins, M.D., Ph.D.
Director, National Institutes of Health

Trial By Error: Happy Tenth Anniversary, PACE Trial!

18 February 2021 by David Tuller 6 Comments

By David Tuller, DrPH

It’s been ten years since The Lancet published the first results of the PACE trial. Wow!

Ten years ago, I was 54 and still a graduate student in public health at UC Berkeley. I was also busy writing stories for The New York Times about the mouse retrovirus study that had roiled the field of research into chronic fatigue syndrome—the then-standard name for the illness now referred to as ME/CFS by US government agencies. The mouse retrovirus, XMRV, turned out to be a lab contaminant. The story had struck such a nerve at least in part because of long-standing and lingering speculations that a retrovirus could be involved—a position that retains some strong adherents.

Trial By Error: Hughes-Tuller Comment on Wessely-Chalder CBT Study Rejected by Journal, Posted Here

15 February 2021 by David Tuller 10 Comments

By David Tuller, DrPH

Last fall, Professor Sir Simon Wessely and Professor Trudie Chalder were among several co-authors of a study published in the Journal of the Royal Society of Medicine. The study purported to prove that years of provision of cognitive behavior therapy (CBT) to patients with “chronic fatigue” and “chronic fatigue syndrome” proved that the intervention was a success. I previously pointed out myriad problem in this post last August.

Trial By Error: New Biopsychosocial Paper on Long-Covid and Somatic Symptom Disorder

9 February 2021 by David Tuller 13 Comments

By David Tuller, DrPH

Biopsychosocial Campaigners Target Long-Covid

New papers from the biopsychosocial campaigners often provide opportunities to highlight unwarranted assertions, misleading use of data, and—in particular–associations interpreted as if they were causal relationships and not, well, associations. An article co-authored by Trudie Chalder, a professor of cognitive behavioural therapy at King’s Collge London, and published recently in the Journal of Mental Health, deploys such strategies in its efforts to suggest that large swaths of those experiencing what is being called long-Covid are suffering from “somatic symptom disorder” (SSD).

Trial By Error: An Excerpt from Jamison Hill’s Memoir

3 February 2021 by David Tuller

By David Tuller, DrPH

Three years ago, The New York Times‘ popular Modern Love column published a beautifully written tear-jerker by Jamison Hill–a former bodybuilder with ME. The column, “Love Means Never Having to Say…Anything,” described in tender prose how two people with debilitating chronic illness were able to find and nurture a very special love. It was later selected for oral treatment by Boston’s WBUR radio and read by Pedro Pascal. (I’ve never watched Game of Thrones or The Mandalorian, so I’d never heard of him before, but I’m likely an outlier.)

Anyway, last month Inkshares published Hill’s memoir, When Force Meets Fate: A Mission to Solve an Invisible Illness. Below is a short excerpt. I’m sure many patients will relate to the experience described.

**********

Trial By Error: Professor Paul Garner’s BMJ Blog Post on His Powerful Cognitions

28 January 2021 by David Tuller

By David Tuller, DrPH

Professor Paul Garner, an infectious disease doctor at the Liverpool School of Tropical Medicine, has had a rough time with long-Covid. He has written about his experiences in a series of compelling blog posts on BMJ’s site. At first, he bonded with members of the ME/CFS community and expressed shock at the shoddy treatment these patients have experienced for years. He even invited me to give a presentation to his research group on the discredited PACE trial, which I was pleased to do.

Trial By Error: Professor Lubet’s Inquiry to Medical Center Holding ME/CFS Patient

27 January 2021 by David Tuller

By David Tuller, DrPH

Yesterday, I posted my exchange with a spokeswoman for Hennepin Healthcare about Thane Fredrickson, the ME/CFS patient who is under threat of involuntary psychiatric commitment. Today, Northwestern University law professor Steven Lubet, e-mailed her the following letter.

**********

Dear Ms. Hill: I am a law professor at Northwestern University in Chicago, and I often write articles, for both scholarly and popular publications, about the legal issues relating to ME/CFS. I understand that Hennepin Healthcare has recently referred an ME/CFS patient for involuntary psychiatric commitment, and I have some questions for you. As a law professor, I am fully aware of HIPAA and other privacy requirements. Please be assured that my questions are general, so you may answer without regard to an identified patient:

Trial By Error: My Exchange with Minnesota Medical Center Holding ME/CFS Patient

26 January 2021 by David Tuller

By David Tuller, DrPH

On Monday afternoon, I had an e-mail exchange with Christine Hill, a spokeswoman for the Hennepin Healthcare in Minnesota. Thane Fredrickson, an ME/CFS patient, is currently under threat of involuntary psychiatric commitment. Because of patient privacy concerns, it was clear the medical center would not be able to provide any specific information. But reporters are still obliged to ask questions even when the answer will inevitably be “no comment.” Receiving such an inquiry also alerts an institution, in this case Hennepin Healthcare, that an issue is of public concern and that its actions are being observed.

Thane has lawyers working on his behalf and, as I understand it, is still awaiting a hearing on his case.

The e-mail exchange with the medical center is below.