virology blog

About viruses and viral disease

exercise

Trial By Error: Why Are Exercise Studies for Long Covid Still Failing to Address PEM Adequately?

by 3 Comments

By David Tuller, DrPH

I have written previously about the issue of exercise studies for long Covid patients. As with the discredited PACE trial for the illness or cluster of illnesses currently being referred to as ME/CFS, the theory behind these studies posits deconditioning as a central cause of patients’ disability. This approach tends to overlook the phenomenon of post-exertional malaise (PEM) or post-exertional symptom exacerbation (PESE), which is characteristic of ME/CFS and reported by many patients with long Covid.

When such studies do mention PEM/PESE, it is not clear they have properly distinguished it from the tiredness that often accompanies exercise and is exacerbated in those who are in fact decondition. In December, for example, I wrote a post about a study called “Effect of using a structured pacing protocol on post-exertional symptom exacerbation and health status in a longitudinal cohort with the post-COVID-19 syndrome.” Among many other methodological issues, some of the PEM episodes documented lasted just one to three hours. Were these examples of PEM, or just standard fatigue?

In 2015, a report from the Institute of Medicine (now the National Academy of Sciences) identified PEM as characteristic of the illness, although it referred to it as “exertion intolerance” when it rechristened the condition as “systemic exertion intolerance disease”–a name that did not catch on. The US Centers for Disease Control and Prevention identifies PEM as a significant factor in developing treatment strategies. Since the emergence of long Covid, PEM has received widespread news coverage. The New York Times published this story by journalist Melinda Wenner Moyer on exercise and long Covid more than a year ago. And Zeynep Tufekci, a sociologist and Times columnist, highlighted PEM in a compelling piece about last fall’s ME/CFS protest at the White House.

Yet many researchers and clinicians still seem not to understand the importance of this symptom. Since last fall, the National Institutes of Health’s RECOVER Initiative, which was set up to investigate the long-term effects of Covid, has been embroiled in a bit of a controversy about a proposed exercise study, highlighted in an announcement page of planned research. Few details of this study, including the study protocol, have been made publicly available.

Long Covid advocates have been seeking more information about the proposed study, and #MEAction has also raised concerns, sending a letter to RECOVER last month. After receiving no response, the organization sent a follow-up letter last week.

Here’s a key paragraph from the latest letter:

“People with Long COVID who meet the criteria for myalgic encephalomyelitis / chronic fatigue syndrome (ME/CFS) and those with post-exertional malaise (PEM) must be excluded from this study. There is no justification for using exercise therapies designed for patients who do not have ME/CFS or PEM on a Long COVID population with ME/CFS or PEM.”

**********

And now a PEM-less “exercise reporting template” for long Covid

And now the journal Archives of Physical Medicine and Rehabilitation has just published an article called “Exercise Reporting Template for long COVID Patients: A Rehabilitation Practitioner Guide.” The article touts the merits of the Consensus on Exercise Reporting Template (CERT), a 16-item checklist “that reports on the quality of the exercise interventions in the management of acute and chronic conditions.”

Ok, then. Here are some of the key points related to prescribing exercise for long Covid:

“Exercise is associated with improved immune system function, psychological and mental health, neural plasticity (healing of the nerves in the brain), reduced pulmonary (lung) complications, and enhanced cardiovascular (heart) outcomes.”

“Most patients presenting with long COVID can begin with supervised breathing and light exercise programs in the early phase of rehabilitation in the clinical setting. Using the 6-20 rating of perceived exertion scale, it has been recommended that patients commence exercise as low as 6 to 8, ranging from ‘no exertion’ to ‘extremely light’ and then progressing by ∼2 to 3 points on the scale every week as tolerated.”

“Although exercise promotes good health despite the health condition, sedentary patients and those with existing chronic medical conditions should undergo medical examination and approval before starting an exercise routine.”

The article makes no mention of PEM/PESE, and that’s a major problem. No one seriously disputes that exercise in general is beneficial, but it is critical to remember that it can be contra-indicated in those with PEM/PESE. In other words, it can cause harm.

No ”exercise reporting template” for long Covid is acceptable if it fails to include a recommendation of rigorous pre-screening for PEM/PESE. Since many patients themselves might not be aware of this symptom and might not understand why their efforts to improve were being met with serious setbacks.

When it comes to any form of “rehabilitation,” patients with and without PEM/PESE should obviously be handled differently. Everyone involved in treating and researching long Covid patients should be aware of this. After all, we have now been dealing with this for years.

Trial By Error: Now a Scottish Long Covid Study Fails to Mention the Risk of Post-Exertional Malaise

by

By David Tuller, DrPH

*April is crowdfunding month at UC Berkeley. If you like my work, consider making a tax-deductible donation to Berkeley’s School of Public Health to support the Trial By Error project: https://crowdfund.berkeley.edu/project/31347

And here’s another major trial designed by investigators who think that an exercise program might be the optimal strategy for treating the complex grab-bag of conditions known as long Covid. As with other long Covid research, these investigators seem either unaware of or unconcerned about the core ME/CFS symptom known as post-exertional malaise (PEM)–also called post-exertional symptom exacerbation (PESE) or, in the ME-ICC case definition, post-exertional neuroimmune exhaustion (PENE).

[Read more…] about Trial By Error: Now a Scottish Long Covid Study Fails to Mention the Risk of Post-Exertional Malaise

An open letter to Psychological Medicine, again!

by

Last week, Virology Blog posted an open letter to the editors of Psychological Medicine. The letter called on them to retract the misleading findings that participants in the PACE trial for ME/CFS had “recovered” from cognitive behavior therapy and graded exercise therapy. More than 100 scientists, clinicians, other experts and patient organizations signed the letter.

Three days later, I received a response from Sir Robin Murray, the UK editor of Psychological Medicine. Here’s what he wrote:

 Thank you for your letter and your continuing interest in the paper on the PACE Trial which Psychological Medicine published. I was interested to learn that Wilshire and colleagues have now published a reanalysis of the original data from the PACE Trial in the journal Fatigue: Biomedicine, Health & Behavior, a publication that I was not previously aware of. Presumably, interested parties will now be able to read this reanalysis and compare the scientific qualiity of the re-analysis with that of the original. My understanding is that this is the way that science advances.

This is an unacceptable response.  Sir Robin Murray is misguided if he believes that science advances by allowing misleading claims based on manipulated data to stand in the literature. When researchers include participants who were already “recovered” on key indicators at baseline, the findings are by definition so flawed and nonsensical they must be retracted.

That the editors of Psychological Medicine do not grasp that it is impossible to be “disabled” and “recovered” simultaneously on an outcome measure is astonishing and deeply troubling. It is equally astonishing that the PACE authors now defend themselves, as noted in a New York Times opinion piece on Sunday, by arguing that this overlap doesn’t matter because there were also other recovery criteria.

In response to the comments from Psychological Medicine, we are reposting the open letter with 17 added individuals and 24 more organizations, for a total of 142 signatories altogether. These include two lawyers from Queen Mary University of London, the academic home of lead PACE investigator Peter White, along with other experts and ME/CFS patient groups from around the world.

 

Sir Robin Murray and Dr. Kenneth Kendler
Psychological Medicine
Cambridge University Press
University Printing House
Shaftesbury Road
Cambridge CB2 8BS
UK

Dear Sir Robin Murray and Dr. Kendler:

In 2013, Psychological Medicine published an article called “Recovery from chronic fatigue syndrome after treatments given in the PACE trial.”[1] In the paper, White et al. reported that graded exercise therapy (GET) and cognitive behavioural therapy (CBT) each led to recovery in 22% of patients, compared with only 7% in a comparison group. The two treatments, they concluded, offered patients “the best chance of recovery.”

PACE was the largest clinical trial ever conducted for chronic fatigue syndrome (also known as myalgic encephalomyelitis, or ME/CFS), with the first results published in The Lancet in 2011.[2] It was an open-label study with subjective primary outcomes, a design that requires strict vigilance to prevent the possibility of bias. Yet PACE suffered from major flaws that have raised serious concerns about the validity, reliability and integrity of the findings.[3] Despite these flaws, White et al.’s claims of recovery in Psychological Medicine have greatly impacted treatment, research, and public attitudes towards ME/CFS.

According to the protocol for the PACE trial, participants needed to meet specific benchmarks on four different measures in order to be defined as having achieved “recovery.”[4] But in Psychological Medicine, White et al. significantly relaxed each of the four required outcomes, making “recovery” far easier to achieve. No PACE oversight committees appear to have approved the redefinition of recovery; at least, no such approvals were mentioned. White et al. did not publish the results they would have gotten using the original protocol approach, nor did they include sensitivity analyses, the standard statistical method for assessing the impact of such changes.

Patients, advocates and some scientists quickly pointed out these and other problems. In October of 2015, Virology Blog published an investigation of PACE, by David Tuller of the University of California, Berkeley, that confirmed the trial’s methodological lapses.[5] Since then, more than 12,000 patients and supporters have signed a petition calling for Psychological Medicine to retract the questionable recovery claims. Yet the journal has taken no steps to address the issues.

Last summer, Queen Mary University of London released anonymized PACE trial data under a tribunal order arising from a patient’s freedom-of-information request. In December, an independent research group used that newly released data to calculate the recovery results per the original methodology outlined in the protocol.[6] This reanalysis documented what was already clear: that the claims of recovery could not be taken at face value.

In the reanalysis, which appeared in the journal Fatigue: Biomedicine, Health & Behavior, Wilshire et al. reported that the PACE protocol’s definition of “recovery” yielded recovery rates of 7 % or less for all arms of the trial. Moreover, in contrast to the findings reported in Psychological Medicine, the PACE interventions offered no statistically significant benefits. In conclusion, noted Wilshire et al., “the claim that patients can recover as a result of CBT and GET is not justified by the data, and is highly misleading to clinicians and patients considering these treatments.”

In short, the PACE trial had null results for recovery, according to the protocol definition selected by the authors themselves. Besides the inflated recovery results reported in Psychological Medicine, the study suffered from a host of other problems, including the following:

*In a paradox, the revised recovery thresholds for physical function and fatigue–two of the four recovery measures–were so lax that patients could deteriorate during the trial and yet be counted as “recovered” on these outcomes. In fact, 13 % of participants met one or both of these recovery thresholds at baseline. White et al. did not disclose these salient facts in Psychological Medicine. We know of no other studies in the clinical trial literature in which recovery thresholds for an indicator actually represented worse health status than the entry thresholds for serious disability on the same indicator.

*During the trial, the authors published a newsletter for participants that included glowing testimonials from earlier participants about their positive outcomes in the trial.[7] An article in the same newsletter reported that a national clinical guidelines committee had already recommended CBT and GET as effective; the newsletter article did not mention adaptive pacing therapy, an intervention developed specifically for the PACE trial. The participant testimonials and the newsletter article could have biased the responses of an unknown number of the two hundred or more people still undergoing assessments—about a third of the total sample.

*The PACE protocol included a promise that the investigators would inform prospective participants of “any possible conflicts of interest.” Key PACE investigators have had longstanding relationships with major insurance companies, advising them on how to handle disability claims related to ME/CFS. However, the trial’s consent forms did not mention these self-evident conflicts of interest. It is irrelevant that insurance companies were not directly involved in the trial and insufficient that the investigators disclosed these links in their published research. Given this serious omission, the consent obtained from the 641 trial participants is of questionable legitimacy.

Such flaws are unacceptable in published research; they cannot be defended or explained away. The PACE investigators have repeatedly tried to address these concerns. Yet their efforts to date—in journal correspondence, news articles, blog posts, and most recently in their response to Wilshire et al. in Fatigue[8]—have been incomplete and unconvincing.

The PACE trial compounded these errors by using a case definition for the illness that required only one symptom–six months of disabling, unexplained fatigue. A 2015 report from the U.S. National Institutes of Health recommended abandoning this single-symptom approach for identifying patients.[9] The NIH report concluded that this broad case definition generated heterogeneous samples of people with a variety of fatiguing illnesses, and that using it to study ME/CFS could “impair progress and cause harm.”

PACE included sub-group analyses of two alternate and more specific case definitions, but these case definitions were modified in ways that could have impacted the results. Moreover, an unknown number of prospective participants might have met these alternate criteria but been excluded from the study by the initial screening.

To protect patients from ineffective and possibly harmful treatments, White et al.’s recovery claims cannot stand in the literature. Therefore, we are asking Psychological Medicine to retract the paper immediately. Patients and clinicians deserve and expect accurate and unbiased information on which to base their treatment decisions. We urge you to take action without further delay.

Sincerely,

Dharam V. Ablashi, DVM, MS, Dip Bact
Scientific Director
HHV-6 Foundation
Former Senior Investigator
National Cancer Institute
National Institutes of Health
Bethesda, Maryland, USA

James N. Baraniuk, MD
Professor, Department of Medicine
Georgetown University
Washington, D.C., USA

Lisa F. Barcellos, MPH, PhD
Professor of Epidemiology
School of Public Health
California Institute for Quantitative Biosciences
University of California, Berkeley
Berkeley, California, USA

Lucinda Bateman, MD
Medical Director
Bateman Horne Center
Salt Lake City, Utah, USA

Alison C. Bested, MD, FRCPC
Clinical Associate Professor
Faculty of Medicine
University of British Columbia
Vancouver, British Columbia, Canada

Molly Brown, PhD
Assistant Professor
Department of Psychology
DePaul University
Chicago, Illinois, USA

John Chia, MD
Clinician and Researcher
EVMED Research
Lomita, California, USA

Todd E. Davenport, PT, DPT, MPH, OCS
Associate Professor
Department of Physical Therapy
University of the Pacific
Stockton, California, USA

Ronald W. Davis, PhD
Professor of Biochemistry and Genetics
Stanford University
Stanford, California, USA

Simon Duffy, PhD, FRSA
Director
Centre for Welfare Reform
Sheffield, UK

Jonathan C.W. Edwards, MD
Emeritus Professor of Medicine
University College London
London, UK

Derek Enlander, MD
New York, New York, USA

Meredyth Evans, PhD
Clinical Psychologist and Researcher
Chicago, Illinois, USA

Kenneth J. Friedman, PhD
Associate Professor of Physiology and Pharmacology (retired)
New Jersey Medical School
University of Medicine and Dentistry of New Jersey
Newark, New Jersey, USA

Robert F. Garry, PhD
Professor of Microbiology and Immunology
Tulane University School of Medicine
New Orleans, Louisiana, USA

Keith Geraghty, PhD
Honorary Research Fellow
Division of Population Health, Health Services Research & Primary Care
School of Health Sciences
University of Manchester
Manchester, UK

Ian Gibson, PhD
Former Member of Parliament for Norwich North
Former Dean, School of Biological Sciences
University of East Anglia
Honorary Senior Lecturer and Associate Tutor
Norwich Medical School
University of East Anglia
Norwich, UK

Rebecca Goldin, PhD
Professor of Mathematics
George Mason University
Fairfax, Virginia, USA

Ellen Goudsmit, PhD, FBPsS
Health Psychologist (retired)
Former Visiting Research Fellow
University of East London
London, UK

Maureen Hanson, PhD
Liberty Hyde Bailey Professor
Department of Molecular Biology and Genetics
Cornell University
Ithaca, New York, USA

Malcolm Hooper, PhD
Emeritus Professor of Medicinal Chemistry
University of Sunderland
Sunderland, UK

Leonard A. Jason, PhD
Professor of Psychology
DePaul University
Chicago, Illinois, USA

Michael W. Kahn, MD
Assistant Professor of Psychiatry
Harvard Medical School
Boston, Massachusetts, USA

Jon D. Kaiser, MD
Clinical Faculty
Department of Medicine
University of California, San Francisco
San Francisco, California, USA

David L. Kaufman, MD
Medical Director
Open Medicine Institute
Mountain View, California, USA

Betsy Keller, PhD
Department of Exercise and Sports Sciences
Ithaca College
Ithaca, New York, USA

Nancy Klimas, MD
Director, Institute for Neuro-Immune Medicine
Nova Southeastern University
Director, Miami VA Medical Center GWI and CFS/ME Program
Miami, Florida, USA

Andreas M. Kogelnik, MD, PhD
Director and Chief Executive Officer
Open Medicine Institute
Mountain View, California, USA

Eliana M. Lacerda, MD, MSc, PhD
Clinical Assistant Professor
Disability & Eye Health Group/Clinical Research Department
Faculty of Infectious and Tropical Diseases
London School of Hygiene & Tropical Medicine
London, UK

Charles W. Lapp, MD
Medical Director
Hunter-Hopkins Center
Charlotte, North Carolina, USA
Assistant Consulting Professor
Department of Community and Family Medicine
Duke University School of Medicine
Durham, North Carolina, USA

Bruce Levin, PhD
Professor of Biostatistics
Columbia University
New York, New York, USA

Alan R. Light, PhD
Professor of Anesthesiology
Professor of Neurobiology and Anatomy
University of Utah
Salt Lake City, Utah, USA

Vincent C. Lombardi, PhD
Director of Research
Nevada Center for Biomedical Research
Reno, Nevada, USA

Alex Lubet, PhD
Professor of Music
Head, Interdisciplinary Graduate Group in Disability Studies
Affiliate Faculty, Center for Bioethics
Affiliate Faculty, Center for Cognitive Sciences
University of Minnesota
Minneapolis, Minnesota, USA

Steven Lubet
Williams Memorial Professor of Law
Northwestern University Pritzker School of Law
Chicago, Illinois, USA

Sonya Marshall-Gradisnik, PhD
Professor of Immunology
Co-Director, National Centre for Neuroimmunology and Emerging Diseases
Griffith University
Queensland, Australia

Patrick E. McKnight, PhD
Professor of Psychology
George Mason University
Fairfax, Virginia, USA

Jose G. Montoya, MD, FACP, FIDSA
Professor of Medicine
Division of Infectious Diseases and Geographic Medicine
Stanford University School of Medicine
Stanford, California, USA

Zaher Nahle, PhD, MPA
Vice President for Research and Scientific Programs
Solve ME/CFS Initiative
Los Angeles, California, USA

Henrik Nielsen, MD
Specialist in Internal Medicine and Rheumatology
Copenhagen, Denmark

James M. Oleske, MD, MPH
François-Xavier Bagnoud Professor of Pediatrics
Senator of RBHS Research Centers, Bureaus, and Institutes
Director, Division of Pediatrics Allergy, Immunology & Infectious Diseases
Department of Pediatrics
Rutgers New Jersey Medical School
Newark, New Jersey, USA

Elisa Oltra, PhD
Professor of Molecular and Cellular Biology
Catholic University of Valencia School of Medicine
Valencia, Spain

Richard Podell, MD, MPH
Clinical Professor
Department of Family Medicine
Rutgers Robert Wood Johnson Medical School
New Brunswick, New Jersey, USA

Nicole Porter, PhD
Psychologist in Private Practice
Rolling Ground, Wisconsin, USA

Vincent R. Racaniello, PhD
Professor of Microbiology and Immunology
Columbia University
New York, New York, USA

Arthur L. Reingold, MD
Professor of Epidemiology
University of California, Berkeley
Berkeley, California, USA

Anders Rosén, MD
Professor of Inflammation and Tumor Biology
Department of Clinical and Experimental Medicine
Division of Cell Biology
Linköping University
Linköping, Sweden

Peter C. Rowe, MD
Professor of Pediatrics
Johns Hopkins University School of Medicine
Baltimore, Maryland, USA

William Satariano, PhD
Professor of Epidemiology and Community Health
University of California, Berkeley
Berkeley, California, USA

Ola Didrik Saugstad, MD, PhD, FRCPE
Professor of Pediatrics
University of Oslo
Director and Department Head
Department of Pediatric Research
University of Oslo and Oslo University Hospital
Oslo, Norway

Charles Shepherd, MB, BS
Honorary Medical Adviser to the ME Association
Buckingham, UK

Christopher R. Snell, PhD
Scientific Director
WorkWell Foundation
Ripon, California, USA

Donald R. Staines, MBBS, MPH, FAFPHM, FAFOEM
Clinical Professor
Menzies Health Institute Queensland
Co-Director, National Centre for Neuroimmunology and Emerging Diseases
Griffith University
Queensland, Australia

Philip B. Stark, PhD
Professor of Statistics
University of California, Berkeley
Berkeley, California, USA

Eleanor Stein, MD, FRCP(C)
Psychiatrist in Private Practice
Assistant Clinical Professor
University of Calgary
Calgary, Alberta, Canada

Staci Stevens, MA
Founder, Exercise Physiologist
Workwell Foundation
Ripon, California, USA

Julian Stewart, MD, PhD
Professor of Pediatrics, Physiology and Medicine
Associate Chairman for Patient Oriented Research
Director, Center for Hypotension
New York Medical College
Hawthorne, NY, USA

Leonie Sugarman, PhD
Emeritus Associate Professor of Applied Psychology
University of Cumbria
Carlisle, UK

John Swartzberg, MD
Clinical Professor Emeritus
School of Public Health
University of California, Berkeley
Berkeley, California, USA

Ronald G. Tompkins, MD, ScD
Summer M Redstone Professor of Surgery
Harvard Medical School
Boston, Massachusetts, USA

David Tuller, DrPH
Lecturer in Public Health and Journalism
University of California, Berkeley
Berkeley, California, USA

Rosemary A. Underhill, MB, BS, MRCOG, FRCSE
Physician and Independent Researcher
Palm Coast, Florida, USA

Rosamund Vallings, MNZM, MB, BS
General Practitioner
Auckland, New Zealand

Michael VanElzakker, PhD
Research Fellow, Psychiatric Neuroscience Division
Harvard Medical School & Massachusetts General Hospital
Instructor, Tufts University Psychology
Boston, Massachusetts, USA

Mark VanNess, PhD
Professor of Health, Exercise & Sports Sciences
University of the Pacific
Stockton, California, USA
Workwell Foundation
Ripon, California, USA

Mark Vink, MD
Family Physician
Soerabaja Research Center
Amsterdam, Netherlands

Frans Visser, MD
Cardiologist
Stichting Cardiozorg
Hoofddorp, Netherlands

Tony Ward, MA (Hons), PhD, DipClinPsyc
Registered Clinical Psychologist
Professor of Clinical Psychology
School of Psychology
Victoria University of Wellington
Wellington, New Zealand
Adjunct Professor, School of Psychology
University of Birmingham
Birmingham, UK
Adjunct Professor, School of Psychology
University of Kent
Canterbury, UK

William Weir, FRCP
Infectious Disease Consultant
London, UK

John Whiting, MD
Specialist Physician
Private Practice
Brisbane, Australia

Carolyn Wilshire, PhD
Senior Lecturer
School of Psychology
Victoria University of Wellington
Wellington, New Zealand

Michael Zeineh, MD, PhD
Assistant Professor
Department of Radiology
Stanford University
Stanford, California, USA

Marcie Zinn, PhD
Research Consultant in Experimental Electrical Neuroimaging and Statistics
Center for Community Research
DePaul University
Chicago, Illinois, USA
Executive Director
Society for Neuroscience and Psychology in the Performing Arts
Dublin, California, USA

Mark Zinn, MM
Research Consultant in Experimental Electrophysiology
Center for Community Research
DePaul University
Chicago, Illinois, USA

New individuals added 23 March 2017

Norman E. Booth, PhD, FInstP
Emeritus Fellow in Physics
Mansfield College
University of Oxford
Oxford, UK

Joan Crawford, CPsychol, CEng, CSci, MA, MSc
Chartered Counselling Psychologist
Chronic Pain Management Service
St Helens Hospital
St Helens, UK

Lucy Dechene, PhD
Professor of Mathematics (retired)
Fitchburg State University
Fitchburg, Massachusetts, USA

Valerie Eliot Smith
Barrister and Visiting Scholar
Centre for Commercial Law Studies
Queen Mary University of London
London, UK

Margaret C. Fernald, PhD
Clinical and Research Psychologist
University of Maine
Orono, Maine, USA

Simin Ghatineh, MSc, PhD
Biochemist
London, UK

Alan Gurwitt, M.D.
Former Clinical Child Psychiatry Faculty Member
Yale Child Study Center, New Haven, Connecticut
University of Connecticut School of Medicine, Farmington, Connecticut
Harvard Medical School, Boston, Massachusetts
Co-author of primers on Adult and Pediatric ME/CFS
Clinician in Private Practice (retired)
Boston, Massachusetts, USA

Geoffrey Hallmann, LLB, DipLegPrac
Former Laywer, (Disability And Compensation)
Lismore, Australia

Susan Levine, MD
Clinician in Private Practice
New York, New York, USA
Visiting Fellow
Cornell University
Ithaca, New York, USA

Marvin S. Medow, Ph.D.
Professor of Pediatrics and Physiology
Chairman, New York Medical College IRB
Associate Director of The Center for Hypotension
New York Medical College
Hawthorne, New York, USA

Sarah Myhill MB BS
Clinician in Private Practice
Knighton, UK

Pamela Phillips, Dip, Dip. MSc MBACP (registered)
Counsellor in Private Practice
London, UK

Gwenda L Schmidt-Snoek, PhD
Researcher
Former Assistant Professor of Psychology
Hope College
Holland, Michigan, USA

Robin Callender Smith, PhD
Professor of Media Law
Centre for Commercial Law Studies
Queen Mary University of London.
Barrister and Information Rights Judge
London, UK

Samuel Tucker, MD
Former Assistant Clinical Professor of Psychiatry
University of California, San Francisco
San Francisco, California, USA

AM Uyttersprot, MD
Neuropsychiatrist
AZ Jan Portaels
Vilvoorde, Belgium

Paul Wadeson, Bsc, MBChB, MRCGP
GP Principal
Ash Trees Surgery
Carnforth, UK

 

ME/CFS Patient Organizations

25% ME Group
UK

Emerge Australia
Australia

European ME Alliance:

Belgium ME/CFS Association
Belgium

ME Foreningen
Denmark

Suomen CFS-Yhdistys
Finland

Fatigatio e.V.
Germany

Het Alternatief
Netherlands

Icelandic ME Association
Iceland

Irish ME Trust
Ireland

Associazione Malati di CFS
Italy

Norges ME-forening
Norway

Liga SFC
Spain

Riksföreningen för ME-patienter
Sweden

Verein ME/CFS Schweiz
Switzerland

Invest in ME Research
UK

Hope 4 ME & Fibro Northern Ireland
UK

Irish ME/CFS Association
Ireland

Massachusetts CFIDS/ME & FM Association
USA

ME Association
UK

ME/cvs Vereniging
Netherlands

National ME/FM Action Network
Canada

New Jersey ME/CFS Association
USA

Pandora Org
USA

Phoenix Rising
International membership representing many countries

Solve ME/CFS Initiative
USA

Tymes Trust (The Young ME Sufferers Trust)
UK

Wisconsin ME and CFS Association
USA

New Organizations added 23 March 2017

Action CND
Canada

Associated New Zealand ME Society
New Zealand

Chester MESH (ME self-help) group
Chester, UK

German Society for ME/CFS (Deutsche Gesellschaft für ME/CFS)
Germany

Lost Voices Stiftung
Germany

M.E. Victoria Association
Canada

ME North East
UK

ME Research UK
UK

ME Self Help Group Nottingham
UK

ME/CFS and Lyme Association of WA, Inc.
Australia

ME/CFS (Australia) Ltd
Australia

ME/CFS Australia (SA), Inc.
Australia

ME/CVS Stichting Nederland
Netherlands

ME/FM Myalgic Encephalomyelitis and Fibromyalgia Society of British Columbia
Canada

MEAction
International membership representing many countries 
 
Millions Missing Canada
Canada
 
National CFIDS Foundation, Inc.
USA
 
North London ME Network
UK
 
OMEGA (Oxfordshire ME Group for Action)
UK
 
Open Medicine Foundation
USA

Quebec ME Association
Canada
 
The York ME Community
UK
 
Welsh Association of ME & CFS Support
UK
Organization added 29 March 2017
Supportgroup ME and Disability
(Steungroep ME en Arbeidsongeschiktheid)
Groningen, Netherlands

[1] White PD, Goldsmith K, Johnson AL, et al. 2013. Recovery from chronic fatigue syndrome after treatments given in the PACE trial. Psychological Medicine 43(10): 2227-2235.

[2] White PD, Goldsmith KA, Johnson AL, et al. 2011. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. The Lancet 377: 823–836

[3] Racaniello V. 2016. An open letter to The Lancet, again. Virology Blog, 10 Feb. Available at: https://www.virology.ws/2016/02/10/open-letter-lancet-again/ (accessed on 2/24/17).

[4] White PD, Sharpe MC, Chalder T, et al. 2007. Protocol for the PACE trial: a randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise, as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy. BMC Neurology 7: 6.

[5] Tuller D. 2015. Trial by error: the troubling case of the PACE chronic fatigue syndrome trial. Virology Blog, 21-23 Oct. Available at: https://www.virology.ws/2015/10/21/trial-by-error-i/ (accessed on 2/24/17)

[6] Wilshire C, Kindlon T, Matthees A, McGrath S. 2016. Can patients with chronic fatigue syndrome really recover after graded exercise or cognitive behavioural therapy? A critical commentary and preliminary re-analysis of the PACE trial. Fatigue: Biomedicine, Health & Behavior; published online 14 Dec. Available at: http://www.tandfonline.com/doi/full/10.1080/21641846.2017.1259724 (accessed on 2/24/17)

[7] PACE Participants Newsletter. December 2008. Issue 3. Available at: http://www.wolfson.qmul.ac.uk/images/pdfs/participantsnewsletter3.pdf (accessed on 2/24/17).

[8] Sharpe M, Chalder T, Johnson AL, et al. 2017. Do more people recover from chronic fatigue syndrome with cognitive behaviour therapy or graded exercise therapy than with other treatments? Fatigue: Biomedicine, Health & Behavior; published online 15 Feb. Available at: http://www.tandfonline.com/doi/full/10.1080/21641846.2017.1288629 (accessed on 2/24/17).

[9] Green CR, Cowan P, Elk R. 2015. National Institutes of Health Pathways to Prevention Workshop: Advancing the research on myalgic encephalomyelitis/chronic fatigue syndrome. Annals of Internal Medicine 162: 860-865.

An open letter to Psychological Medicine about “recovery” and the PACE trial

by

Sir Robin Murray and Dr. Kenneth Kendler
Psychological Medicine
Cambridge University Press
University Printing House
Shaftesbury Road
Cambridge CB2 8BS
UK

Dear Sir Robin Murray and Dr. Kendler:

In 2013, Psychological Medicine published an article called “Recovery from chronic fatigue syndrome after treatments given in the PACE trial.”[1] In the paper, White et al. reported that graded exercise therapy (GET) and cognitive behavioural therapy (CBT) each led to recovery in 22% of patients, compared with only 7% in a comparison group. The two treatments, they concluded, offered patients “the best chance of recovery.”

PACE was the largest clinical trial ever conducted for chronic fatigue syndrome (also known as myalgic encephalomyelitis, or ME/CFS), with the first results published in The Lancet in 2011.[2] It was an open-label study with subjective primary outcomes, a design that requires strict vigilance to prevent the possibility of bias. Yet PACE suffered from major flaws that have raised serious concerns about the validity, reliability and integrity of the findings.[3] Despite these flaws, White et al.’s claims of recovery in Psychological Medicine have greatly impacted treatment, research, and public attitudes towards ME/CFS.

According to the protocol for the PACE trial, participants needed to meet specific benchmarks on four different measures in order to be defined as having achieved “recovery.”[4] But in Psychological Medicine, White et al. significantly relaxed each of the four required outcomes, making “recovery” far easier to achieve. No PACE oversight committees appear to have approved the redefinition of recovery; at least, no such approvals were mentioned. White et al. did not publish the results they would have gotten using the original protocol approach, nor did they include sensitivity analyses, the standard statistical method for assessing the impact of such changes.

Patients, advocates and some scientists quickly pointed out these and other problems. In October of 2015, Virology Blog published an investigation of PACE, by David Tuller of the University of California, Berkeley, that confirmed the trial’s methodological lapses.[5] Since then, more than 12,000 patients and supporters have signed a petition calling for Psychological Medicine to retract the questionable recovery claims. Yet the journal has taken no steps to address the issues.

Last summer, Queen Mary University of London released anonymized PACE trial data under a tribunal order arising from a patient’s freedom-of-information request. In December, an independent research group used that newly released data to calculate the recovery results per the original methodology outlined in the protocol.[6] This reanalysis documented what was already clear: that the claims of recovery could not be taken at face value.

In the reanalysis, which appeared in the journal Fatigue: Biomedicine, Health & Behavior, Wilshire et al. reported that the PACE protocol’s definition of “recovery” yielded recovery rates of 7 % or less for all arms of the trial. Moreover, in contrast to the findings reported in Psychological Medicine, the PACE interventions offered no statistically significant benefits. In conclusion, noted Wilshire et al., “the claim that patients can recover as a result of CBT and GET is not justified by the data, and is highly misleading to clinicians and patients considering these treatments.”

In short, the PACE trial had null results for recovery, according to the protocol definition selected by the authors themselves. Besides the inflated recovery results reported in Psychological Medicine, the study suffered from a host of other problems, including the following:

*In a paradox, the revised recovery thresholds for physical function and fatigue–two of the four recovery measures–were so lax that patients could deteriorate during the trial and yet be counted as “recovered” on these outcomes. In fact, 13 % of participants met one or both of these recovery thresholds at baseline. White et al. did not disclose these salient facts in Psychological Medicine. We know of no other studies in the clinical trial literature in which recovery thresholds for an indicator actually represented worse health status than the entry thresholds for serious disability on the same indicator.

*During the trial, the authors published a newsletter for participants that included glowing testimonials from earlier participants about their positive outcomes in the trial.[7] An article in the same newsletter reported that a national clinical guidelines committee had already recommended CBT and GET as effective; the newsletter article did not mention adaptive pacing therapy, an intervention developed specifically for the PACE trial. The participant testimonials and the newsletter article could have biased the responses of an unknown number of the two hundred or more people still undergoing assessments—about a third of the total sample.

*The PACE protocol included a promise that the investigators would inform prospective participants of “any possible conflicts of interest.” Key PACE investigators have had longstanding relationships with major insurance companies, advising them on how to handle disability claims related to ME/CFS. However, the trial’s consent forms did not mention these self-evident conflicts of interest. It is irrelevant that insurance companies were not directly involved in the trial and insufficient that the investigators disclosed these links in their published research. Given this serious omission, the consent obtained from the 641 trial participants is of questionable legitimacy.

Such flaws are unacceptable in published research; they cannot be defended or explained away. The PACE investigators have repeatedly tried to address these concerns. Yet their efforts to date—in journal correspondence, news articles, blog posts, and most recently in their response to Wilshire et al. in Fatigue[8]—have been incomplete and unconvincing.

The PACE trial compounded these errors by using a case definition for the illness that required only one symptom–six months of disabling, unexplained fatigue. A 2015 report from the U.S. National Institutes of Health recommended abandoning this single-symptom approach for identifying patients.[9] The NIH report concluded that this broad case definition generated heterogeneous samples of people with a variety of fatiguing illnesses, and that using it to study ME/CFS could “impair progress and cause harm.”

PACE included sub-group analyses of two alternate and more specific case definitions, but these case definitions were modified in ways that could have impacted the results. Moreover, an unknown number of prospective participants might have met these alternate criteria but been excluded from the study by the initial screening.

To protect patients from ineffective and possibly harmful treatments, White et al.’s recovery claims cannot stand in the literature. Therefore, we are asking Psychological Medicine to retract the paper immediately. Patients and clinicians deserve and expect accurate and unbiased information on which to base their treatment decisions. We urge you to take action without further delay.

Sincerely,

Dharam V. Ablashi, DVM, MS, Dip Bact
Scientific Director
HHV-6 Foundation
Former Senior Investigator
National Cancer Institute
National Institutes of Health
Bethesda, Maryland, USA

James N. Baraniuk, MD
Professor, Department of Medicine
Georgetown University
Washington, D.C., USA

Lisa F. Barcellos, MPH, PhD
Professor of Epidemiology
School of Public Health
California Institute for Quantitative Biosciences
University of California, Berkeley
Berkeley, California, USA

Lucinda Bateman, MD
Medical Director
Bateman Horne Center
Salt Lake City, Utah, USA

Alison C. Bested, MD, FRCPC
Clinical Associate Professor
Faculty of Medicine
University of British Columbia
Vancouver, British Columbia, Canada

Molly Brown, PhD
Assistant Professor
Department of Psychology
DePaul University
Chicago, Illinois, USA

John Chia, MD
Clinician and Researcher
EVMED Research
Lomita, California, USA

Todd E. Davenport, PT, DPT, MPH, OCS
Associate Professor
Department of Physical Therapy
University of the Pacific
Stockton, California, USA

Ronald W. Davis, PhD
Professor of Biochemistry and Genetics
Stanford University
Stanford, California, USA

Simon Duffy, PhD, FRSA
Director
Centre for Welfare Reform
Sheffield, UK

Jonathan C.W. Edwards, MD
Emeritus Professor of Medicine
University College London
London, UK

Derek Enlander, MD
New York, New York, USA

Meredyth Evans, PhD
Clinical Psychologist and Researcher
Chicago, Illinois, USA

Kenneth J. Friedman, PhD
Associate Professor of Physiology and Pharmacology (retired)
New Jersey Medical School
University of Medicine and Dentistry of New Jersey
Newark, New Jersey, USA

Robert F. Garry, PhD
Professor of Microbiology and Immunology
Tulane University School of Medicine
New Orleans, Louisiana, USA

Keith Geraghty, PhD
Honorary Research Fellow
Division of Population Health, Health Services Research & Primary Care
School of Health Sciences
University of Manchester
Manchester, UK

Ian Gibson, PhD
Former Member of Parliament for Norwich North
Former Dean, School of Biological Sciences
University of East Anglia
Honorary Senior Lecturer and Associate Tutor
Norwich Medical School
University of East Anglia
Norwich, UK

Rebecca Goldin, PhD
Professor of Mathematics
George Mason University
Fairfax, Virginia, USA

Ellen Goudsmit, PhD, FBPsS
Health Psychologist (retired)
Former Visiting Research Fellow
University of East London
London, UK

Maureen Hanson, PhD
Liberty Hyde Bailey Professor
Department of Molecular Biology and Genetics
Cornell University
Ithaca, New York, USA

Malcolm Hooper, PhD
Emeritus Professor of Medicinal Chemistry
University of Sunderland
Sunderland, UK

Leonard A. Jason, PhD
Professor of Psychology
DePaul University
Chicago, Illinois, USA

Michael W. Kahn, MD
Assistant Professor of Psychiatry
Harvard Medical School
Boston, Massachusetts, USA

Jon D. Kaiser, MD
Clinical Faculty
Department of Medicine
University of California, San Francisco
San Francisco, California, USA

David L. Kaufman, MD
Medical Director
Open Medicine Institute
Mountain View, California, USA

Betsy Keller, PhD
Department of Exercise and Sports Sciences
Ithaca College
Ithaca, New York, USA

Nancy Klimas, MD
Director, Institute for Neuro-Immune Medicine
Nova Southeastern University
Director, Miami VA Medical Center GWI and CFS/ME Program
Miami, Florida, USA

Andreas M. Kogelnik, MD, PhD
Director and Chief Executive Officer
Open Medicine Institute
Mountain View, California, USA

Eliana M. Lacerda, MD, MSc, PhD
Clinical Assistant Professor
Disability & Eye Health Group/Clinical Research Department
Faculty of Infectious and Tropical Diseases
London School of Hygiene & Tropical Medicine
London, UK

Charles W. Lapp, MD
Medical Director
Hunter-Hopkins Center
Charlotte, North Carolina, USA
Assistant Consulting Professor
Department of Community and Family Medicine
Duke University School of Medicine
Durham, North Carolina, USA

Bruce Levin, PhD
Professor of Biostatistics
Columbia University
New York, New York, USA

Alan R. Light, PhD
Professor of Anesthesiology
Professor of Neurobiology and Anatomy
University of Utah
Salt Lake City, Utah, USA

Vincent C. Lombardi, PhD
Director of Research
Nevada Center for Biomedical Research
Reno, Nevada, USA

Alex Lubet, PhD
Professor of Music
Head, Interdisciplinary Graduate Group in Disability Studies
Affiliate Faculty, Center for Bioethics
Affiliate Faculty, Center for Cognitive Sciences
University of Minnesota
Minneapolis, Minnesota, USA

Steven Lubet
Williams Memorial Professor of Law
Northwestern University Pritzker School of Law
Chicago, Illinois, USA

Sonya Marshall-Gradisnik, PhD
Professor of Immunology
Co-Director, National Centre for Neuroimmunology and Emerging Diseases
Griffith University
Queensland, Australia

Patrick E. McKnight, PhD
Professor of Psychology
George Mason University
Fairfax, Virginia, USA

Jose G. Montoya, MD, FACP, FIDSA
Professor of Medicine
Division of Infectious Diseases and Geographic Medicine
Stanford University School of Medicine
Stanford, California, USA

Zaher Nahle, PhD, MPA
Vice President for Research and Scientific Programs
Solve ME/CFS Initiative
Los Angeles, California, USA

Henrik Nielsen, MD
Specialist in Internal Medicine and Rheumatology
Copenhagen, Denmark

James M. Oleske, MD, MPH
François-Xavier Bagnoud Professor of Pediatrics
Senator of RBHS Research Centers, Bureaus, and Institutes
Director, Division of Pediatrics Allergy, Immunology & Infectious Diseases
Department of Pediatrics
Rutgers New Jersey Medical School
Newark, New Jersey, USA

Elisa Oltra, PhD
Professor of Molecular and Cellular Biology
Catholic University of Valencia School of Medicine
Valencia, Spain

Richard Podell, MD, MPH
Clinical Professor
Department of Family Medicine
Rutgers Robert Wood Johnson Medical School
New Brunswick, New Jersey, USA

Nicole Porter, PhD
Psychologist in Private Practice
Rolling Ground, Wisconsin, USA

Vincent R. Racaniello, PhD
Professor of Microbiology and Immunology
Columbia University
New York, New York, USA

Arthur L. Reingold, MD
Professor of Epidemiology
University of California, Berkeley
Berkeley, California, USA

Anders Rosén, MD
Professor of Inflammation and Tumor Biology
Department of Clinical and Experimental Medicine
Division of Cell Biology
Linköping University
Linköping, Sweden

Peter C. Rowe, MD
Professor of Pediatrics
Johns Hopkins University School of Medicine
Baltimore, Maryland, USA

William Satariano, PhD
Professor of Epidemiology and Community Health
University of California, Berkeley
Berkeley, California, USA

Ola Didrik Saugstad, MD, PhD, FRCPE
Professor of Pediatrics
University of Oslo
Director and Department Head
Department of Pediatric Research
University of Oslo and Oslo University Hospital
Oslo, Norway

Charles Shepherd, MB, BS
Honorary Medical Adviser to the ME Association
Buckingham, UK

Christopher R. Snell, PhD
Scientific Director
WorkWell Foundation
Ripon, California, USA

Donald R. Staines, MBBS, MPH, FAFPHM, FAFOEM
Clinical Professor
Menzies Health Institute Queensland
Co-Director, National Centre for Neuroimmunology and Emerging Diseases
Griffith University
Queensland, Australia

Philip B. Stark, PhD
Professor of Statistics
University of California, Berkeley
Berkeley, California, USA

Eleanor Stein, MD, FRCP(C)
Psychiatrist in Private Practice
Assistant Clinical Professor
University of Calgary
Calgary, Alberta, Canada

Staci Stevens, MA
Founder, Exercise Physiologist
Workwell Foundation
Ripon, California, USA

Julian Stewart, MD, PhD
Professor of Pediatrics, Physiology and Medicine
Associate Chairman for Patient Oriented Research
Director, Center for Hypotension
New York Medical College
Hawthorne, NY, USA

Leonie Sugarman, PhD
Emeritus Associate Professor of Applied Psychology
University of Cumbria
Carlisle, UK

John Swartzberg, MD
Clinical Professor Emeritus
School of Public Health
University of California, Berkeley
Berkeley, California, USA

Ronald G. Tompkins, MD, ScD
Summer M Redstone Professor of Surgery
Harvard Medical School
Boston, Massachusetts, USA

David Tuller, DrPH
Lecturer in Public Health and Journalism
University of California, Berkeley
Berkeley, California, USA

Rosemary A. Underhill, MB, BS, MRCOG, FRCSE
Physician and Independent Researcher
Palm Coast, Florida, USA

Rosamund Vallings, MNZM, MB, BS
General Practitioner
Auckland, New Zealand

Michael VanElzakker, PhD
Research Fellow, Psychiatric Neuroscience Division
Harvard Medical School & Massachusetts General Hospital
Instructor, Tufts University Psychology
Boston, Massachusetts, USA

Mark VanNess, PhD
Professor of Health, Exercise & Sports Sciences
University of the Pacific
Stockton, California, USA
Workwell Foundation
Ripon, California, USA

Mark Vink, MD
Family Physician
Soerabaja Research Center
Amsterdam, Netherlands

Frans Visser, MD
Cardiologist
Stichting Cardiozorg
Hoofddorp, Netherlands

Tony Ward, MA (Hons), PhD, DipClinPsyc
Registered Clinical Psychologist
Professor of Clinical Psychology
School of Psychology
Victoria University of Wellington
Wellington, New Zealand
Adjunct Professor, School of Psychology
University of Birmingham
Birmingham, UK
Adjunct Professor, School of Psychology
University of Kent
Canterbury, UK

William Weir, FRCP
Infectious Disease Consultant
London, UK

John Whiting, MD
Specialist Physician
Private Practice
Brisbane, Australia

Carolyn Wilshire, PhD
Senior Lecturer
School of Psychology
Victoria University of Wellington
Wellington, New Zealand

Michael Zeineh, MD, PhD
Assistant Professor
Department of Radiology
Stanford University
Stanford, California, USA

Marcie Zinn, PhD
Research Consultant in Experimental Electrical Neuroimaging and Statistics
Center for Community Research
DePaul University
Chicago, Illinois, USA
Executive Director
Society for Neuroscience and Psychology in the Performing Arts
Dublin, California, USA

Mark Zinn, MM
Research Consultant in Experimental Electrophysiology
Center for Community Research
DePaul University
Chicago, Illinois, USA

 

ME/CFS Patient Organizations

25% ME Group
UK

Emerge Australia
Australia

European ME Alliance:

Belgium ME/CFS Association
Belgium

ME Foreningen
Denmark

Suomen CFS-Yhdistys
Finland

Fatigatio e.V.
Germany

Het Alternatief
Netherlands

Icelandic ME Association
Iceland

Irish ME Trust
Ireland

Associazione Malati di CFS
Italy

Norges ME-forening
Norway

Liga SFC
Spain

Riksföreningen för ME-patienter
Sweden

Verein ME/CFS Schweiz
Switzerland

Invest in ME Research
UK

Hope 4 ME & Fibro Northern Ireland
UK

Irish ME/CFS Association
Ireland

Massachusetts CFIDS/ME & FM Association
USA

ME Association
UK

ME/cvs Vereniging
Netherlands

National ME/FM Action Network
Canada

New Jersey ME/CFS Association
USA

Pandora Org
USA

Phoenix Rising
International membership representing many countries

Solve ME/CFS Initiative
USA

Tymes Trust (The Young ME Sufferers Trust)
UK

Wisconsin ME and CFS Association
USA

[1] White PD, Goldsmith K, Johnson AL, et al. 2013. Recovery from chronic fatigue syndrome after treatments given in the PACE trial. Psychological Medicine 43(10): 2227-2235.

[2] White PD, Goldsmith KA, Johnson AL, et al. 2011. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. The Lancet 377: 823–836

[3] Racaniello V. 2016. An open letter to The Lancet, again. Virology Blog, 10 Feb. Available at: https://www.virology.ws/2016/02/10/open-letter-lancet-again/ (accessed on 2/24/17).

[4] White PD, Sharpe MC, Chalder T, et al. 2007. Protocol for the PACE trial: a randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise, as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy. BMC Neurology 7: 6.

[5] Tuller D. 2015. Trial by error: the troubling case of the PACE chronic fatigue syndrome trial. Virology Blog, 21-23 Oct. Available at: https://www.virology.ws/2015/10/21/trial-by-error-i/ (accessed on 2/24/17)

[6] Wilshire C, Kindlon T, Matthees A, McGrath S. 2016. Can patients with chronic fatigue syndrome really recover after graded exercise or cognitive behavioural therapy? A critical commentary and preliminary re-analysis of the PACE trial. Fatigue: Biomedicine, Health & Behavior; published online 14 Dec. Available at: http://www.tandfonline.com/doi/full/10.1080/21641846.2017.1259724 (accessed on 2/24/17)

[7] PACE Participants Newsletter. December 2008. Issue 3. Available at: http://www.wolfson.qmul.ac.uk/images/pdfs/participantsnewsletter3.pdf (accessed on 2/24/17).

[8] Sharpe M, Chalder T, Johnson AL, et al. 2017. Do more people recover from chronic fatigue syndrome with cognitive behaviour therapy or graded exercise therapy than with other treatments? Fatigue: Biomedicine, Health & Behavior; published online 15 Feb. Available at: http://www.tandfonline.com/doi/full/10.1080/21641846.2017.1288629 (accessed on 2/24/17).

[9] Green CR, Cowan P, Elk R. 2015. National Institutes of Health Pathways to Prevention Workshop: Advancing the research on myalgic encephalomyelitis/chronic fatigue syndrome. Annals of Internal Medicine 162: 860-865.

Trial By Error, Continued: Why has the PACE Study’s “Sister Trial” Been “Disappeared” and Forgotten?

by

By David Tuller, DrPH

David Tuller is academic coordinator of the concurrent masters degree program in public health and journalism at the University of California, Berkeley.

In 2010, the BMJ published the results of the Fatigue Intervention by Nurses Evaluation, or FINE. The investigators for this companion trial to PACE, also funded by the Medical Research Council, reported no benefits to ME/CFS patients from the interventions tested.

In medical research, null findings often get ignored in favor or more exciting “positive” results. In this vein, the FINE trial seems to have vanished from the public discussion over the controversial findings from the PACE study. I thought it was important to re-focus some attention on this related effort to prove that “deconditioning” is the cause of the devastating symptoms of ME/CFS. (This piece is also too long but hopefully not quite as dense.)

An update on something else: I want to thank the public relations manager from Queen Mary University of London for clarifying his previous assertion that I did not seek comment from the PACE investigators before Virology Blog posted my story. In an e-mail, he explained that he did not mean to suggest that I hadn’t contacted them for interviews. He only meant, he wrote, that I hadn’t sent them my draft posts for comment before publication. He apologized for the misunderstanding.

I accept his apology, so that’s the end of the matter. In my return e-mail, however, I did let him know I was surprised at the expectation that I might have shared the draft with the PACE investigators before publication. I would not have done that whether or not they had granted me interviews. This is journalism, not peer-review. Different rules.

************************************************************************

In 2003, with much fanfare, the U.K. Medical Research Council announced that it would fund two major studies of non-pharmacological treatments for chronic fatigue syndrome. In addition to PACE, the agency decided to back a second, smaller study called “Fatigue Intervention by Nurses Evaluation,” or FINE. Because the PACE trial was targeting patients well enough to attend sessions at a medical clinic, the complementary FINE study was designed to test treatments for more severely ill patients.

(Chronic fatigue syndrome is also known as myalgic encephalomyelitis, CFS/ME, and ME/CFS, which has now been adopted by U.S. government agencies. The British investigators of FINE and PACE prefer to call it chronic fatigue syndrome, or sometimes CFS/ME.)

Alison Wearden, a psychologist at the University of Manchester, was the lead FINE investigator. She also sat on the PACE Trial Steering Committee and wrote an article about FINE for one of the PACE trial’s participant newsletters. The Medical Research Council and the PACE team referred to FINE as PACE’s “sister” trial. The two studies included the same two primary outcome measures, self-reported fatigue and physical function, and used the same scales to assess them.

The FINE results were published in BMJ in April, 2010. Yet when the first PACE results were published in The Lancet the following year, the investigators did not mention the FINE trial in the text. The trial has also been virtually ignored in the subsequent public debate over the results of the PACE trial and the effectiveness, or lack thereof, of the PACE approach.

What happened? Why has the FINE trial been “disappeared”?

*****

The main goal of the FINE trial was to test a treatment for homebound patients that adapted and combined elements of cognitive behavior therapy and graded exercise therapy, the two rehabilitative therapies being tested in PACE. The approach, called “pragmatic rehabilitation,” had been successfully tested in a small previous study. In FINE, the investigators planned to compare “pragmatic rehabilitation” with another intervention and with standard care from a general practitioner.

Here’s what the Medical Research Council wrote about the main intervention in an article in its newsletter, MRC Network, in the summer of 2003: “Pragmatic rehabilitation…is delivered by specially trained nurses, who give patients a detailed physiological explanation of symptom patterns. This is followed by a treatment programme focussing on graded exercise, sleep and relaxation.”

The second intervention arm featured a treatment called “supportive listening,” a patient-centered and non-directive counseling approach. This treatment presumed that patients might improve if they felt that the therapist empathized with them, took their concerns seriously, and allowed them to find their own approach to addressing the illness.

The Medical Research Council committed 1.3 million pounds to the FINE trial. The study was conducted in northwest England, with 296 patients recruited from primary care. Each intervention took place over 18 weeks and consisted of ten sessions–five home visits lasting up to 90 minutes alternating with five telephone conversations of up to 30 minutes.

As in the PACE trial, patients were selected using the Oxford criteria for chronic fatigue syndrome, defined as the presence of six months of medically unexplained fatigue, with no other symptoms required. The Oxford criteria have been widely criticized for yielding heterogeneous samples, and a report commissioned by the National Institutes of Health this year recommended by the case definition be “retired” for that reason.

More specific case definitions for the illness require the presence of core symptoms like post-exertional malaise, cognitive problems and sleep disorders, rather than just fatigue per se. Because the symptom called post-exertional malaise means that patients can suffer severe relapses after minimal exertion, many patients and advocacy organizations consider increases in activity to be potentially dangerous.

To be eligible for the FINE trial, participants needed to score 70 or less out of 100 on the physical function scale, the Medical Outcomes Study 36-Item Short Form Health Survey, known as the SF-36. They also needed to score a 4 or more out of 11 on the 11-item Chalder Fatigue Scale, with each item scored as either 0 or 1. On the fatigue scale, a higher score indicated greater fatigue.

Among other measures, the trial also included a key objective outcome–the time to take 20 steps, (or number of steps taken, if this is not achieved) and maximum heart rate reached on a step-test.

Participants were to be assessed on these measures at 20 weeks, which was right after the end of the treatment period, and again at 70 weeks, which was one year after the end of treatment. According to the FINE trial protocol, published in the journal BMC Medicine in 2006, “short-term assessments of outcome in a chronic health condition such as CFS/ME can be misleading” and declared the 70-week assessment to be the “primary outcome point.”

*****

The theoretical model behind the FINE trial and pragmatic rehabilitation paralleled the PACE concept. The physical symptoms were presumed to be the result not of a pathological disease process but of “deconditioning” or “dysregulation” caused by sedentary behavior, accompanied by disrupted sleep cycles and stress. The sedentary behavior was itself presumed to be triggered by patients’ “unhelpful” conviction that they suffered from a progressive medical illness. Counteracting the deconditioning involved re-establishing normal sleep cycles, reducing anxiety levels and gently increasing physical exertion, even if patients remained homebound.

“The treatment [pragmatic rehabilitation] is based on a model proposing that CFS/ME is best understood as a consequence of physiological dysregulation associated with inactivity and disturbance of sleep and circadian rhythms,” stated the FINE trial protocol. “We have argued that these conditions are often maintained by illness beliefs that lead to exercise-avoidance. The essential feature of the treatment is the provision of a detailed explanation for patients’ symptoms, couched in terms of the physiological dysregulation model, from which flows the rationale for a graded return to activity.”

On the FINE trial website, a 2004 presentation about pragmatic rehabilitation explained the illness in somewhat simpler terms, comparing it to “very severe jetlag.” After explaining how and why pragmatic rehabilitation led to physical improvement, the presentation offered this hopeful message, in boldface: “There is no disease–you have a right to full health. This is a good news diagnosis. Carefully built up exercise can reverse the condition. Go for 100% recovery.”

In contrast, patients, advocates and many leading scientists have completely rejected the PACE and FINE approach. They believe the evidence overwhelmingly points to an immunological and neurological disorder triggered by an initial infection or some other physiological insult. Last month, the National Institutes of Health ratified this perspective when it announced a major new push to seek biomedical answers to the disease, which it refers to as ME/CFS.

As in PACE, patients in the FINE trial were issued different treatment manuals depending upon their assigned study arm. The treatment manual for pragmatic rehabilitation repeatedly informed participants that the therapy could help them get better–even though the trial itself was designed to test the effectiveness of the therapy. (In the PACE trial, the manuals for the cognitive behavior and graded therapy arms also included many statements promoting the idea that the therapies could successfully treat the illness.)

“This booklet has been written with the help of patients who have made a full recovery from Chronic Fatigue Syndrome,” stated the FINE pragmatic rehabilitation manual on its second page. “Facts and information which were important to them in making this recovery have been included.” The manual noted that the patients who helped write it had been treated at the Royal Liverpool University Hospital but did not include more specific details about their “full recovery” from the illness.

Among the “facts and information” included in the manual were assertions that the trial participants, contrary to what they might themselves believe, had no persistent viral infection and “no underlying serious disease.” The manual promised them that pragmatic rehabilitation could help them overcome the illness and the deconditioning perpetuating it. “Instead of CFS controlling you, you can start to regain control of your body and your life,” stated the manual.

Finally, as in PACE, participants were encouraged to change their beliefs about their condition by “building the right thoughts for your recovery.” Participants were warned that “unhelpful thoughts”–such as the idea that continued symptoms indicated the presence of an organic disease and could not be attributed to deconditioning–“can put you off parts of the treatment programme and so delay or prevent recovery.”

The supportive listening manual did not similarly promote the idea that “recovery” from the illness was possible. During the sessions, the manual explained, “The listener, your therapist, will provide support and encourage you to find ways to cope by using your own resources to change, manage or adapt to difficulties…She will not tell you what to do, advise, coach or direct you.”

*****

A qualitative study about the challenges of the FINE research process, published by the investigators in the journal Implementation Science in 2011, shed light on how much the theoretical framework and the treatment approaches frustrated and angered trial participants. According to the interviews with some of the nurses, nurse supervisors, and participants involved in FINE, the home visits often bristled with tension over the different perceptions of what caused the illness and which interventions could help.

“At times, this lack of agreement over the nature of the condition and lack of acceptance as to the rationale behind the treatment led to conflict,” noted the FINE investigators in the qualitative paper. “A particularly difficult challenge of interacting with patients for the nurses and their supervisors was managing patients’ resistance to the treatment.”

One participant in the pragmatic rehabilitation arm, who apparently found it difficult to do what was expected, attributed this resistance to the insistence that deconditioning caused the symptoms and that activity would reverse them. “If all that was standing between me and recovery was the reconditioning I could work it out and do it, but what I have got is not just a reconditioning problem,” the participant said. “I have got something where there is damage and a complete lack of strength actually getting into the muscles and you can’t work with what you haven’t got in terms of energy.”

Another participant in the pragmatic rehabilitation arm was more blunt. “I kept arguing with her [the nurse administering the treatment] all the time because I didn’t agree with what she said,” said the participant, who ended up dropping out of the trial.

Some participants in the supportive listening arm also questioned the value of the treatment they were receiving, according to the study. “I mostly believe it was more physical than anything else, and I didn’t see how talking could truthfully, you know, if it was physical, do anything,” said one.

In fact, the theoretical orientation also alienated some prospective participants as well, according to interviews the investigators conducted with some patients who declined to enter the trial. “It [the PR intervention] insisted that physiologically there was nothing wrong,” said one such patient. “There was nothing wrong with my glands, there was nothing wrong, that it was just deconditioned muscles. And I didn’t believe that…I can’t get well with treatment you don’t believe in.”

When patients challenged or criticized the therapeutic interventions, the study found, nurses sometimes felt their authority and expertise to be under threat. “They are testing you all the time,” said one nurse. Another reported: “That anger…it’s very wearing and demoralizing.”

One nurse remembered the difficulties she faced with a particular participant. “I used to go there and she would totally block me, she would sit with her arms folded, total silence in the house,” said the nurse. “It was tortuous for both of us.”

At times, nurses themselves responded to these difficult interactions with bouts of anger directed at the participants, according to a supervisor.

“Their frustration has reached the point where they sort of boiled over,” said the supervisor. “There is sort of feeling that the patient should be grateful and follow your advice, and in actual fact, what happens is the patient is quite resistant and there is this thing like you know, ‘The bastards don’t want to get better.'”

*****

BMJ published the FINE results in 2010. The FINE investigators found no statistically significant benefits to either pragmatic rehabilitation or supportive listening at 70 weeks. Despite these null findings one year after the end of the 18-week course of treatment, the mean scores of those in the pragmatic rehabilitative arm demonstrated at 20 weeks a “clinically modest” but statistically significant reduction in fatigue–a drop of one point (plus a little) on the 11-point fatigue scale. The slight improvement still meant that participants were much more fatigued than the initial entry threshold for disability, and any benefits were no longer statistically significant by the final assessment.

Despite the null findings at 70 weeks, the authors put a positive gloss on the results, reporting first in the abstract that fatigue was “significantly improved” at 20 weeks. Given the very modest one-point change in average fatigue scores, perhaps the FINE investigators intended to report instead that there was a “statistically significant improvement” at 20 weeks–an accurate phrase with a somewhat different meaning.

The abstract included another interesting linguistic element. While the trial protocol had designated the 70-week assessment as “the primary outcome point,” the abstract of the paper itself now stated that “the primary clinical outcomes were fatigue and physical functioning at the end of treatment (20 weeks) and 70 weeks from recruitment.”

After redefining their primary outcome points to include the 20-week as well as the 70-week assessment, the abstract promoted the positive effects found at the earlier point as the study’s main finding. Only after communicating the initial benefits did they note that these advantages for pragmatic rehabilitation later wore off. The FINE paper cited no oversight committee approval for this expanded interpretation of the trial’s primary outcome points to include the 20-week assessment, nor did it mention the protocol’s caveat about the “misleading” nature of short-term assessments in chronic health conditions.

In fact, within the text of the paper, the investigators noted that the “pre-designated outcome point” was 70 weeks. But they did not explain why they then decided to highlight most in the abstract what was not the pre-designated but instead a post-hoc “primary” outcome point–the 20-week assessment.

A BMJ editorial that accompanied the FINE trial also accentuated the positive results at 20 weeks rather than the bad news at 70 weeks. According to the editorial’s subhead, pragmatic rehabilitation has a short term benefit, but supportive listening does not.” The editorial did not note that this was not the pre-designated primary outcome point. The null results for that outcome point–the 70-week assessment–were not mentioned until later in the editorial.

*****

Patients and advocates soon began criticizing the study in the “rapid response” section of the BMJ website, citing its theoretical framework, the use of the broad Oxford criteria as a case definition, and the failure to provide the step-test outcomes, among other issues.

“The data provide strong evidence that the anxiety and deconditioning model of CFS/ME on which the trial is predicated is either wrong or, at best, incomplete,” wrote one patient. “These results are immensely important because they demonstrate that if a cure for CFS/ME is to be found, one must look beyond the psycho-behavioural paradigm.”

Another patient wrote that the study was “a wake-up call to the whole of the medical establishment” to take the illness seriously. One predicted “that there will be those who say that this trial failed because the patients were not trying hard enough.”

A physician from Australia sought to defend the interests not of patients but of the English language, decrying the lack of hyphens in the paper’s full title: “Nurse led, home based self help treatment for patients in primary care with chronic fatigue syndrome: randomised controlled trial.”

“The hyphen is a coupling between carriages of words to ensure unambiguous transmission of thought,” wrote the doctor. “Surely this should read ‘Nurse-led, home-based, self-help’…Lest English sink further into the Great Despond of ambiguity and non-sense [hyphen included in the original comment], may I implore the co-editors of the BMJ to be the vigilant watchdogs of our mother tongue which at the hands of a younger texting  generation is heading towards anarchy.” [The original comment did not include the expected comma between “tongue” and “which.”]

*****

In a response on the BMJ website a month after publishing the study, the FINE investigators reported that they had conducted a post-hoc analysis with a different kind of scoring for the Chalder Fatigue Scale.

Instead of scoring the answers as 0 or 1 using what was called a bimodal scale, they rescored them using what was called a continuous scale, with values ranging from 0 to 3. The full range of possible scores now ran from 0 to 33, rather than 0 to 11. (As collected, the data for the Chalder Fatigue Scale allowed for either scoring system; however, the original entry criteria of 4 on the bimodal scale would translate into a range from 4 to as high as 19 on the revised scale.)

With the revised scoring, they now reported a “clinically modest, but statistically significant effect” of pragmatic rehabilitation at 70 weeks–a reduction from baseline of about 2.5 points on the 0 to 33 scale. This final score represented some increase in fatigue from the 20-week interim assessment point.

In their comment on the website, the FINE investigators now reaffirmed that the 70-week assessment was “our primary outcome point.” This statement conformed to the protocol but differed from the suggestion in the BMJ paper that the 20-week results also represented “primary” outcomes. Given that the post-hoc rescoring allowed the investigators to report statistically significant results at the 70-week endpoint, this zig-zag back to the protocol language was perhaps not surprising.

In their comment, the FINE investigators also explained that they did not report their step-test results–their one objective measure of physical capacity–“due to a significant amount of missing data.” They did not provide an explanation for the missing data. (One obvious possible reason for missing data on an objective fitness test is that participants were too disabled to perform it at all.)

The FINE investigators did not address the question of whether the title of their paper should have included hyphens.

In the rapid comments, Tom Kindlon, a patient and advocate from a Dublin suburb, responded to the FINE investigators’ decision to report their new post-hoc analysis of the fatigue scale. He noted that the investigators themselves had chosen the bimodal scoring system for their study rather than the continuous method.

“I’m sure many pharmacological and non-pharmacological studies could look different if investigators decided to use a different scoring method or scale at the end, if the results weren’t as impressive as they’d hoped,” he wrote. “But that is not normally how medicine works. So, while it is interesting that the researchers have shared this data, I think the data in the main paper should be seen as the main data.”

*****

The FINE investigators have published a number of other papers arising from their study. In a 2013 paper on mediators of the effects of pragmatic rehabilitation, they reported that there were no differences between the three groups on the objective measure of physical capacity, the step test, despite their earlier decision not to publish the data in the BMJ paper.

Wearden herself presented the trial as a high point of her professional career in a 2013 interview for the website of the University of Manchester’s School of Psychological Sciences. :”I suppose the thing I did that I’m most proud of is I ran a large treatment trial of pragmatic rehabilitation treatment for patients with chronic fatigue syndrome,” she said in the interview. “We successfully carried that trial out and found a treatment that improved patients’ fatigue, so that’s probably the thing that I’m most proud of.”

The interview did not mention that the improvement at 20 weeks was transient until the investigators performed a post-hoc-analysis and rescored the fatigue scale.

*****

The Science Media Centre, a self-styled “independent” purveyor of information about science and scientific research to journalists, has consistently shown an interest in research on what it calls CFS/ME. It held a press briefing for the first PACE results published in The Lancet in 2011, and has helped publicize the release of subsequent studies from the PACE team.

However, the Science Media Centre does not appear to have done anything to publicize the 2010 release of the FINE trial, despite its interest in the topic. A search of the center’s website for the lead FINE investigator, Alison Wearden, yielded no results. And a search for CFS/ME indicated that the first study embraced by the center’s publicity machine was the 2011 Lancet paper.

That might help explain why the FINE trial was virtually ignored by the media. A search on the LexisNexis database for “PACE trial” and “chronic fatigue syndrome” yielded 21 “newspaper” articles (I use the apostrophes here because I don’t know if that number includes articles on newspaper websites that did not appear in the print product; the accuracy of the number is also in question because the list did not include two PACE-related articles that I wrote for The New York Times).

Searches on the database combining “chronic fatigue syndrome” with either “FINE trial” or “pragmatic rehabilitation” yielded no results. (I used the version of LexisNexis Academic available to me through the University of California library system.)

Other researchers have also paid scant attention to the FINE trial, especially when compared to the PACE study. According to Google Scholar, the 2011 PACE paper in The Lancet has been cited 355 times. In contrast, the 2010 FINE paper in BMJ has only been cited 39 times.

*****

The PACE investigators likely exacerbated this virtual disappearance of the FINE trial by their decision not to mention it in their Lancet paper, despite its longstanding status as a “sister trial” and the relevance of the findings to their own study of cognitive behavior therapy and graded exercise therapy. The PACE investigators have not explained their reasons for ignoring the FINE trial. (I wrote about this lapse in my Virology Blog story, but in their response the PACE investigators did not mention it.)

This absence is particularly striking in light of the decision made by the PACE investigators to drop their protocol method of assessing the Chalder Fatigue Scale. In the protocol, their primary fatigue outcome was based on bimodal scoring on the 11-item fatigue scale. The protocol included continuous scoring on the fatigue scale, with the 0 to 33 scale, as a secondary outcome.

In the PACE paper itself, the investigators announced that they had dropped the bimodal scoring in favor of the continuous scoring “to more sensitively test our hypotheses of effectiveness.” They did not explain why they simply didn’t provide the findings under both scoring methods, since the data as collected allowed for both analyses. They also did not cite any references to support this mid-trial decision, nor did they explain what prompted it.

They certainly did not mention that PACE’s “sister” study, the FINE trial, had reported null results at the 70-week endpoint–that is, until the investigators rescored the data using a continuous scale rather than the bimodal scale used in the original paper.

The three main PACE investigators–psychiatrists Peter White and Michael Sharpe, and Trudie Chalder, a professor of. cognitive behavior therapy–did not respond to an e-mail request for comment on why their Lancet paper did not mention the FINE study, especially in reference to their post-hoc decision to change the method of scoring the fatigue scale. Lancet editor Richard Horton also did not respond to an e-mail request for an interview on whether he believed the Lancet paper should have included information about the FINE trial and its results.

*****

Update 11/9/15 10:46 PM: According to a list of published and in-process papers on the FINE trial website, the main FINE study was rejected by The Lancet before being accepted by BMJ, suggesting that the journal was at least aware of the trial well before it published the PACE study. That raises further questions about the absence of any mention of FINE and its null findings in the text of the PACE paper.

Trial By Error, Continued: Did the PACE Study Really Adopt a ‘Strict Criterion’ for Recovery?

by

By David Tuller, DrPH

David Tuller is academic coordinator of the concurrent masters degree program in public health and journalism at the University of California, Berkeley.

First, some comments: When Virology Blog posted my very, very, very long investigation of the PACE trial two weeks ago, I hoped that the information would gradually leak out beyond the ME/CFS world. So I’ve been overwhelmed by the response, to say the least, and technologically unprepared for my viral moment. I didn’t even have a photo on my Twitter profile until yesterday.

Given the speed at which events are unfolding, I thought it made sense to share a few thoughts, prompted by some of the reactions and comments and subsequent developments.

I approached this story as a journalist, not an academic. I read as much as I could and talked to a lot of people. I did not set out to write the definitive story about the PACE trial, document every single one of its many oddities, or credit everyone involved in bringing these problems to light. My goal was to explain what I recognized as some truly indefensible flaws in a clear, readable way that would resonate with scientists, public health and medical professionals, and others not necessarily immersed in the complicated history of this terrible disease.

To do that most effectively and maximize the impact, I had to find a story arc, some sort of narrative, to carry readers through 14,000 words and many dense explanations of statistical and epidemiologic concepts. After a couple of false starts, I settled on a patient and advocate, Tom Kindlon, as my “protagonist”—someone readers could understand and empathize with. Tom is smart, articulate, and passionate about good science–and he knows the PACE saga inside out. He was a terrific choice whose presence in the story, I think, made reading it a lot more bearable.

That decision in no way implied that Tom was the only possible choice or even the best possible choice. I built my work on the work of others, including many that James Coyne recently referred to as “citizen-scientists.” Tom’s dedication to tracking and critiquing the research has been heroic, given his health struggles. But the same could be said, and should be said, of many others who have fought to raise awareness about the problems with PACE since the trial was announced in 2003.

The PACE study has generated many peer-reviewed publications and a healthy paper trail. My account of the story, notwithstanding its length, has significant gaps. I haven’t finished writing about PACE, so I hope to fill in some of them myself—as with today’s story on the 2011 Lancet commentary written by colleagues of Peter White, the lead PACE investigator. But I have no monopoly on this story, nor would I want one—the stakes are too high and too many years have already been wasted. Given the trial’s wealth of problems and its enormous influence and ramifications, there are plenty of PACE-related stories left for everyone to tackle.

I am, obviously, indebted to Tom—for his good humor, his willingness to trust me given so many unfair media portrayals of ME/CFS, and his patience when I peppered him with question after question via Facebook, Twitter, and e-mail.

I am also indebted to my friend Valerie Eliot Smith. We met when I began research on this project in July, 2014; since then, she has become an indispensible resource, offering transatlantic support across multiple domains. Valerie has given me invaluable legal counsel, making sure that what I was writing was verifiable and, just as important, defendable—especially in the U.K. (I don’t want to know how many billable hours she has invested!) She has provided keen strategic advice. She has been a terrific editor, whose input greatly improved the story’s flow and readability. She has done all this, I realize, at some risk to her own health. I am lucky she decided to join me on this unexpected journey.

I would like to thank, as well, Dr. Malcolm Hooper, Margaret Williams, Dr. Nigel Speight, Dr. William Weir, Natalie Boulton, Lois Addy, and the Countess of Mar for their help and hospitality while I was in England researching the story last year. I will always cherish the House of Lords plastic bag that I received from the Countess. (The bag was stuffed with PACE-related reports and documents.)

So far, Richard Horton, the editor of The Lancet, has not responded to the criticisms documented in my story. As for the PACE investigators, they provided their own response last Friday on Virology Blog, followed by my rebuttal.

In seeking that opportunity for the PACE investigators to respond, a public relations representative from Queen Mary University of London, or QMUL, had approached Virology Blog. In e-mails to Dr. Racaniello, the public relations representative had suggested that “misinformation” and “inaccuracies” in my article had triggered social media “abuse” and could cause “reputational damage.”

These are serious charges, not to be taken lightly. Last Friday’s exchange has hopefully put an end to such claims. It seems unlikely that calling rituximab an “anti-inflammatory” rather than an “immunomodulatory” drug would trigger social media abuse or cause reputational damage.

Last week, in an effort to expedite Virology Blog’s publication of the PACE investigators’ response, the QMUL public relations representative further charged that I had not sought their input before the article was posted. This accusation goes to the heart of my professional integrity as a journalist. It is also untrue—as the public relations representative would have known had he read my piece or talked to the PACE investigators themselves. (Whether earlier publication of their response would have helped their case is another question.)

Disseminating false information to achieve goals is not usually an effective PR strategy. I have asked the QMUL public relations representative for an explanation as to why he conveyed false information to Dr. Racaniello in his attempt to advance the interests of the PACE investigators. I have also asked for an apology.

 

Since 2011, the PACE investigators have released several papers, repeatedly generating enthusiastic news coverage about the possibility of “recovery”–coverage that has often drawn conclusions beyond what the publications themselves have reported.

The PACE researchers can’t control the media and don’t write headlines. But in at least one case, their actions appeared to stimulate inaccurate media accounts–and they made no apparent effort immediately afterwards to correct the resulting international coverage. The misinformation spread to medical and public health journals as well.

(I mentioned this episode, regarding the Lancet “comment” that accompanied the first PACE results in 2011, in my excruciatingly long series two weeks ago on Virology Blog. However, that series focused on the PACE study, and the comment itself raised additional issues that I did not have the chance to explore. Because the Lancet comment had such an impact on media coverage, and ultimately most likely on patient care, I felt it was important to return to it.)

The Lancet comment, written by Gils Bleijenberg and Hans Knoop from the Expert Centre for Chronic Fatigue at Radboud University Nijmegen in the Netherlan was called “Chronic fatigue syndrome: where to PACE from here?” It reported that 30 percent of those receiving the two rehabilitative interventions favored by the PACE investigators–cognitive behavior therapy and graded exercise therapy–had “recovered.” Moreover, these participants had “recovered” according to what the comment stated was the “strict criterion” used by the PACE study itself.

Yet the PACE investigators themselves did not make this claim in their paper. Rather, they reported that participants in the two rehabilitative arms were more likely to improve and to be within what they referred to as “the normal range” for physical function and fatigue, the study’s two primary outcome measures. (“Normal range” is a statistical concept that has no inherent connection to “normal functioning” or “recovery.” More on that below.)

In addition, the comment did not mention that 15 percent of those receiving only the baseline condition of “specialist medical care” also “recovered” according to the same criterion. Thus, only half of this 30 percent “recovery” rate could actually be attributed to the interventions.

The PACE investigators themselves reviewed the comment before publication.

Thanks to this inaccurate account of the PACE study’s reported findings, the claim of a 30 percent “recovery” rate dominated much of the news coverage. Trudie Chalder, one of the key PACE investigators, reinforced the message of the Lancet comment when she declared at the press conference announcing the PACE results that participants in the two rehabilitative interventions got “back to normal.”

Just as the PACE paper did not report that anyone had “recovered,” it also did not report that anyone got “back to normal.”

Three months later, the PACE authors acknowledged in correspondence in The Lancet that the paper did not discuss “recovery” at all and that they would be presenting “recovery” data in a subsequent paper. They did not explain, however, why they had not taken earlier steps to correct the apparently inaccurate news coverage about how patients in the trial had “recovered” and gotten “back to normal.”

*****

It is not unusual for journals, when they publish studies of significance, to also commission commentaries or editorials that discuss the implications of the findings. It is also not unusual for colleagues of a study’s authors to be asked to write such commentaries. In this case, Bleijenberg and Knoop were colleagues of Peter White, the lead PACE investigator.  In 2007, the three had published, along with two other colleagues, a paper called “Is a full recovery possible after cognitive behavior therapy for chronic fatigue syndrome?” in the journal Psychotherapy and Psychosomatics.

(In their response last Friday to my Virology Blog story, the PACE investigators noted that they had published a “correction” to clarify that the 2011 Lancet paper was not about “recovery”; presumably, they were referring to the Lancet correspondence three months later. In their response to Virology Blog, they blamed the misconception on an “editorial…written by others.” But they did not mention that those “others” were White’s colleagues. In their response, they also did not explain why they did not “correct” this “recovery” claim during their pre-publication review of the comment, nor why Chalder spoke at the press conference of participants getting “back to normal.”)

In the Lancet comment, Bleijenberg and Knoop hailed the PACE team for its work. And here’s what they wrote about the trial’s primary outcome measures for physical function and fatigue: “PACE used a strict criterion for recovery: a score on both fatigue and physical function within the range of the mean plus (or minus) one standard deviation of a healthy person’s score.”

This statement was problematic for a number of reasons. Given that the PACE paper itself made no claims for “recovery,” Bleijenberg and Knoop’s assertion that it “used” any criterion for “recovery” at all was false. The PACE study protocol had outlined four specific criteria that constituted what the investigators referred to as “recovery.” Two of them were thresholds on the physical function and fatigue measures, but the Lancet paper did not present data for the other criteria and so could not report “recovery” rates.

Instead, the Lancet paper reported the rates of participants in all the groups who finished the study within what the researchers referred to as “the normal ranges” for physical function and fatigue. But as noted immediately by some in the patient community, these “normal ranges” featured a bizarre paradox: the thresholds for being “within the normal range” on both the physical function and fatigue scales indicated worse health than the entry thresholds required to demonstrate enough disability to qualify for the trial in the first place.

*****

To many patients and other readers, for the Lancet comment to refer to “normal range” scales in which entry and outcome criteria overlapped as a “strict criterion for recovery” defied logic and common sense. (According to data not included in the Lancet paper but obtained later by a patient through a freedom-of-information request, 13 percent of the total sample was already “within normal range” for physical function, fatigue or both at baseline, before any treatment began.)

In the Lancet comment, Bleijenberg and Knoop also noted that these “normal ranges” were based on “a healthy person’s score.” In other words, the “normal ranges” were purportedly derived from responses to the physical function and fatigue questionnaires by population-based samples of healthy people.

But this statement was also at odds with the fact. The source for the fatigue scale was a population of attendees at a medical practice—a population that could easily have had more health issues than a sample from the general population. And as the PACE authors themselves acknowledged in the Lancet correspondence several months after the initial publication, the SF-36 population-based scores they used to determine the physical function “normal range” were from an “adult” population, not the healthier, working-age population they had inaccurately referred to in The Lancet. (An “adult” population includes the elderly.)

The Lancet has never corrected this factual mistake in the PACE paper itself. The authors had described—inaccurately–how they derived a key outcome for one of their two primary measures. This error indisputably made the results appear better than they were, but only those who scrutinized the correspondence were aware of this discrepancy.

The Lancet comment, like the Lancet paper itself, has also never been corrected to indicate that the source population for the SF-36 responses was not a “healthy” population after all, but an “adult” one that included many elderly. The comment’s parallel claim that the source population for the fatigue scale “normal range” was “healthy” as well has also not been corrected.

Richard Horton, the editor of The Lancet, did not respond to a request for an interview to discuss whether he agreed that the “normal range” thresholds represented “a strict criterion for recovery.” Peter White, Trudie Chalder and Michael Sharpe, the lead PACE investigators, and Gils Bleijenberg, the lead author of the Lancet comment, also did not respond to requests for interviews for this story.

*****

How did the PACE study end up with “normal ranges” in which participants could get worse and still be counted as having achieved the designated thresholds?

Here’s how: The investigators committed a major statistical error in determining the PACE “normal ranges.” They used a standard statistical formula designed for normally distributed populations — that is, populations in which most people score somewhere in the middle, with the rest falling off evenly on each side. When normally distributed populations are graphed, they form the classic bell curve. In PACE, however, the data they were analyzing was far from normally distributed. The population-based responses to the physical function and fatigue questionnaires were skewed—that is, clustered toward the healthy end rather than symmetrically spread around a mean value.

With a normally distributed set of data, a “normal range” using the standard formula used in PACE—taking the mean, plus/minus one standard deviation–contains 68 percent of the values. But when the values are clustered toward one end, as in the source populations for physical function and fatigue, a larger percentage ends up being included in a “normal range” calculated using this same formula. Other statistical methods can be used to calculate 68 percent of the values when a dataset does not form a normal distribution.

If the standard formula is used on a population-based survey with scores clustered toward the healthier end, the result is an expanded “normal range” that pushes the lower threshold even lower, as happened with the PACE physical function scale. And in PACE, the threshold wasn’t just low–it was lower than the score required for entry into the trial. This score, of course, already represented severe disability, not “recovery” or being “back to normal”—and certainly not a “strict criterion” for anything.

Bleijenberg and Knoop, the comment authors, were themselves aware of the challenges faced in calculating accurate “normal ranges,” since the issue was addressed in the 2007 paper they co-wrote with Peter White. In this paper, White, Bleijenberg, and Knoop discussed the concerns related to determining a “normal range” from population data that was heavily clustered toward the healthy end of the scale. The paper noted that using the standard formula “assumed a normal distribution of scores” and generated different results under the “violation of the assumptions of normality.”

*****

Despite the caveats the three scientists included in this 2007 paper, Bleijenberg and Knoop’s 2011 Lancet comment did not mention these concerns about distortion arising from applying the standard statistical formula to values that were not normally distributed. (White and his colleagues also did not mention this problem in the PACE study itself.)

Moreover, the 2007 paper from White, Bleijenberg, and Knoop had identified a score of 80 on the SF-36 as representing “recovery”—a much higher “recovery” threshold than the SF-36 score of 60 that Bleijenberg and Knoop now declared to be a “strict criterion” In the Lancet comment, the authors did not mention this major discrepancy, nor did they explain how and when they had changed their minds about whether an SF-36 score of 60 or 80 best represented “recovery.” (In 2011, White and his colleagues also did not mention this discrepancy between the score for “recovery” in the 2007 paper and the much lower “normal range” threshold in the PACE paper.)

Along with the PACE paper, The Lancet comment caused an uproar in the patient and advocacy communities–especially since the claim that 30 percent of participants in the rehabilitative arms “recovered” per a “strict criterion” was widely disseminated.

The comment apparently caused some internal consternation at The Lancet as well. In an e-mail to Margaret Williams, the pseudonym for a longtime clinical manager in the National Health Service who had complained about the Lancet comment, an editor at the journal, Zoe Mullan, agreed that the reference to “recovery” was problematic.

“Yes I do think we should correct the Bleijenberg and Knoop Comment, since White et al explicitly state that recovery will be reported in a separate report,” wrote Mullan in the e-mail. “I will let you know when we have done this.”

No correction was made, however.

*****

In 2012, to press the issue, the Countess of Mar pursued a complaint about the comment’s claim of “recovery” with the (now-defunct) Press Complaints Commission, a regulatory body established by the media industry that was authorized to investigate the conduct of news organizations. The countess, who frequently championed the cause of the ME/CFS patient community in Parliament’s House of Lords, had long questioned the scientific basis of support of cognitive behavior therapy and graded exercise therapy, and she believed the Lancet’s comment’s claims of “recovery” contradicted the study itself.

In defending itself to the Press Complaints Commission, The Lancet acknowledged the earlier suggestion by a journal editor that the comment should be corrected.

“I can confirm that our editor of our Correspondence section, Zoe Mullan, did offer her personal opinion at the time, in which she said that she thought that we should correct the Comment,” wrote Lancet deputy editor Astrid James to the Press Complaints Commission, in an e-mail.

“Zoe made a mistake in not discussing this approach with a more senior member of our editorial team,” continued James in the e-mail. “Now, however, we have discussed this case at length with all members of The Lancet’s senior editorial team, and with Zoe, and we do not agree that there is a need to publish a correction.”

The Lancet now rejected the notion that the comment was inaccurate. Despite the explicit language in the comment identifying the “normal range” thresholds as the PACE trial’s own “strict criterion for recovery,” The Lancet argued in its response to the Press Complaints Commission that the authors were only expressing their personal opinion about what constituted “recovery.”

In other words, according to The Lancet, Bleijenberg and Knoop were not describing—wrongly–the conclusions of the PACE paper itself. They were describing their own interpretation of the findings. Therefore, the comment was not inaccurate and did not need to be corrected.

(In its response to the Press Complaints Commission, The Lancet did not explain why thresholds that purportedly represented a “strict criterion for recovery” overlapped with the entry criteria for disability.)

*****

The Press Complaints Commission issued its findings in early 2013. The commission agreed with the Countess of Mar that the statement about “recovery” in the Lancet comment was inaccurate. But the commission gave a slightly different reason. The commission accepted the Lancet’s argument that Bleijenberg and Knoop were trying to express their own opinion. The problem, the commission ruled, was that the comment itself didn’t make that point clear.

“The authors of the comment piece were clearly entitled to take a view on how “recovery” should be defined among the patients in the trial,” wrote the commission. However, continued the decision: “The authors of the comment had failed to make clear that the 30 per cent figure for ‘recovery’ reflected their view that function within “normal range’ was an appropriate way of ‘operationalising’ recovery–rather than statistical analysis by the researchers based on the definition for recovery provided. This was a distinction of significance, particularly in the context of a comment on a clinical trial published in a medical journal. The comment was misleading on this point and raised a breach of Clause 1 (Accuracy) of the Code.”

However, this determination seemed based on a msreading of what Bleijenberg and Knoop had actually written: “PACE used a strict criterion for recovery.” That phrasing did not suggest that the authors were expressing their own opinion about “recovery.” Rather, it was a statement about how the PACE study itself purportedly defined “recovery.” And the statement was demonstrably untrue.

Compounding the confusion, the Press Complaints Commission decision noted that the Lancet comment had been discussed with the PACE investigators prior to publication. Since the phrase “strict criterion for recovery” had thus apparently been vetted by the PACE team itself, it remained unclear why the commission determined that Bleijenberg and Knoop were only expressing their own opinion.

The commission’s response left other questions unanswered. The commission noted that the Countess had pointed out that the “recovery” score for physical function cited by the commenters was lower than the score required for entry. Despite this obvious anomaly, the commission did not indicate whether it had asked The Lancet or Bleijenberg and Knoop to explain how such a nonsensical scale could be used to assess “recovery.”.

*****

Notwithstanding the inaccuracy of the Lancet comment’s “recovery” claim, the commission also found that the journal had already taken “sufficient remedial action” to rectify the problem. The commission noted that the correspondence published after the trial had provided a prominent forum to debate concerns over the definition of “recovery.” The decision also noted that the PACE authors themselves had clarified in the correspondence that the actual “recovery” findings would be published in a subsequent paper.

In ruling that “sufficient remedial action” had already been taken, however, the commission did not mention the potential damage that already might have been caused by this inaccurate “recovery” claim. Given the comment’s declaration that 30 percent of participants in the cognitive behavior and graded exercise therapy arms had “recovered” according to a “strict criterion,” the message received worldwide dissemination—even though the PACE paper itself made no such claim.

Medical and public health journals, conflating the Lancet comment and the PACE study itself, also transmitted the 30 percent “recovery” rate directly to clinicians and others who treat or otherwise deal with ME/CFS patients.

The BMJ referred to the approximately 30 percent of patients who met the “normal range” thresholds as “cured.” A study in BMC Health Services Research cited PACE as having demonstrated “a recovery rate of 30-40%”—months after the PACE authors had issued their “correction” that their paper did not report on “recovery” at all. (Another mystery about the BMC Health Services Research report is the source of the 40 percent figure for “recovery.”) A 2013 paper in PLoS One similarly cited the PACE study—not the Lancet comment—and noted that 30 percent achieved a “full recovery.”

Given that relapsing after too much exertion is a core symptom of the illness, it is impossible to calculate the possible harms that could have arisen from this widespread dissemination of misinformation to health care professionals—all based on the flawed claim from the comment that 30 percent of participants had recovered according to the PACE study’s “strict criterion for recovery.”

And that “strict criterion,” it should be remembered, allowed participants to get worse and still be counted as better.