Trial By Error: My 2011 Exchange with White et al about Case Definition

By David Tuller, DrPH

My first exchange of views with the PACE authors involved the issue of case definition–the criteria used to identify the illness they  called chronic fatigue syndrome. This exchange took place courtesy of The New York Times, not long after The Lancet published the results of the PACE trial. In March, 2011, the Times ran a piece about the role of disease criteria in epidemiology, in which I analyzed the PACE trial’s use of the problematic fatigue-based Oxford criteria. (This was actually my second piece involving PACE; the first, a news story, was a piece of crap because I took the study at face value, having never heard of it before the findings were announced.)

After my case definition story ran, I received an e-mail from Professor Michael Sharpe, requesting a correction and–if I remember accurately–offering to send me a copy of the paper. Professor Sharpe’s complaint was that I had not mentioned the PACE trial’s sub-group analyses of participants who had met two other case definitions. I forwarded Professor Sharpe’s note to my editor, who agreed to run a letter from the PACE authors, along with my response. I thought it would be interesting to revisit that exchange, so I’ve posted it below.

Meanwhile, case definition continues to be a major source of debate and contention in the community of people–patients, advocates, researchers–interested in this domain. The issue also gets entangled with the back-and-forth on the name or names used for the illness/disease as well as the core symptom of “post-exertional malaise” or “post-exertional neuro-immune exhaustion” or “exertion intolerance” or whatever.

In these discussions, the distinctions between clinical and research case definitions often get overlooked. As a general rule, clinical case definitions are broader because you want to make sure everyone who might have the condition gets medical care, even if they are eventually determined to be suffering from something else. Research case definitions can be tighter because you want a more homogeneous group to reduce the possibility that extraneous factors might muddy the clarity of the results.

For example, a clinical case definition for an illness might include people with co-morbidities that can generate overlapping symptoms. However, the parallel research case definition might exclude these patients, since their presence could make it harder to interpret the findings.

I’m not going to address specifics of the various criteria in this post. For now, here are a few helpful case definition resources for ME, ME/CFS, etc:

#MEAction recently posted “Demystifying the Diagnostic Criteria for ME and Related Disease,” a thorough account of six key case definitions.

MEAdvocacy, which supports the 2011 ME International Consensus Criteria, has a helpful side-by-side comparison of that case definition with the one from the Institute of Medicine (now the National Academy of Medicine), among other resources.

Last August, the US ME/CFS Clinician Coalition released a set of diagnostic and treatment guidelines based on the IOM/NAM case definition.

Last year, I posted a Q-and-A with Leonard Jason, a professor of psychology at DePaul University, who has published many studies about case definition in this domain. In our conversation, he expressed concerns about use of the IOM case definition in research because it does not exclude those with primary psychiatric disorders.

**********

On March 4, 2011, The New York Times published “Defining an Illness is Fodder for Debate,” my story on case definitions in epidemiology. (The same story ran in the print edition four days later.) On March 14, the news site published the following exchange:

To the Editor:

In “Defining an Illness Is Fodder for Debate” (March 8), your reporter David Tuller correctly noted that the way an illness is defined can often determine what is found in studies of it. He also suggested that this problem of definition had limited the interpretation of our trial of treatments for chronic fatigue syndrome, published recently in The Lancet. That is not the case.

The patients in this trial had a disabling chronic illness in which fatigue was their main symptom and for which no alternative had been found; that is the definition of the syndrome used in Britain. But we also assessed trial participants to see if they met two other definitions of the illness that are favored by some scientists.

We found that both cognitive behavior therapy and graded exercise therapy, when added to specialist medical care, were most effective not only in the whole sample but also in the participants who met these alternative criteria. In addition, these treatments were the most effective whether or not a patient was depressed, a not uncommon accompaniment to this chronic and misunderstood illness. The trial also found that these treatments were safe so long as they were provided by appropriate therapists trained to help patients with chronic fatigue syndrome.

So to Mr. Tuller’s question “Does the evidence from that study prove that these strategies would help patients identified as having chronic fatigue syndrome through very different criteria?,” the answer is “Yes, it does.” Patients and their doctors now have robust evidence that there are two safe treatments that can improve both symptoms and quality of life, however the illness is defined.

P.D. White

T. Chalder

M. Sharpe

London and Edinburgh

The writers are the principal investigators of the PACE trial.

***

David Tuller replies: The article asked whether findings among a population defined by one set of criteria would apply to populations defined by “very different criteria.” In this study, all participants were first defined, identified and selected not by different criteria but by the same criteria, the so-called Oxford criteria used in Britain. Subgroups within that already screened population who also meet secondary criteria are not easily compared to patients who have not been screened, since an unknown number who met the secondary criteria might not have met the study’s criteria for inclusion. The gold standard for making comparisons across groups of patients identified by three varying case definitions would be a study with three completely separate cohorts, not one large sample with two embedded subgroups.

 

 

 

 

 

Comments on this entry are closed.

  • Paul Fox 31 December 2019, 7:03 am

    According to White et al, PACE “… found that [GET and CBT] were safe so long as they were provided by appropriate therapists trained to help patients with chronic fatigue syndrome.” So, according to these PACE authors, what are “appropriate therapists”? Do they know, and have they thought through what they are suggesting, or is this just a lot of vaguely plausible-sounding old tosh that they have dreamt up in the absence of real understanding? What else can it be when PACE is, patently, worse than worthless and that the “training” given to these “appropriate therapists” is based on nothing other than dogma, speculation and fancy?

  • Richard Vallee 31 December 2019, 1:31 pm

    “that can improve both symptoms and quality of life”

    Symptoms plural, despite exclusive focus over the one symptom of “fatigue”, itself redefined to mean lack of motivation instead of, you know, what it actually means. PACE did not measure “quality of life” so as typical just making stuff up.

    “however the illness is defined”

    Says everything about how unserious the effort was, a dog and pony show that replaced substance with style and sunk cost too large not to commit to it. In hindsight, after Sharpe’s own Oxford criteria have been downgraded by AHRQ as being unreliable, of basically being too much “however”, which was always evident, this is particularly damning of their own incompetence and inability to apply to the task other than fabricating a conclusion that had been reached many years prior.

    I wonder what their peers would think of research over “however this illness is defined”. Well, actually we know, they award and applaud it, and it makes for a very poor outlook over Wessely’s dream of placing psychiatry as frontline medicine. Can’t even scratch their own ass yet and promising a minute mile run out of the crib with a full diaper.

  • jimells 31 December 2019, 3:33 pm

    “a dog and pony show”

    Indeed. This is marketing, not research. It’s hardly different from the NIH and CDC who have a ME public relations program but no ME research program worthy of the name.

    This is the unavoidable end result of the financialization of everything by the Wall Street crowd and their global associates. There is now little effort to actually explore and understand the world around and within us. Instead the universities have been turned into knowledge factories that produce junk research papers at an ever faster rate in order to stay in the “publish or perish” grant-seeking race. If anything actually useful comes out of research, that seems to be more of a fortunate side-effect.

    When the number of new reliable papers drops to zero, will anyone notice?

    Soon there will be artificial “intelligence” software where Sharpe and buddies can type in a hypothesis and a conclusion. The software will fill in the rest with gibberish and it won’t look much different from the Wessely School papers we get now. No one will notice the difference except for patients and advocates but who cares about them anyway.

    People want to keep politics and science separate but that is impossible. What has happened to us has not happened in a vacuum. We have been and continue to be severely harmed by policies of deliberate indifference instituted to protect profits.

    The institutions and people at the top of the pyramid are squeezing everybody else harder and tighter every year. They are also successfully dividing-and-conquering everybody else into small atomized groups competing to be the biggest most deserving victims in hopes of gathering a few more crumbs.

    Where I live the decade is ending with a major winter storm with another storm close behind. A perfect metaphor for greeting the new decade.

    (just so it’s clear, I’m referring to much more than just BPS research)

  • jimells 31 December 2019, 4:55 pm

    If Rip Van Winkle with ME had laid down for a nap on Dec 31, 1999 and woke up today, he’d still be treated with disdain, still be denied social benefits, and still have no effective treatments.

    Happy New Year?

  • CT 1 January 2020, 12:15 pm

    I wholeheartedly agree jimells, this is all so wrong and yet nothing ever changes. I’m beginning to think that it never will. It’s not how I’d wish to feel when seeing in a new year and decade.

  • Wendy Boutilier 4 January 2020, 12:42 pm

    Dr BM Carruthers led the team to produce the CCC-2003 for ME/CFS and the group of International ME Experts who meticulously researched and designed the ICC-2011 specifically to diagnose ME patients accurately & quickly with no 6 month Chronicity required. The ICC can diagnose patients ranging from Mild to Very Severe. There is an ICPrimer for Dr’s clinical use which makes this criteria useful for both clinical & research. Dr Carruthers believed each patient needs to be individually assessed & treated because we all have our personalized envelope of Symptoms & Severity. To ignore the patient’s individual needs so they can be mass diagnosed to build prevalence, only attracts funding. It does not explore the individual’s needs and research will always be inconclusive. CFS/SEID research can’t help the severe patient who lives in purgatory.

    “In writing the history of a disease, every philosophical hypothesis whatsoever, that has previously occupied the mind of the author, should lie in abeyance. This being done, the clear and natural phenomena of the disease should be noted—these and these only. They should be noted accurately, and in all their minuteness; … No man can state the errors that have been occasioned by these physiological hypotheses. Writers, whose minds have taken a false colour under their influence, have saddled diseases with phenomena which existed in their own brains only; but which would have been clear and visible to the whole world had the assumed hypotheses been true.” Dr BM Carruthers
    https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1860613/

  • boolybooly 5 January 2020, 9:15 am

    Yes exactly, looks to me like the PACE authors are building on methodology which lacks experimental rigor in confounding broad clinical definitions with, if not substituting them for, research definitions.

    My sense of this is it has been going on a long time since before the game changing Mental Health Parity Act (MHPA) 1996 superceded by the Mental Health Parity and Addiction Equity Act (MHPAEA) 2008 as a technical sleight of hand which has the intended outcome of persuading medical and legal professions that CFS is a mental condition and therefore requires less expensive treatment.

    This repurposing of psychological research set out to utilise rather than clarify typical intellectual errors which often obfuscate scientific experimentation and set precedents with a natural momentum which continue today. Which is why we need hawkeyes like you David to point these errors out.

    Which is why IMHO it cannot be said too many times, we need better criteria, to distinguish subtypes effectively to enable scientific discovery.

  • Steve Hawkins 7 January 2020, 4:45 pm

    “…both cognitive behavior therapy and graded exercise therapy, when added to specialist medical care, were most effective…”

    And stop right there. This tells you you are being hoodwinked:

    1) There is *no* ‘specialist medical care’, and the authors know it.

    The most common treatments meted out to putative M.E. patients by the medical profession, are disbelief, suspicion, disdain, sarcasm, vexation, character assassination, and avoidance, in various combinations and degrees.

    It is grossly misleading to cause readers to believe that there is any ‘standard care’ for M.E. patients: in all too many cases, the standard is medical and psychological abuse: anything but care.

    If Sharpe et al felt a need to do a supplementary breakdown of patients by case definition, then they should, rightly, have also compared these with cohorts of patients separated by their various degrees of mistreatment through the *lack* of there being any specialist care, or even any definition of specialist care, and;

    2) Telling readers CBT/GET were “most effective” is not the same as telling them this treatment was effective, nor even what it was effective at, or to what degree!

    This trial model is just the model drug manufacturers use to play rival companies’ ‘me too’ products off against each other, where the ‘benefits’ are usually marginal, and not even intended to try and establish any cure (for most diseases, finding a cure would be a financial disaster for drug cos).

    It may be ‘job done’ for *drug co* researchers to claim any ‘treatment’ under this kind of trial was the ‘most effective’, but patients want to be cured: not just cost the social and health services slightly more or less for remaining just as ill and unemployed as they were to start with. (And just as unwelcome at their typical GP’s surgery (if they can get there!)).

    There is scarcely any output from these people that isn’t full of weasel words ‘normally’ (another favourite) found in advertorial on company drug PR, and woo, sites.

  • Bryan Thomas 13 January 2020, 7:04 am

    This article seems to be great!
    We are organizing a Global Summit on Infectious Diseases.
    If you are interested you can join the summit.
    Website link: https://coalesceresearchgroup.com/conferences/infectiousdiseases