Trial By Error: Some Thoughts About NICE

By David Tuller, DrPH

The UK’s National Institute for Health and Care Excellence (NICE), which develops clinical guidelines for a range of medical conditions, is currently selecting a committee to develop a new guidance for the illness it refers to as myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS). The new guidance will replace one written in 2007, when the organization was calling the illness CFS/ME. That 2007 guidance recommended graded exercise therapy (GET) and cognitive behavior therapy (CBT) as standard-of-care interventions and has been routinely cited as evidence that these treatments are effective.

Patients and advocates have long criticized the guidance and its recommendations. Last year, after a protracted public debate, NICE agreed to conduct a full-scale overhaul. Given NICE’s history of support for the long-prevailing but controversial GET/CBT treatment paradigm, patients and advocates have been skeptical that this process will result in a fair review of the science and substantive changes to the guidance. They have worried that NICE could direct the process in a way likely to advantage the status quo. NICE has declined requests to remove the 2007 version pending the expected publication of the new one in 2020, although a note on the main page for the guidance indicates that an update is in progress.

It was always going to be difficult for NICE to overcome patient and advocacy wariness about its intentions. Emerging turmoil over the composition of the new guidance development committee is likely to make that even harder. The most recent committee membership list, posted November 1, included appointments for 10 out of the 14 or 15 slots for medical and health care professionals, as well as for all five slots reserved for lay people. The roster featured quite a few professionals who appear likely to be GET/CBT proponents. Their selection has raised justifiable consternation among many who are observing the proceedings.

NICE needs to take this issue to heart. Any new guidance will not be well-served by a committee that appears stacked with promoters of the very treatments whose failing reputation triggered this overhaul in the first place. It is of course impossible to know exactly what people will do or how they will vote once they find themselves engaged in group discussion, but it is fair to make at least some assumptions about their present views based on the documented facts.

I don’t know much about NICE’s formulas for generating these committees and screening out those perceived to have unacceptable conflicts of interest. These processes were likely not designed to accommodate such contentious and divisive situations. It is possible that NICE’s standard method of designating people works well in many contexts but in this case over-selects for those aligned with dominant perspectives at the UK National Health Service. It is also possible that high-level connections have been leveraged to influence the decision-making.

**********

Group-Think in the UK

Over the last few years, the GET/CBT treatment paradigm for this illness has been exposed internationally as grounded in shoddy science. Yet the UK academic and medical establishment has continued to stand behind it. This sort of group-think about ME/CFS has prevailed for decades among interlocking circles of scientists, health care providers, government agencies, and insurance interests. It has been promoted and enabled by the supposedly independent arbiters of research at the Science Media Centre, Cochrane, and elsewhere. A lot of people have a lot of interests at stake in the battle over any new NICE guidance for ME/CFS.

Let’s be clear: NICE is pursuing this update in the first place because the evidence base for GET and CBT is crumbling under sustained and legitimate critique. Many leading scientists from outside this field and outside the UK now recognize that PACE and related GET/CBT studies suffer from an insurmountable problem. As open-label trials relying on subjective outcomes, they are infused with potential bias and cannot provide robust and trustworthy data. That potential bias is compounded by another bias-inducing factor–both GET and CBT, at least as described in PACE, actively encourage patients to believe the treatments will work.

Here’s what the NICE scoping document stated about the reasons for the guidance overhaul: “Concerns have…been raised about these interventions [GET and CBT], including challenges to the evidence supporting them and reports that people with ME/CFS have been pressured to participate in exercise programmes, leading to a worsening of symptoms.” This statement suggests that members of the guidance development committee need to be able to recognize and address these concerns and challenges, not just ignore and dismiss them.

The GET/CBT proponents on the NICE committee should of course have the chance to make their case that their favored treatments are effective. But they cannot base this case on the argument that patients and others who raise concerns about the research are irrational or engaged in harassment. These defenders must make the case based on the quality of the evidence, not the reputations of the investigators or anecdotal reports from their own clinical experiences.

**********

Tough task for GET/CBT proponents

In fact, these GET/CBT proponents have a formidable job. They need to convince the rest of the NICE committee that open-label studies with subjective outcomes can provide reliable data—even though these studies cannot do that. They need to explain why studies of people with unexplained “fatigue”—which provide most of the evidence for GET and CBT–should apply to patients diagnosed through more specific case definitions. They need to explain why objective measures in these studies have failed to support the subjective claims of success, and why these failures have been downplayed or ignored.

If NICE committee members cite PACE favorably, they need to explain why more than 100 scientists, clinicians, academics and other experts signed Virology Blog’s most recent open letter to The Lancet, which referenced the study’s “unacceptable methodological lapses.” They need to explain why 13 % of PACE participants met the outcome threshold for physical function (“within normal range” per The Lancet, “recovered” per Psychological Medicine) before receiving any treatment at all. That paradox alone strips the study of validity; concealing such details from readers suggests possible research misconduct. Any PACE co-authors on the NICE committee should be asked about these and related aspects of the trial.

It appears that a couple of those selected for the NICE committee might be involved in either FITNET-NHS or MAGENTA, two pediatric trials from Bristol investigators. If either of these studies results in publications during the next two years, the findings might be tossed into the mix for NICE consideration. Both are open-label trials with subjective outcomes, so like other studies in this domain they are at enormous risk of bias and should not be used as the basis for clinical guidelines and public health policy.

Beyond having to grapple with that core reality, NICE committee members involved with these studies should address some other peculiarities. In the case of FITNET-NHS, that means explaining why the trial has been recruiting participants by essentially promising them that most who receive the intervention will achieve “recovery.” Recruiting participants by appearing to promise “recovery” would induce an unknown amount of bias—on top of the bias already inherent in the basic study design of an open-label trial with subjective outcomes.

With MAGENTA, the investigators have also managed to add more bias to the bias already inherent in the basic study design. In this case, 100 children were recruited as part of a feasibility study, which was then extended into a full trial of more than 220 participants. Yet the full trial’s primary and secondary outcomes were designated only after the feasibility study participants had provided data–an excellent way to generate bias in the reported findings.

Any effort to portray MAGENTA results as data from a prospective trial should be countered. Prospective trials are supposed to have pre-designated primary and secondary outcomes to avoid the possibility of selective outcome reporting. Prospective trials are not, like MAGENTA, supposed to engage in selective outcome reporting by designating primary and secondary outcomes after almost half their participants have been recruited and assessed. The reported findings from MAGENTA, as from FITNET-NHS, are likely to be so fraught with bias as to be meaningless.

**********

The role of “eminence-based” medicine

I have little or no personal knowledge of most committee members, their degree of involvement in past and current research efforts, and their depth of support for the CBT/GET paradigm. Some patients and advocates would like members to be removed from the committee because of their affiliations, published writings and employment histories, and I can understand why community members would feel that way. In this post, I am focused on the need for all committee members—whether these or others–to back up any claims with research that can withstand reasonable scientific scrutiny.

Unfortunately, some leading lights of the GET/CBT ideological brigades seem so used to deference that they have appeared surprised and even offended at having their work challenged by well-regarded scientists from outside their intellectual and academic networks. As Brian Hughes, a psychology professor at National University of Ireland, Galway, recently explained, the domain of “eminence-based” medicine has played a key role in disseminating the PACE/GET/CBT framework. This treatment approach achieved hegemony largely through the status and prestige of those who promoted it, not through the power and integrity of the research.

Previous publications in this field—including the 2007 NICE guidance—often read as if those allowed into an exclusive club were reviewing, approving, and citing each others’ works. No one in a position of authority appears to have seriously pushed back against the self-serving assumptions and methodological missteps that mar many of these publications. Perhaps this sort of coddling is why some of these investigators appear incapable of presenting coherent explanations for such startling anomalies as the inclusion in a study of participants who are already “recovered” on key variables at baseline.

Instead, these investigators respond that the various concerns have all been addressed multiple times. They maintain that others are misrepresenting what they wrote. They repeat stories about being targeted by aggressive patients. In the UK, these diversionary tactics have often helped to stifle honest debate.

**********

As others move on…

Elsewhere, the scientific community has intensified the search for biomedical causes and treatments while moving beyond the shortcomings of PACE and other GET/CBT studies. The US Centers for Disease Control and Prevention removed these two interventions from its recommendations more than a year ago, given the lack of convincing evidence to support claims of effectiveness. That does not mean, of course, that no one has ever benefited from them, including some people diagnosed with some variant of the illness or cluster of illnesses under the ME/CFS umbrella. It means that the available data do not justify presenting them in clinical guidelines or public health recommendations as the standard-of-care or as “evidence-based” treatments.

UK authorities have stonewalled these issues, as I know from my own three-year effort to breach this alternate zone of reality. Heavy-weights and grandees still applaud PACE and other GET/CBT studies as high quality research. I assume some of these smart people do not understand that they are wrong. Perhaps the NICE process could be a turning point, if it evolves into a forum for assessing the science with real-world metrics and standards.

These standards would have to include what should be a non-controversial point–that studies in which participants have met outcomes thresholds at baseline cannot be taken seriously. Another non-controversial point should be that open-label trials relying on subjective outcomes are at high risk of bias—especially when the treatments themselves include claims that they are already proven to be effective. A third non-controversial point should be that patients and others have a right to raise issues about research and file freedom-of-information requests without being labeled “vexatious” or anti-scientific.

In fact, a robust debate about ME/CFS treatments rooted in scientific principles would be welcome, since such a debate has been largely absent in the UK. But it would require the NICE committee to include, in addition to a group of apparent GET/CBT proponents, a comparable cohort of biomedical professionals with some knowledge of clinical trials—that is, experts capable of rebutting claims of treatment effectiveness that exceed the evidence. The current list of committee members seems slim on that front. NICE should address this apparent discrepancy if it has any hope or expectation that the new guidance might prove credible and acceptable to a broad audience of clinicians, patients, care-givers, and others.

{ 19 comments… add one }
  • Wendy Boutilier 19 November 2018, 6:25 pm

    Thank you David.

  • Steve Hawkins 19 November 2018, 7:08 pm

    A ‘nicely’ crafted piece: if I may say so. (y)

  • Alex Young aka alex3619 19 November 2018, 7:43 pm

    Eminence based medicine used to be discussed as Zombie Science, though to be fair ZS includes monetary and political influence and gain, not just prestige.

  • Anonymous 19 November 2018, 7:56 pm

    The true reason that NICE suddenly decided to update the guideline was the petition of 15,000 and the public and professional outcry when they announced that the “CFS/ME” guideline was to move to the static list – it had nothing to do with their acceptance of the science or evidence against GET and CBT

    Things have gone badly – shockingly so – and the stitch up began long before the review committee were announced – see NICE’s document
    Appendix C: stakeholder consultation comments table
    Consultation dates: 10 to 24 July 2017

    also found on M.E. Research and the M.E. Association websites.

    The Countess of Mar still hasn’t had an answer about who the secretive “surveillance committee” are – or the “topic experts” who are in fact those who have already decided what scientific evidence does count and rejected anything not in line with BPS views.
    As highlighted in the Appendix C (mentioned above) the secretive Topic Experts have read only abstracts and not summaries!
    Invest in M.E have commented on being given under 24 hrs to attend some meetings, a very short time to review huge documents, and raised commented issues with the scope.
    Many stakeholders have commented on the breach of the Equalties Act law, ethical issues, NICE not meeting NICE’s own standards etc.

    As Charlotte Blease recently pointed out the “epistemic injustice” that people with ME/CFS have dealt with for decades is likely to continue.

    I want to know how many complaints NICE have received about their Guidelines Review Committee – and are they even acknowledging them?

    Of particular concern are those who recommend safeguarding procedures are used to force children into GET or CBT against both child and parental wishes, the PACE trial author, someone who worked on the FINE trial data, and those who insist that ME/CFS is not neurological but “medically unexplained symptoms”. Just how does a co-author of Per Fink get on a panel? Or someone who will almost certainly lose her job should CBT be withdrawn?
    This is not simply about the BPS model, it is about abuse of patients, people with fundamental disagreements with the illness’s classification as neurological – which the NHS must follow as a World Health Organization member who has adopted the ICD for diagnosis.

    There is no transparency and secret dealings continue – as revealed by Invest in M.E.

    This must stop.
    Please keep investigating – there’s so more that would reveal exactly how the Wessely school are in control of NICE.

  • Graham McPhee 19 November 2018, 9:45 pm

    Thanks David, once again, for a clear and comprehensive explanation of the situation, and of our concerns.

    I’d like to add one further concern though. When we focus on PACE, we take the focus away from the fact that the guidelines were not based on PACE, but on a large number of smaller, similar trials. PACE was seen to confirm their findings, and in a way it did. All of them suffered from the same problem: only those that concentrated solely on subjective answers claimed these therapies were effective. Those few studies that used objective assessments found they were not. Somehow NICE thought that if a lot of poor studies said the same thing, they must be right. Has NICE yet learned the difference between quantity and quality, and will that message be allowed to percolate through?

    We do have some very good members on that committee. They aren’t going to give in to shoddy science. I remain optimistic.

  • Paul Kayes 20 November 2018, 1:49 am

    Thank you David a brilliant summation of the situation we face here in the U.K.

  • Ellen Goudsmit 20 November 2018, 3:35 am

    You’re largely wrong. But if you won’t learn from history, and seem predisposed to narratives from men, even if new kids on the block, then little should change. You’re doing the same as journals. You prefer views of certain individuals. And they don’t know history of this and weren’t there. All this is guess eork, yet we have facts. Keep ignoring those, David. Who rund the world? Boys! Well, certain boys. Not the experts like the late Dr Ho-Yen.

  • Barbara R 20 November 2018, 7:08 am

    Can you specify more specifically in what way wrong Ellen?
    I am unclear as to the point you are trying to make, and I have been around this for some 25 years!
    “…seem predisposed to narratives from men…” I really don’t think so!-What about the pursuit of Bath and it’s female ‘acolites’? FG at the BMJ, FF at the SMC?

    What has preference got to do with this here?
    It’s true that those who know the history and who were there, (many are here no more – Alan Franklin, Terry Mitchell, Bruce Carruthers etc) have been sidelined and ignored by the powers that be.
    That’s power politics in medicine like everywhere else, – good decent doctors rarely play that game after all. But there are many females currently scrabbling to play like CG, ( SW’s….) TC, EC, …..
    I think David’s appraoch is spot on personally and I am grateful for his attention to NICE/RCP and their processes.

  • Barbara R 20 November 2018, 7:10 am

    I apologise for my spelling mistakes- as done in haste!

  • Freda 20 November 2018, 8:04 am

    In 1997 papers white made objective measurements these showed little advantage from GET and CBT. Presumably this is why objective measured were dropped. Why can’t the UK clinics objectively prove the impact of their programs?

  • Erik Johnson 21 November 2018, 12:26 am

    As an Incline Village member of the original CFS cohort, I had been 100% convinced that people would want the real story.
    To my total amazement, it was the opposite.
    The “CFS community” and now “ME/CFS” or “ME” community had no interest in getting the facts of how and why “CFS” was coined and have been more deceitful, vicious and malevolent toward us than the CDC ever was.
    So I “‘Don’t have a dog in this battle”
    To me, this is just a pack of mad jackals ripping each other to pieces.
    No matter which who wins or loses, all have proven they are antagonists to the history of ME and the history of CFS, and wish to see our evidence destroyed.

    And while I don’t care if the “ME advocates” get anywhere, it is worth noting that by portraying British psychs as having done nothing more than “shoddy research”, this is essentially forgiving them of the science fraud they are really guilty of.

    However, this is what the “patient community” wants, so this is what they are going to do.
    Personally, I think they have scuttled their own best evidence.

  • Kelly McLellan 21 November 2018, 3:56 pm

    I would like to know what everyone thinks that the NICE guidelines should say and why they should say it.
    Is anyone (or has anyone) actually reviewing the literature, in the way that NICE is supposed to, in an attempt to come up with something or are we all just leaving it to NICE? If we are leaving it to NICE then we aren’t really having a discussion, let alone an open one, because no matter what NICE come back with it seems to me that people will be unhappy. That isn’t a very positive way of going about things.

  • Dream11 Football Prediction 22 November 2018, 12:32 am

    It’s amazing to visit this website and reading the views
    of all friends about this piece of writing, while I am also keen of getting
    know-how.

  • Anonymous 22 November 2018, 2:27 pm

    Good to raise this topic but I feel, David, you have held back here. Could be more cutting, like your previous blogs.
    Having Dr Murphy, a PACE Trial author, on the committee is an outrage – no patient would ever trust her – we should demand nothing short of her removal.
    Same for the member who has authored books about ME being “medically unexplained symptoms” and writing about ME patients in a derogatory way.

  • Another 23 November 2018, 6:17 am

    Petition against biased committee members of the NICE Guidelines?

  • Mary 24 November 2018, 3:21 am

    When asked what do we want in the guidelines look to the IOM clinical guide and ICC for diagnostic criteria and self management. Look to Work well for CPET tests for the less ill to Myhill and Shepherd to food and chemical intolerances heart rate based testing and pacing…. Physiogy and biology not psychology. Science and evidence not eminence. Quality dissection of papers not quick reads of abstracts and conclusions.

  • Nancy Blake 26 November 2018, 10:27 am

    The IOM have read the research and state that ME/CFS is not psychogenic, which ends any justification for the BPS model, involvement of anyone adhering to that model. No place for anyone involved with psychiatry or any type of therapy aimed at ‘changing beliefs’. It is not a case for ‘balance’, or a level playing field, the psychogenic idea is just wrong.
    The IOM also state that the defining feature of ME/CFS is that ‘exertion of any type – physical, cognitive or emotional- may adversely affect many organ systems…’
    Our belief that we have a serious medical disease and that exercise will harm us is correct.
    GET is as damaging as patients say it is.
    It follows that maximum conservation of exertion should e the fundamental principle of management. It should be noted that when six months of rest was the first line of treatment, only a proportion ontined to get worse… now most do. GET is culpable medical abuse.
    The facts are clear. uncomplicated and unequivocal.
    So is the fact that the power and prestige of those who would suffer great financial and reputational loss from acknowledgement of these facts means the review conclusions have been determined before it began.
    The elaborate processes have one aim: to make sure this happens.

Leave a Comment

Next post:

Previous post: