Trial By Error: My Letter to Professor Chew-Graham About METRIC

By David Tuller, DrPH

Earlier this evening, I sent the following e-mail to Carolyn Crew-Graham, a professor of general practice research at Keele University. Professor Chew-Graham is the lead author of METRIC, the atrocious online training course hailed last week by Steve Brine MP as addressing “misconceptions” about the illness variously called ME, CFS, CFS/ME and ME/CFS.

Professor Chew-Graham is a co-author of the FINE study, which tested a combined form of CBT and GET called “pragmatic rehabilitation” for home-bound patients. As reported in The BMJ, FINE produced null results for its two primary outcomes of fatigue and physical function.

I look forward to Professor Chew-Graham’s response–not that I actually expect to receive one.

**********

Dear Professor Chew-Graham–

As a public health researcher and investigative journalist at UC Berkeley, I have spent the last 3+ [years] examining the many methodological and ethical failings of PACE and related studies in the body of CBT/GET research for the illness variously known as ME, CFS, ME/CFS and CFS/ME.

That includes the FINE trial, which had null results for its reported findings and modestly positive results for post-hoc, non-peer-reviewed findings published as part of rapid-response, post-publication correspondence. These data were not peer-reviewed and did not conform to the assessment methods promised in the protocol, so it is of course inappropriate to cite those findings as the authoritative conclusions from FINE. Yet that little detail doesn’t seem to have stopped you and others from doing just that.

In last week’s parliamentary debate, Steve Brine MP cited the Royal College of General Practitioners’ online course as evidence of efforts to correct the “misconceptions” about the illness. In response to this interesting claim, I posted a critique of METRIC this afternoon:

Trial By Error: Steve Brine’s Troubling Claim in Parliamentary Debate on ME

Given that my critique is scathing about METRIC’s deficiencies, I want to offer you an opportunity to respond. I would be happy to post your comments, at any length, on Virology Blog, should you choose to send them to me. In addition to your responses about METRIC, I am also interested if you share the opinion, expressed in RCGP’s 2012 document about health care decision-making and discussed in my post, that patients with this illness are apparently too confused or irrational to make their own treatment choices. (Just to be clear–I would of course include my own response to your comments.)

As I mentioned in the post, I am also preparing another blog on repeated inaccurate claims about the cost to the English branch of NHS of so-called “medically unexplained symptoms.” (I hope to post this on Wednesday, or possibly next week.)

You have stated, as have others, that around 10% of “total NHS expenditures” or “total NHS spend” goes toward services related to treating MUS. But this statement is demonstrably not based on the available facts. The relevant 2010 study, Bermingham et al in the journal Mental Health in Family Medicine, asserts that around 10% of NHS funds spent on the working-age population go toward treating MUS. (I am not necessarily endorsing Bermingham et al’s method of assessing costs; I’m just citing the data produced by the study. Bermingham et al did not examine data from other demographic groups.)

The estimated cost of these MUS services for the working-age population as a percentage of “total NHS expenditures” would obviously be much, much lower than 10%. To mischaracterize this data in the way you have serves to dramatically inflate the apparent impact of MUS services for the working-age population on the overall NHS budget. You have made this erroneous claim at least twice–in a 2017 blog post and in a paper published the same year in the British Journal of General Practice. Perhaps there are other examples I haven’t seen.

Here is what you wrote on the blog post for Keele’s Research institute for Primary Care and Health Sciences: “MUS actually accounts for a considerably high proportion of NHS activity, with approximately 10% of total NHS expenditure being spent on services for the working age population in England with medically unexplained symptoms.”

The meaning of this statement is not ambiguous: MUS services for the working-age population account for 10% of the total NHS budget, not just 10% of the amount spent by NHS on the working-age population. A commenter on the blog post pointed out this mistake, but you have not issued a correction or clarification. Just to let you know, I intend to write to the British Journal of General Practice to request a correction. It is obviously unacceptable to promote public health policies by citing data inaccurately, whether the misstatement was made in error or intentionally.

Thank you for your quick attention to this important matter. Given their interest in the issue, I have cc’d Carol Monaghan MP, Nicky Morgan MP, Darren Jones MP, and Steve Brines MP on this e-mail.

Best–David

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley

Comments on this entry are closed.

  • Sean 28 January 2019, 4:50 pm

    I guess the hapless Junior Minister Steve Brine will be getting a letter on the same theme too soon. 🙂

  • Richard Vallee 28 January 2019, 5:35 pm

    :chef’s kiss:

    Are there other organisations that would be concerned by an official training program being built upon research that itself concluded to be useless? If only for the purpose of putting on official record notice of this, quite frankly, shocking systemic malpractice.

    It just seems to breach basically every ethical problem medicine deals with, in just about the worst possible way.

    I guess technically that would be NICE but the training material itself is created by the NHS so it’s unclear who is ultimately responsible for reviewing this content.

  • Couch Turnip 28 January 2019, 5:38 pm

    “You have made this erroneous claim at least twice–in a 2017 blog post and in a paper published the same year in the British Journal of General Practice. ”

    So she’s been merrily telling GPs that MUS patients take up 10% of the NHS budget? No wonder they’re set on weeding out their so-called ‘MUS’ patients by hook or by crook and on sending them off to IAPT/CBT. 10% of NHS budget would go a long way to making life more bearable for them, wouldn’t it? What’s the real figure, I wonder? I guess we might find out in the next instalment. Can’t wait!

  • Justin Reilly 28 January 2019, 8:12 pm

    Thanks for being on the case, as always!

  • Graham McPhee 28 January 2019, 8:14 pm

    So someone who cannot understand that his 10% figure is utterly wrong (in much the same way that the PACE trial authors could not understand that a mean average value does not give a middle value if data is skewed) thinks that patients like me are unable to make rational decisions about our treatment?

    If my A-level students had displayed this level of statistical ignorance, I would have gone home despondent.

    Sadly I think this inability to be self-critical, combined with a habit of demeaning patients’ ability to understand what is happening, lies at the heart of much of our problems with proponents of these therapies for ME.

  • Couch Turnip 29 January 2019, 4:53 am

    She’s admitted that she’s ‘not very good with numbers’ – (Helen Lester Memorial lecture – see from 21.30 on video ‘Making a difference….’). I suspect that’s why she concentrates on ‘qualitative’ rather than ‘quantitative’ research.

  • Wendy 30 January 2019, 12:43 pm

    One has to wonder how and why so, so much effort and money is being poured into avoiding scientific realities ? The tax payer is a loser, the patients are losers, the families are losers – the only winner appears to be insurance companies.