Trial By Error: A Letter to KCL, Another Letter to BMJ

By David Tuller, DrPH

Last week, I wrote about a troubling press release issued by King’s College London regarding a major study of cognitive behavioural therapy as a treatment for so-called dissociative seizures. On Friday, I sent a letter to the two communications people listed on the press release about the study, as well as to the corresponding author.

I have also written some recent posts about BMJ’s methodologically and ethically challenged pediatric study of CBT plus music therapy–herehere and here. Two weeks ago, several colleagues and I wrote a letter of concern to Professor Imti Choonara, editor-in-chief of BMJ Paediatrics Open, and Dr Fiona Godlee, editorial director of BMJ. We have not received a reply, so this morning I sent a follow-up nudge.

Both letters are posted below.

**********

Subject: Misleading press release about CODES trial

Dear Mr Booth and Ms Rianjongdee–

I am a journalist and academic fellow at the School of Public Health at the University of California, Berkeley. I frequently write about research in the domains of so-called medically unexplained symptoms. Much of my work appears on Virology Blog, a science site hosted by Professor Vincent Racaniello, a microbiologist at Columbia University. (I have cc’d Professor Racaniello on this e-mail.)

I am writing because of concern that a recent press release–about findings from the CODES trial of cognitive behavioural therapy for dissociative seizures–has presented a misleading picture of the research. Lancet Psychiatry published the study last month. The press release about the trial was posted on the Maudsley Biomedical Research Centre site on May 21st and reposted on June 3 on the KCL news site. 
 
I wrote about troubling discrepancies between the trial results and the press release on Virology Blog this week: https://www.virology.ws/2020/06/11/trial-by-error-a-kings-college-london-press-release-hides-the-bad-news/

In a clinical trial, the primary outcome is the primary outcome for a reason–it is the outcome designated by the investigators as the most significant indicator of treatment efficacy. Secondary outcomes are just that–of secondary importance.

In CODES, the primary outcome was seizure frequency–a salient fact not disclosed in the press release, presumably because the intervention did not produce positive results for this measure. These null results for seizure frequency were mentioned in passing in the bottom half of the press release–but the status of seizure frequency as the primary outcome was not. At the same time, the modest reported improvements in some secondary indicators–out of a selection of more than a dozen–were hailed as proof of effectiveness, including in a statement from Professor Trudie Chalder, one of the co-investigators. 

I plan to write more about this matter soon. To that end, here are a few questions: 

1) Why did the press release not disclose that seizure frequency at 12 months after randomization was the primary outcome? 

2) Why did the press release not disclose that all the reported benefits were from secondary outcomes and not the primary outcome?

3) Did the lead investigators review the press release for accuracy before it was posted? 

4) Does KCL believe it is appropriate to disseminate information about clinical trial results without disclosing which outcome was designated as most significant by the investigators before they started the study?

5) In KCL’s view, does the decision to withhold key information about the primary outcome from the press release represent a commitment to core academic values, such as transparency and integrity, or a violation of them? 

Thanks. I look forward to hearing from you. In addition to Professor Racaniello, I have cc’d Professor Goldstein, the corresponding author of the CODES study. For full transparency, I plan to post this letter on Virology Blog.

Best–David Tuller

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley

**********

Subject: Awaiting response to letter of concern about CBT-music therapy paper

Dear Professor Choonara and Dr Godlee—

Two weeks ago, several colleagues and I wrote about serious methodological and ethical concerns involving a recent study in BMJ Paediatrics Open on cognitive behavioural therapy plus music therapy as a treatment for adolescents with chronic fatigue following acute EBV infection. 

Since we haven’t heard back, I am following up. As we noted in our May 31st letter, the paper should be withdrawn pending a re-review and then retracted if our concerns are confirmed. Children with serious health issues—an extremely vulnerable group—deserve at least that much consideration. 

(I have cc’d my colleagues: Vincent Racaniello and Mady Hornig from Columbia, John Swartzberg from Berkeley, and Jonathan Edwards from University College London.)

The science site Virology Blog, hosted by Professor Racaniello, has published three posts I have written that are critical of the paper–herehere and here. A cogent and meticulously argued comment that has been posted under the article itself raises many related points.

It is not acceptable to publish a fully powered trial that failed to meet expectations as if it were designed from the start to be a feasibility study seeking data to support a fully powered trial, as seems to have happened in this case. The other issues we raised include the following: 

*The addition of a key outcome measure, post-exertional malaise, not mentioned in the trial protocol and statistical analysis plan

*The construction of a definition of “recovery” that excluded the objectively measured primary outcome, average steps taken, and relied solely on a subjective secondary outcome 

*The omission in the conclusion of any mention of the poor results for the primary outcome, with both groups taking fewer steps after the trial and the intervention group performing even worse than the comparison group

*The acknowledgement by one reviewer that he had not read “beyond the abstract”

Again, I urge you to treat this matter with the seriousness it deserves. Decisions about children’s medical care must be based on sound and robust research–not studies marred by the kinds of flaws documented here. 

For full transparency, I will post this letter on Virology Blog.

Best–David

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley 

Comments on this entry are closed.

  • Alicia Butcher Ehrhardt, PhD 15 June 2020, 6:21 pm

    Good. The last thing they want is transparency. They want to not have their murky reasoning questioned, nor anyone looking into finances and appointments.

    Keep poking – you’re very good at it. And we desperately need it.

  • CT 16 June 2020, 2:37 am

    Isn’t Godlee supposed to be keen to promote Medical Investigative Journalism -https://www.bmj.com/content/medical-investigative-journalism-staying-cutting-edge and https://twitter.com/fgodlee/status/1196494014771580930? I can only conclude that Fiona continues not to give a flying flamingo when it comes to child health. These people, (including those who composed that KCL press release), are a disgrace to British science and medicine.

  • CT 16 June 2020, 3:47 am

    David – Alex Booth is a woman -https://uk.linkedin.com/in/alexbevisbooth

  • Martin 16 June 2020, 4:35 pm

    Surely they would be better off asking the question ” what causes fatigue after EBV infection has resolved?” I have been thinking about this. I think that when you get an infection, fatigue is the brain telling you to sleep. When you sleep, your body produces things to help you recover. A Post Fatigue state then is likely to be that your brain thinks you are still sleeping/need to sleep, and does not switch off the fatigue for a while until it gets used to you being up and about again. just my little idea.

  • lucie webb 17 June 2020, 7:50 am

    Martin, on this blog of David’s from 11 June you wisely pose the question “What does one do about a viral attack on the CNS once it has happened?” From your comments I infer that you want to treat the endocrine system to which my response is “enough already”. We have damaged our bowel flora (antibiotics) and cast our immune systems out of the Garden of Eden so that they will never regain equilibrium (vaccines). Amazonian indians in the remotest jungle have twice the variety of organisms in their guts than we do. Thank you David for providing this forum.

  • Martin 18 June 2020, 5:45 pm

    Lucie, good post. do Amazonian Indians get ME/CFS/MS do you know? 😉

  • Suzannah Williams 18 June 2020, 8:46 pm

    Want to get cured from herpes you can also contact him on his address: https://robinbuckler. com…………….

  • lucie webb 20 June 2020, 10:52 am

    Unlike our immune systems which are collapsing before our sell by date, surviving Amazonians’ biggest health risk comes from us. No point in asking them for a fecal transplant as developed by Professor Borody in Sydney at the Centre for Digestive Diseases. He was saving lives decades ago whilst English gastros were holding their noses. Pointless because our agricultural soils, tap water, food and drink are increasingly toxic. (Phosphate is added to our drinking water to counteract lead).
    To begin to answer Martin’s exasperated plea – what to do when the CNS/PNS is invaded. You have 2 years to try to get your guts in order before your MAST cells are damaged and you are permanently disabled. Don’t go to the NHS. They are at least 20 years out of date. I don’t think they have been getting funding for professional development. They will send you away with IBS which is a MUS= we can’t be asked to fund the necessary labs and technicians to discover what has overwhelmed your body’s normal function or test your microbes for what they are responsive/resistant to in terms of medication. THIS IS CRIMINAL because they are leaving you to deteriorate.
    I would like to complain about the po-faced, ignorant and domineering attitude of the mental health services in the U.K. Do they think that by-passing the oesophagus is the way of anoerexic women to gain weight? Don’t they realise that our Krebs Energy Cycle is NOT working? Don’t pour substances in to the stomach when they can’t be processed
    As for GET – they have instruments of torture which they strap our bodies into. Listen up, I am an old woman and what I want are regular sports massages from a nice, fit young man where an exchange of energy (Chinese=peng) passes from him to a very recumbent me. Strangely, this is not on offer from the NHS.
    The tow-rag (nautical term) Johnson is talking about building more hospitals. The modern hospital is designed to spread disease. It is a must-storey, air-conditioned – no windows open – no sunlight building. Our local geriatric ward smells of faeces. When I was 12 (1964) our local hospital was spread out all on the ground floor with huge windows which were often open. Each ward was run by a matron and the whole place smelt of disinfectant.
    If those “we know better than you” types at the Maudesley, S.London and Warneford, Oxford will step aside there will be room for small, local, cottage hospitals that could offer Fred Kahn’s cold laser and LED light treatments which use light to reawaken dormant neural circuits (Norman Doidge The Brains Way of Healing). These people must know that when the bones in the head are misaligned mental/physical health problems are created that are beyond their competence to deal with. Here is the man who has helped me:https://dramir.com/blog/archives/882-Explaining-the-concept-of-Cranio-Dental-Skeletal-Symmetry-Your-questions-answered.html