Trial By Error: My Letter to Author of CODES Commentary

By David Tuller, DrPH

Earlier today, I sent the following letter to Dr David Perez, a neurologist and psychiatrist at Massachusetts General Hospital in Boston. Dr Perez, an expert on functional neurological disorders, wrote a commentary for Lancet Psychiatry that accompanied the publication of the results for CODES, a major study of cognitive behavior therapy as a treatment for dissociative seizures.

My critique of the commentary is here. My previous posts on CODES are here and here. 

**********

Subject: Invitation to respond to my critique of Lancet Psychiatry commentary

Dear Dr Perez–

I am a journalist and academic fellow at Berkeley’s School of Public Health. I frequently write about research on ME/CFS and other conditions that seem to fall under the category of “medically unexplained symptoms.” Much of my work appears on Virology Blog, a science site hosted by Professor Vincent Racaniello, a microbiologist at Columbia University. (I have cc’d Professor Racaniello on this e-mail.)

Yesterday, I posted a critique of the commentary you recently wrote for Lancet Psychiatry about the CODES study of cognitive behavior therapy as a treatment for dissociative seizures. You can read the critique here. My previous posts on CODES are here and here. 

By way of background, I initially became concerned about how the CODES results were being presented to the public when the King’s College London press release failed to distinguish between the primary and secondary outcomes. I found that to be a disturbing omission–a clumsy attempt to hide the bad news.

Furthermore, one of the lead investigators stated in the press release that the study provided “evidence for the effectiveness” of the intervention. From my perspective, such a statement is inconsistent with the results of a trial in which participants who did not receive the specialized seizure-reduction intervention showed a (not statistically significant) trend toward more improvement on the primary outcome than those who did receive the intervention. 

The statement also seemed to violate the spirit and letter of the investigators’ own admonition in the trial’s statistical analysis plan that “care should be taken when interpreting the numerous secondary outcomes.” I would suggest your commentary also overlooks that admonition, given that only five of 16 self-reported secondary outcomes survived the correction for multiple comparisons. None of them involved assessments of health-related quality of life.

CODES was an open-label trial relying on self-reported outcomes–a study design known to be vulnerable to bias. In my posts, I have included an assessment of CODES from my Berkeley colleague Philip Stark, a professor of statistics. That assessment bears repeating:

“The trial did not support the primary clinical outcome, only secondary outcomes that involve subjective ratings by the subjects and their physicians, who knew their treatment status. This is a situation in which the placebo effect is especially likely to be confounded with treatment efficacy. The design of the trial evidently made no attempt to reduce confounding from the placebo effect. As a result, it is not clear whether CBT per se is responsible for any of the observed improvements in secondary outcomes.”

Given the importance of open debate on these issues (and because Lancet Psychiatry gave an unconvincing reason for rejecting a cogent letter about the commentary), I want to offer you a chance to respond to the issues I have raised in the venue where I raised them. If you send me your remarks, at whatever length you choose, I will be happy to post them in full on Virology Blog, without interruption or editorial comment. (If I decide to respond, I will do so in a separate post.) 

I made the same offer to lead investigators of the CODES study recently, but I have not heard back from them. I also wrote to the King’s College London communications department about the problematic press release, but have not heard back from anyone there either.

I look forward to posting your remarks on Virology Blog, if you decide to respond. For full transparency, I will post this letter on Virology Blog.

Best–David

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley