Trial By Error: The Lightning Process Is “Effective”? Really?

By David Tuller, DrPH

BMJ seems to be in a state of paralysis over what to do about the Lightning Process study. The fact that this study was cited positively last month in a major review of pediatric CFS/ME (as the review called the illness) raises the stakes all around. The other day I sent the following letter to Dr Terry Segal, a pediatrician at London’s University College Hospital and the senior author of the review, which appeared in the journal Current Opinion in Pediatrics.

**********

Dear Dr Segal—

You recently co-authored a major review in Current Opinion in Pediatrics called “Child and adolescent chronic fatigue syndrome/myalgic encephalomyelitis: where are we now?” This review cites the Lightning Process as having been shown to be “effective” in treating children with the illness. The basis of this claim is a 2017 paper published in Archives of Disease in Childhood, a BMJ journal, called “Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial.” Yet the Lightning Process trial and paper are rife with serious methodological and ethical lapses that cast doubt on the validity and reliability of its reported findings.

By way of introduction, I am a senior fellow in public health and journalism at UC Berkeley’s Center for Global Public Health, which is part of the School of Public Health. I have been a journalist for decades and more recently received a public health doctorate from Berkeley. For the last few years, I have examined–and critiqued harshly–some of the research in the ME/CFS field, including the PACE trial.

I have chronicled my findings in more than 150 posts and articles on the site Virology Blog, hosted by Professor Vincent Racaniello, a microbiologist at Columbia University. I have also written about the issue for The New York Times, the health policy journal Health Affairs, and elsewhere. My investigation of ME/CFS research has been mentioned in Science, Nature, The Wall Street Journal, The Guardian, and Slate, among other publications.

In December of 2017, I documented on Virology Blog that the investigators of the Lightning Process study recruited more than half the participants before trial registration, swapped primary and secondary outcomes based on the early results, and then failed to mention these details in the published paper. The following month, I alerted Archives of Disease in Childhood to these issues in an open letter signed by more than 20 experts from Columbia, Berkeley, Harvard, University College London, Queen Mary University of London, Stanford and elsewhere.

Last June, Archives posted an editor’s note about the study, which essentially confirmed the concerns I had raised. The note also informed readers that the investigators had provided the journal with “clarifications” and that the matter was now under “editorial consideration.” It is unclear why this period of “editorial consideration” has now lasted 11 months, since it takes no more than ten minutes to verify the details from the relevant trial documentation. You and your co-authors likely did not notice this editor’s note, since it is not linked directly to the paper itself and has been placed on a separate page in the editorial equivalent of Siberia. (The publisher has not explained why the editor’s note is so obscurely located that readers of the paper are unlikely to see it.)

I have been vigorously urging BMJ to resolve the matter, so far to no avail. It is important to note that all the major medical publishers agreed more than a decade ago, as part of efforts to ensure research integrity, not to consider any trials in which participants were recruited before registration. Fiona Godlee, BMJ’s editorial director, reaffirmed her commitment to this key principle in parliamentary testimony several years ago. BMJ’s apparent unwillingness to adequately deal with this present situation raises some questions about the depth of that commitment. It also suggests that the publisher is more concerned about avoiding reputational damage than protecting the health and well-being of children. It does not make me happy to draw that conclusion, but I see no other way of reading the available facts.

Under the circumstances, it goes without saying that the Lightning Process trial cannot be appropriately cited as an authoritative source for any claims that this pseudo-scientific intervention is “effective.” I think it is especially important to highlight this point now because the National Institute for Health and Care Excellence is currently developing new guidance for the illness, which it calls ME/CFS. Your review, or future publications unwittingly promoting the Lightning Process as a credible piece of scientific research, could easily influence the opinions of those involved with the NICE process. I am therefore urging you to immediately revise your review in a way that adequately addresses the problems with the Lightning Process paper, as documented at length on Virology Blog and acknowledged more briefly in the editor’s note posted by Archives of Disease in Childhood.

This is a matter of current public health and medical concern involving members of a highly vulnerable population–children with a debilitating and stigmatizing illness. For maximum transparency, I am therefore cc-ing this letter to several people: Professor Philip Pizzo, the editor of Current Opinion in Pediatrics; Dr Godlee, BMJ’s editorial director; Professor Alan Montgomery, the senior author of the Lightning Process paper; four physicians involved in the process of developing the NICE guidance for ME/CFS; and Sue Paterson, director of legal services at Bristol University. I also plan to post this letter on Facebook and on Virology Blog.

Thank you for your quick attention to this matter.

Best–David

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley

{ 11 comments… add one }
  • Margaret Laverick 20 May 2019, 6:25 am

    Once again, thank you for your perseverance.

  • Couch Turnip 20 May 2019, 6:58 am

    Will Dr Segal revise her review? I’m not holding my breath -https://twitter.com/terryheller1/status/1049051503137370113

  • Peter 20 May 2019, 7:09 am

    Thanks great letter, and it’s really really important to address this non-documented junk on every level. One thing is discussion of effectiveness, which data doesn’t support at all. Another important thing that the letter ought to address, (sure there be plenty of opportunities later) is the undisputed harm of LP for many ME-patients, here meaning patients diagnosed properly. When applying LP to these patients, that’s a fundamental failure of what these patients actually need, parents that already manage disease, so often, almost every day go beyond their limits. If LP on top of that, the result is a given. They get harmed bad, often with long-standing (irreversible) effect.

  • Wendy Boutilier 20 May 2019, 9:38 am

    It all boils down to “LDS” Lazy Doctor Syndrome. This is a contagious disease which sucks the integrity out of primary care Dr’s. Some symptoms include; doesn’t listen to Patient, doesn’t care about patients, verbally abuses the patient, doesn’t upgrade knowledge of new science, refers patient to pseudoscience just to rid of them and so much more.

  • Janet Maclennan 20 May 2019, 10:10 am

    You are a true champion, not just of people with ME but also scientific integrity. Thank you!

  • jimells 20 May 2019, 11:42 am

    Can these researchers, as well as physicians in general, really not see that any treatment with a name like “Lightning Process” should be viewed with very deep suspicion?

    I often wonder if we are dealing with corruption, incompetence, or corrupt incompetence. If I were a gambler I would put my money on the last.

  • Andrew Kewley 21 May 2019, 1:06 am

    If all it takes is a single unblinded study with subjective outcomes to be considered “evidence”, then there are many quack therapies that could be considered effective for many conditions. It is the same standard of evidence.

  • MK 21 May 2019, 2:14 pm

    Thank you again, for being a champion of integrity in medical research. Without integrity in medical research, there is “science”. There is just snake oil and con men, and very sick patients who are being actively harmed by pseudo-science in the guise of “medicine”.

  • Margaret 21 May 2019, 2:17 pm

    Beyond the harm done to patients is the concern that there are unethical and unscrupulous researchers who are not being policed by the scientific and medical community for unethical practices. Beyond what they’ve already done, we should be asking ourselves… What are they going to do NEXT, and who ELSE is going to get hurt by their actions?

  • Valerie 21 May 2019, 8:12 pm

    Thank you. It is terrible what is being done to patients, I equate it with back in dark days when electrotherapy was given willy nilly. I still can’t believe they are allowed to experiment on children.

Leave a Comment