By David Tuller, DrPH
On May 15th, I sent a letter to Dr Fiona Godlee, BMJ’s editorial director, alerting her that a new review in Current Opinion in Pediatrics had highlighted the Lightning Process as an “effective” treatment, based on a flawed study in one of her journals–Archives of Disease in Childhood. The subject line: “a plea about addressing the flawed Lightning Process trial”
I cc-d many people, including Professor Jonathan Edwards, who is currently serving as an advisor to the committee developing new guidelines for ME/CFS under the auspices of the National Institute for Health and Care Excellence. A couple of days later, Professor Edwards followed up with his own letter to Dr Godlee, which was also cc-d to the same group of people. I am posting Professor Edwards’ letter below, with his permission.
I thought I should comment on the email from David Tuller, copied below. I think David is right to raise this as a matter of serious concern and urgency and I think that those of us who may be in a position to influence things have a duty of care to do what we can. I see this as a safeguarding issue, not so different from that faced recently by senior clergy.
Up until now I have limited public statements about issues relating to ME/CFS almost entirely to responses to invitations to give my view on research quality, either from journals or administrative bodies. I have done that because people ask me as a disinterested observer and I want to make sure that is justified.
A number of colleagues and patients asked me to contribute to the NICE ME/CFS guideline committee and, my having indicated that I would be prepared to do so, the committee chair, Peter Barry, asked me to act as an expert witness on the difficulties associated with clinical trials. I have submitted a testimonial indicating my views and will present this to the committee later in the year.
The content of the testimonial is confidential but I discuss ethical issues relating to trials that involve cognitive manipulation, which include SMILE. I allude to SMILE briefly but did not seriously think the Lightning Process would be raised in the context of NICE guidelines. I submitted the testimony before being aware of the review of treatment in minors by Gregorowski, Simpson and Segal that David refers to, which makes an unqualified statement, in the specific context of the current NICE guidelines review, that the Lightning Process has been shown to be effective when added to medical care.
In many ways I would prefer to continue in a passive advisory position. However, it has struck me that having accepted a role that, like your own, has an implicit commitment to medical governance, and with it a responsibility to flag up potential ethical problems, I may need to be more proactive. I appreciate that in the present context you may not have a direct responsibility for editorial decisions at sister journals but I think we have a duty to do what we can to ensure problems are flagged up publicly if we are aware of them.
There are several problems with the SMILE trial and although I agree with David I tend to focus on some different ones. My understanding is that there was disquiet because the Lightning Process had not been shown to be effective in adults prior to a paediatric trial. This seems a valid and serious concern. Perhaps the most worrying aspect is that we are told that Lightning Process subjects are told to â€˜keep secretâ€™ information about treatment and progress. â€˜Keeping secretsâ€™ is a central red flag in safeguarding of children. There are, in my view, more general ethical issues about trials of unproven treatments that involve cognitive manipulation, particularly in minors, that apply to SMILE, but it is the two above features that worry me most specifically.
In terms of demonstration of efficacy I regard the SMILE trial, along with essentially all trials of therapist-delivered treatment in ME/CFS, as valueless because it was unblinded and used outcomes, which, in the context, were bound to be subject to major expectation bias via role-playing. Further problems with lack of prospective definition of outcome measures and prospective registration David has alluded to and would, I assume, make the trial fall short on journal policy requirements. We have no idea which part of the â€˜Processâ€™ has efficacy, if any. All this ought to be obvious to a trained physician, although I have been surprised how many (all with some form of professional interest) claim not to recognise it.
I think we are faced with a major issue of potential harm to minors if the problems with the SMILE trial are not publicly acknowledged. Adverse effects from cognitive and, in particular, exercise based, interventions are a serious concern in the patient community. The key reason why there is a review of NICE guidelines, as I understand it, is that there is a lack of confidence that previous advice has been well grounded in evidence on safety and efficacy. The variation in recommendations for ME/CFS by government health care systems in other countries does not inspire confidence and I suspect other European countries often look to the UK for a lead. I have also recently refereed a manuscript, which an editor has recommended for publication, that casts doubt on the adequacy of surveillance for harms in this area by clinical service units.
I have come to the view that the SMILE trial publication should either be retracted or the journal should ensure that readers are made immediately aware that serious ethical and methodological problems have been raised with the study. I think it is up to those of us directly or indirectly involved in governance to flag this sort of problem up and make sure it is dealt with. I hope you would agree.
Department of Medicine
University College London