Trial By Error: Our Latest Tango with BMJ Open…

By David Tuller, DrPH

On March 27th, Professor Racaniello and I sent an e-mail to Dr. Trish Groves, editor-in-chief at BMJ Open, declining her request that we send in an e-letter for publication to ensure “maximum transparency.” Two days letter, we received yet another appeal from Dr. Groves. In this e-mail, she promised for the first time that BMJ Open would seek a “point by point” response from the authors as well as a comment from the University of Bristol.

Here is Dr. Groves’ e-mail:

Dear Professor Racaniello and colleagues
Thank you for your email of 26 March.

I am inviting you again to submit an eletter to the paper in BMJ Open.

While we appreciate that Virology Blog is widely read, at BMJ we try hard to ensure that post publication debate takes place in the journal where the research was published, not least because the scholarly systems of indexing and archiving mean that this is where the millions of users of PubMed worldwide will always most easily find the debate.
Asking readers and authors to respond in the journal to issues that arise after publication is also an essential part of ongoing peer review, and is a key pillar of good editorial practice.

We do hope, therefore, that you will be willing to send an eletter for publication at BMJ Open. We will then invite the authors to respond, point by point. This will also enable us to write to the authors’ institution and ask them to review your concerns and the authors’ reply.

With best wishes
Trish Groves

**********

And here is the response Professor Racaniello and I sent today:

Dear Dr. Groves:

Thank you for your most recent invitation to submit an e-letter for publication. This time, you have invoked “good editorial practice.” Our view is that “good editorial practice” involves acknowledging facts rather than ignoring them. There can be no legitimate academic “debate” about whether BMJ Open’s 2011 school absence study was service evaluation or research. Since it included a formal hypothesis and generalizable conclusions, it was indisputably research; presumably that is why BMJ Open published it under the heading of “research” in the first place.

The investigators justified their decision to bypass ethical review on false grounds. When a peer reviewer raised serious questions about the claim that the study was service evaluation and therefore exempt from ethical review, the authors provided non-responsive answers to justify their position, as we documented in our initial post about the paper last August. For reasons that remain unclear, BMJ Open published the paper anyway.

We fail to see why a published e-letter from us is necessary in order for you to fulfill your editorial responsibilities and seek a point-by-point response from the investigators and the University of Bristol about this troubling abuse of the ethical review process. Instead of pursuing that strategy when Virology Blog expressed concerns, you chose to submit an opaque and disingenuous defense of your actions to the COPE Forum. In doing so, BMJ Open cited statements from Bristol and the local research ethics committee that Virology Blog had already proven were irrelevant to the dispute.

In the description of the situation provided to the COPE Forum, BMJ Open further claimed that the data for the school absence study were collected “anonymously.” Had you and your editorial team familiarized yourselves with the paper your journal published, you would have known that this claim was untrue. BMJ Open also accused Virology Blog of publishing “inaccurate” and “misrepresentative” information without documenting a single inaccuracy or misrepresentation. It goes without saying that if anything we have published was factually inaccurate, we would be happy to correct the record.

The COPE Forum suggested that BMJ Open needed to base its determination about whether ethical review was required on the actual methodology of the study. That response was apparently not to your liking, since you decided instead to declare the COPE Forum case resolved. As we noted in our last letter, you could have sought guidance from the Health Research Authority, the arbiter in the U.K. of whether studies qualify as service evaluation. But you have not taken that step.

The authors have already demonstrated that they are either unable or unwilling to recognize the difference between service evaluation of routine care using anonymous databases and a “pilot clinical service” that involved the primary collection of data by the lead investigator. The University of Bristol, for its part, does not appear to have made any significant effort to investigate the problem. We therefore have little faith in the integrity of any possible responses from either the authors or Bristol.

Our concerns are compounded by BMJ’s failure to address another case of unethical research—the Lightning Process study published in September in Archives of Disease in Childhood. As we noted in our letter to Archives editor Nick Brown on January 30th, this trial was registered in July 2012, yet more than half the participating children were recruited before the trial registration date, starting in September 2010.

This methodological anomaly occurred because the investigators sought ethical approval to extend their feasibility trial into the full trial while simultaneously swapping primary and secondary outcome measures based on those feasibility trial results. This strategy was an obvious recipe for bias, so we remain perplexed that it received approval from the local research ethics committee. The paper itself failed to disclose this maneuver. Yet the outcome swap allowed the researchers to highlight positive findings for the revised primary outcome, self-reported physical function, rather than null results from the original primary outcome of school attendance.

BMJ has touted its longstanding policy of rejecting all trials in which participants were recruited before the registration date. This policy does not appear to include any exemptions. BMJ Open itself published a study earlier this year that once again reaffirmed the importance of this approach. At that time, you personally tweeted that “prospective registration of #clinicaltrials isn’t red tape: it’s done for sound ethical and scientific reasons.”

We wholeheartedly agree. Given that statement, we fail to understand why Archives of Disease in Childhood has taken more than two months to assess the matter—and why you or someone else at BMJ has not stepped in to enforce the company’s unequivocal policy not to publish such studies. This trial is already impacting treatment of children with ME/CFS and has provided bragging rights to practitioners of the Lightning Process, a commercial program unavailable through the National Health Service. Yet Dr. Brown has declined to offer a deadline for completing the journal’s review. We see no reason why it should not be retracted immediately.

The publication of these studies and BMJ’s subsequent refusal to take responsibility for such errors in judgement suggest that something is seriously amiss with the editorial processes and ethical compass of your organization. Nonetheless, we are happy to allow BMJ Open to publish an e-letter as long as you agree to seek a determination about the study from the national HRA leadership. The submission to HRA should include the letter sent by schools sent to parents of children with significant absences, the application that resulted in the 2007 REC opinion cited to claim exemption from ethical review, the peer reviewer’s comments and the investigators’ response, BMJ Open’s exchange with the COPE Forum, and Virology Blog’s many posts on the issue.

In the U.K., the HRA—not a local research ethics committee–is the ultimate authority on these matters. Once we have received your assurance that you will offer HRA officials a chance to analyze the situation and offer their assessment, we will submit an e-letter for publication to BMJ Open. We thank you for the opportunity to do so.

Vincent R. Racaniello, PhD
Professor of Microbiology and Immunology
Columbia University
New York, NY, USA

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
University of California, Berkeley
Berkeley, CA, USA

Comments on this entry are closed.