Trial By Error: The School Absence Study, Revisited

By David Tuller, DrPH

This post is about a serious issue–ethical approval for research studies involving children. It is also about how powerful institutions, like leading medical journals, respond to concerns. But the story is really too long and complicated. I recommend it only for those following things pretty closely or who for whatever reason like this kind of granular, somewhat obsessive analysis.

Elsewhere, Tom Chivers at BuzzFeed has written a terrific piece on the cult-like Lightning Process, its founder Phil Parker, and Professor Esther Crawley’s SMILE trial—the story was posted over the weekend. Chivers captures the woo-woo-ness of the Lightning Process with lots of details about what it entails, from the perspective of patients who have undergone it.

The piece doesn’t delve into SMILE’s methodological flaws (outlined in this Virology Blog post). But Chivers includes key observations from Jonathan Edwards, Charles Shepherd, Peter Rowe and others. He also highlights Professor Crawley’s unsupported accusation that I have libeled her and Bristol University’s effort to pressure me by complaining to Berkeley. Definitely worth reading.

**********

In August, I posted a critical analysis of a 2011 study by Professor Esther Crawley and colleagues. The article, published in BMJ Open, was called “Unidentified Chronic Fatigue Syndrome/myalgic encephalomyelitis (CFS/ME) is a major cause of school absence: surveillance outcomes from school-based clinics.” In my post, I criticized BMJ Open for how it handled the paper and also for how its editors handled concerns raised about the paper.

Studies that involve human subjects and draw broad conclusions normally need approval from a research ethics review committee. Remarkably, Professor Crawley and her co-investigators decided that this school absence study was exempt from such a review. The paper included the creative claim that the study did not need one because it qualified as what was called “service evaluation.” In publishing the paper, BMJ Open implicitly endorsed that questionable argument.

Why am I reprising the issue now? After my post about the school absence paper, a journal presented a summary of the exact same situation to the Committee on Publication Ethics. The journal’s narrative was published as part of the agenda for the November meeting of what is called the COPE forum—a group of editors who debate publishing dilemmas.

These COPE submissions are anonymous—they do not include the names of journals, editors, or study authors. No one from BMJ Open or COPE has officially informed me this submission to the forum involves Professor Crawley’s school absence paper. But the specifics coincide. Moreover, the statement to COPE is initialed AA. BMJ Open’s editor is Adrian Aldcroft, who last year tried to deflect concerns raised about the paper by an observant reader.

Unfortunately, when read as an anonymized version of the events involving BMJ Open and Professor Crawley’s school absence study, the account presented to the COPE forum is misleading and inaccurate. Post-hoc rationalizations cannot undo what happened or obscure the journal’s lapses in editorial judgement.

In this post, I will recap the study itself. Then I will explain the official guidelines on the differences between “research” and “service evaluation,” and discuss how these guidelines relate to the study. Next I will look at BMJ Open’s earlier responses to the concerns. Finally, I will post the statement to COPE, and respond to it.

In a subsequent post, I will review the COPE forum’s answer to the questions posed by BMJ Open. The group’s response was reasonable, given the information or misinformation provided.

(For the sake of clarity and convenience, I will substitute the terms “BMJ Open” and “Professor Crawley” while discussing the “journal” and “author” mentioned in the submission to COPE. Given the parallels between the anonymous account and publicly documented events, the substitute terms would be accurate even if it were to turn out that a different journal was describing a different case altogether.)

**********

The Study:

Professor Crawley’s school absence study investigated what it called a “pilot clinical service”–a novel intervention to use school absence records as a way of identifying students suffering from undiagnosed and/or untreated CFS/ME. Working in coordination with the research team, three schools in southwest England reviewed their records and identified a total of 146 students who had experienced significant absences for unexplained reasons.

The schools sent letters to the families of these students, asking them to attend a private meeting with a school officer (in most cases this was the attendance officer) and a pediatrician—i.e. Professor Crawley. In these meetings, Professor Crawley personally evaluated the students, with those suspected of having CFS/ME referred to her Bath specialist CFS/ME service for further diagnosis.

Of the total who attended these meetings, 28 were ultimately found to have the illness. Let’s put that another way: For this pilot clinical service, the families of 118 students who were not ultimately diagnosed as having CFS/ME received potentially troubling letters inviting them to a meeting about a sensitive issue with a physician and a school officer.

The study included both a “hypothesis” (“Many children with CFS/ME remain undiagnosed and untreated, despite evidence that treatment is effective in children”) and a “research question” (“Are school-based clinics a feasible way to identify children with CFS/ME and offer treatment?”) The study compared students diagnosed with CFS/ME through the pilot intervention with those referred to the CFS/ME specialist clinic through health networks.

Among the study’s “conclusions”: “Chronic fatigue is an important cause of unexplained absence from school” and school-based settings “have the potential to identify children with CFS/ME.” BMJ Open published this study under the following slug: “Research.”

Under “ethics approval,” the paper included the following statement: “The clinical service in this study was provided as an outreach from the Bath specialist CFS/ME service. The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of data from children and young people seen by the CFS/ME specialist service were part of service evaluation and as such did not require ethical review by the NHS Research Ethics Committee or approval from the NHS R&D office (REC reference number 07/Q2006/48).”

In other words, the study exempted itself from ethical review because the data were being collected as part of what was being called service evaluation. From the phrasing of this ethics approval statement, it was not clear whether the research ethics committee specifically reviewed the school absence study.

**********

Health Research Authority Guidelines

According to guidelines from the U.K.’s Health Research Authority, “research” studies involve “the attempt to derive generalisable or transferable new knowledge to answer questions with scientifically sound methods including studies that aim to generate hypotheses as well as studies that aim to test them, in addition to simply descriptive studies.” In contrast, “service evaluation” studies are “designed and conducted solely to define or judge current care” and “involve minimal additional risk, burden or intrusion for participants.” Moreover, service evaluation “involves an intervention in use only.”

(These official definitions of what constitutes “research” and “service evaluation” for the purpose of obtaining ethical approval have remained essentially unchanged for years.)

With the school absence study, the facts are clear. The study was not about current care being provided to patients already attending clinics, so it could not reasonably be categorized as service evaluation. Moreover, the intervention—sending letters to families of children who were not patients but had high school absence rates—was presented as a “pilot clinical service,” so it could not reasonably be called “an intervention in use,” as required for service evaluation.

The paper also included a hypothesis and generalizable conclusions—hallmarks of what the Health Research Authority considers to be “research.” The study did include major caveats about the conclusions, declaring that “results may not be generalisable to regions without a CFS/ME service or to regions with different socioeconomic factors that impact on school attendance.” But many studies include limitations on the generalizability of their findings. The salient point is that the findings were not presented as restricted to students from these three specific schools.

The University of Bristol itself promoted the study as research that produced generalizable conclusions. Here’s the opening of the university’s press release: “New research into the cause of school absence finds that up to one per cent of secondary pupils could be suffering from chronic fatigue…”

The Bristol press release included quotations from Professor Crawley, in which she made similarly broad claims: “These findings reveal the scale of how many children are affected by disabling chronic fatigue that prevents them attending school, and how few are diagnosed and offered help… Our findings suggest that school-based surveillance for fatigue could be of potential benefit.”

Following the lead of Professor Crawley and Bristol University, media outlets touted these conclusions as newsworthy. Here’s how The Guardian characterized the study: “Far more children than previously thought miss a lot of schooling because of chronic fatigue syndrome which has not been diagnosed, according to research.”

By any reasonable measure, this school absence study cannot be considered a service evaluation. No journal should have accepted the argument that it was exempt from ethical review on those grounds.

In fact, such a claim should have raised questions that could have been clarified by examining the 2007 research ethics committee letter referenced in the paper’s ethics approval statement. Professor Crawley presumably could have made this documentation available to BMJ Open, had anyone requested it.

The 2007 research ethics committee letter did not support the investigators’ assertion that no ethical review was required. The letter involved a different set of circumstances and had nothing to do with the pilot intervention reported in the school absence study.

The 2007 letter was written in response to an application seeking to expand data collection among children attending the Bath CFS/ME specialist clinic, run by Professor Crawley. Patients at the clinic already filled out self-reported questionnaires for assessment at entry and at 12 months. The application sought approval to add additional assessments for children at six weeks and six months, which were expected to take no more than an additional 20 minutes to complete each time. The questionnaires included measures for school attendance, which was listed as the primary outcome.

The 2007 application included a question about how patients for this expanded questionnaire regimen would be “identified,” “approached,” and “recruited.” In response, the investigators stated that “there will be no change in the way potential participants are identified.” In other words, the application did not anticipate, mention or seek approval for the new actions involved in implementing and evaluating the pilot intervention.

After reviewing the application, the North Somerset & South Bristol Research Ethics Committee sent a letter dated May 1, 2007. Here’s the operative phrasing: “Members [of the REC] considered this project to be service evaluation. Therefore it does not require ethical review by a NHS Research Ethics Committee or approval from the NHS R&D office.”

The approval letter referred specifically to “this project” as “service evaluation”—that is, the approval was for the extra questionnaires for children who were current patients of the CFS/ME specialist service. Professor Crawley and her co-authors, therefore, appear to have over-interpreted the letter’s meaning when they wrote, in their paper, that the research ethics committee “decided that the collection and analysis of data from children and young people seen by the CFS/ME specialist service were part of service evaluation and as such did not require ethical review.”

The research ethics committee letter did not make such a blanket decision. The letter did not grant Professor Crawley and her co-investigators the right or permission to apply the term “service evaluation” to any other data-collection projects undertaken by the specialist service, including extensive outreach to students not currently in clinical care. In short, the research ethics committee did not review the school absence study conducted by Professor Crawley or determine that it was “service evaluation.” The claim that the 2007 letter applies to the data collection for the pilot intervention cannot withstand scrutiny.

I did not discover this discrepancy. As is often the case, a perceptive observer alerted me to the issue. She had accessed the key documents through Freedom of Information requests and had attempted to seek clarity from BMJ Open. Dissatisfied with the journal’s response, she contacted me and provided copies of the 2007 research ethics committee letter and related documentation.

**********

BMJ Open’s Initial Response

When this perceptive observer first wrote to BMJ Open, editor Adrian Aldcroft acknowledged in his response that “the article published in BMJ Open is not strictly a service evaluation.” But Aldcroft argued that the article was nonetheless exempt from ethical review, citing a May 2017 statement of support for Professor Crawley’s research from the University of Bristol. This statement, like the research ethics committee’s 2007 letter, had nothing to do with the data collection for the school absence study.

Bristol’s May 2017 statement involved a separate issue. It was headlined “University of Bristol statement about the CFS/ME National Outcome Database (NOD).” This National Outcome Database included information collected from regular clinic patients. The pilot intervention was an outreach to non-patients and was not part of the National Outcome Database. Yet in his response to the observant reader who had noted the issue, BMJ Open editor Aldcroft cited the following sentence from Bristol’s CFS/ME National Outcome Database statement:

“REC review is not required for the following types of research: Research limited to secondary use of information previously collected in the course of normal care (without an intention to use it for research at the time of collection), provided that the patients or service users are not identifiable to the research team in carrying out the research.”

Aldcroft then declared in his e-mail: “The data in the BMJ Open article meet these criteria.”

Aldcroft’s declaration was false—the data in the BMJ Open article did not meet the criteria mentioned in Bristol’s statement. The pilot intervention described in the BMJ Open paper provided new data for analysis, so the study was not based on “secondary use of information previously collected in the course of normal care.” And as I’ve noted, the students in the school absence study were clearly “identifiable,” not anonymous, since Professor Crawley assessed them personally and carried out the research.

Last summer, I wrote to BMJ Open editor Aldcroft while preparing my Virology Blog post. I heard back not from him but from BMJ Open’s editor-in-chief, Dr. Trish Groves. She argued—unconvincingly, from my perspective–that the authors of the school absence study had satisfactorily answered the concerns and had followed the proper procedures in making their decisions about ethical review.

Dr. Groves did not explain why Aldcroft had already acknowledged, in responding to the observant reader, that the study was “not strictly” a service evaluation. She did not explain why Aldcroft had cited in defense a Bristol University statement that was unrelated to the school absence study. Nor did she explain why the journal had chosen to publish the paper under the heading of “research” if it considered it to be service evaluation.

Here are some of the questions I posed for Dr. Groves in that Virology Blog post, plus some comments:

“Is she [Dr. Groves] really comfortable that–as part of a study defined as service evaluation–more than one hundred families whose children did not have CFS/ME were nonetheless sent school letters on a sensitive issue and invited to meet with Professor Crawley? Does Dr. Groves really believe that testing out a new strategy to identify patients unknown to the clinical service qualifies as service evaluation for routine care? I doubt she actually does believe that, but who knows? Smart people can convince themselves to believe a lot of stupid things. In any event, in dismissing these concerns, BMJ Open has demonstrated that something is seriously amiss with its ethical compass.”

**********

The Anonymous Journal’s Statement to COPE

Service evaluation as research in a controversial area of medicine

We received an email from a reader relating to the ethics statement in a research article published in 2011. The article presented data collected at a clinic relating to a controversial area in medicine. The ethics statement in the article indicates that, in accordance with regional guidelines, the research ethics committee deemed that the study was a service evaluation and formal ethical review was not required.

Using the reference number cited in the article, the reader obtained the relevant documents from the research ethics committee via a freedom of information request. The reader argued that the documents from the ethics committee related to data that predated what was presented in the article. A review of the documents indicated that this appeared to be the case. In addition, the reader argued that service evaluations should not be presented as research articles as these are two separate things.

The editor of the journal wrote to the author of the article and asked for comment on the issues raised. The author replied that there had been regular contact with the ethics committee as the service period of the clinic was extended, and the ethics committee continued to indicate that the data were being collected as part of a service evaluation and further ethical review was not required. In addition, the data were collected anonymously, which would further exempt the study from requiring formal ethical approval. The ethics committee also provided the authors with a letter indicating that “this letter …may be provided to a journal or other body as evidence that ethical approval is not required under [the regional] research governance arrangements.”

The author indicated that similar requests had been made in the past and that, due to the controversial area of the work, many attempts were being made to retract articles that used the data from the clinic. Attempting to prevent further queries, the author asked the institutional head of research to post a public statement indicating that the work was conducted appropriately and met the highest ethical standards. As requested, the head of research issued a statement on the institutional website in support of the work.

The editor then responded to the reader indicating that the journal was satisfied with the author’s response and the support of the head of research. The reader was not satisfied with the editor’s response and forwarded the details of the case to a high profile blogger who writes extensively on this controversial area of medicine. The blogger then posted a blog criticising both the article and the journal’s handling of the case. The blog was shared widely on social media. From the journal’s perspective, the blog was inaccurate, misrepresentative and damaging to the publisher’s reputation.

Question(s) for the COPE Forum

• Should we allow data collected in service evaluations to be published as research articles? In medical journals, this is often seen as an acceptable exception; however, if research ethics committees are declaring a study “not research”, should journals do the same?

• Should the journal have posted a correction on the article to provide a more detailed ethics statement, bearing in mind that anything labelled a “correction” in a controversial area would be misinterpreted as an error in the research by the critics?

• How should journals respond to blog posts that they feel portray them unfairly and are damaging to the publisher’s reputation?

**********

My Response to the Journal’s Statement to COPE

(Again, the statement to the COPE forum is anonymous. For the sake of argument, I am using “BMJ Open” in lieu of “journal” and Professor Crawley in lieu of “author.” The circumstances fit, even if it were to turn out that this account was written by a different journal about a different paper altogether.)

In constructing this account for the COPE forum, BMJ Open has chosen to ignore the testimony of the school absence paper itself, the claims of Bristol University’s press release, and the documentation from my August blog post. The journal has relied instead on statements from the regional ethics committee and Professor Crawley’s university that fail to support its argument.

Let’s look at this account in more detail.

BMJ Open: “Service evaluation as research in a controversial area of medicine…We received an email from a reader relating to the ethics statement in a research article published in 2011. The article presented data collected at a clinic relating to a controversial area in medicine. The ethics statement in the article indicates that, in accordance with regional guidelines, the research ethics committee deemed that the study was a service evaluation and formal ethical review was not required.

Using the reference number cited in the article, the reader obtained the relevant documents from the research ethics committee via a freedom of information request. The reader argued that the documents from the ethics committee related to data that predated what was presented in the article. A review of the documents indicated that this appeared to be the case. In addition, the reader argued that service evaluations should not be presented as research articles as these are two separate things.”

RESPONSE: The initial description of the material at hand—the headline itself and the second sentence’s statement that the article presented “data collected at a clinic”—bias the discussion. The headline suggests that the matter is settled—that this was a service evaluation. But that is the issue being contested. And the phrasing in the second sentence implies that the data were collected during normal functioning of a clinic, which would be consistent with service evaluation.

But the implication is incorrect. The people contacted were not identified through their attendance at clinical services, so they were not current patients receiving regular care. The study participants were identified through their school absence records. Their data were gathered while road-testing a novel method of engaging non-patients with clinical services. In this context, to refer to these records simply as “data collected at a clinic” is not only vague but misleading.

Moreover, most of those impacted by the study’s intervention and outreach were not ultimately diagnosed with the illness. Service evaluation is supposed to pose no more than a “minimal additional risk, burden or intrusion on participants.” The use of the word “additional” here is important. Professor Crawley’s school absence study was not posing an “additional” risk on current patients. In sending potentially troubling letters to the families of high-absentee students, it was creating an entirely new “risk, burden or intrusion” for an entirely new set of people.

It is unclear how or even whether these families provided informed consent to participate in Professor Crawley’s innovative pilot program. The paper itself does not mention any consent obtained before or after these school letters were sent.
The BMJ Open’s account mentions the questions about the paper’s ethical review status. But the account is silent on why Professor Crawley believed she could cite the 2007 letter, which involved unrelated circumstances, to exempt this pilot intervention from ethical review. The account is also silent on why experienced editors did not recognize that this paper was not a service evaluation, whatever the authors claimed, and that it therefore required an ethical review.

BMJ Open: “The editor of the journal wrote to the author of the article and asked for comment on the issues raised. The author replied that there had been regular contact with the ethics committee as the service period of the clinic was extended, and the ethics committee continued to indicate that the data were being collected as part of a service evaluation and further ethical review was not required. In addition, the data were collected anonymously, which would further exempt the study from requiring formal ethical approval. The ethics committee also provided the authors with a letter indicating that ‘this letter …may be provided to a journal or other body as evidence that ethical approval is not required under [the regional] research governance arrangements.’”

RESPONSE: Let’s dispense first with this statement: “the data were collected anonymously.” This is untrue, as any review of the school absence study would indicate. Professor Crawley gathered information in person–directly from the students and families who came to the school to meet with her. Is it possible that BMJ Open editors are defending this study without having re-read or even read what they published? Beyond that, are they really suggesting that anonymous collection of data alone qualifies a study for exemption from ethical review?

Next, “regular contact” between an author and an ethics committee cannot substitute for formal ethics approval. Absent any documentation, we have no idea what information was presented to the ethics committee during these informal communications, so BMJ Open’s third-hand accounting of the interactions between Professor Crawley and the committee is meaningless. The way that Professor Crawley and other investigators characterized what they were doing to ethics committee members would have impacted the responses.

For example, were members of the ethics committee told simply that the data were collected during routine school clinics? Or were they told that this was a pilot program to identify previously undiagnosed students using their school absence records? Were they told that most of the families identified and impacted by this recruitment process turned out not to have children with CFS/ME after all?

Besides citing unverifiable exchanges, BMJ Open’s account includes as back-up evidence a snippet of a letter from the research ethics committee itself. This snippet is presented as if it were possibly from a post-2007 or updated ethics committee letter based on new information arising from “regular contact” with Professor Crawley. Instead, it actually appears to have been taken from the 2007 letter already cited in the paper as the source of the ethical review exemption.

That 2007 letter ended with this paragraph, from which the snippet could have been extracted: “This letter should not be interpreted as giving a form of ethical approval to the project, but it may be provided to a journal or other body as evidence that ethical approval is not required under NHS research governance arrangements.” In the context of the 2007 letter, “the project” is solely referring to the collection from current patients of additional data at six weeks and six months. Since this does not describe the data collection involved in the pilot intervention, it is unclear why BMJ Open is citing the letter as evidence to support the case for service evaluation.

BMJ Open: “The author indicated that similar requests had been made in the past and that, due to the controversial area of the work, many attempts were being made to retract articles that used the data from the clinic. Attempting to prevent further queries, the author asked the institutional head of research to post a public statement indicating that the work was conducted appropriately and met the highest ethical standards. As requested, the head of research issued a statement on the institutional website in support of the work.”

RESPONSE: This line of argument also has nothing to do with the school absence study. To be used in service evaluation studies, “data from the clinic”–such as referenced in this passage–would have to be data gathered from current patients receiving routine care. But data for the school absence study were gathered differently–in specially arranged meetings between Professor Crawley, a school official, and families whose children were identified through school absence records.

Instead of recognizing this distinction, BMJ Open is again presenting Bristol’s irrelevant CFS/ME National Outcome Database statement as a defense for the school absence study. As I have explained, in my August blog and earlier in this one, the data generated by the pilot intervention were not from this national database, so Bristol’s statement has nothing to do with the matter at hand. BMJ Open’s decision to reference it once more in the account to the COPE forum does not enhance the credibility or integrity of the journal’s position.

BMJ Open: “The editor then responded to the reader indicating that the journal was satisfied with the author’s response and the support of the head of research. The reader was not satisfied with the editor’s response and forwarded the details of the case to a high profile blogger who writes extensively on this controversial area of medicine. The blogger then posted a blog criticising both the article and the journal’s handling of the case. The blog was shared widely on social media. From the journal’s perspective, the blog was inaccurate, misrepresentative and damaging to the publisher’s reputation.”

RESPONSE: BMJ Open apparently believes that medical journals should not be expected to make their own independent judgements—even when study authors, ethics panels or academic offices provide evidence or information that is irrelevant or untrue. Others would hold that a major journal like BMJ Open not only has a right but a responsibility to reject testimony that is clearly at odds with the facts, no matter the source.

Instead of taking that approach here, BMJ Open’s editors have expressed themselves satisfied with Professor Crawley’s deficient answers and with a Bristol statement about “secondary use” of data and “not identifiable” participants that is not germane to the data collection for the pilot intervention described in the school absence study. Given the journal’s apparent obtuseness in addressing this matter objectively, it is understandable that the observant reader was dissatisfied with the response and contacted the blogger—me.

I’m glad that BMJ Open considers me “high profile” and thinks my critical post was “shared widely.” Moreover, it is possible that what I wrote was “damaging to the publisher’s reputation.” When editors and journals twist themselves upside down to avoid admitting mistakes, they should not be surprised if they suffer reputational damage.

But I don’t agree that my post was inaccurate and misrepresentative. No one–including BMJ Open, its editor-in-chief (Dr. Groves), or its editor (Aldcroft)–has let me know of inaccuracies that need to be corrected. In this case, the misrepresentations have been made by Professor Crawley, with her claim that this research qualified as service evaluation, and by BMJ Open, with its clumsy effort to air-brush the editorial history.

BMJ Open:
Question(s) for the COPE Forum

• Should we allow data collected in service evaluations to be published as research articles? In medical journals, this is often seen as an acceptable exception; however, if research ethics committees are declaring a study “not research”, should journals do the same?

• Should the journal have posted a correction on the article to provide a more detailed ethics statement, bearing in mind that anything labelled a “correction” in a controversial area would be misinterpreted as an error in the research by the critics?

• How should journals respond to blog posts that they feel portray them unfairly and are damaging to the publisher’s reputation?

RESPONSE:
BMJ Open’s questions to the COPE forum rest on the fiction that Professor Crawley’s article described a service evaluation. From this perspective, the journal’s main goals would be to mitigate the embarrassment of having published the paper as “research” and to deal with pesky bloggers who have misinterpreted these actions and written bad things.

I don’t accept the premise of the BMJ Open’s questions. So I’ll ignore them and pose my own questions:

1) Why did the author exempt the school absence study from ethical review as a service evaluation for current care, even though the study tested a new intervention to engage students unknown to the clinical service?

2) Why did the journal accept the claim that investigating a pilot intervention to identify new patients could be exempt from ethical review as a service evaluation?

3) Are journals supposed to accept at face value the claims of authors, ethics review boards and university committees, even if such claims are irrelevant, unconvincing or based on a tenuous relationship to the facts? Or are journals supposed to exercise independent judgement in making their decisions?

Comments on this entry are closed.

  • Wendy Boutilier

    As far as I’m concerned any member of the Psychiatric Collaborative should be criminally charged when manipulating families of Myalgic Encephalomyelitis stricken children including young adults. I feel my heart drop every time they start pushing their pseudoscience theories. I don’t understand how any Association of Medical Professionals can turn a blind eye to what is happening. I’m not buying any excuse they have to offer. They are just as aware of what is going on as we are. At least we are speaking out.

  • Barry

    Excellent as always David. Many thanks. The ethical approval issue is as fundamental as it gets, given the pilot study newly researched 146 children who were not currently attending specialist ME clinics. For a researcher and professional journal to somehow think the original ethical approval could legitimately be “extrapolated” to the new pilot research study … itself feels very unethical. There used to be a BBC medical documentary series “Your Life in Their Hands” … the phrase feels relevant here somehow.

  • Pingback: Trial By Error: The School Absence Study, Revisited - Virology()

  • mesupport

    Granular detail the journal and author are strangely unfamiliar with having questioned. Moving a simple question into an area that has suddenly become contentious with a smattering of reputational damage without evidence of either also requires justification as fantastic claims require fantastic evidence.

  • Pingback: Trial By Error: The School Absence Study, Revisited - VETMEDICS()

  • jimells

    I’m very familiar with the technique of obtaining written third party permission for one event and then using it for another. I did it in high school — more than 40 years ago! But it was only so I could leave the lunch room before the end of the period, not for avoiding ethical review while conducting research on children.

    I wonder if Professor Crawley understands that then push comes to shove, she will be the patsy for the whole BPS scandal. Regius Professor Sir Simon Wessely, Professor Peter White, and the rest have quietly faded into the background while shoving Professor Crawley into the spotlight to take the heat for the lot of them.

    Dr Tuller, don’t let them slip away into comfortable retirement. They have been, and continue to be, responsible for immense suffering while protecting the profits of disability insurers. Justice demands that they be held to account.

  • pinklil

    Another excellent blog..thank you David.

    I’m very pleased to see that the S4ME forum have uploaded a link to the 2007 letter/documents referred to in the blog above here: https://www.s4me.info/index.php?threads/trial-by-error-the-school-absence-study-revisited.1758/ (comment 3). I’m sure others will find them fascinating reading.

  • pinklil

    I see a member of S4ME forum has also crated a Janet and John version of the salient points here: https://www.s4me.info/index.php?threads/trial-by-error-the-school-absence-study-revisited.1758/#post-30350

  • Paul_Watton

    Thanks @pinklil Those documents are very helpful in understanding what happened, particularly with regards to the significance of the changes which Prof Crawley made after initially applying for Research Ethics Committee approval.

    The answer given to question A10-1 of the NHS REC Application form for the study (see page 14 of the Scribd documents set) is clearly not what happened at all.
    (If they had read it, that misleading answer alone ought to have caused BMJ Open to realise that they had made an error of judgement.)

    Furthermore, in answer to Q1 of the REC application form (P9 of the Scribd documents) “Is your project an audit or service evaluation?” the response in clearly “No”.

    On the basis of what’s stated in the REC application form, it seems that it wasn’t Prof Crawley’s original intention to go out into schools in order to “trawl” for more potential subjects for her CFS/ME clinic to gather data upon. However, if numbers through the doors of the clinic were below expectation, and with a self-imposed target of 150 to meet (see answer to Question A9), she probably felt the need to do something to boost those numbers. The moment she did that, the ethics situation changed.

    I wonder how Prof. Crawley subsequently explained away to BMJ Open, the obvious conflict in the answers given on the application form, with what actually happened and her subsequent statements about the project only being a service evaluation, and therefore exempt from ethical approval?

    Maybe she wasn’t asked to do so.

    Maybe Trish Groves and the Editor of BMJ Open, didn’t bother reading the documents properly.

  • pinklil

    We can only conjecture whether or not Trish Groves bothered to read the documents properly but what we do know is that they were most DEFINITELY sent to her by the reader who made the complaint in the FIRST INSTANCE.

  • pinklil

    As you’ve noticed, Paul, the devil is most definitely in the detail and is rather damning imo….

  • Paul_Watton

    Trish Groves attended the 6th Invest in ME Conference in May 2011 as a guest and took part in the plenary session at the end – which I recall became quite a heated exchange between herself and patients / advocates. http://www.investinme.eu/IIMEC6.shtml
    Having been made well aware at that event, of the suspected chicanery that the Psyche brigade engage in – particularly re PACE – she ought to have been much more suspicious about (and hence more willing to delve deeper into) what went on in this case.

  • Brenda Vreeswijk

    I’m wondering what the ethics committee has to say about this debate. It’s unbelievable how this group still tries to shove it all under the rug and looks away like nothing happened. Defending the work of a person who has absolutely no scruples regarding doing harm to children. Exposing them to pseudoscience and manipulate parents…schools and the whole system.
    In my opinion, and I keep saying this…Dr Crawely should be held accountable for her actions, she should be ordered to resign and legal actions should take place for experimenting on children and young adults with pseudoscience and denying the ethical committee.
    Just like a lot of others that caused immense harm to the ME community or protecting the ones that caused this harm…willingly and knowingly …

    Thanks again David, unfortunately I couldn’t read all of it…to much text.

  • pinklil

    But instead, just after that Invest in ME conference in May 2011 that you mention, in June 2011 she contributed to this -http://www.bmj.com/bmj/section-pdf/187262?path=/bmj/342/7812/Feature.full.pdf- “The Dangers of Research into CFS/ME”

  • pinklil

    The link doesn’t seem to want to work here. Instead google “The dangers of research into CFS/ME Nigel Hawkes bmj.com”

  • Paul_Watton

    Here’s the link: http://www.bmj.com/content/342/bmj.d3780
    It’s interesting that Crawley features quite prominently in the article.
    In what way did Dr Groves contribute towards it?

  • Seven

    Excellent work, as always. This is crazy, I am glad you started to ask the questions like “what does it take to…” This whole situation is beyond ridiculous at this point:
    So you see how crazy this club is: EC is invited to talk here:
    http://espidmeeting.org/2018/scientific-information/confirmed-invited-speakers#.WkvCJd9l-yK
    ESPID’s Board President is Adam Finn, Professor of Paediatrics at University of Bristol. Quote from thread here: http://forums.phoenixrising.me/index.php?threads/crawley-to-spread-misinformation-about-me-to-international-group-of-paeds-in-sweden.56926/
    Are you starting to see the relationship of universities / establishments and the top club spreading this ME misinformation???

  • pinklil

    Sorry I should have said google “The dangers of research into CFS/ME Nigel Hawkes bmj.com pdf” and the scroll down (not far) to the pdf labelled ‘Download PDF – the BMJ’. This shows the original BMJ article, I believe, complete with photo of Simon Wessely. Trish Groves provided a substantial Commentary at the end entitled ‘Heading for a Therapeutic Stalemate’. It’s a matter of personal opinion how biased that account was, and how it may have influenced those reading it.

  • Peter Trewhitt

    There can be little doubt that setting up a new screening process to identify what proportion of children with low attendance records may display the symptom of chronic fatigue was a new venture for Prof Crawley’s CFS/ME service. There also is no doubt that the consultations about the need for ethical approval for service evaluation undertaken by Prof Crawley did not relate to this research.

    It is possible that Prof Crawley’s failure to follow due procedures resulted from woolly thinking and was a genuine mistake. However it is concerning that she did not consider several ethical issues involved in inviting children with low school attendance records to what amounts to a medical assessment: firstly there is the matter of undertaking a preliminary medical assessment of children for a condition you expect a significant proportion not to have, secondly it is not clear what constitutes free and informed consent in a situation where parents/guardians may fear legal repercussions and thirdly all the paper work refers only to the symptom of chronic fatigue and provides no information on how and when any subsequent differential diagnosis of CFS/ME might take place.

    However there can be no doubt that Prof Crawley, the University of Bristol and the BMJ failed to respond appropriately when the lack of any real evidence that Prof Crawley followed due process in considering ethical issues.

  • Peter Trewhitt

    The research then involved looking at how the children identified as having chronic fatigue responded to the service provided by Prof Crawley’s clinic in comparison to children referred by the existing processes. Again this would seem to be research rather than evaluation of current provision as it is comparing the existing service to a new source of referrals.

    Again there does not seem to be any mention of differential diagnosis as to what was the cause of the symptom of chronic fatigue identified in the previous screening. Again there is the issue of free and informed consent to any intervention in a situation where low school attendance places parents/guardians at risk of legal action. This is particularly relevant as included in the interventions offered by the Clinic are GET and CBT which are particularly controversial for people who might actually have CFS/ME. If no differential diagnosis was undertaken can anyone be considered as giving informed consent.

    Although many people might consider this as nitpicking it does raise very important issues of genuine ethical aproval and informed consent, but also the apparent lack of openness and honesty in the British research establishment. So thank you to the person that raised these issues and to David, both of whom have put a lot of work into clarifying the obfuscation of Prof Crawley, Bristol University and the BMJ.

  • Peter Trewhitt

    DISQUS seems to be eating up my comments again, though I divided them into smaller chunks and avoided any web links, so I have copied them to the Science4ME thread referred to in the other comments.

  • Peter Trewhitt

    Here is the link to the S4ME thread

    https://www.s4me.info/index.php?threads/trial-by-error-the-school-absence-study-revisited.1758/

    Though DISCUS usually deletes any thing I add with a link in it.

  • jimells

    Yes, Disqus really bites the big one. I still have four comments marked as spam. Just today I figured out how to see a list of all my comments, including the alleged spam. None of them have links.

    Perhaps Disqus sells a blacklisting service, just as Google sells search result rankings. I don’t trust any of these companies any further than I can throw them. I wish Dr Racaniello used a different comment system.

    I resent that I have to give data to Disqus for them to sell to the highest bidder just so I can comment on and support Dr Tuller’s work. But I do appreciate that Dr Racaniello allows Dr Tuller (and us) to post on his website. Some days I swear I will not log into Disqus ever again. And now here I am posting another message.

    As soon as I’m done I’ll delete the cookies and remove Disqus from the cookie exception list. I have cookies disabled by default, which is a hassle, but it makes me feel like I am fighting back, at least a little.

  • If anyone is having trouble with their comments not showing up here, please send me an email and I’ll add your email to the whitelist. vincent@virology.ws

  • nancyblake

    From the perspective of a long-time psychotherapist in the community therapy/groupwork tradition…

    • How should journals respond to blog posts that they feel portray them unfairly and are damaging to the publisher’s reputation?

    ‘How do you feel’ is the Rogerian therapist’s way of not paying attention to ‘what has actually happened’. Whether they ‘feel’ they have been portrayed unfairly is entirely irrelevant. What are the facts? Tuller brings them up, again and again, the factual information. If a blog post has portrayed them unfairly, the relevant question is, was the blog post accurate or not. They may very well ‘feel’ unfairly portrayed because some accurate but unpleasant truths have been expressed. If so, they need to clean up their act, not cry ‘it’s not fair’.

    From the group-dynamics, systems approach, if discussing a situation in exhaustive, minute detail doesn’t solve the problem, it is because there is an agenda which is operating more powerfully, and not based on facts and rational argument, which is not being discussed.

    In this case, the over-riding system that is calling the shots is the way that the Wessely agenda is embedded in the system of status, privilege and power, which one could loosely call the ‘British Establishment’, and means that the Wessely agenda dominates the politics, of both parties, the media, the medical establishment, including the press, research funding, the giving of awards to books promoting this agenda (‘It’s All In Your Head’, recently, and ‘Exhaustion’…and in the 80’s, Showalter’s ‘Hystories’, blatantly influenced by Wessely, and a US best-seller)…at all levels of society, they seem to have the power to determine what goes on. This influence extends internationally….Fukuda went on to become head of WHO, in Denmark, the diagnosis of WHO 93.3 myalgic encephalomyelitis has been taken away from patients, leaving them victims of Fink and BDD (Bodily Distress Disorder)…I think with the advice of Peter White. Who remains medical advisor to Swiss Re.

    If someone had the research skills of a historian/social analyst, it is this web of power relations which needs investigation and challenge. The very first place to look would be to find out who, and how, managed to get the ‘mass hysteria’ theory proposed by McEvedy and Beard in 1970 set out in a contemptuous TIME article.(Jan 1971)..about 500 words, which influenced their 4,000,000 readers. This article was under a heading ‘Behavior’, so not in any section connected to health, or medicine. It did not refer to the article, so readers had no basis for researching the background. The illness was also not named. It simply compared the illness to medieval dances, demeaned the doctors, and concluded that the final proof that it was just hysteria was that most of the patients were women TIME is recorded in the Encyclopedia Britannica as the most influential publication of that time. Ramsay records that from that time all research funding dried up.

    The subsequent history of psychiatric dominance, reiterated in the face of medical research developments…and emphasising the ‘dangers of rest’ and the importance of ‘increasing activity’ (still the stated goal of the current NICE Guidelines) would surely be worth some history and analysis of the power relations behind the scenes.

    Part of the strategy is to turn valid criticism into ‘harassment’, while trying, behind the scenes, to attack the critic via their employing agency, of which the Bristol effort is only the latest example. It is certainly career-threatening to study ME/CFS medically, or challenge their treatment practices ..the attack will come, not from patients or advocates, but from these individuals, for example by complaints to the GMC.

    We need a scholar of these behind-the-scenes power dynamics. George Monbiot is ‘too busy’, maybe an investigative historian could take this as a thesis topic?

  • nancyblake

    Other bits and pieces…Wessely is recorded as having advised the German Government after reunification about privatising the German medical system. Going back to 2010, there is a report giving to the US IOM, or some bit of the CDC asking about research into ME/CFS, written by White and Komaroff (why were they working together) in which White points out that there should be attention paid to ‘directionality’…as in, the patient must have had a psychiatric problem first which led to the ‘CFS’. Problem being that the narrative of ME people is one of being committed to high-exertion activities (professional athletes, or housewives with small children/disabled family members, or nurses…marathonners, hikers, backpackers, children happy, popular and very good at sports) and not having any history of mental health problems.

    If someone wants to do the research, I can provide links to quite a lot of this!

  • Pingback: Trial By Error: COPE To BMJ Open: More Details, Please ! – The York ME Community()

  • Klaas van Dijk

    David wrote: “No one from BMJ Open or COPE has officially informed me this submission to the forum involves Professor Crawley’s school absence paper.” It seems to me that David will have contacted COPE and/or BMJ and/or others with a request to confirm this assumption. Does this quote implies that COPE and/or BMJ Open and/or others have not responded on such a request?

  • pinklil

    I doubt that if David contacted (or had contacted) COPE that they would confirm or deny to him whether this case does relate to Crawley et al’s School Absence Study – the cases brought before the COPE Forum are supposed to be anonymous. However, Tuller informed us of the following in his original blog on this study:
    “Dr. Groves did leave the door open a tiny crack. She wrote: “BMJ Open is a member of the Committee on Publication Ethics (COPE) and follows its best practice guidance and policies. In light of the matters that have been raised, we will submit this case (anonymised, as always) to the COPE Forum.”
    The submission to the COPE Forum meeting in November was presumably BMJ fulfilling that intention/ honouring that promise.

  • Klaas van Dijk

    Thanks for the response. I have extensive experiences with communicating with COPE about the herculean efforts to retract a fraudulent study in Iraq on the breeding biology of the Basra Reed Warbler in a Taylor & Francis journal, a member of COPE. Communicating about this issue with COPE is tough. Communicating bout this issue with TF is impossible.

    There is towards the best of my knowledge no indication that Trish Groves is holding a PhD (and/or a MD). Copy/pasted from http://www.bmj.com/about-bmj/editorial-staff/trish-groves “Trish Groves (MBBS, MRCPsych)”. I am on the other hand in the possession of an e-mail from Trish Groves in which Trish Groves admits that she had at that time a side-job at COPE which was at that time not indicated at her personal profile, and which is right now stlll not indicated at her personal profile. So it is not excluded that Trish Groves is allowed to use the title Dr. (but that the details are not listed at her personal profile).
    Details about this hidden side-job of Trish Groves at COPE and about my efforts to get a copy of the ICMJE form of a guarantor of another paper in the journal BMJ Open are published in an article (“Is partial behaviour a plausible explanation for the unavailability of the ICMJE disclosure form of an author in a BMJ journal?) at https://riviste.unimi.it/index.php/roars/article/view/9073

  • pinklil
  • pinklil

    The COPE site https://publicationethics.org/about/cope-alumni gives Trish Groves’ involvement with COPE as being a Council member from 2007 up to 2010, but no later than this. From the COPE site she does not appear to have been working for COPE after 2010. She would however have been working for BMJ/ BMJ Open, a COPE member journal, but that is not the same thing as being a COPE Council member and COPE representative.

  • 1SarahL1

    Professor Racaniello, this came up in ‘my memories’ section on fb from 2010 and I was wondering if you would consider having a TWIV with Dr Judy Mikovits now? It would seem to be appropriate considering all the developments and further retroviral findings eg Montoya’s results from the Lipkin Multicentre Study. It could be another way of exposing Crawley and others of her ilk from wasting time and research funds whilst the retroviral elephant in the room continues to stampede. Lives are being ruined and lost, not least those of children, whilst some folk on antiretroviral drugs and supplements are faring well – maybe you could interview some of those folk as well? In response to ‘Kati’ 7 years ago, you wrote: “I will try to have a TWiV next year with Dr. Mikovits – it’s up to her to agree. I do agree that the speed with which the press has dismissed XMRV based on yesterday’s reports is disheartening. I think most of the stories were simply repeating press releases which were negative.”

  • Klaas van Dijk

    See https://web.archive.org/web/20170202173037/https://publicationethics.org/about/cope-alumni for an archived copy of https://publicationethics.org/about/cope-alumni in which it is clearly stated that Trish Groves has “agreed on occasion to continue to represent and speak on behalf of COPE.”

    This archived copy is dated 2 February 2017. The contents of this url had been changed some time after I had contacted Trish Groves about this issue (dates are in my paper at https://riviste.unimi.it/index.php/roars/article/view/9073 , “Is partial behaviour a plausible explanation for the unavailability of the ICMJE disclosure form of an author in a BMJ journal?”).

    Trish Groves, and/or anyone else from BMJ, and in the widest possible sense, have until now not informed me that there are errors and/or mistakes in the text of my paper. This paper was published on 2 November 2017. Trish Groves and others at BMJ were informed about this paper a few days after it was published. A short e-letter to BMJ Open in which I refer to this publication was accepted and can be accessed through
    http://bmjopen.bmj.com/content/6/11/e012047.responses

  • Klaas van Dijk

    A GP in the UK has a medical degree and the same is the case for a ‘huisarts’ in The Netherlands. You might call these medical practionizers doctors, but that does not imply that they are allowed to use the academic title Dr. Within The Netherlands, using the title Dr. is only allowed when you have a PhD (or a MD). People in The Netherlands can get a fine when they are using the title Dr. when they don’t have a PhD (or a MD). MBBS is a Bachelor of Medicine / Bachelor of Surgery and thus not equal to a PhD and/or a MD.

  • pinklil

    That’s not the case in the UK.

  • pinklil

    Thank you very much for the web archive link, that IS revealing. It clearly shows that Trish Groves still potentially has her fingers in the COPE pie. It seems that you have had significant issues with COPE too, I can sense your utter frustration that these people can get away with such underhand and cliquey (bordering on incestuous even) ways of conducting their business. Science is not in safe hands.

  • Seven

    Can somebody please start a “family tree” or diagram for me of these people / institutions? My brain fog is getting crazy and the interrelationships are soooo crazy at this point, I don’t even understand how they can get away with this incest (love your description of their intertwined relationship).

  • Pingback: Trial By Error: COPE to BMJ Open: More Details, Please!()

  • pinklil

    Not a diagram or ‘family tree’ but I hope this helps you to put names to faces and to see what’s going on:

    COPE = Committee on Publication Ethics. Most reputable journals will be members of COPE and so demonstrate their intention that they will abide by COPE’s ethical guidelines. More information about COPE here – https://www.s4me.info/index.php?threads/is-cope-committee-on-publication-ethics-fit-for-purpose.1775/

    BMJ Open published Crawley et al’s School Absence Study.
    BMJ Open – Editor Adrian Aldcroft , see Aldcroft at Crawley’s presentation https://www.youtube.com/watch?v=jeujHnwsYPs – 29 mins to 31.30 mins
    BMJ Open – Editor-in-chief Trish Groves, also Deputy Editor BMJ (see her here – https://www.youtube.com/watch?v=GLG0T9qnzCs – where she apparently just loves everything to be open, and also here demonstrating her appreciation of social media – https://www.youtube.com/watch?v=5jXPOgK0guM ) . She was a COPE Council member from 2007 to 2010 but has also worked for them ‘on occasion’ since – https://web.archive.org/web/20170202173037/https://publicationethics.org/about/cope-alumni

    Trish Groves said that she would take Tuller’s concerns about the School Absence Study to the COPE Forum meeting where COPE members (journal editors etc) discuss anonymized cases between themselves. Tuller argues here that the case taken to the COPE FORUM meeting, if it was the School Absence Study case, was not portrayed accurately by whoever raised it. It seems likely that Adrian Aldcroft presented the case to the COPE Forum, because the case was initialled ‘AA’.