By David Tuller, DrPH
Update: I have sent the following letter to the responsible person listed on the clinical trial registration for the Finnish study of “amygdala and insula retraining”:
Dear Dr Lira–
I am a public health researcher and journalist with the Center for Global Public Health, part of UC Berkeley’s School of Public Health. I frequently comment on research in the field of ME/CFS and related diseases, now including long Covid. Much of this commentary appears on Virology Blog, a popular science site hosted by Professor Vincent Racaniello, Higgins Professor of Microbiology at Columbia University. (I have cc’d Professor Racaniello here.)
I have recently seen the clinical trial registration for “Amygdala and Insula Retraining (AIR) Program and HUS Internet Therapy Compared to Treatment as Usual in Bodily Stress Syndrome, Fibromyalgia, Long Covid, and Chronic Fatigue Syndrome (ME/CFS).” Yesterday, I posted a Virology Blog commentary in which I criticized the research on a couple of fronts.
If you would like to send a response to the concerns I have raised, I would be happy to post it on Virology Blog, alongside my critique. If you do so, perhaps you could address these questions:
1) Given the interventions, the study is of course unblinded. However, all the outcomes are subjective. This combination of elements—being unblinded and relying solely on subjective outcomes—is a recipe for generating bias. Participants who know they have received interventions that promise significant benefits are understandably more likely for any number of reasons to report benefits than participants who do not receive the interventions.
Are you concerned about the bias inherent in the study design, and that this issue could create doubt about the accuracy of the results? If not, why not? Did you consider including any biomarkers or objective outcomes of physical function or movement? If not, why not?
2) The study is presented as investigating “amygdala and insula retraining” even though it is not assessing participants’ amygdalae and insulae in any way, much less whether they have been “retrained.” As I assume you understand, the use of this sort of language by so-called “brain rewiring” programs is a marketing strategy designed to enhance their appeal and imply they are grounded in sound science.
Beyond hypothetical musings, can you provide any serious evidence that the intervention in fact involves “amygdala and insula retraining”? (Modest and hard-to-interpret changes in BDNF levels in a small pilot study do not justify use of the phrase; moreover, this trial is not measuring BDNF levels.)
Are you concerned that giving an intervention an unwarranted scientific-sounding name could lead to bias and unrealistic expectations among participants? Did anyone involved in the design or approval of the study raise concerns that adoption of the phrase “amygdala and brain retraining” was deceptive and perhaps unethical?
As mentioned, I would be happy to post any response you would like to send. Thank you!
Best–David
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley
**********
I have written a couple of times about a Dutch trial of internet-delivered cognitive behavior therapy to prevent severe fatigue in long Covid patients. The goal is to address factors like “dysfunctional beliefs about fatigue,” “perceived low social support, “problems with processing the acute phase of COVID-19, and “fears and worries regarding COVID-19″ that are purportedly driving the prolonged symptoms. My posts—here and here–were not favorably disposed toward the research and suggested the design was biased toward producing positive results.
Last month, advocates from three Dutch patient organizations published their own report about the study. The researchers, from the Dutch Knowledge Center for Chronic Fatigue (NKCV), were awarded €308,000 by ZonMw, a major health care research agency. The lead investigator, psychologist Hans Knoop, is a well-known proponent of the CBT/GET treatment paradigm for ME/CFS. In 2011, he and a Dutch colleague wrote a commentary for The Lancet that accompanied the PACE trial and declared that participants had met “a strict criterion of recovery”—a demonstrably untrue statement.
The three patient organizations–Group ME The Hague, ME and Disability Support Group, and the ME/CFS Association—explained in their report why they decided to conduct their review:
“Our goal was to prevent that in the future grants will be awarded to a treatment that is not based on scientific knowledge about the nature and cause of the disease, but on the assumption that the patient maintains his own fatigue through unhelpful thoughts and wrong behaviour. The treatment in the ReCover study has the same principles as ‘CBT for CFS’. The British NICE [National Institute for Health and Care Excellence, which published new ME/CFS guidelines last October] has determined that the quality of the scientific evidence for this treatment is low to very low. It is also clear from Dutch and foreign patient surveys that this form of CBT does not help the majority of patients and leads to deterioration in a considerable number of patients. We aim to prevent this for longCOVID patients too.”
The report also included the transcript of a somewhat testy exchange between representatives of the three organizations and Professor Knoop. Here’s a revealing slice:
“[Organizations] It is known that measuring only subjective outcome parameters in such an open trial has a high risk for bias. Therefore, objective outcome parameters must be used also. How do you handle that?
(K) Why are patient organizations so obsessed with objective measures? Objective measures are subjective too. They depend on how you interpret them.
[Organizations] It is well known that the outcomes on subjective measures such as fatigue are directly influenced by the treatment (we have used the term ‘manipulated’ in the conversation).
(K) So you’re basically saying you don’t trust the outcomes reported by the patients themselves? How does that relate to the fact that you as a patient organization represent these patients while you do not take their answers seriously?
[Organizations] We take patients very seriously. But with regard to your research, we indicate that the treatment is aimed directly at influencing the patient’s thoughts about fatigue, the main outcome measure. That leads to bias.
(K) We have only subjective primary and secondary outcome parameters. In medical research, more and more researchers are moving away from objective outcomes and they are looking at what is relevant for the patient and how he feels…In previous research, we have shown that there is no correlation between the objective and subjective outcome parameters.“
Professor Knoop’s argument that medical researchers are “moving away from objective outcomes” is a new one to me. Apart from that, his apparent rejection of objective outcomes in his studies because they have “no correlation” to subjective outcomes is hard to justify. What’s the rationale for dumping the objective findings instead of the subjective ones?
I don’t get it.
********
Finnish Study of “Amygdala and Insula Retraining”
And now here’s a clinical trial of a self-management program grandiosely called Amygdala and Insula Retraining (AIR). The trial is officially titled “Amygdala and Insula Retraining (AIR) Program and HUS Internet Therapy Compared to Treatment as Usual in Bodily Stress Syndrome, Fibromyalgia, Long Covid, and Chronic Fatigue Syndrome (ME/CFS).”
This Finnish trial, which is currently recruiting and seeks a total of 360 participants for three arms, is being conducted by the HUS Clinic for Functional Disorders at the Helsinki University Central Hospital. Per the clinical trial registration information, BSS includes fibromyalgia, irritable bowel syndrome, multiple chemical sensitivities, and other functional disorders.
Besides the AIR arm, a second arm is receiving another self-management program. Both groups will also receive treatment as usual. The third group will only get treatment as usual. The primary outcome of “functional ability” and seven secondary outcomes are all subjective. No objective measures of function or physical ability appear to be included. To be clear, no one is measuring the impact of the interventions on anyone’s amygdala and insula—the nominal targets of the interventions.
Given that the treatments are necessarily unblinded, the study presents enormous opportunity for bias. After all, people enrolled in a program claiming to retrain their amygdala and insula will undoubtedly be motivated to provide answers that support the notion that such retraining has occurred. Unblinded studies relying on subjective outcomes simply cannot be considered reliable evidence of anything other than that such trial provide unreliable results.
Like the Lightning Process, the Gupta Program, and the Dynamic Neural Retraining System, AIR falls into the category of so-called “braining rewiring” programs. Of course, the program creators have no evidence that anyone’s amygdala and insula are being retrained. This is a hypothesis—nothing more—and the scientific-sounding words are marketing ploys based on the buzzy concept of neuroplasticity.
It is disturbing that a clinical trial would highlight such an unevidenced claim for an intervention and even include such hype in its formal title. It is also disturbing that any ethics committee would approve a trial with such an obviously bogus and unevidenced promise.
Here’s information about AIR from the trial registration:
“The Amygdala and insula retraining program consists of novel brain retraining approaches focused on hypothetically interrupting and retraining the conditioned defensive hyper-stimulation of the sympathetic nervous system and aspects of the immune system by the amygdala and insula, to bring the brain and body back to homeostasis. It includes supportive techniques such as breathing, meditation, and neurolinguistic programming.”
And here’s information about the other self-management program:
“The HUS internet therapy for bodily stress syndromes (iHUSbss) includes psychoeducation about autonomic nervous system and of the effects of patient’s own thinking and action on the nervous system. The exercises aim at relaxing the body, at novel ways to observe the symptoms, and at developing acceptance and self-compassion. The program includes exercises that are done regularly in everyday life.”
To repeat what I have written before: Some of the strategies used in these programs are likely helpful for many people. Meditation, relaxation exercises, breathing strategies, positive affirmations—these modalities all have their place and can help prompt positive changes in thought and behavior. Unfortunately, beyond personal anecdotes from devoted fans, there is no legitimate scientific evidence that they cure chronic medical diseases. To create a study that is bound to generate attractive results for so-called “brain rewiring” because of its poor design is a farce.
Brain retraining by altering beliefs! Really? More fakers & sham ‘science’. FakeNews is fully entrenched in the BPS brigade
“Objective measures are subjective too. They depend on how you interpret them.”
A measurement is both objective and subjective? That sounds like an interesting topic for philosophers to debate. For medical research it is nonsensical. The phrase “That does not compute” comes to mind. Why does Knoop remind me of Dr Zachary Smith on “Lost In Space”?
I made banana bread yesterday and objectively measured all the ingredients and those measurements didn’t change no matter how I felt about them or how much I wanted to finish so I could lay down and rest. (The results were subjectively tasty.)
Knoop really should get a refund on his college tuition. Even without any degrees I am able to understand that the *conclusion* depends on interpreting the data. The data depends on careful methodology. The data does not depend on interpreting the data.
“Professor Knoop’s argument that medical researchers are ‘moving away from objective outcomes’ is a new one to me. … What’s the rationale for dumping the objective findings instead of the subjective ones? … I don’t get it.”
Knoop’s view has been growing in many areas of medical research for 20+ years. Although it is not seen in ME/CFS research, it is seen in other areas of medical research. You can research terms like “patient oriented outcomes”, “patient centered outcomes”, “quality of life” and so on. I think this idea comes from Cancer research in the 1980s and 1990s, where much of the research was focused on keeping people alive (often with bad side effects) for extra months or years. The patient-advocates lobbied that quality of life measures were more important and the research community grew to the same position. These quality of life measures were often subjective and measured by questionnaires.
I’ve been following ME/CFS for over 10 years, and it has always been weird to me that the ME/CFS advocates took the opposite approach to the rest of medical research, which was moving in the direction of “whole body health” and “quality of life” and similar “subjective from the point of view of the patient” results.
Years ago, I was at a meeting for another disease (not ME/CFS or Cancer), and a rep from the FDA gave a talk focused on how they wanted more “patient oriented outcomes” to be measured and reported in clinical trials.
The situation is even worse for ME/CFS. In cancer, the disease is identified objectively. Size of tumor, that sort of thing, so researchers started out measuring treatments objectively. Later, patient advocates pushed them to add subjective measures. But ME/CFS is diagnosed using subjective measures, so it makes total sense to measure treatments subjectively also.
For ME/CFS patient advocates accept subjective measurements for diagnosis, but then object loudly to subjective measurements of treatment. It is a pretty obvious double standard when you think about it. Once you have a diagnosis, and you want to measure if a treatment makes you better or worse, suddenly subjective measures are banned, and only objective measurements are OK. If subjective measures are the tool used for measuring diagnosis, they must also be the tool used for measuring treatment. There is no other way forward, until you can do objective diagnosis.
Joshua Levy, I don’t think ME patients want a subjective diagnosis. Most want labs and scans in the diagnostic tool kit to aid diagnosis, and providers do too.
And, while everyone wants quality of life it sounds like you’re conflating treatment side effects from cancer treatment with… CBT. Ironically, chemotherapy induces a lot of ME like symptoms, which most people would wish to avoid. Would you advocate brain retraining for people recovering from cancer?
If you’ve been following for 10 years you know about PACE and understand somr ME researchers can’t be trusted with subjective outcomes.
Brain retraining sounds like the schools they’d send people to in the USSR for objecting to state propaganda.
Good job, Tuller. Keep at it.
Indeed, I agree with Robin. Subjective and wishy-washy diagnostic criteria and subjective and wishy-washy treatment objectives/outcomes don’t cut it. It’s reasonable for cancer patients to ask to be listened to, ME patients having been asking for that for decades to no avail, but this is akin to asking cancer patients to guess whether their disease has gone – no tests, no scans, just lick a finger, stick it in the air and decide by that.
I was given what I think were called feedback forms after my CBT therapy ended. The sweet, smiling therapist asked me to fill in the “anonymous” forms! How could they be anonymous when the therapist personally handed mine to me and stayed, still smiling sweetly, while I completed the forms? How many people would write a more positive, favourable review so as not to hurt the feelings of such a sweet therapist? And how many more, realising the staff could identify who said what, would write a favourable review so that their medical records did not include anything unfavourable written by that no longer sweet, smiling therapist?
Joshua Levy asserts that advocates want to ban subjective measures. This is a mischaracterization. Advocates want to ban poor research. When Fluge and Mella designed their *blinded* retuximab trial they specified subjective primary endpoints and no one complained.
What we want to know is, does the treatment work? Does it cause harm? For *unblinded* trials with subjective *primary* endpoints, the answer is always: we don’t know, because bias. That is well understood by honest competent investigators.
Endless repetition of the phrase, “The PACE trial is a Thing of Beauty” does not make it so. And endless repetition of trials of failed and harmful treatments will never improve the lives of patients.
Well done. Keep being attentive to these Presstitutes of Science sponsored by P€R$ONAL agenda.
The Wikipedia entry calls Neuro-Linguistic Programming a pseudoscience:
https://en.wikipedia.org/wiki/Neuro-linguistic_programming
Hello Joshua!
I remember you! You were the guy who signed Joshua (not Jay) Levy!
Has it really been ten years? How time flies. I will not get into a debate with you as obviously you haven’t listened as you are still spouting the same type of nonsense and misrepresentation of patients concerns you did all those years ago. Shame you couldn’t or refused to learn, but hey we have all become accustomed to people like you. How about leaving patients to represent their own concerns until you actually understand them properly? How about advocating for decent research for M.E patients instead of being an apologist for quacks? No? Well no surprise there! I think we need more research to figure out what makes people like you tick. It’s certainly not for the benefit of seriously Ill people’s health, for an increase in their Quality of Life, nor academic improvement or scientific understanding, so what is it? Time for some CBT for you methinks.
jmelli says:
“Joshua Levy asserts that advocates want to ban subjective measures. This is a mischaracterization. Advocates want to ban poor research.”
If you read David’s original posting he talks about subject and objective and never uses words like poor, good, or bad. Here is his last paragraph in its entirety. All “subjective” and “objective” nothing about overall quality: “Professor Knoop’s argument that medical researchers are “moving away from objective outcomes” is a new one to me. Apart from that, his apparent rejection of objective outcomes in his studies because they have “no correlation” to subjective outcomes is hard to justify. What’s the rationale for dumping the objective findings instead of the subjective ones?”
Obviously, other advocates might say different things, but David is clearly talking about objective and subjective in this posting.
Joshua–My point has nothing to do with “banning” subjective outcomes. As far as I know, no one is advocating “banning” subjective outcomes. That’s a red herring. Patient-reported outcomes are fine when you have a blinded study. The problem isn’t with these outcomes, it is with relying exclusively on them when you have an unblinded study. Even with an unblinded study, subjective outcomes add value IF there are also objective measures. But it is not okay to toss out your objective measures just because you like the subjective results better.
Pingback: Trial By Error: My Exchange of Letters on Amygdala Retraining; That Undead Lightning Process Study
So this is your rule: “The problem isn’t with these outcomes, it is with relying exclusively on them when you have an unblinded study.”
If so, the you would think that PACE did not have a problem, because it used both subjective and objective results, so it did not “rely exclusively on [subjective measures]”. It would be totally cool under this rule. I’m sure you did not mean that. Oh, wait a second, you did: “Even with an unblinded study, subjective outcomes add value IF there are also objective measures.” That is exactly the situation with PACE, if you assume it is unblinded: subjective outcomes to add value together with the objective measures (walking and stair climbing). Do you want to rephrase that in some way?
Joshua Levy, you cannot be that stupid…
Pingback: Trial By Error: The Lancet Publishes Whine de Coeur from Impassioned GET/CBT Defenders
In reply to Joshua Levy and the PACE trial objective outcomes, I’m sure you know that there was no evidence from any of the objective outcomes of any benefit from the GET and CBT treatments. The patients were not fitter, had no more stamina and were no more able to return to work. The problem was not the lack of objective outcomes in the trial, it was the dishonest way the researchers reported the trial results. The main papers didn’t mention these objective outcomes, they focused entirely on subjective fatigue and physical function, and claimed on that basis that large numbers of patients health had improved or recovered. That was clearly, from the objective evidence, false.
I don’t know about you, but if I did a treatment that persuaded me to fill in questionnaires a bit differently about how fatigued I felt, but had no impact on my ability to function, I would still regard myself as being just as sick as when I started. If I still can’t walk to the local shop, sit or stand for more than a few minutes without fainting, and have all over body pain and if I still find that every time I push myself to do a few more things like having a shower and cooking a meal, I become so sick I can’t get out of bed for a few days, then objectively I have not recovered, however a nice kind therapist may have persuaded me to interpret my fatigue differently.
ME has two main aspects – unpleasant symptoms which by definition are subjective, and therefore only able to be recorded subjectively, and ability to function physically and cognitively/be upright/do cognitive challenges all of which can be measured objectively. We need both types to be measured and included in outcome measures of clinical trials.
As for the cancer argument, I don’t know about you, but if I were offered a choice of two treatments, one of which (homeopathy and mindfulness meditation) made me feel subjectively nice for a while, and had no nasty side effects but had no effect on my cancer; and another (chemotherapy, radiotherapy or surgery) which subjectively makes me feel dreadful for a while but gives me a good chance of recovery and longer life, I’d want to know about objective cancer recovery data for each when deciding which path to take, not just patients subjective experience of how they felt during the treatment.
Are you seriously suggesting that cancer treatment should only be assessed subjectively? That seems to be what you are suggesting for ME/CFS and for cancer. Really? Sure, it’s good to find out the patients’ subjective responses to cancer treatments, so doctors can help patients make informed decisions about whether they want to keep doing a therapy that has a low chance of success and is reducing quality of remaining life unacceptably. But they need the objective information about the efficacy of the treatment too.