Trial By Error: My Letter to Mahana’s CEO and Co-Founder

By David Tuller, DrPH

Venture capitalist Robert Paull is listed on the Mahana Therapeutics website as the company’s co-founder and CEO. I have no idea how he got involved with this group of UK researchers and King’s College London. Given my own experiences, I would certainly have advised him against getting in bed with them, had he asked me–which he didn’t.

As it is, Mr Paull’s company has now licensed a product that generates minimal and transient reported benefits in reducing symptom severity in irritable bowel syndrome–yet is promoting the product based on its purported success in long-term reduction of symptom severity. I pointed out this dilemma in the letter I sent Mr Paull earlier today.

**********

Dear Mr Paull–

I am a senior fellow in public health and journalism at UC Berkeley’s Center for Global Public Health, which is part of the School of Public Health. I write frequently about research on illnesses in the category of so-called “medically unexplained symptoms.” Much of my work appears on Virology Blog, a science site hosted by Vincent Racaniello, a professor of microbiology at Columbia. (I have cc’d Dr Racaniello on this e-mail.)

I have recently written about Mahana Therapeutics’ licensing deal with King’s College London, which was announced last month. The deal is for a web-based cognitive behavior therapy program for treating irritable bowel syndrome, which is apparently slated to be Mahana’s first product. I initially critiqued the ACTIB study that road-tested the web-based program in this post, following up with two posts–here and here–about the licensing deal itself.

It is true, as you are quoted as saying in the press release about the deal, that there are “extensive clinical data” about the web-based program. Unfortunately, both the press release and the Mahana website convey seriously misleading information about what these clinical data from the ACTIB study mean.

Mahana’s press department has not responded to three e-mail messages I have sent in my efforts to seek responses to my concerns. I have also reached out to Professor Rona Moss-Morris, ACTIB’s co-lead investigator, and some of the gastroenterology advisors listed on the Mahana site.

According to the study, the web-based program demonstrated very modest reported benefits in reducing symptom severity on an outcome measure called the IBS-SSS. At 12 months, the mean score of patients in the web-based arm on this measure was 35.2 points lower than the mean score for those who received treatment-as-usual. (A reduction of 50 points or more on the IBS-SSS is considered clinically significant; a reduction of less than 50 points is not considered clinically significant.) At 24 months, the 12.9-point difference in mean scores between the two groups was not statistically significant.

In its press release about the licensing deal, Mahana described these improvements in symptom severity as “substantial” and “durable.” On its website, Mahana is promoting them as “dramatic and potentially game-changing for patients.” I do not think these statements accurately reflect the fact that the difference between the mean scores for the web-based and treatment-as-usual groups was less than the designated threshold for reaching clinical significance, and in fact was non-existent at long-term follow-up.

Moreover, the website notes that, at 12 months, 66% of those in the web-based arm had “clinically meaningful reduction” in IBS severity–in this case, that means their IBS-SSS scores were reduced by 50 or more points. Yet fhe website fails to mention that 44% of participants who received treatment-as-usual and provided data at 12 months also achieved that threshold.

Do you agree that the omission of such an important detail is likely to create the impression among readers that all the reported improvements should be attributed to the web-based program? To be fully transparent, do you believe the website should also mention that 30% of the trial participants did not provide these 12-month responses, so their IBS-SSS outcomes are unknown?

The IBS-SSS was the only instrument in the study specifically designed to assess IBS symptoms rather than more generic domains also assessed by self-reported questionnaire, such as depression and social adjustment. Modestly positive reports on these additional measures might be expected from a round of CBT and should not be taken as signs of successful treatment of IBS or remission of IBS symptoms. In any event, Mahana is touting the IBS-SSS results themselves as evidence of robust impacts–even though these results clearly do not provide such evidence.

Since I have criticized Mahana’s promotional approach, I would be happy to post any response you want to make on Virology Blog–at full length and without editorial interruption from me. As well, here are some other specific questions:

1) Do you really believe the ACTIB findings represent “substantial,” “durable,” “dramatic,” and “potentially game-changing” reductions in symptom severity? If so, what data have you seen that can support these claims?

2) Do you think that this sort of rhetoric, which would seem likely to induce hopes of improvement in symptom severity that go beyond the study findings, is acceptable and appropriate when marketing medical products?

3) Can you explain why Mahana decided to license a program that produces such minimal and transient benefits in reducing symptom severity, only to promote the program based on its purported success in long-term reduction of symptom severity?

Thanks much! I look forward to hearing from you. To keep Virology Blog readers informed of my reporting efforts, I plan to post this letter.

Best–David
 
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley
 
{ 5 comments… add one }
  • Alicia Butcher Ehrhardt 12 February 2020, 12:53 am

    I hope he has the guts to answer you – and change his ways. Someone sold him a fake, if he is a good guy. IF.

  • CT 12 February 2020, 6:08 am

    I had to laugh at your opening paragraph, David, but this whole thing is one big joke. Have you considered writing to NICE about it? After all, they are supposed to be an independent body which is honestly and independently evaluating the app, and they should therefore be interested to learn that the people involved with the development of the app appear to date to have avoided answering legitimate questions in the public interest that have been put to them about it. Of course, it would be nice if you got some replies to your letters…..I hope you do. Perhaps they’re just a bit slow in answering.

  • Lady Shambles 12 February 2020, 6:26 am

    I’d be more inclined to think that someone who has money at stake as well as monetary promise might be willing to respond to you. I’m guessing he hasn’t invested in this jiggery-pokery for the promised health gains to people with IBS because it wouldn’t take much digging to find that the average IBS sufferer will simply laugh at the notion that such a product could possibly help them in their most painful or explosive moments 😉
    I also imagine Robert Paull isn’t au fait with the machinations of those who have created this wonderful piece of software (heavy irony wrt to ‘wonderful’) because if he was he’d already know that their track record isn’t the best. I wonder how big his equity stake happens to be and whether it’s worth the gamble given the projected high growth of this application is based on pretty dodgy data..the sort of data that the BPS regularly churn out…. we know all about this but I guess a venture capitalist hasn’t necessarily got his ear to the ground when it comes to junk science?
    Money talks (according to Neil Diamond at least) which is why i) the BPS crew have created this artifice, ii) a venture capitalist would think about investing, and iii) a venture capitalist might decide it’s better not to invest.. although it sounds as if that ship has already sailed.

  • jimells 12 February 2020, 1:49 pm

    The goal is to reduce the number of clinic visits, not the severity of symptoms. One look at this idiotic program will convince most patients to not bother to return for more abuse.

    Therefore, success.

Leave a Comment