Trial By Error: Bruce Levin on “How Not to Conduct a Randomized Clinical Trial”

By David Tuller, DrPH

When I first began examining the PACE trial in detail, I turned to clinical trial experts to vet my concerns. One of them was biostatistician Bruce Levin, a professor at Columbia University’s Mailman School of Public Health, to whom I was referred by a mutual colleague. After he reviewed the trial, he pronounced it to be a mess—a conclusion that fueled my determination to investigate the matter.

Professor Levin identified many faults in the PACE trial’s design and conduct. In particular, he found the overlaps between the entry and outcome thresholds for the key outcomes of physical function and fatigue to be unacceptable peculiarities of the study. “I have never seen a trial design where eligibility requirements for a disease alone would qualify some patients for having had a successful treatment,” said Levin told me for my initial series of articles. “It calls into question the diagnosis of an illness whose patients already rate as ‘recovered’ or ‘within normal range.’ I find it nearly inconceivable that a trial’s data monitoring committee would have approved such a protocol problem if they were aware of it.”

Levin also said the mid-trial publication of a newsletter featuring glowing testimonials from earlier participants and positive news about interventions under investigation created legitimate concerns that subsequent responses might have been biased, especially in an unblinded study with subjective outcomes like PACE. “It is highly inappropriate to publish anything during an ongoing clinical trial,” he told me. “To let participants know that interventions have been selected by a government committee ‘based on the best available evidence’ strikes me as the height of clinical trial amateurism.” (In fact, “the height of clinical trial amateurism” is probably one of my favorite quotes of all time.)

Anyway, last week Professor Levin took his criticism a step further with a public talk about PACE at Columbia. The talk was called, appropriately enough, “How Not to Conduct a Randomized Clinical Trial.” For the last couple of years, I have been pointing out that PACE has been used at Berkeley as a case study of bad science in epidemiology courses. Now it is legitimately possible to say that it has been presented at multiple major American universities as a case study of bad science. Thanks, Bruce!!!

Here is a link to his slide presentation:

http://irvinginstitute.columbia.edu/pdf/How_Not_to_Conduct_in_Randomized_Clinical_Trial_Levin.pdf