• Skip to main content
  • Skip to primary sidebar
virology blog

virology blog

About viruses and viral disease

Replicability of scientific results

3 February 2011 by Vincent Racaniello

Excellent description by Jonah Lehrer in “The Truth Wears Off” (The New Yorker) on the importance of confirming experimental findings in different laboratories:

Before the effectiveness of a drug can be confirmed, it must be tested and tested again. Different scientists in different labs need to repeat the protocols and publish their results. The test of replicability, as it’s known, is the foundation of modern research. Replicability is how the community enforces itself. It’s a safeguard for the creep of subjectivity. Most of the time, scientists know what results they want, and that can influence the results they get. The premise of replicability is that the scientific community can correct for these flaws.

Mr. Lehrer’s article is about drug effectiveness, but his words apply equally to other types of science, including virology.

Filed Under: Information Tagged With: repetition, replicate, science, scientific method, subjectivity, viral, virology, virus

Reader Interactions

Comments

  1. Anonymous says

    5 February 2011 at 6:02 pm

    Utterly fascinating article, but where exactly does it leave us? It actually poses more questions than it answers. The last graph captures the essence of the problem because these arguments can be used by various points of view to discredit other points of view. But how do you know for sure?

    Most interesting was the case cited where the scientist controlled for every possible variable (something that many scientists do not do either because of conformation bias orbecause theyare unaware of variables for which they are not controlling). But in the end it was random – or was it?

    The deceptive assumption is that at any given point in time we “know” all the basics there are to know and have reliable methods that “prove” what we know. Right up until we are proven wrong. Or are we?

    Excellent post. What are your thoughts?

  2. profvrr says

    10 February 2011 at 12:18 pm

    I find it hard t accept that all scientists are sloppy, that they bias results to obtain the results they want, that ‘a lot of extraordinary data are nothing but noise’. Is it really true that we have to choose what to believe? The article would lead us to believe that nothing that science produces is incorrect. But that’s clearly not the case. I believe that in the majority of cases, over time, science finds what is correct. Perhaps in the case of therapeutics for which the end point is hard to define (e.g. schizophrenia as in the article) and subjective, drugs that aren’t effective are used. But there are many other examples where therapeutics have been tested, shown to be effective, and then used for years in people with excellent benefits. The poliovaccine comes to mind first – subjected to large clinical trials, shown to be effective, then used in populations to the point where we have nearly eliminated the disease. In basic science, discoveries build upon one another; if one is wrong, the field does not progress until it is corrected. For example: for years it was thought that protein was the genetic material of cells, but subsequent research could not prove it. Then it was shown that DNA is the genetic material; and study after study validated this, and the conclusion is clearly correct. There are so many other examples. So while there is certainly bad science (just as in other fields), there is still a great deal of well done science that has influenced our view of nature. As my colleague Dickson Despommier has said, don’t trust scientists – trust science.

  3. MthNrd says

    15 February 2011 at 5:59 pm

    Comment more on the New Yorker article than the above posts…The issue here is that the scientific method only truly gives replicable results when the conducted research falls within the realm of hard/physical science where there are “laws” governing the processes being studied. In the fields of medicine, psychology, and especially behavior there are no hard and fast laws…just generalities. This is due to the infinitessimally large number of factors that contribute to behavior. The larger the set of variables, the greater the possibiilty that one outlier will skew the results. Researchers try to create artificial, closed systems in which to conduct the experiments that in reality are far from closed.

    The tests Crabbe conducted may have appeared to be identical (with all variables set equal), but there were many other factors not considered/not controlled including: the lab personnel handling the mice were different in each lab (mentioned in the Crabbe article), the gender of the lab personnel, the “smell” of the lab personnel (whether perfumes/fragrances, naturally occuring pheremones, or residual smells from lab personnel behavior (what they had to eat, did they smoke, etc.))

    As to the question above of what do we really know…with any certainty, nothing.

Primary Sidebar

by Vincent Racaniello

Earth’s virology Professor
Questions? virology@virology.ws

With David Tuller and
Gertrud U. Rey

Follow

Facebook, Twitter, YouTube, Instagram
Get updates by RSS or Email

Contents

Table of Contents
ME/CFS
Inside a BSL-4
The Wall of Polio
Microbe Art
Interviews With Virologists

Earth’s Virology Course

Virology Live
Columbia U
Virologia en Español
Virology 101
Influenza 101

Podcasts

This Week in Virology
This Week in Microbiology
This Week in Parasitism
This Week in Evolution
Immune
This Week in Neuroscience
All at MicrobeTV

Useful Resources

Lecturio Online Courses
HealthMap
Polio eradication
Promed-Mail
Small Things Considered
ViralZone
Virus Particle Explorer
The Living River
Parasites Without Borders

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.