The price of transparency can mean failure

Kaplan RM & Irvin VL Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time. PLOS one. Published: August, 2015. DOI: 10.1371/journal.pone.0132382

Background: We explore whether the number of null results in large National Heart Lung, and Blood Institute (NHLBI) funded trials has increased over time.
Methods:We identified all large NHLBI supported RCTs between 1970 and 2012 evaluating drugs or dietary supplements for the treatment or prevention of cardiovascular disease. Trials were included if direct costs >$500,000/year, participants were adult humans, and the primary outcome was cardiovascular risk, disease or death. The 55 trials meeting these criteria were coded for whether they were published prior to or after the year 2000, whether they registered in prior to publication, used active or placebo comparator, and whether or not the trial had industry co-sponsorship. We tabulated whether the study reported a positive, negative, or null result on the primary outcome variable and for total mortality.
Results:17 of 30 studies (57%) published prior to 2000 showed a significant benefit of intervention on the primary outcome in comparison to only 2 among the 25 (8%) trials published after 2000 (χ2=12.2,df= 1, p=0.0005). There has been no change in the proportion of trials that compared treatment to placebo versus active comparator. Industry co-sponsorship was unrelated to the probability of reporting a significant benefit. Pre-registration in clinical was strongly associated with the trend toward null findings.
Conclusions:The number NHLBI trials reporting positive results declined after the year 2000. Prospective declaration of outcomes in RCTs, and the adoption of transparent reporting standards, as required by, may have contributed to the trend toward null findings.

What does this mean for MS. Well you could argue not very much. However there maybe similarities to MS, so brace your self for negative findings

In the scientific literature there is a bias for reporting of positive results. 

You may say “who cares its good to have good news”. However, the problem is when this so called good news is tried in MS and it fails, there is clear disapointment to say the least

If there was transparency and you knew that the drug was tried ten times and it failed in nine times (unpublished) verses the one time (published) it worked would you be willing to try it?

Maybe not.

This happens in science alot. High profile paper published, others failing to repeat it never see the light of day.

There has been a feeling that pharma trials that fail, never see the light of day and so there is a rosy picture presented in the literature and media. 

What happened to the phase III oral tolerance trial in MS for example? So much high profile immunology, hype and promise and then what? 

So there is a move for more transparency and unless you register your trials, with with the likes of clinical  before you treat anyone then many journals refuse to publish the data. If it was a failure you may phew. However, if it is a positive and this is your marketing strategy then you can’t afford not to register the trial.

Now there is nothing to make the negative trial data to be published, but because you know about it you will aske questions. Often the negative stuff arrives,  in the minor journals at a snails pace..but never the less it arrives.

So this study looks at what was published in yesteryear and there was a bias for positive results, but now there is greater transparency and with this more trials are found to fail. This may happen in MS too.

There is now a move for pharma to make the trial data open access. Some are resisting tooth and nail, where as other are embracing this but not allowing historical data be examined, as we would want pharma to be caught fiddling the data would we? 

However, it is amazing what you can get with freedom of information requests;-).

About the author



  • I see your point. If you worked on a drug previously that never made it to market, all of the future work done by these people need to be banished without looking at the data.

    I'll keep this in mind for anyone who worked on Clabradine so I can pre-judge any work or opinions they may have.

    Thanks for the tip. I'm starting to understand how science works.

    • Cladribine is a poor choice as the failure there was pharma made.
      Cut corners and paid the commercial price i.e apparently $800,000,000 down the toilet.

    • Wrong. It wasn't approved by the FDA. Looking at it from the surface as you are doing it was a FALLURE.

      But choosing Clabradine as a therapy to investigate was a no brainer. Any immunosuppressive drug is going to have an effect on an autoimmune disease. You may as well trialed Azathioprine.

      The difference between Team G and the other lab that failed with oral tolerance was that they were looking to find a cure instead of looking for a candidate drug that Pharma could market. They failed but the data was valueable because it added to the knowledge of what causes MS.

      You failed Pharma, which was a big waste time and money and didn't contribute anything to the understanding of the disease.

    • Problem facts get in the way of an argument.

      Azathiorpine is in the league of beta interferon but cladribine was in the league of natalizumab. It failed because Merck Serono would not do a second trial to address perceived safety issues.

      The people doing the oral tolerance of bovine myelin was actually a company called Autoimmune Inc. So it was pharma!.

      In terms of MS, did the failure of oral tolerance tell us that MS has nothing to do with myelin or Th2 /Th3 cells are not relevant or that oral tolerance is not good at inhibiting established immune responses.

      However, I can't be bothered to argue with you, we had all this rot last week:-)

By MouseDoctor



Recent Posts

Recent Comments