Tsilidis KK, Panagiotou OA, Sena ES, Aretouli E, Evangelou E, et al. (2013) Evaluation of Excess Significance Bias in Animal Studies of Neurological Diseases. PLoS Biol 11(7): e1001609. doi:10.1371/journal.pbio.1001609
Studies have shown that the results of animal biomedical experiments fail to translate into human clinical trials; this could be attributed either to real differences in the underlying biology between humans and animals, to shortcomings in the experimental design, or to bias in the reporting of results from the animal studies (including EAE). We use a statistical technique to evaluate whether the number of published animal studies with “positive” (statistically significant) results is too large to be true. We assess 4,445 animal studies for 160 candidate treatments of neurological disorders, and observe that 1,719 of them have a “positive” result, whereas only 919 studies would a priori be expected to have such a result. According to our methodology, only eight of the 160 evaluated treatments should have been subsequently tested in humans. In summary, we judge that there are too many animal studies with “positive” results in the neurological disorder literature, and we discuss the reasons and potential remedies for this phenomenon.
Testing a new drug on human subjects is expensive, risky and ethically complex, so most of them are tried on animals first. The scientific literature is littered with studies that are either uncertain in their outcomes or which appear to flatly contradict each other, but when looked it was found that there is a bias as there are more positive than negative studies. In fact this shows there are twice as many studies as expected appeared to have statistically significant conclusions – something known as excess significance bias.
Well I say “No S**t Sherlock
Anyone who works in science can tell you this from personal experience, no-one is interested in what does work they are interested in what does work. It is therefore much more difficult to publish negative stuff and sometimes quite impossible. I remember trying to repeat some stuff published in Nature but could not get it to work, it turns out about 4 other groups could not do it either but we could get together to put it out there and if you try publish the negative stuff, you get asked to do more experiments to show why it does not work, rather than accept it was a dodgy idea in the first place.
The animal experiment reformers are going to try use this information to push through the idea of a registry for animal experiments a bit like clinical trials.gov so that you know about negative studies. So they can argue this will help work out what stuff to take into human trials.
This idea is bonkers, but I bet it will happen.
Many ideas in science evolve over years so you would show your hands years in advance of publication. These meddlers have to realise that most animal studies are not clinical trials aimed at developing drugs…many scientists are just preoccupied with mechanisms
Would the pharmaceutical industry buy into this idea..so I tell you all experiments before they do them…..Don’t think so!
Oh now here is a problem if you use the work to support the patent when you find a drug that works, so you will have to file before you do the study coz it goes in public domain, if it is in public domain before you file it isn’t patentable in many countries.
For these type of EAE studies you could have a reputable testing lab or few to repeat the studies in more than one strain before drugs go into man……I would put money on that many would fail.