Fanelli D, Ioannidis JP. US studies may overestimate effect sizes in softer research. Proc Natl Acad Sci U S A. 2013 Aug 26. [Epub ahead of print]
Many biases affect scientific research, causing a waste of resources, posing a threat to human health, and hampering scientific progress. These problems are hypothesized to be worsened by lack of consensus on theories and methods, by selective publication processes, and by career systems too heavily oriented toward productivity, such as those adopted in the United States (US). Here, we extracted 1,174 primary outcomes appearing in 82 meta-analyses published in health-related biological and behavioral research sampled from the Web of Science categories Genetics & Heredity and Psychiatry and measured how individual results deviated from the overall summary effect size within their respective meta-analysis. We found that primary studies whose outcome included behavioral parameters were generally more likely to report extreme effects, and those with a corresponding author based in the US were more likely to deviate in the direction predicted by their experimental hypotheses, particularly when their outcome did not include additional biological parameters. Non-behavioral studies showed no such “US effect” and were subject mainly to sampling variance and small-study effects, which were stronger for non-US countries. Although this latter finding could be interpreted as a publication bias against non-US authors, the US effect observed in behavioral research is unlikely to be generated by editorial biases. Behavioral studies have lower methodological consensus and higher noise, making US researchers potentially more likely to express an underlying propensity to report strong and significant findings.
So non-behavioural science such as genetics are no so easy to big-up. There it is difficult to manipulate the results such as say reading a genetic code, but behavioural sciences such as drug tests are woolly and one can pick different ways to analyse the results.
The results in behavioural studies had more extreme positive biases, suggesting that people are sexing-up their results by over extending their analysis.
Furthermore the work that had US-based authors (remember the US is a magnet for the World’s best scientists) showed a greater tendency for the more extreme examples to occur. Therefore they are more likely to find that Whacky drug effect.
Why? Probably pressure to get that all important result. This can be the make or break for the people doing the work. Dull results…no grants. It can be the survival of the fittest.
I suspect in a few years this regional influence will decrease as more and more pressure is heaped on global scientists to deliver the goods and get those eye catching headlining papers.
Taylor-made for non-reproducible studies.