Politics: scientific impact, what is it?

How to measure scientific impact in the era of pop science? Will altmetric turn out to be a curse? #MSPolitics #MSResearch #MSBlog

“Research impact; what does it mean? We as researchers are increasingly getting judged, and then hired and fired, based on our research performance. The 2014 UK research excellence framework (REF) include a metric on impact. We are told  that going forward research impact is going to be increasingly important. It is a great pity we can’t rely in the tried and tested way of assessing research impact based on citations and longevity of research over time. Short-term impact may turn out to be a fad; i.e. pop science, and disappear with time. In comparison transformational discoveries stand the the test of time; they are adopted and become entrenched in our culture.”

“A good example, of pop science is our recent publication on the negative association between HIV infection and MS: Gold et al. HIV and lower risk of multiple sclerosis: beginning to unravel a mystery using a record-linked database study. J Neurol Neurosurg Psychiatry. 2014 Aug 4. pii: jnnp-2014-307932. When you track it using Altmetric, it is flying.”

“When you look at it using the JNNP Marmite-ometer it is not doing so well.”

“If you ask me this paper will only have real impact if what we have described is real, i.e. confirmed, and it leads to new insights and treatments for multiple sclerosis. What is needed and is currently lacking is the time vector? As I sit on my sabbatical in Lima, Peru, at the LACTRIMS meeting contemplating life, the universe and everything, it is essential for us to take the HIV-MS association as a starting point to look at MS in a different way. Could this observation that started off with an anecdotal observation that someone’s MS went into long-term remission after he went onto antiretroviral therapy having become HIV-positive. Could the association between MS and HIV be the clue to the cause of MS and autoimmunity in general? What we really need is; (1) for a register of people who have MS and HIV who are on anti-retrovirals to see what happens to their MS; (2) the results of the INSPIRE trial; (3) and a full-blown well-powered phase 2 trial of HAART (highly-active antiretroviral therapy) in MS.”

“You may be interested in reading an article on the Wellcome Trust’s website about the issue of impact: Alternative impact: Can we track the impact of research outside of academia? It is based on  the following publication in PLoS Biology”. 

Dinsmore et al. Alternative Perspectives on Impact: The Potential of ALMs and Altmetrics to Inform Fundersabout Research Impact. PLoS Biol. 2014 Nov 25;12(11):e1002003.

More evidence of the meaning and validity of ALMs and altmetrics, coupled with greater consistency and transparency in their presentation, would enable research funders to explore their potential value and identify appropriate use cases.

CoI: multiple, please note that Merck have kindly funded the INSPIRE Trial

About the author

Prof G

Professor of Neurology, Barts & The London. MS & Preventive Neurology thinker, blogger, runner, vegetable gardener, husband, father, cook and wine & food lover.


  • http://www.digitopoly.org/2014/10/13/tirole-and-pasteur/
    (The link is a blogpost about the Economics Nobel winner)

    "The rarer case, however, is the economist who is more like Louis Pasteur. Pasteur, famously, founded the field of microbiology and with it vaccination: A fundamental advance in science alongside an incredible invention that impacts directly on people. Donald Stokes, the famed historian of Science, liked to think of scientist impact as having two dimensions: academic and applied. When an idea managed to have impact on both fronts, he termed in Pasteur’s Quadrant."

    If the Charcot project leads to a treatment and possibly a cure it will be in 'Pasteur's Quadrant'

  • Marmite is a British spread made from yeast (tastes like concentrated salty beef stock)..which is yummy. However it is an acquired taste and you either "like it" or "loathe it" Like it and the swing-o-meter goes one way and loathe it and the swing-o-meter goes the otherway.

    In Australia they have vegimite and in Denmark they don't have any Marmite because it is is apparently banned because it contains vitamin B12 and is classed as a supplement not a food (at least in the zenophobic British Press that likes to knock our EU chums

    However the world is metrics mad.

    We are have become a useful tool to the MS community because every paper we comment on, it gets an altimetric. So donatation to our Swiss bank accounts to get you papers mentioned 🙂

    Maybe when we post our papers on the blog site and then tweet it, all you have to do is retweet the paper and there we go Altimetrics superstars, our employers happy….

    Maybe if we add a few comments in our papers about ejaculating mice (yes one of the drugs we tested did this…we didn't mention it and I am still suffering flashbacks of something that I witnessed a week ago…it would be viral if I filmed it) and a few more knob or Pooh stories so the media will pick it up and off we go:-). We know what they do with a story i.e. kill the truth, but altimetric heaven….By the way I saw so data on the high salt diet and EAE story….180% opposite effect with high salt diet reported in Nature…so beware of pop science and take it with a pinch.

    We had a PLOS one paper and within a week it had a few thousand tweets/Facebooks and within about four weeks it was up to over 16,000 facebooks, a week later it was down to 1 so what happened?…I always wondered if Facebook does the right thing:-)

    • Nope…..however they are planning on using this class of drug (PDE4 inhibitor) in up and coming trials in progressive so it will be interesting to if this side effect occurs in people with spasticity.

    • I will never forget that day, MD looking like he'd participated in a mouse bukake session!
      If we'd put that image in the paper our Altmetric score would have been huge!

    • yes but look at the reason….it said by mistake "shall we cite that crappy paper" by…..weird thing it apparently took 4 months for someone to notice..which may be says something about the interest in the contents?

  • I suspect you have troll who does not like marmite hitting the dislike button repeatedly. What is there to dislike in your article?

By Prof G



Recent Posts

Recent Comments