The publication process is broken. We are all desperate to get our papers in high-impact factor journals, because we are being judged by what we publish. The journals know this and either have a set of reviewers that can ask the earth before the work is published and then they charge you for the pleasure of publishing. These journals get so many papers they are picky and can ask you to do lots of extra experiments before the work is accepted. This has resulted in “science three ways”, that you do the experiments in different ways to get the same result and end up with loads of figures…so many in fact you don’t know where to look and that some times helps when the figures look abit dodgey.
It used to be that science work was published in print in scientific journals run by publishing houses. This was after they had been peer-reviewed using snail-mail (the post). Stuff would be type set, figures were taken from photographs and you would go down to the library and read through journal after journal to get your knowledge.
You pay to do the work and you pay to publish the work, academics are editors and often work for free, the reviewers do the review for free and the publishers now pay often South East Asians, I guess not a lot, to format into journal style. The journals that are paid for by the libraries.
If you were lucky the paper would appear about 6 months after you submitted it. This is similar now but technology have changed all and has speeded the whole process up. In addition now we are all too lazy to go to libraries and graze through journals. So we just sit on our bums and download stuff onto our computers.
Now, I think the quickest publication I have had from submission to being refereeed, changed, accepted and online in less than 2 weeks. Not bad from submission to availability. Sadly another group published the same work 10 months later and should I say they repeated what we had found, but they simply failed to mention our work:-(. They knew about it as we sent them a copy. No wonder I am bitter and twisted:-). However we are having another revolution.
Now the publishing process can occur in a few days .
In yesteryear, you the public would get their science fix by reading “Scientific American” or “New Scientist” and in my youth you could pop down to W.H.Smith (A News agent) and get your weekly fix from Nature. Now the powers that be want the people who fund the work (you the public) to be given access to the work and so Open Access publishing was born. The publishing houses thought “suckers” as we can get those mug scientists to pay more to publish the work.
Did you know that the Dutch government is now paying for manuscripts from Dutch Scientists, in every paper journal, to be open-access. At £1000-£3000 a pop, it is going to to take a lot of research money away from research budget to pay fat cat publishers, who are laughing all the way to the Bank. UK Government sponsored research, likewise does the same as does the National Institute of Health in the USA.
The powers that be, want publications to go open access so the public get to read them. Are you bothered? We decided to go that one step further and cut out the middle man and so you have our latest paper before it was been submitted to….I was not sure where, when this post was written. It was a while ago.
Now there are some predatory journals who will take any old rubbish just so they get their hands on the Open access fee ($1000-4,000) and this has really tainted the Open Access approach. The high impact factor journals reject your papers, but then they offer you an expedited slot in the fee-for publish open access sister journal so Ker Ching, Ker Ching. Some have got good impact factors now, so they may spawn their own cousing journal (sister of the sister)
So now we have the next development and this is now called the pre-print site. Currently they are free and open access, but this won’t last. The idea is that you get the information into the public domain quickly. This also allows you to get the work reviewed before you submit the work. Some journals won’t accept this model and so won’t publish the work as they argue it loses novelty, others are are willing to accept this practice and this is forced upon them by some of the big players. With time this will become the accepted approach and then the pre-print sites will charge a fee. There will be loads of pre-print sites popping up to fill the free-void, but the easy solution is to for a central government to house such a site and core fund it. Research costs are reduced. Alternatively Universities will house their own papers…What a mess we are creating.
These pre-print sites are essentially no different than some open access journals as they are not linked to indexing sites like scopus and pubmed. You say they are not refereed, but the level of refereeing of some papers is not good.
Once accepted by a journal it will go on an indexing site. Now for the publishing revolution to really happen, you deposit on a pre-print site, it gets reviewed by the community and goes on an indexing site…you cut the journals out of the equation, impact factor is irrelevant, citation index is key. If it is good the work, it gets cited. If it it isn’t, it doesn’t. If central governments fund the pre-print sites you save thousands in publishing costs, you kill the journals…Booh Booh. Do we really care about Kodak and other 35mm film makers, with mobile phones and technology, film is really a thing of the past. I guess this is why publishers are creating their own pre-print sites. Do we lose the online publication literature as the publishing house goes under?
How will we know what a good paper is? Well you will just have to read them, you should not rely on the names of the authors, as many people do now. I think this can be the new revolution. People will have to read and the blinkers may come off.
The referee of today’s paper seems concerned that we challenge classical peer review…..but there is nothing new in meta analysis and there is nothing new with reviewing papers, but where this study is challenging, it reviews work that has not been properly published. It was done over 8 years ago by the neurology big fish. It seems to us to have merit as it speaks to risk benefit, but it could dint pharma profits by not prescribing, so is this the reason why it has not been published and become searchable in indexing sites? Alternatively is the work not good enough to publish? But if that is, so why was the information presented in at least three Major International Meetings, with well known Neuros as the presenting authors?
So now you can have the paper before it goes to a journal. Here is the preprint address for an open access paper. Have a read if interested.
COI: Prof G knows nothing of this.
Here is the paper,
Although it is indicated that pre-prints in pre-print sites are not peer reviewed, but this is actually not true as this paper has been submitted to two high impact factor journals. Generally the reviewers comments are not seen. Whilst this is changing, today I am being a bad mouse, I give you the reviewers comments. I also give my take on them. Each time it has been rejected, it has taken 4-5 months.
So it has now taken 12 months for this data to surface. We could go again, but we have said “schtop” let the World read the paper. Let’s cite the paper and try and do a clinical study, or someone else can read this and do the study..Sure they may ignore the fact that we gave them the idea but this happens. But importantly it stops wasting time before the studies are potentially done. The company said they were going to publish when we told them we wanted the data, we have waited a year and a half and nothing.
Maybe it will spur the neuros and company to actually write the work up and so if there are elements they don’t like, they will be eager to set the record straight.
Just so you know,the editor may not read the paper. They rely on the reviewers to read the paper and make comments, so the editor can make then make a decision, if it is suitable to published Rejection is part of the peer-review process. It always hurts but you can live with this if the comments are constructive. However, if they are destructive or half-truths, it is much more irritating and gets you down. You get rejected you pick yourself up and go again. But sometimes you just get fed up and today (when writing this a short while ago) was one of those days
You know me. I like to have a rant and today I want to give you some insight into the publication process and give you the reviewers comments. I will be chastised for doing this, hopefully ProfG won’t get too mad, but the reviewers are anonymous and if they see this and are horrified…just think how I felt when I read their comments. Likewise rememeber they wrote this and need to stand by what was written. I must say remember they are not here to defend themselves adn my interpretations may be wrong. However, they won’t be reading this as I am sure they won’t read the blog. However read, reviewer #1 is you can’t be bothered to read the paper it does give a synopis. My comments are in italics
The hypothesis underpinning the present work is that depletion of memory B cells is the main mechanism of action of B cell depleting therapies (and perhaps to a variable degree also other cell depleting DMTs such as alemtuzumab and cladribine, or HSCT). While substantial evidence suggests that MS is mainly a T cell driven disease, (Referee is a T cell immunologist… and probably American or American-trained as a Brit wouldn’t be bothered to do a long review like this) the quite remarkable efficacy of B cell depleting therapies demonstrates that B cells play a non-redundant accessory role somewhere in the disease process. (Referee is a T cell Immunologists, who struggles to think that anything other than a T cell is important, But the trial data can’t be ignored). Whether this is through cytokines, co-stimulatory T cell activation or presentation of autoantigens is not yet clarified (They still can’t believe B cells are important and have to explain it some how and don’t want to admit they have or haven’t read any of our papers, which have already made these points. But who cares about this? The paper is not about how B cells work, but the referee is telling the editor how much they know). However, experimental work by Roland Martin´s lab (The referee says My mate Rolly is ace..the authors are not worthy and how dare they not cite him or maybe how dare they not cite my superior work:-), recently published in Cell (not included among the references of this paper) (Naughty boy for not citing a “Cell” paper by the referee’s scientific hero and telling the Editor that we can’t do a literature review. I guess I would reply, why bother doing this as there are plenty of earlier papers saying this, before Prof Martin and if you have read my blog post on that paper there are problems with Prof Martins view in that paper…Yes and I have said this to his face at a meeting in Sweden), suggests direct physical antigen-dependent contact between bona fede encephalitogenic B and T cells (both bearing memory cell surface markers) to be a crucial component (OK, so what has this got to do with the paper? Still nothing). Still, unambiguous demonstration awaits the exact definition of the antigen or antigens causing MS (We have been looking for antigens forever, but the referee is happy to tell the editor how much they know and again what has this got to do with the paper…Again a very big nothing). The clinical impact of defining memory B cells, rather than B cells in general, as the main treatment target are quite far reaching, implying that after an initial course of B cell depleting drug additional dosings could be given at long interspaced intervals (perhaps with monitoring of the CD27+ memory B cell population, as has been suggested in neuromyelitis optica with rituximab). Or perhaps by adopting a protocol similar to dosing regimes for alemtuzumab and cladribine (I say OK you have got the point, but the Editor will have already rejected the paper by now. They are so bored and have slashed their wrists already) . The authors of this paper approach this angle by taking advantage of existing data from the ocrelizumab registration program, in particular the phase II trial, which included a drug-free safety follow up lasting until B cells had normalized in most subjects. Hence, the presented data show that even if CD19+ B cell numbers had returned to within normal limits, there were no sign of return of neuroradiological disease activity (Repeating the study findings to the editor to prove they can read). In addition, a point is made about the fact that both fewer adverse events overall, but also infections occured during this stage of the study, which could indicate a better risk-benefit with more infrequent drug dosing Repeating the study findings to the editor to prove they can read . It is not for our benefit as I know what was written in the paper. Why bother? Half a page of someone saying how much they know.
As a first note (Now the boot is coming you think but), this is a laudable initiative to extract results from existing data, especially since any active support from the drug manufacturer in this direction is unlikely (They recognize the issues but……..). The key trial included 220 RRMS patients that were randomized to into four arms to receive placebo or IFNb for the first 24 weeks, or ocrelizumab (600 or 2000mg, the latter not included here). As stated in the manuscript, 151 entered the extension study, with ocrelizumab dosings until week 96 and drug-free follow up until week 144, however, only n=33 contributing data for the end of study time point. The results section is divided into three parts, the first regarding B cell repletion (but lacking data on memory B cell populations), a second paragraph on effect of treatment (showing peristent suppression of disease activity throughout the safety follow up, with similar ARR numbers as in the earlier phase and in the phase III studies) and a last part on reported adverse events (much reduced in the drug-free follow up compared to the first part). (They are still showing the editor that they can read, but now the editor really is bored, thinks they have died and gone to hell, so a rejection is in order). In my opinion there is a substantial weakness (Reject, reject, reject) with the results part in supporting the message conveyed (Reject, Reject Reject), not least with the successively smaller number of individuals contributing data (The Boot goes in …Reject…They think its a rubbish study… . Yes the numbers got notably smaller. Was it because they didn’t finish the study or the data analysis finished before the majority of people reached that point. Whilst the numbers were low at 18 months they were not at 12 months). The text doesn´t entirely make it clear to me how many remained at each step of follow up, and how many were lost to follow up (That is because the reviewer is too lazy to read the supplementary information supplied to the reviewers that answer the very question, but the editor thinks this paper is really rubbish because it doesn’t give enough information-Reject. OK next time give more detail on numbers). With regard to safety reporting, was the follow up routine identical for those remaining in the study and those that droped out? (Again the reviewer is too lazy to read the references to the trial protocol that would give this answer if they were really interested, but again this paper is rubbish because the methods aren’t good enough-reject) In the phase III trial, the safety follow up differed in frequency for those stopping drug dosing and those that remained on drug (So what, this isn’t about the phase III study, maybe it is a typo but that would mean they had read the protocol, so they could answer their own question). It is likely that a reduced frequency of visits leads to less AE reporting, at least regarding non-severe events (It says these lot can’t analyse data. Reject. OK deal with this in next version) . Furthermore, some individuals are likely to be more susceptible to adverse events, driving up AE frequency early on and making comparisons across initial and later phases perilous (at least in terms of inferring that the drop in AE frequency is caused by stopping drug, in particular since one of the main drivers of increased susceptibility to infection, hypogammaglobulinaemia, normally develops over time). (OK point taken I will incorporate this into the next version) This is also relevant for relapse frequency. (Wiggle wiggle) Taken together the data do suggest that there is no rebound in disease activity even if B cells have repleted, but falls short (by quite a bit) (The karate kick comes in. Reject this paper now!) of demonstrating that efficacy with a shorter treatment course is similar to continuous dosing and with less risk of adverse events. (It is a hypothesis, this work does not disprove it).
So if we look at what was said there was very little constructive critisim about the paper. We can adjust the discussion abit to accomodate some of the useful points and add abit more detail. However, nothing major to fix
The authors have reviewed meeting abstracts, posters and regulatory documents from a Phase 2 studies as a modified meta-analytic review. While they are unable to present the primary data (Isn’t this obvious from the above sentence) nor elaborate on the quality of the primary data (We need to say there are some rubbish aspects), they report that ARR reduction and MRI lesion activity remain reduced during B cell repletion. (OK should I say the trial was rubbish because it wasn’t reported. But if it is rubbish why is this information being presented at the two most prestigious MS meetings in the World and one gets a platform presentation to do it). They data as presented suggest there is no infection issue, but as presented suggestsst that there are markedly fewer adverse events and infections. Unfortunately, since there is no access to the unpublished data, the authors are not able to provide any statistical comparison nor elaborate on the nature of the infections nor their potential relationship to drug therapy (This paper says nothing as they don’t have the data. Reject). The authors ignore that ARR and MRI activity respond quickly to B cell depletion, but other critical components of MS therapeutic efficacy are more variable (Not ignored but in my opinion the reviewer does not get it that relapse and MRI respond to one thing and disability progression is a composite. So now who’s the person that doesn’t know what they are talking about?). For instance, reduction in the risk of disability progression appears to be exquisitely dependent of the extent of B cell suppression (unpublished abstract presenting at meeting from OPERA trial data) (Nice one… the reviewer blasts us for not quoting unpublished data that was presented at a meeting that we and most other people may not have even been at….So we are wrong for something that does not exist in the current science literature..However, actually I was at the meeting and I did see the presentation. The B cell depletion was unrelated to relapses but it related to progression. The most logical suggestion is that the effect on progression is not peripheral B cell depletion but because the high dose of antibody got into the CNS, so it is irrelevant to this paper. However, if you look on the blog ProfG has argued that we are underdosing and that there should be double dosing but that is another story). Hence, the conclusion that therapeutic efficacy is “maintained” during repletion is based on a limited dataset that has only partial clinical relevance. (Reject…based on my “dubious interpretation ) The authors state (suggest/hypothesise) that therapeutic efficacy is due to suppression of memory B cells. Unfortunately there is no primary data presented on memory B cell repopulation in the data presented (Is this surprising we say that this data has not been presented. Some of the data on early rituximab studies are presented in supplementary data but the file is corrupt and shows nothing. The journal nor the authors have the data, so sometimes even when things are reported they still can’t be seen.. However, surely you can assimilate knowledge?…Oh I’m sorry I forgot you can’t. If you see the squashed pheasant in the middle of the road, you can’t guess that it was hit by a car and run over buy a few more vehicles. Good job they are scientists and not police officers) .
The authors have a hypothesis that warrants testing; however, the current report is not sufficient to support their conclusion. (Reject) A primary study should be designed and initiated. (Do a trial!….Yes but we need preliminary supporting data would help us to secure funding to do a trial. I would love to do a trial..We should do a trial, Importantly Roche should do a trail).
So again very little constuctive critisim, but we can explain the dosing issue
This is a paper which addresses a number of observations made in the ocrelizumab phase II extension studies, mostly by obtaining information from data bases and meeting abstracts, since much of this information was unpublished. The authors make a number of suggestions, many which have merit, but often do not have much data to support their hypotheses. (Reject) For example, the authors point out that in some disorders, anti-CD20 strategies resulted in increased infections. However, this does not appear to be a major feature in anti-CD20 strategies used in multiple sclerosis. Since T cells, NK cells, and myeloid cells are essentially unaffected by CD20 B cell depletion, it would stand to reason that patients would respond appropriately to most immunological challenges, which is consistent with previous studies indicating that infection rates and severity are not significantly changed in MS patients on B cell depleting therapy (Hauser et al., 2008; Bar-Or et al., 2008; Hawker et al., 2009; Kappos et al., 2011; Montalban et al., 2017; Hauser et al., 2017). (Fair point and serious infections are rare, but if you are one of the people with the serious infection you may not think they are trivial). Another point made by the authors is that therapeutic efficacy seems to outlive the half life of the depleting antibodies and that this is likely due to long-lasting depletion of memory B cells. Interestingly, with rituximab and ublituximab, other B cell depleting monoclonal antibodies, have shown a shift in the Treg to Teffector ratio, which may be a factor in the longer lived therapeutic effect. (So the referee is a T cell immunologists hooked on the T regulatory cell controls everything idea) For example, in immune thrombocytopenia treatment with rituximab and steroids increased Treg compared to treatment with steroids alone (Kidney International92:227).In treatment of membranous nephropathy with rituximab, both Treg and CD56hi CD16lo NK cells were increased (Int J Hematol 93:91). In multiple sclerosis, a recent study with another anti-CD20 monocloncal, ublituximab, also increased Treg and reduced Teffector cells (JNI 332:187 see below). (Therefore it is T regs. T regs T regs that control everything. T cell immunologists reign supreme) This is an example where the authors really have little data to support their hypothesis (Reject) and there are examples where effects on other immune cell populations may mediate long-term treatment effects. (Reject, as these lot don’t know their bottom from their elbow, they can’t see the wood for the trees. I say really……I mean really?. So lets look at this massive T reg increase in the papers they cite. Now you have to remember that CD20 inhibits MRI lesions within a couple of weeks not months. Ocrelizumab inhibits MRI lesions within weeks, anti-CD20 depleting inhbits natalizumab-cessation associated rebound within 8 weeks)
So I think rather a half-truth used to reject an idea.
Treatment strategies in MS that could effectively treat women in their child-bearing years would be a great advance. While the authors again put forth reasons why ocrelizumab might be a beneficial strategy to manage women with MS who wish to become pregnant, they actually have no data regarding this scenario (Not with ocrelizumab but there is data with rituximab. I can see that what happens with rituximab will probably happen with ocreliziumab) . Overall, this manuscript in many ways reads like an opinion piece under the guise of a research paper (Yep it is an opinion piece/review so what! That is what was in the letter to the editor). While many of the points are reasonable, many are highly speculative (Oh speculation…..I have just soiled myself) and do not really have much support at present, although future studies to address many of these issues could be warranted. (Yes, please let’s do some future studies)
If we went back we could correct the referee, but they probably have fixed views, but in a new version we can address this issue. We could try an argue but we won’t get far with it,
So we take the constructive comments on-board and try again, this time the responses don’t seem quiet as negative, this is what peer-review does. It improves the manuscript
The paper addresses an important issue for our future therapeutic repertoire in MS. Obviously, B cell depletion over many years (if not decades) is a relevant issue for long-term safety but there are no prospective trials investigating this aspect of ocrelizumab, rituximab or future ofatumumab therapy in a systematic manner. Therefore, all attempts to improve our understanding of lasting biological, clinical and immunological changes by chronic B cell depletion are welcome (Looking good Billy-Ray)
However, as the authors frankly discuss, access to the clinical and laboratory data of the phase II extension trial for ocrelizumab was requested but not granted to them. This explains why the paper does not represent classical original research (Who said it was original research? Not I) but rather a compilation of so far presented but not yet published data sets of researchers and the pharmaceutical company (OK) which are still running the open label extension trial (Due to finish in 2021 when there will be no participants left and says it is OK to keep the results quiet for over 10 years after the trial finished). Clearly, the presented data provide very useful insights into the running trial but may not represent the final results (So What? Most trials are published with 2-3 year data when they run for 5 years a trial running for a decade is unheard of). This approach challenges classical peer review along our established routes for primary (original research) (Why is it challenging? Are you afraid of pharma if you say accept? So you will say reject) and the paper may be more appropriate as a personal view or commentary (OK its a commentary, but what is really said…Nothing).
From a constructive critism point of view there is nothing to correct, but the reviewer is cautious about approving the approach. I had not problem with this view.
The authors are to be commended for the painstaking working of piecing together the data and examining a very important question. The manuscript is thoughtful and well-written, a pleasure to read. (At last:-). One minor question, are the authors suggesting that high doses of ocrelizumab prior to pregnancy would deplete neonatal B-cells? (Maybe this is why European Union want 12 month contraception after end of treatment verses 6 months in USA and Australia)
From a constructive critism point of view there is nothing to correct and we can add detail about depletion of B cells during pregnancy.
This is a very interesting assessment of the phase 2 data from the ocrelizumab studies. As such it provides interesting data that emphasise the problems with drug studies in that very little importance is placed on what is the minimum dose and what are the criteria for stopping DMTs in MS. Pharmaceutical companies have no interest in pursuing these outcomes and do not interrogate their data to provide these answers. The authors have done an excellent job in finding these data for ocrelizumab and provide some of these insights which may have very important implications for people with MS. From this point of view the paper provides a strong argument for further studies of whether in fact ocrelizumab is potentially an immune reconstituting therapy that can be used les intensively and for a shorter time frame than currently indicated. which may reduce the risk of complications particularly viral infections and reduced immunoglobulin levels. (Ta).
The discussion goes into a lot of detail on B cell immunology and this could be significantly shortened although providing a potential explanation for the findings of the paper it has been well described before and does distract from the overall impact of the paper. (I guess you have to see that this is added in response to referees above notably referee #2. This referee was unaware of what referee#2 had written) Additionally the overall tone of the paper could be adjusted down as currently it comes across as over the top (Which bits do you think are OTT?) and this distracts from the impact of the findings. Much of the figures on B cell biology could be supplementary figures. (I would disagree as the duration of repopulation is probably the central difference between rituximab and ocrelizumab)
From a constructive critism point of view there is nothing much to correct, we could streamline the discussion abit but them risk comments like we got from referee#3. We could do back but the editor had cold feet.
So now the paper is on the web you can become referee#7, although maybe it has already been sent out and you can be referee#10 maybe it is too late and it has been accepted. I hope this gives you some insight into the peer review process. What do you think?
Right to Rant or Shutupayerface. Yep I will get chastised for this Post:-( Sure I apologise to the reviewers for putting their comments out there. In some journals like the Faculty1000 the reviewing process is out in the open and completely transparent.