Hróbjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Rasmussen JV, Hilden J, Boutron I, Ravaud P, Brorson S. Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. Int J Epidemiol. 2014 Jan. [Epub ahead of print]
BACKGROUND:We wanted to evaluate the impact of nonblinded outcome assessors on estimated treatment effects in time-to-event trials.
METHODS: Systematic review of randomized clinical trials with both blinded and non-blinded assessors of the same time-to-event outcome. Two authors agreed on inclusion of trials and outcomes. We compared hazard ratios based on nonblinded and blinded assessments. A ratio of hazard ratios (RHR) <1 indicated that nonblinded assessors generated more optimistic effect estimates. We pooled RHRs with inverse variance random-effects meta-analysis.
RESULTS: We included 18 trials. Eleven trials (1969 patients) with subjective outcomes provided hazard ratios, RHR 0.88 (0.69 to 1.12), (I2 = 44%, P = 0.06), but unconditional pooling was problematic because of qualitative heterogeneity. Four atypical cytomegalovirus retinitis trials compared experimental oral administration with control intravenous administration of the same drug, resulting in bias favouring the control intervention, RHR 1.33 (0.98 to 1.82). Seven trials of cytomegalovirus retinitis, tibial fracture and multiple sclerosis compared experimental interventions with standard control interventions, e.g. placebo, no-treatment or active control, resulting in bias favouring the experimental intervention, RHR 0.73 (0.57 to 0.93), indicating an average exaggeration of non-blinded hazard ratios by 27% (7% to 43%).
CONCLUSIONS: Lack of blinded outcome assessors in randomized trials with subjective time-to-event outcomes causes high risk of observer bias. Nonblinded outcome assessors typically favour the experimental intervention, exaggerating the hazard ratio by an average of approximately 27%; but in special situations, non-blinded outcome assessors favour control interventions, inducing a comparable degree of observer bias in the reversed direction.
This is perhaps timely given the recent FDA rejection of Alemtuzumab, Some of the problems high lighted was the fact that the study was not blinded. Without blinding there is a chance an increased chance of bias to report overly positive effects. However, we are getting to a stage where it is not ethical to do placebo trials and you need to compare to an active. So if it is a beta interferon or glaterimer acetate you are going to have to manufacture dummy needles and you then inject nothing for a few years. So if you are a company who don’t have an injectable you are going to have to manufacture these? Will this be a barrier to entering into the MS space for any company Novartis, Teva, Merck Serono, Biogen Idec all have injectables…Genzyme-sanofi doesn’t.Are they paying the price? So we will need to get the pills into the forefront to stop this strangle hold of the big four as it is far easier and less costly and certainly less inconvenient to make a dummy pill.
Who are is it more important to blind the MSers or the Neuros