Most research is about investigating the known unknowns. In other words, investigators have a certain hypothesis, but also a range of plausable possibilities as outcomes. However, in some instances the outcome is unexpected – the unknown unknown. Rare side effects are an example of this, and not always evident in the initial randomized-placebo (dummy treatment) controlled clinical trials. They are normally picked up in subsequent observational studies. Examples of this include the development of inflammatory brain disorders following dalclizumab treatment, enterovirus infection in ocrelizumab, and more recently the report of strokes in alemtuzumab.
I knew all of this already, but what I didn’t appreciate is that upon reviewing tables upon tables of side effects from clinical trials, was that they were not always that informative. According to the study authors here, in trying to develop a model to review the side effect profiles of drugs from clinical trials, they disappointingly discovered that there was no standardized methodology for reporting. Instead, the focus in reporting clinical trials seems to be on efficacy outcomes, including relapse rate reduction, delay in disease progression, decrease in the number of MS lesions etc.
“The inspection of the geometry of the networks showed that we found lower values than the reported in the literature for other NMAs (Network meta analyses), when concerning the median number of studies per edge, number of nodes and edges, thickness, common comparator and strong edges. These results suggest weakly connected networks with a limited amount of direct evidence. Therefore, the data should be interpreted with caution. Despite of the high number of safety outcomes, they were not reported in a standardized manner throughout the studies, which precluded the construction of a precise network with a high number of studies, resulting, instead, in several less robust small networks” – say the authors.
They also point out that clinical trial design is inherently flawed in its ability to accurately report on side effects. Primarily, due to the short study duration (i.e. most trials are only two years long), but also the small number of people included (thousands instead of the tens of thousands needed to pick up side effects with precision), and the exclusion of those with co-morbidities or other medical disorders (thereby not representative of real-life usage). Observational studies that follow, in the form of post-marketing surveys, therefore become the main way of monitoring side effects in the in the real world, over the long-term. However, observational studies are not comparative studies, and therefore are able provide little data beyond that of the drug in question.
Maybe, it’s time that the FDA (US Food and Drug Administration) and the EMA (European Medicines Agency) that license these drugs in fact invested in prospective randomized trials specifically designed, of sufficient duration and adequately powered to test the safety hypothesis. I’m certain that to make a balanced treatment decision, both efficacy and safety trials are needed, taking most of the guess work out of it.
I wonder who will take up the gauntlet first, if at all?
Mult Scler Relat Disord. 2019 Jun 29;35:7-15. doi: 10.1016/j.msard.2019.06.036. [Epub ahead of print]
Safety outcomes of disease-modifying therapies for relapsing-remitting multiple sclerosis: A network meta-analysis.
Randomised clinical trials (RCTs) and observational studies have reported adverse events that preclude the use of disease-modifying therapies (DMTs) in relapsing-remitting multiple sclerosis (RRMS) in the long term or in specific populations, however, little is known about the relationship between the use of DMTs and frequency of undesirable events. We aimed to conduct a systematic review and network meta-analyses (NMAs) of RCTs and observational studies to synthesise the evidence on the safety of all available DMTs for patients with RRMS.
PubMed, Scopus and a manual search were performed. Bayesian NMAs of safety outcomes reported in RCTs and observational studies assessing DMTs as monotherapies were conducted.
Forty-seven studies were included in the systematic review. Considering all studies, 368 and 149 different safety outcomes were reported for at least one study and two studies, respectively. Considering clinical trials, 22 NMAs were conducted for 16 outcomes. Regarding geometry metrics, the median number of studies, DMTs, common comparator, strong edge, and patients were 5 (IQR 5-9), 5 (IQR 4-8), 44%, 33%, and 3998 (IQR 3380-6761). In summary, most comparisons showed similar risk of safety events for DMTs and placebo for all outcomes. Considering cohort studies, only three meta-analyses were conducted.
Safety outcomes are poorly reported in primary studies of DMTs in RRMS, precluding the conduction of robust meta-analyses. Therefore, the current available data on safety of these drugs is not contributing to regulatory and clinical decision making, with adverse event reports underbalanced compared to efficacy outcomes.