Why does the EDSS allow one to rise from the dead? A flawed gold-standard #ResearchSpeak #MSResearch #MSBlog
“The study below confirms many of my suspicions I have been having about progressive MS and its definitions. In short is a mess. What the study below shows that when somebody has a confirmed progression in a large proportion it is not sustained (30%); i.e. they regress or improve. Wow! Are you surprised? I am not at all. The statistical method used for analysing this is called a survival analysis or Kaplan Meier curve. The latter was designed for a black-and-white outcome; alive or dead. With the exception of Lazarus you can’t become undead. The problem with the EDSS is you can improve after you have been confirmed as having progressed; this improvement may be due to real biology (remyelination, axonal & synaptic plasticity, cortical plasticity, etc.), but it can also be due to noise in the outcome measure. The EDSS is a very noisy measure with high intra- and inter-rater variability, it is non-linear, it has so called ceiling and floor effects, it is mainly an impairment scale (abnormal neurological examination) at low scores and becomes a disability scale higher-up (mobility) and it is rarely done properly in routine clinical practice. In short the EDSS is a flawed gold-standard and is probably the single biggest reason holding back treatments for progressive MS.”
“It is reassuring to note that regression, or improvement, after confirmed progression (Lazarus effect) was more common in younger MSers, MSers with a relapsing-remitting disease course, and after a smaller change in disability, and was inflated by higher visit frequency presumably due to more frequent EDSS measurements, i.e. more noise. Younger age, and relapses, predicting regression hints at this being due to improvement in disability as a result of functional reserve and recovery. The latter is linked to age; unfortunately, recovery mechanisms drop off with age. The latter is well studied in both humans and animal models.”
“In clinical trials and natural history studies disease progression in a population of MSers is only meaningful if it is backed up significant differences in the average entry and exit EDSS scores over the time period being studied. For example, if a group of MSers starts a trial with an average EDSS of 2.0 and proportion of subjects have disability progression in the trial then you would expect the average EDSS to get worse in the study to say 2.5. However, in many studies this does not happen and the average entry and exit EDSS scores remain the same. This implies that what has happened in the trial is noise and if you are in a placebo arm with more relapses, more pseudo-progressions will occur compared to an active comparator group. George Ebers and colleagues elegantly showed that in the placebo arms of phase 3 clinical trials you are as likely to have disability regression (improvements) as progressions (worsening) illustrating the point I am trying to make in this presentation.”
“In principle I am against defining progressive MS using the EDSS; we really need a biological definition of progressive MS using more sensitive biomarkers. It is clear that progressive MS does not suddenly begin at some point in time. The pathological processes that drive progressive disease, i.e. neuroaxonal loss and gliosis, are present from the earliest stages of the disease. Progressive MS only manifests clinically when you can’t compensate for further damage, in a particular pathway, because you have run out of reserve capacity. What I am really saying is that relapsing-remitting and progressive MS are the same beast and trying to separate them by a measuring scale is futile process. What we need to do is come up with biological definitions of the processes we are trying to measure, and change with our treatments; i.e. focal inflammation, focal axonal loss, diffuse inflammation, diffuse neuroaxonal loss, gliosis, demyelination, remyelination, etc. In this way we will be able to ‘spread hope‘ and not ‘write-off‘ people with ‘progressive MS’ with terms such as ‘therapeutic windows‘, etc. As I have said many times before the fact that someone is in a wheelchair does not mean we can’t do something to change the natural history of their disease; these people need their upper limb and bulbar function (speech and swallowing) protected and preserved.”
Epub: Kalincik et al. Defining reliable disability outcomes in multiple sclerosis. Brain. 2015 . pii: awv258.
Background: Prevention of irreversible disability is currently the most important goal of disease modifying therapy for multiple sclerosis. The disability outcomes used in most clinical trials rely on progression of Expanded Disability Status Scale score confirmed over 3 or 6 months. However, sensitivity and stability of this metric has not been extensively evaluated (really why not?).
Objective: Using the global MSBase cohort study, we evaluated 48 criteria of disability progression, testing three definitions of baseline disability, two definitions of progression magnitude, two definitions of long-term irreversibility and four definitions of event confirmation period. The study outcomes comprised the rates of detected progression events per 10 years and the proportions of the recorded events persistent at later time points.
Methods: To evaluate the ratio of progression frequency and stability for each criterion, we calculated the proportion of events persistent over the five subsequent years once progression was achieved. Finally, we evaluated the clinical and demographic determinants characterising progression events and, for those that regressed back to baseline, determinants of their subsequent regression. The study population consisted of 16 636 patients with the minimum of three recorded disability scores, totalling 112 584 patient-years.
Results: The progression rates varied between 0.41 and 1.14 events per 10 years, with the length of required confirmation interval as the most important determinant of the observed variance. The concordance among all tested progression criteria was only 17.3%. Regression of disability occurred in 11-34% of the progression events over the five subsequent years. The most important determinant of progression stability was the length of the confirmation period. For the most accurate set of the progression criteria, the proportions of 3-, 6-, 12- or 24-month confirmed events persistent over 5 years reached 70%, 74%, 80% and 89%, respectively. Regression post progression was more common in younger patients, relapsing-remitting disease course, and after a smaller change in disability, and was inflated by higher visit frequency.
Conclusions: These results suggest that the disability outcomes based on 3-6-month confirmed disability progression overestimate the accumulation of permanent disability by up to 30%. This could lead to spurious results in short-term clinical trials, and the issue may be magnified further in cohorts consisting predominantly of younger patients and patients with relapsing-remitting disease. Extension of the required confirmation period increases the persistence of progression events.
Ebers et al. Disability as an outcome in MS clinical trials. Neurology. 2008 Aug 26;71(9):624-31.
BACKGROUND: Inferences about long-term effects of therapies in multiple sclerosis (MS) have been based on surrogate markers studied in short-term trials. Preventing progressive disability is the key therapeutic goal but there remains no validated definition for its measurement in a trial context. Meanwhile, MS trials continue to shorten and to depend on unvalidated surrogates. Since there have been no treatment claims for improving unremitting disability, worsening of disability in the placebo/control arm must occur for effectiveness on this outcome to be shown.
METHODS: We examined widely-used clinical surrogates of long-term disability progression in individual patients with MS within a unique database from the placebo arms of 31 randomized clinical trials.
RESULTS: Detection of treatment effects in secondary progressive MS trials is undermined by noise in disability measurement. Whereas existing measures can be partially validated in secondary progressive MS, this is not the case in relapsing-remitting MS. Here, examination of widely used definitions of treatment failure demonstrated that disability progression was no more likely than similarly defined improvement. Existing definitions of disease progression in short-term intervention trials in relapsing-remitting patients reflect random variation, measurement error, and remitting relapses.
CONCLUSION: Clinical surrogates of unremitting disability used in trials of relapsing-remitting multiple sclerosis cannot be validated. Trials have been too short or degrees of disability change too small to measure the key outcomes. These analyses highlight the difficulty in determining effectiveness of therapy in chronic diseases.