Content

Issues

Research

Permissions

Community Content

Videos and Multimedia

IJMSC Awards

For Authors

Information for Authors

Manuscript Submission

Patient Consent Form

About IJMSC

Aim and Scope

Ethics

Editorial Board

Peer Review

Contact Us

Home

Advertising

IJMSC

1rem

Authors

 Abiodun Emmanuel Akinwuntan, PhD, MPH, Department of Physical Therapy, Georgia Health Sciences University, 987 St. Sebastian Way, Augusta GA 30912; e-mail: 

Prediction of Driving Ability in People with Relapsing-Remitting Multiple Sclerosis Using the Stroke Driver Screening Assessment

Individuals with advanced multiple sclerosis (MS) have complex care requirements and are more likely to use long-term facilities. This study determined the associations between mood and social care–related quality of life (SCRQOL), and health-related quality of life (HRQOL) and examined the association between HRQOL and SCRQOL.

Baseline data from a cohort study were used. Patients completed questionnaires, including the Hospital Anxiety and Depression Scale (HADS), Adult Social Care Outcomes Toolkit (ASCOT), and EuroQOL 5D-5L (EQ-5D-5L) and EQ-Visual Analogue Scale (EQ-VAS). Linear regression analyses were employed to assess the relationships between mood and both outcomes of QOL while controlling for relevant confounding factors (βs; 95% CI). The cross-sectional association between SCRQOL and HRQOL was examined using Pearson correlation coefficients (

A total of 75 patients, with a mean age of 56.1 years and a disease duration of 17.3 years, were enrolled from a long-term care facility in the Netherlands. Results showed that after controlling for confounders, HADS is an independent determinant of ASCOT (βs = −.368; 95% CI, −.581 to −.154) and EQ-5D-5L (βs = −.297; 95% CI, −.507 to −.087). Also, there are significant but weak correlations between ASCOT and EQ-5D-5L (

 = 0.242; 95% CI, .015-.468), between ASCOT and EQ-VAS (

 = 0.230; 95% CI, .003-.457) and between EQ-5D-5L and EQ-VAS (

Mood, especially the depression component, is an important determinant of both HRQOL and SCRQOL in advanced MS. Focusing on mood in health care and social care may contribute to the improvement of QOL in a broader sense.

From the Department of Rehabilitation Medicine, Amsterdam Movement Sciences, MS Center Amsterdam, Amsterdam University Medical Center, Amsterdam, The Netherlands (MBR, EEHW, VG); Department of Neurology, Neurosciences, MS Center Amsterdam, Amsterdam University Medical Center, Amsterdam, The Netherlands (BMJU); Department of Rehabilitation Medicine, Neurosciences, MS Center Amsterdam, Amsterdam University Medical Center, Amsterdam, The Netherlands (MO, MMWL); Nieuw Unicum, Zandvoort, The Netherlands (MO, MMWL). 

: Maaike Ouwerkerk, MSc, Amsterdam UMC, VU Amsterdam, Department of Rehabilitation Medicine, De Boelelaan 1117, 1081 HZ Amsterdam, The Netherlands; 

We would like to thank the study participants.

 All authors declare no conflict of interest.

 This study was funded by Stichting Vrienden van Nieuw Unicum.

Multidisciplinary rehabilitation and symptom management is critical for people with more advanced multiple sclerosis (MS)

 because progression of the disease is associated with complex physical and psychosocial needs and decreased health-related quality of life (HRQOL).

Individuals in the advanced phases of MS have complex care requirements and more extensive use of long-term care facilities, including both health care and social care services.

 Although no uniform definition of advanced MS exists,

 it is characterized by physical impairments, multiple MS symptoms at the same time, and dependence on others for many care and support needs. Because HRQOL does not measure the impact of these complex care and support needs, it has been argued that a wider concept of QOL should be used. Objectives of many social care services should include aspects of QOL beyond HRQOL, such as control over daily life, supporting physical functioning, cleanliness and comfort, personal safety, and social participation and involvement. These aspects are primarily concerned with reducing the effects of impairments on daily life

 and are collectively called social-care–related quality of life (SCRQOL). SCRQOL is defined as “the aspects of an individual’s quality of life that can be affected by social care.” Earlier studies in frail, older adults (ie, age 64 or older) have shown the value of taking both HRQOL and SCRQOL into account when considering QOL.

 Similar to frail, older adults, advanced MS threatens aspects of SCRQOL, like control over daily life, occupation, and social participation and involvement.

 It is therefore important to consider the wider concept of health and take into account both HRQOL and SCRQOL when MS progresses.

Mood disturbances, ie, the presence of anxiety and depression symptoms, are among the most common psychological symptoms in MS, with reported prevalence rates of 22.1% for anxiety and 30.5% for depression.

 It is well known that both MS and mood have a major impact on HRQOL. Reduced HRQOL among people with MS is significantly associated with mood disturbances.

 To our knowledge, no studies have investigated the association between mood and SCRQOL in MS and the association between mood and HRQOL in a population with advanced MS.

Other factors have been found to be related to both mood and HRQOL in MS, including gender,

 These factors may confound the association between mood and HRQOL and SCRQOL.

Understanding the impact of mood on the wider concept of QOL may contribute to developing, improving, and evaluating interventions to enhance QOL in advanced MS

 and can inform professionals providing social care services. Therefore, the aim of this study is to determine the relationship between mood and HRQOL and SCRQOL among patients with advanced MS. First, we examine the association between HRQOL and SCRQOL. Second, we assess whether mood is associated with HRQOL and SCRQOL.

Baseline data were from a previous 1.5-year-long longitudinal cohort study. The study cohort was adults diagnosed with MS and in the later stages of the disease who visited the long-term care facility Nieuw Unicum (NU) in the Netherlands for first-time treatment consultation between January 2018 and January 2020. There were no restrictions for referral to the center based on the Dutch health care system. The participants came from 1 of 3 groups of center-offered care: (1) remote care (ie, an online coaching program including advice and support provided by an MS nursing consultant), (2) multidisciplinary screening (ie, an individually tailored 1- or 2-day assessment by a multidisciplinary team of different medical and allied health care professionals with treatment advice as an outcome), and (3) short-stay care for 1 to 6 weeks to optimize health and functioning and/or to offer relief to caregivers at home. Written informed consent was obtained from all patients.

All individuals with MS who were referred to NU from December 2017 until December 2019 were asked to participate (

). After assessing outliers and after consideration within the research team, 1 participant was excluded because they did not meet the criteria for an advanced MS diagnosis.

Study size estimation was based on the previous longitudinal cohort study. The study was approved by the Medical Research Ethics Committee (METc Amsterdam UMC, VUmc; registration number 2017.519).

Demographic information (ie, age, gender, education) and disease characteristics (ie, type of MS, disease duration, MS severity, motor and cognitive functioning) were collected by telephone or in individual meetings at the center. Education levels were defined using the International Standard Classification of Education 2011.

 contains the sociodemographic and clinical characteristics. Seventy-five participants with a mean age of 56.1 years (SD 9.2) were included. Slightly more women (53%) than men participated. The mean disease duration was 17.3 years (SD 9.6), and the median EDSS score was 7.0 (range 3.5-9.0).

Participants completed questionnaires at study onset and at 3, 6, 12, and 18 months. Questionnaires could be filled in on paper or digitally, and researcher assistance was available.

The Hospital Anxiety and Depression Scale (HADS) was used to measure mood state.

 This self-reported scale consists of 14 items rated on a 4-point Likert scale ranging from 0 to 3 with an anxiety (HADS-A) and depression (HADS-D) subscale (7 items each). This results in a total score between 0 and 42, and a subscale score between 0 and 21, with higher scores indicating higher levels of distress. The HADS has a high sensitivity and specificity in relation to clinical interviews in MS patients.

HRQOL was measured using the EuroQOL 5D questionnaire (EQ-5D) which has 2 parts, the EQ-5D 5-level (EQ-5D-5L) descriptive system and the EuroQOL Visual Analogue Scale (EQ-VAS). 

 The EQ-5D-5L measures health in 5 dimensions—mobility, self-care, usual activities, pain/discomfort, and anxiety/depression—and, by applying the Dutch Tariff valuation set, can be converted into an overall utility index score ranging from −.446 for the worst state to 1 for the mildest state.

 The UK Tariff valuation set was used to compare index scores with other populations.

 The EQ-VAS is a vertical visual analog scale to indicate current overall health. The participant rates their health on a scale ranging from 0, “the worst health you can imagine,” to 100, “the best health you can imagine.” The EQ-5D is found to be valid and reliable in disabled populations, such as individuals with acute stroke and Parkinson disease.

The Dutch version of the Adult Social Care Outcomes Toolkit (ASCOT) was used to measure SCRQOL.

 The preference-based ASCOT was developed to help users broadly evaluate social care services. It quantifies the impact of services in domains of quality of life most affected by social care with 9 items and 8 domains: control over daily life, personal cleanliness and comfort, meals and nutrition, personal safety, social participation and involvement, occupation, accommodation cleanliness and comfort, and dignity. Each domain is scored on a 4-point Likert scale, ranging from ideal to worst imaginable state with high needs.

 Overall utility index scores can be calculated and range from –0.171, “high needs on all domains,” to 1, “the ideal state of SCRQOL.” The Dutch translation of the ASCOT is valid, reliable, and comparable to the original English version.

Severity of MS was measured using a telephone version of the Expanded Disability Status Scale (EDSS).

 EDSS scores range from 0, “no disability,” to 10, “death due to MS.”

Motor and cognitive functioning were assessed using the Functional Independence Measure (FIM).

 The FIM consists of a 13-category motor subscale and 5-category cognitive sub-scale that assess the level of functional independence. Each item is scored ranging from 1, “total assistance,” to 7, “complete independence.” Motor subscores can range from 13 to 91 and cognitive subscores can range from 5 to 35. The FIM is a responsive, reliable, and valid measure of disability.

Pain and spasticity were assessed using a numerical rating scale (NRS).

 Patients rated average pain intensity during the last 24 hours from 0, “no pain,” to 10, “pain as bad as could be.” Patients rated their spasticity over the last 24 hours from 0, “no spasticity,” to 10, “worst possible spasticity.” High test-retest reliability and good validity for pain were observed in patients with rheumatoid arthritis

 and the scale was found to be valid and reliable for spasticity in MS research.

Fatigue was rated using the subjective fatigue subscale of the Checklist Individual Strength (CIS20R-f).

 The subscale consists of 8 items on a 7-point Likert scale. A summary score is the sum of points scored on the 8 items, which results in a score ranging between 8 and 56. Higher scores reflect higher levels of perceived fatigue. Test-retest reliability of the fatigue subscale of the CIS20R-f is good.

Descriptive statistics were calculated for demographic and disease characteristics.

The association between HRQOL and SCRQOL concepts was examined using Pearson correlation coefficients (

) with 95% CI. Correlations were interpreted as weak (< 0.3), moderate (≥ 0.3, < 0.5), or strong (≥ 0.5).

Linear regression analyses were used to investigate whether mood is a determinant of both QOL outcomes. To determine differences in strength of association between mood and QOL constructs, standardized regression coefficients (βs) and 95% CIs were compared. No difference in strength was presumed when the standardized regression coefficient was included in the association 95% CI of the other QOL construct.

In step 1, univariable linear regression analyses were performed to assess the associations between mood and QOL. In step 2, these univariable models were subsequently adjusted for sex, disease duration, MS severity, motor function, cognitive function, pain, spasticity, and fatigue. When the unstandardized regression coefficient of mood changed by more than 15% after adjustment, this variable was considered a confounder.

 In step 3, all relevant confounders from step 2 were added to a final model to determine the uncon-founded association of mood with QOL. In step 4, the standardized regression coefficients of anxiety and mood were compared to study their relative contribution to QOL.

The mutual relationship between the QOL measures appeared to be weak with significant Pearson correlation coefficients between ASCOT and EQ-5D-5L (

 = 0.242; 0.015-0.468), between ASCOT and EQ-VAS (

 = 0.230; 0.003-0.457), and between EQ-5D-5L and EQ-VAS (

 shows the proportion of participants responding to each domain of the ASCOT and EQ-5D-5L.

 shows the results of the regression analyses between mood and SCRQOL and HRQOL. In step 1, significant inverse associations were found between mood, SCRQOL, and HRQOL, but not between mood and personal overall health status. Fatigue was an important confounder in the relationships between mood and SCRQOL and HRQOL, and pain was an important confounder only in the relationship between mood and HRQOL. After correcting for confounding (step 3), mood was still a significant determinant of SCRQOL (βs = –0.368; −0.581 to –0.154) and HRQOL (βs = -0.297; −0.507 to −.087). In step 4, the relative effect of HADS-D was found to be considerably stronger than HADS-A for both SCRQOL and HRQOL.

Our study shows that mood, especially as measured by HADS-D, is inversely and equally associated with both HRQOL and SCRQOL among patients with advanced MS. HRQOL and SCRQOL were only weakly correlated, indicating that these concepts measure different aspects of QOL.

Our results in an advanced MS population are in line with findings in other populations; our data show the mean utility index score of the HRQOL measure EQ-5D-5L is lower than the utility index score of the SCRQOL measure ASCOT. This emphasizes that these QOL measures have different underlying constructs. Comprehensive QOL assessments should therefore include measurement of these different concepts of QOL, especially in people requiring complex social care services.

Although HRQOL and SCRQOL are likely measuring different aspects of QOL, both are strongly associated with mood, with a stronger relative effect for depression. An increase of 1 standard deviation in mood leads to a 0.474 standard deviation decrease in SCRQOL index score and a 0.428 standard deviation decrease in HRQOL index score. These results are in line with a recent systematic review that shows mood is a strong risk factor for a lower HRQOL in MS,

 although none of the studies consisted only of people with advanced MS.

A relationship was found between SCRQOL, as measured with ASCOT, and mood, measured by different instruments, in frail, older adults in 3 studies.

 One study showed that the ASCOT was correlated with the 12-Item Short Form Health Survey mental health component (

 = 0.50) in frail adults aged 65 years or older in the Netherlands.

 Other studies conducted in England found a correlation with the Geriatric Depression Scale (

 = –0.69) in social care users 70 years of age and older,

 and with the General Health Questionnaire (

 = –0.58) in people 65 years or older who are using social care services.

 Next to HRQOL, generating more knowledge about SCRQOL in advanced MS and its related factors (eg, mood) may contribute insight into the broader picture of QOL for this population.

No significant relationship was found between mood and HRQOL via EQ-VAS assessment. Since the relationship with the other QOL instruments was so robust, we argue that this may be because overall health status is a more encompassing construct than general QOL, which is not fully captured by the EQ-5D-5L and ASCOT separately.

According to the World Health Organization, QOL is a broad concept affected in a complex way by a person’s physical health, psychological state, level of independence, social relationships, personal beliefs, and their relationship to salient features of their environment.

 As such, not all aspects of this definition can be captured in the QOL instruments (eg, personal beliefs, relationship to salient environmental features). In addition, in an online survey, 97% of individuals with MS (n = 237) felt that the EQ-5D-5L did not capture all important aspects of well-being that added significant burden to their HRQOL.

 They cited fatigue, balance, and dizziness, which are not included in the ASCOT. Therefore, other factors might play a more prominent role in judgment of personal overall health status.

In our study, fatigue confounded the association of mood with HRQOL as well as the association of mood with SCRQOL. Pain had only a confounding effect in the association of mood with HRQOL. Although it is known that fatigue and pain are associated with HRQOL and mood in individuals with MS,

 the role of these factors in SCRQOL has not been investigated. A possible reason that pain was not a confounder in the association of mood with SCRQOL is that the EQ-5D-5L specifically assesses the level of pain/discomfort, while the ASCOT does not. Pain and fatigue are important factors in QOL, but their influence depends on the QOL construct used. These factors, next to mobility and upper-extremity function, are highlighted as the key symptoms in MS symptom management and rehabilitation.

No confounding effects of sex, disease duration, severity of MS, cognitive functioning, or spasticity were found in the association between QOL and mood. In the literature, findings about their role in mood and QOL are inconsistent; some studies

 found an association between the variables and mood or QOL, while other studies did not.

Patients in our cohort have varying levels of physical impairment (EDSS ≥ 3.5). However, most have several complex needs and utilize both health and social care services to maintain their QOL, which likely indicates an advanced stage of MS. Considering the characteristics of advanced MS in the literature, it can be argued whether advanced MS should be defined only when physical impairment is present or whether the definition of advanced MS should also include more intangible characteristics like psychosocial needs.

Another limitation is the collection of data from 1 center; the study population may not be representative of the larger population of individuals with advanced MS.

The present study is cross-sectional and did not have this capability, but future research should use longitudinal assessments to study time-dependent cause-and-effect relationships, as it is likely that QOL and mood are not stable but vary over time.

In addition, future research could consider other instruments that focus on aspects of QOL beyond health (eg, the ICEpop CAPability measure for Adults

) and on HRQOL other than the EQ-5D, such as disease-specific instruments, especially since reviewers of the EQ-5D have reported the low coverage of QOL domains relevant to MS.

 However, disease-specific instruments typically do not incorporate utility scores, which are important when conducting economic evaluations of future interventions in health care.

In summary, mood was inversely and equally strongly associated with both HRQOL and SCRQOL, which were found to be distinct but complementary concepts of QOL. Depression was considerably more strongly associated with both HRQOL and SCRQOL than anxiety. Fatigue was a confounder in both models, whereas pain only distorted the association between mood and HRQOL. Focusing on mood in both health care and social care for individuals with advanced MS may contribute to improved QOL in a broader sense.

Mood Associated With Health- and Social Care–Related Quality of Life in Patients With Advanced Multiple Sclerosis

The BeCare MS Link mobile app collects data as users complete different in-app assessments. It was specifically developed to evaluate the symptomatology and neurologic function of patients with multiple sclerosis (MS) and to become a digital equivalent of the Expanded Disability Status Scale (EDSS) and other standard clinical metrics of MS progression.

Our research compared EDSS scores derived from the BeCare MS link app to EDSS scores derived from neurologist assessment for the same cohort of 35 patients diagnosed with MS. App-derived data were supplied to 4 different machine learning algorithms (MLAs) with an independent EDSS score prediction generated from each. These scores were compared with the clinically derived EDSS score to assess the similarity of the scores and to determine an accuracy estimate for each.

Of the 4 MLAs employed, the most accurate MLA produced 19 EDSS score predictions that exactly matched the clinically derived scores, 21 score predictions within 0.5 EDSS points, and 32 score predictions within 1 EDSS point. The remaining MLAs also provided a relatively high level of accuracy in predicting EDSS scores when compared with clinically derived EDSS, with over 80% of scores predicted within 1 point and a mean squared error with a range of 1.05 to 1.37.

The BeCare MS Link app can replicate the clinically derived EDSS assessment of a patient with MS. The app may also offer a more complete evaluation of disability in patients with MS.

From the Department of Neurology, Yale School of Medicine, Yale University, New Haven, CT (SS, CL); Yale New Haven Hospital, New Haven, CT (SS, CL); BeCare Link, Rumson, NJ (NR, LR); and Weill Cornell Medicine, New York, NY (TV). 

Sharon Stoll, DO, Department of Neurology, Yale School of Medicine, Yale University, New Haven, CT 06510; 

Medical writing support was provided by Mark Elms, PhD, of PharmaGenesis London, London, UK, and Adeline Rosenberg, MSc, of Oxford PharmaGenesis, Oxford, UK, and was funded by BeCare Link, LLC.

 Dr Stoll has served on scientific advisory boards for Bristol Myers Squibb; F. Hoffmann-La Roche Ltd; Forepont Capital Partners; Genentech, Inc; Horizon Therapeutics, Inc; and TG Therapeutics, Inc; received research support from BeCare Link, LLC and MedDay Pharmaceuticals SA; has received compensation for consulting services, served on scientific advisory boards, and received speaker honorarium for Alexion Pharmaceuticals, Inc; Biogen, Inc; Bristol Myers Squibb; EMD Serono, Inc; Horizon Therapeutics, Inc; Novartis AG; Roche’s Genentech, Inc; and Sanofi Genzyme; is CEO of Global Consultant MD; and has served on the steering committee of Horizon Therapeutics, Inc and Roche’s Genentech, Inc. Dr Lichtman has received personal compensation for consulting, serving on a scientific advisory board, speaking, or other activities with Amgen, Inc; holds stock and/or stock options in BeCare Link, LLC, which sponsored this research; and has received personal compensation for consulting, serving on a scientific advisory board, speaking, or other activities with Amgen, Inc. Dr Noah Rubin has no conflicts of interest to declare. Dr Larry Rubin has received personal compensation for consulting, serving on a scientific advisory board, speaking, or other activities with BeCare Link, LLC; has received compensation for serving on the board of directors of BeCare Link, LLC; and has received royalty, license fees, or contractual rights payments from BeCare Link, LLC. Dr Vartanian has received personal compensation for consulting, serving on a scientific advisory board, speaking, or other activities with Biogen, Inc; Novartis AG; Roche’s Genentech, Inc; and Sanofi Genzyme.

 Funding for this study was provided by BeCare Link, LLC.

Multiple sclerosis (MS) is a chronic neurological disorder affecting more than 2.5 million individuals globally and approximately 1 million individuals in the United States alone.

 Individuals with MS can experience a broad symptomatology including pain, fatigue, depression, memory loss, and difficulty ambulating. There are effective medications available and others are being developed to help to prevent disease progression and associated disability.

To determine when treatment regimens must be modified to optimize care, physicians rely on patient-reported symptoms and on physical changes detected through clinical examination. There are standard disability scales used by MS specialists to help quantify disability progression and neurologic dysfunction.

 The gold standard of these metrics is the Expanded Disability Status Scale (EDSS),

 but this scale and other clinical assessments are time-consuming and cumbersome.

 Reliance on the currently available disability status scores, which have limited sensitivity, combined with infrequent in-person visits, often delay treatment modifications. This is because decline in neurologic function and cognition is not typically detected in patients at the first sign of progression.

During the COVID-19 pandemic, most routine and nonessential health care appointments were switched to telehealth or placed on hold.

 The pandemic has therefore revealed the potential for telemedicine in the care of numerous chronic conditions, such as MS, in order to protect vulnerable individuals from harm, reduce the burden of clinical visits, and improve the early detection of degenerative or progressive disease.

The BeCare MS Link app—compatible with both Android and Apple operating systems—was developed to evaluate the symptomatology and neurologic function of patients and act as a digital equivalent to the clinically derived EDSS (ie, EDSS scores as determined by neurologists). By logging into the app, a user establishes a session in which they perform certain app-directed assessments to evaluate their MS symptoms, and the resultant data are collected and analyzed by machine learning algorithms (MLAs) to produce an app-derived EDSS score. The measurements and data obtained from these assessments correspond to the neurologist-determined EDSS, which is currently the most widely used MS assessment tool.

 Although the EDSS is primarily based on ambulation,

 the app-based assessments evaluate the functionality of a range of nervous system components, including pyramidal, cerebellar, sensory, visual, mental, and motor processes. The app also collects information on diet, mood, memory loss, missed workdays, limiting symptoms, demographics, comorbidities, disease severity, and prior MS treatments. The BeCare MS Link app, as well as being a digital equivalent to a clinically derived EDSS, may provide a more complete evaluation of disability in patients with MS.

The primary objective of this clinical trial was to validate the EDSS scores derived from the BeCare MS Link app compared with the clinically derived EDSS scores of the same patients, as determined by neurologists at the Weill Cornell Institute. The secondary objective was to generate an EDSS score for individual patients that mirrors the EDSS score obtained via neurologist assessment. These objectives differ as the former is a comparison of app versus clinical measurements, while the second measurement is to test the accuracy of the MLAs used by the app.

By using the BeCare MS Link app, patients can gain greater control over the management of their disease. Patients can take the tests on the BeCare MS Link app at home and as often as they choose, meaning that neurologic deterioration can be detected earlier, and that the effectiveness of drug therapy can be followed more quantifiably and longitudinally. Ultimately, this minimizes the need for higher levels of care because physicians may be alerted to problems well before the need for an emergency department visit or inpatient admission.

In total, 35 patients were enrolled in the study over a period of 1 year from the time of institutional review board approval. Patients had the opportunity to complete the study up to 3 months after enrollment, but in most cases enrollment and study completion took place on the same day. Research coordinators at the clinical site screened eligible patients and obtained informed consent. The trial was registered on ClinicalTrials.gov as NCT04281160.

The inclusion criteria consisted of patients aged 18 to 75 years with clinically definite MS based on the revised McDonald criteria.

 Patients needed to have mild to moderate disability in 1 or more of the modalities assessed by the BeCare MS Link app and a baseline clinically derived EDSS score of 0.0 to 6.5. Patients had to be able to speak English and to provide informed consent. Patients were excluded if they had a clinically derived EDSS score higher than 6.5, were unable to provide consent, had congenital or traumatic loss of index finger or thumb, had impaired mobility or function owing to rheumatologic or other illnesses, or had neurologic impairment due to an illness other than MS.

The details of the study protocol were reviewed with each patient and all questions were addressed. If a patient agreed to enroll in the study, an informed consent form was signed. The informed consent was kept in a study binder and a digital copy was made and stored on a secure server in compliance with the Health Insurance Portability and Accountability Act.

At their clinical visit, the patients were evaluated by MS neurologists at the Weill Cornell Institute and a clinically-derived EDSS score was obtained. Additionally, patients were trained to use the BeCare MS Link app on their mobile devices and obtained their first set of scores by completing a sufficient number of app assessments. There were 11 assessments in the app: Arm Elevation, Path, Transcription Test, Contrast Sensitivity, Timed 25-Foot Walk (T25-FW), 6-Minute Walk, Time Up and Go (TUG), Tap Task, Stroop Test, Code Test and Memory Test. These assessments included measurements of motor mobility, fine motor function, upper extremity coordination, auditory comprehension, time to walk predetermined distances, visual acuity, visual tracking of objects, cognitive function, memory, and vibration sense (

; supplemental material is available in a PDF at the end of the online article). Gold standard measurements for determining mobility function in MS, the T25-FW and TUG, were compared with similar evaluation modalities via the app in order to assess the agreement between clinician-derived and app-derived measurements.

 Additional diagnostic procedures measured by the app were compared with the current clinical gold standard for disability in MS, the Kurtzke EDSS.

 The app questionnaires also collected data on the duration and severity of disease, success of various therapies, patient demographics, quality of life, mood, dietary habits, bowel and bladder function, and disease comorbidity (

). Patients completed clinical assessments and the 11 app-based assessments in 1 visit. The app-based assessments were repeated 3 to 7 times to eliminate learning variation.

The app-based assessments were designed to have a unique sequence of steps that the user completes to successfully perform the assessment. Each assessment corresponds to a standard functional test that a clinician could perform during a neurological assessment. The app uses the accelerometer, gyroscope, or magnetic sensor in the patient’s phone to determine when they take a step, make a transition to a different position or orientation, or move their arms.

Users only completed some of the assessments during a particular session and completed additional assessments during later sessions. After an assessment was completed, the data corresponding to the steps of the assessment were sent to the BeCare Link cloud, provided an internet connection from the mobile device to the BeCare Link cloud was available. After the assessment data were stored in the BeCare Link cloud, the MLAs could produce an app-derived EDSS score.

Four different MLAs—Huber regression, linear regression, random forest, and naïve Bayes—were used to analyze the app-derived data and calculate app-derived EDSS scores. Each MLA predicted an independent EDSS score. The app-derived EDSS scores were then compared to the clinically derived EDSS score to determine whether the app could replicate the clinically derived EDSS scores, as well as to verify the accuracy of the app-derived EDSS scores.

The analysis was based upon extracting certain key features from the app-derived raw data that were correlated with clinically obtained EDSS scores. A key feature was one that was an essential determinant of the EDSS score from the point of view of the MLAs.

Key features were generated from the raw data that modeled the state transitions associated with completing each assessment. For example, the TUG assessment was modeled as a sequence of state transitions: (1) sitting to standing, (2) walking, (3) turning around, (4) walking back to the chair, and (5) standing to sitting. Therefore, each assessment produced hundreds of data points that were used to calculate MLA-derived scores.

Another example is the T25-FW assessment for which the app-derived raw data were obtained from statistical measures such as the median time taken between each step and the aggregate variation. Multiple completed instances of the same chosen assessment were aggregated by taking order statistics for each feature, such as the median or 25th percentile. These order statistics became the feature values. This raw data had high dimensionality, which made it difficult to analyze, so principal component analysis (PCA) was used to reduce its dimensionality.

The MLAs were initially trained using a set of control participants (ie, people who do not have MS and no evidence of disability). Using only these control participants, a z-score was computed for each feature value. Data points that exceeded 2 standard deviations from the control population mean were removed as statistical outliers. When collecting data from sensory inputs, it is standard to filter out results that are attributable to either noisy measurements or the participant misunderstanding the directions of the assessment. Using this filtered set of input data, the mean 

 were computed, which allowed the computation of z-scores for each participant and feature value, using the standard calculation (x-

 is the participant’s feature value. These z-scores will be referred to as the feature statistics.

The z-scores for certain feature values had small standard deviations owing to the limited range of data. For each participant, feature threshold tests were calculated, each of which was a binary variable: 1 if the corresponding feature statistic was considered evidence of disability and 0 otherwise.

Each feature was given a threshold value that determined whether a feature statistic was evidence of disease. For instance, a participant with a high feature statistic for the time between steps was assigned a 1 for the corresponding feature threshold test, given that difficulty walking is a symptom of disability. In other cases, low feature statistics were indicative of disability; for example, the coding assessment had a feature statistic corresponding to the number of numeric codes mapped correctly to icons.

Nevertheless, the results obtained from all models were used because the dispersion of results can be used to generate a confidence statistic on the accuracy of the reported score.

Given that other regression algorithms such as random forest and linear regression fail owing to missing data, feature threshold tests of 0 were substituted for those who had not completed the corresponding assessment. Essentially, participants were considered to have no disability unless there was evidence of it.

Overall, only 6 of the 11 BeCare MS Link app-based tasks were included in this analysis (see Results section). All together, these 6 assessments had 16 features, which were too many, relative to the number of eligible patients (N = 35), to perform direct regressions. The number of features is the same as the number of regression coefficients. When the number of regression coefficients is too large relative to the number of samples, the resulting model may become overfitted.

To avoid model overfitting, we sought to reduce the 16-feature threshold tests (regression coefficients) to a smaller number of features without losing too much information. We accomplished this in 2 different ways: We combined feature threshold tests according to (1) assessment and (2) functional system.

For (1), provided the corresponding assessment was completed, the mean value of each threshold test for each assessment was calculated, or was given a regressor value of 0 if the assessment was incomplete. This yielded 6 regressors.

For (2), assessments were grouped by the functional system they measured, allowing for overlap. For each functional system, the regressor was the mean of the means of each completed assessment associated with that functional system. A patient who did not perform any of the assessments associated with a functional system was given a regressor value of 0. Thus, this second method yielded 5 total regressors.

A machine-learning model to output the mean EDSS score was trained for each of these 2 sets of regressors. The model for each set of regressors used linear regression with the Huber loss function with a threshold parameter equal to 1.5 so that the regression was not dominated by outlier effects.

 The results of the models for both sets of regressors were then averaged to produce the final EDSS score for each independent MLA. The performance of the model was assessed with leave-one-out cross-validation, wherein a prediction for each patient was obtained by a model trained on all the other patients.

Overall, 6 of the assessments had key features with high predictive value in determining EDSS scores: TUG, Tap Task, Path, T25-FW, Code Test, and the Contrast Sensitivity tests (

). Although the clinical trial design was intended for all patients to perform the clinical assessments and the 11 BeCare MS Link app-based assessments, many patients were unable to stay for the time required to complete all app assessments; therefore, only 6 of the 11 BeCare MS Link app assessments were included in this analysis. The remaining 5 app assessments were excluded because they did not have sufficient participation by the cohort of patients.

To compare the app-derived EDSS score calculated from the 6 app-based assessments outlined in Table S1, the clinically derived EDSS score was measured for all 35 participants. Their clinically derived EDSS scores ranged between 0 and 3.5 (

), with 0.0 (48.6%) and 1.0 (20.0%) being the most common scores.

The clinically derived EDSS score for each patient was then compared to each of the app-derived EDSS scores that were independently calculated by the 4 different MLAs. The most accurate was the Huber regression. Nineteen patients (54.3%) had scores that were predicted exactly, 21 patients (60.0%) had scores that were predicted within half a point, and 32 patients (91.4%) had scores that were predicted within 1 point (

). The mean squared error of the Huber regression-derived EDSS score compared with the clinically derived EDSS score was 0.86 (Table S3).

The other 3 MLAs also provided a relatively high level of accuracy in predicting EDSS scores when compared with the clinically derived EDSS scores with more than 80% of scores predicted within 1 point and a mean squared error in the range of 1.05 to 1.37 (Table S3).

There are many benefits to using mobile apps to monitor patient health outside the clinic setting, and the COVID-19 pandemic has highlighted this.

 In the literature, studies have analyzed the use of mobile apps to monitor chronic conditions including congestive heart failure, diabetes, and MS.

 To the best of our knowledge, there is yet to be a mobile app capable of evaluating the neurological and physical functioning of patients. Other apps designed to monitor MS rely on patient-reported symptoms and they function more specifically as treatment trackers.

 The BeCare MS Link app is novel in its ability to measure neurologic dysfunction objectively, rather than relying on patient-reported symptoms, and in its use of MLAs to calculate EDSS scores that can mimic clinically derived EDSS scores. Additionally, the BeCare MS Link can reduce the problematic inter-rater variation between 2 clinician assessments of the same patient,

 benefiting outcomes in clinical care as well as in clinical trials.

Patient access to quantifiable assessments that can be performed at home via a mobile app will result in a much greater frequency of data collection. The BeCare MS Link’s MLAs can detect subtle changes in patient performance so that their treating physicians can intervene earlier than with the data derived from standard clinical scoring, which is collected on an infrequent basis. This is a marked improvement in patient monitoring of MS, which has a relapsing-remitting course. As the BeCare MS Link measurements are made every time the app is used, it is possible to study the relationship between real-time assessments, such as medication changes, dietary changes, and rehabilitation, with changes in the BeCare MS Link-derived EDSS score over time.

The data collected by the BeCare MS Link app are not only available to the patient and their physician but can also be used by the MLAs as training in order to improve the app’s output in future EDSS calculations. The MLAs learn by updating their internal state each time they are presented with EDSS-related data. This way the accuracy and reproducibility of the app-derived EDSS scores are intended to improve with time, reducing the variability in EDSS scores that is often seen in the clinic. This analysis used MLAs because of the availability of neurologist-supplied EDSS scores that could be used in the future to further train the algorithms, thereby potentially minimizing computed output errors.

 Historically, assessment of disease progression has been based on the degree of impairment as compared with either a population standard or the patient’s historical performance. This technique suffers from a learning effect as patient scores improve after they become accustomed to the testing.

 We have therefore trained our MLAs to account for this “drift” in the data owing to familiarity.

The app-derived EDSS scores can both objectively validate the patient report of clinical improvement or disability progression and help to discern true improvement from both placebo and learning effect. Changes in app-derived EDSS scores can then be reported to the treating physician who can determine whether an earlier follow-up might be beneficial. It would also be possible for investigators, insurers, and health care systems to study the large and comprehensive collection of data to assess comorbidities and the success of new and established treatments and rehabilitation techniques. Ultimately, because of the wide breadth of data collected by the app, it will be possible to learn much more about MS.

This study does have limitations. Despite confirming the equivalence of clinically derived and app-derived EDSS scores, a number of app-based assessments were excluded from the final analysis because many patients did not complete app-based assessments due to time constraints. In future studies, we hope to perform further analyses on the utility of the BeCare MS Link app, which will include these assessments, and we hope to categorically determine whether the app-based assessments provide a more rounded and thorough evaluation of MS than the clinician-derived EDSS score. Other limitations include the small size of the cohort and that the trial was conducted at a single site. Finally, the patients in this trial did not have EDSS scores greater than 3.5, so future work should focus on and include patients with higher levels of clinician-assessed disability. We hope to expand the size of the cohort to include patients with a range of disability levels and conduct a multinational and multicentered trial to fully characterize the accuracy and utility of the BeCare MS Link app as a digital equivalent, or potential replacement, of the clinician-derived EDSS score.

The BeCare MS Link app can replicate the assessment of a patient with MS by a clinically derived EDSS. The goal of the BeCare MS Link app is to improve the well-being of people with MS through the accurate assessment of neurologic function, which can be used to direct treatment decisions by the patients themselves as well as by their health care providers. The clinically derived EDSS score is heavily weighted for ambulation whereas assessments made by the app via MLAs include evaluations of arm movement, cerebellar function, cranial nerve function, and sensory function, offering a more complete evaluation of disability in patients with MS. These results show that the BeCare MS Link app has utility for both physician and patient to monitor stability, improvement, or progression and support the use of the app in clinical trials to acquire significantly more information at a greater number of time points with marked reduction in cost and reduced inter-rater variability in multicenter trials.

Validated, Quantitative, Machine Learning-Generated Neurologic Assessment of Multiple Sclerosis Using a Mobile Application

The Multiple Sclerosis Resiliency Scale (MSRS) was designed to assess factors connected to resilience when facing MS-related challenges. Although the MSRS has demonstrated good internal consistency and construct validity, its test-retest reliability has yet to be established. Identifying the minimal detectable change (MDC) of the scale will also improve its utility as an outcome measure for resilience-based interventions. This study aimed to determine the test-retest reliability and MDC of the MSRS.

Participants were 62 persons with MS who completed the MSRS twice, with a mean ± SD of 16.60 ± 3.97 days (range, 14–30 days) between assessments. Test-retest reliability was evaluated using a 2-way, random-effects, single-measurement intraclass correlation coefficient (ICC), with agreement between time 1 and time 2 visualized with a Bland-Altman plot. The MDC was calculated using the standard error of measurement with a 95% CI.

At time 1, the mean ± SD MSRS score was 77.19 ± 11.97 (range, 45.83–97.00); at time 2, the mean ± SD score was 76.38 ± 12.75 (range, 46–98). The MSRS total score had good test-retest reliability (ICC = 0.88), with the subscale ICCs ranging from 0.77 (MS Peer Support) to 0.93 (Spirituality). The MDC for the total score was 11.95.

These findings suggest that the MSRS has good test-retest reliability and that persons with MS with a difference of 12 points or more between assessments have experienced a reliable change. The results support the utility of the MSRS as a potential outcome measure for MS-related resilience.

From the Mandell Center for Multiple Sclerosis, Mount Sinai Rehabilitation Hospital, Trinity Health Of New England, Hartford, CT, USA (ESG, LON, JAR, ACL); Department of Rehabilitative Medicine (ESG, LON, JAR) and Department of Medical Sciences (ESG, JAR), Frank H. Netter MD School of Medicine at Quinnipiac University, North Haven, CT, USA; Department of Neurology, University of Connecticut School of Medicine, Farmington, CT, USA (ESG); Multiple Sclerosis Center of Excellence West, Veterans Affairs, Seattle, WA, USA (APT); Rehabilitation Care Service, VA Puget Sound Health Care System, Seattle, WA, USA (APT); Department of Rehabilitation Medicine, University of Washington, Seattle, WA, USA (APT); Department of Family Medicine and the Center for Quantitative Medicine, University of Connecticut Health Center, Farmington, CT, USA (TA); Ferkauf Graduate School of Psychology, Yeshiva University, Bronx, NY, USA (FWF); and Holy Name Medical Center Multiple Sclerosis Center, Teaneck, NJ, USA (FWF). 

: Elizabeth S. Gromisch, PhD, 490 Blue Hills Ave, Hartford, CT 06112, USA; 

: elizabeth.gromisch@trinityhealthofne.org.

The authors thank Dotty Wakefield, MS, for coding and management of the REDCap data collection and Aprille Gangi, Carly Gillen, and Amelia Kroschwitz for their assistance with data verification. The PDDS scale is provided for use by the North American Research Committee on Multiple Sclerosis (NARCOMS) Registry (http://www.narcoms.org/pdds). NARCOMS is supported in part by the Consortium of Multiple Sclerosis Centers (CMSC) and the CMSC Foundation.

 The authors declare no conflicts of interest.

 The views and opinions expressed in this article are those of the authors and do not necessarily reflect those of the US Department of Veterans Affairs.

 grant from St Francis Hospital and Medical Center. Dr Gromisch is a Harry Weaver Scholar of the National Multiple Sclerosis Society.

 The results of this study were presented at the Consortium of Multiple Sclerosis Centers 2021 annual meeting; October 25-28, 2021; Orlando, Florida.

Persons with multiple sclerosis (MS) can be confronted with several challenges related to their disease, to which they can have any number of responses. Among persons with MS, resilience has been described as “bouncing back,”

 Resilience can have several positive effects on the lives of persons with MS. For example, high resilience has been associated with greater levels of physical activity and participation, better diet and psychological well-being, and lower levels of fatigue.

 On the opposite end of the spectrum, persons with lower levels of resilience may experience more psychological distress.

Measuring resilience to MS-related challenges provides the opportunity for clinicians to identify relative strengths as well as areas of needed support, both of which can inform targets for intervention. The Multiple Sclerosis Resiliency Scale (MSRS) is a multidimensional measure based on the conceptual framework that MS-related resilience is an interaction between protective factors and risk factors, which include psychological, physical, and social components.

 The 25-item MSRS produces a total score as well as 5 subscale scores: Emotional and Cognitive Strategies (13 items), Physical Activity and Diet (3 items), MS Peer Support (2 items), Support From Family and Friends (5 items), and Spirituality (2 items).

Abiodun Emmanuel Akinwuntan, PhD, MPH

Articles

Latest Updated Articles