Validation of the NARCOMS Registry

Depending on the population studied, tremor affects 25% to 58% of patients with multiple sclerosis (MS).^{1 2} The areas affected by tremor are variable. In a study of 100 patients selected from an MS clinic, 58 were affected on examination, with tremor affecting the upper limbs in 56%, lower limbs in 10%, head in 9%, and trunk in 7%.¹ All tremors were postural or kinetic.

Tremor, limb ataxia, or impaired balance affects fewer than 15% of patients at MS onset³; however, cerebellar involvement at onset is a negative prognostic factor for disability progression.⁴ Among individuals with secondary progressive MS, severe cerebellar involvement is also associated with an increased risk of respiratory impairment (odds ratio [OR], 6.24; 95% confidence interval [CI], 1.71–22.8).⁵ Tremor secondary to MS can be severely disabling, limiting independence and the ability to perform activities of daily living. Treatments for MS-related tremor include physical therapy, pharmacotherapy, and neurosurgery, but their effectiveness is limited.⁶ Thus tremor and ataxia are clinically important issues.

Despite their negative impact and limited treatments, tremor and ataxia in MS remain relatively understudied. Several methods exist for assessing tremor. Rating scales focus on characteristics of the tremor such as location and severity. Performance-based measures include objective assessments such as the Nine-Hole Peg Test (NHPT), target board tests, and drawing tests.^{7 8} These measures do not assess the functional impact of tremor; thus functional examinations such as the Performance-Based Test of Function, TEMPA (Test d'Evaluation de la performance des Membres Supérieurs des Personnes Agées), and Fahn Tremor Rating Scale focus on tasks important to daily function such as drinking water from a nearly fully glass or using a soup spoon to drink liquid from a bowl.^9–11 Finally, functional questionnaires assess the effect of tremor on activities of daily living and are particularly important to capture the patient perspective.⁷ Patient-reported outcomes are of increasing interest, but often such functional questionnaires consist of multiple items that may not be suitable for studies assessing the impact of multiple symptoms. The North American Research Committee on Multiple Sclerosis (NARCOMS) Registry dataset emphasizes the patient-perceived impact of symptoms on daily function. NARCOMS developed a single-item questionnaire, the Tremor and Coordination Scale (TACS), to assess the impact of tremor and impaired coordination on daily activities. The TACS has not been validated; thus the current study aimed to assess the criterion and construct validity of the TACS.

Methods

As detailed elsewhere, we previously conducted a study to validate the subscales of the Performance Scales.¹² Briefly, we enrolled 44 individuals with definite MS,¹³ aged 18 to 55 years, with an Expanded Disability Status Scale (EDSS) score of 0 to 6.5, and who were capable of completing the study questionnaires. Because this was part of a larger study validating the 6-Minute Walk (6MW) test,¹⁴ we excluded patients with comorbid heart disease or respiratory disease to ensure participant safety. All participants underwent a standardized examination to determine the EDSS and Multiple Sclerosis Functional Composite (MSFC) scores. They completed questionnaires including the NARCOMS TACS, Performance Scales, and Multiple Sclerosis Quality of Life Inventory (MSQLI). The study was approved by the institutional review board of the Cleveland Clinic. All participants provided written informed consent.

NARCOMS Tremor and Coordination Scale (TACS)

The NARCOMS TACS was developed as a single question, scored ordinally from 0 to 5, as follows: 0 (normal), 1 (minimal), 2 (mild), 3 (moderate), 4 (severe), or 5 (total disability). This scoring system makes it similar to the Performance Scales (Appendix 1).

Measures Used for Validation

Validity refers to the degree to which a measurement actually measures what it purports to measure, and has multiple aspects including face validity, criterion validity, and construct validity. The measures used for validation in this study are listed in Table 1.

Description of instruments used in this study to establish criterion or construct validity

Criterion Validity Measures

Criterion validity can be assessed by correlating the measurement of interest with an external criterion or gold standard measure of the phenomenon being studied. The EDSS is an ordinal disability measure based on a comprehensive neurologic examination, and is scored from 0 (no disability) to 10 (death due to MS).¹⁵ The EDSS is derived from scores on visual, brainstem, pyramidal, sensory, cerebellar, sphincter, and mental functional systems. The cerebellar Functional System Score (FSS) measures cerebellar dysfunction affecting the upper and lower extremities, as well as gait, but provides a single score. Thus it does not distinguish between cerebellar dysfunction in the limbs and gait impairment. The MSFC has three components.¹⁶ The Timed 25-Foot Walk (T25FW) measures lower-extremity function and ambulation. The NHPT measures upper-extremity function. Finally, the Paced Auditory Serial Addition Test (PASAT-3) measures cognition. We normalized the component and summary z scores to the National Multiple Sclerosis Society Task Force database, for several reasons, including the small sample size, the cross-sectional study design, and the ability to compare our results with those of other studies. Alusi et al.⁷ evaluated ways to measure tremor in MS and reported that the NHPT was a useful objective measure of upper-extremity tremor, and the NHPT was used as an outcome measure in a clinical trial of cannabis for the treatment of tremor.¹⁷ Therefore, we assessed the criterion validity of the TACS by comparing it to the NHPT using Spearman rank correlations with casewise deletion. The cerebellar FSS captures tremor, ataxia, and other problems with coordination; thus we also examined the correlation of the TACS with the cerebellar FSS.

Construct Validity Measures

Construct validity measures the extent to which the measure of interest correlates with measures of other variables in hypothesized ways. This includes convergent validity, in which variables that are related to each other are expected to be moderately or highly correlated; and divergent validity, in which variables that are not expected to be related are poorly correlated.

Performance Scales is a validated patient-reported measure that uses a single question to assess each of eight domains: mobility, bowel/bladder, fatigue, sensory, vision, cognition, spasticity, and hand.¹⁸ Each subscale except mobility is scored ordinally as follows: 0 (normal), 1 (minimal), 2 (mild), 3 (moderate), 4 (severe), or 5 (total disability). Mobility is scored from 0 to 6. Total scores for the Performance Scales may range from 0 to 41. The MSQLI is a disease-specific quality of life (QOL) measure for MS and includes the 36-item Short Form Health Status Survey (SF-36), a generic QOL measure. The SF-36 has two summary scales, the Physical Component Score (PCS) and the Mental Component Score (MCS). The PCS-36 measures QOL related to physical health, while the MCS-36 measures QOL related to mental health. They are scored from 0 to 100, with a mean of 50 and a standard deviation of 10; higher scores indicate better health. To assess construct validity, we correlated the TACS with hand function and mobility (from the Performance Scales), overall disability as measured by the EDSS, age, and body-mass index (BMI). We expected moderate correlations between the TACS and hand function, the EDSS, and the PCS-36 (convergent validity). We expected low correlations between the TACS and age, BMI, and the MCS-36 (divergent validity).

Statistical Analysis

Categorical variables were summarized as frequency (percent). Continuous variables were summarized as mean (standard deviation; SD) or as median (interquar-tile range; IQR), as appropriate. Associations between variables were assessed using chi-square tests, Fisher exact tests, t tests, and Wilcoxon signed rank tests as appropriate based on cell sizes and the distribution of the variable.

We estimated that enrolling 40 participants would provide 80% power to detect a correlation of 0.44 with the significance level α = .01. We considered a P value of <.01 to be statistically significant, to adjust for multiple comparisons. Statistical analyses were conducted using SAS version 9.3.1 (SAS Institute Inc, Cary, NC).

Results

Thirty-five participants (79.5%) were women, and 38 (86.4%) were white (Table 2). Most participants had a relapsing-remitting course. Their mean ages at symptom onset and diagnosis were consistent with those reported in other American MS populations.¹⁹ Based on their median (IQR) score on the EDSS, they were mildly to moderately disabled (Table 3). Participants' median scores were highest in the pyramidal and sensory functional systems.

Characteristics of study participants by Tremor and Coordination Scale (TACS) score

Participant scores on various measures

All participants completed the TACS. The median (IQR) score on the TACS was 1 (0.5–2.0). Seventy-five percent of participants reported some disability due to tremor or incoordination, including 10 (22.7%) who reported moderate or severe symptoms. No participant reported a TACS score of 5 (total disability). After dichotomizing the study population into those who reported any disability secondary to tremor or incoordination and those who reported no disability, we compared their demographic characteristics. The two groups did not differ with respect to age, sex, or race (Table 2). Although the difference did not reach statistical significance, it appeared that participants reporting tremor were more likely to have a secondary progressive than a relapsing-remitting disease course. Age at onset, age at diagnosis, and disease duration did not differ between the groups.

The median (IQR) for the cerebellar FSS was 1 (0–2) (Table 3). The TACS score correlated moderately with the cerebellar FSS (r = 0.51; 95% CI, 0.24–0.70). The mean (SD) z score on the NHPT was 0.18 (0.97). The TACS score also correlated moderately with the NHPT (r = −0.51; 95% CI, −0.70 to −0.29). This supports the criterion validity of the TACS. Graphical and regression analysis showed that the TACS was reasonably linearly related to the NHPT (NHPT = 0.72 – 0.36 × TACS), although this was less evident at the upper range of the scale. Very few participants reported a TACS score of 4 (n = 3), and none reported a score of 5, contributing to a lack of linearity at the upper range of the scale.

As expected, the TACS was moderately to strongly correlated with the hand and mobility domains of the Performance Scales, as well as with the EDSS (Table 4). The correlations of the criterion measures, the NHPT and cerebellar FSS, with hand, mobility, and the EDSS were also moderate. With increasing TACS score, physical QOL as measured by the PCS-36 declined (r = −0.50; 95% CI, −0.22 to −0.71; Figure 1). This supports the convergent construct validity of the TACS. The TACS was weakly correlated with age and BMI, but these correlations were not statistically significant. Furthermore, the TACS was not correlated with mental QOL, as measured by the MCS-36 (r = −0.085; 95% CI, −0.39 to 0.24), also supporting the divergent construct validity of the scale. The correlations of the NHPT and cerebellar FSS with age and BMI were also weak.

Median (interquartile range) for the Physical Component Score–36 (PCS-36) according to score on the Tremor and Coordination Scale (TACS)

Spearman correlations and 95% CIs for assessment of construct validity: NARCOMS TACS

Discussion

In a population-based study conducted in Rochester, Minnesota, tremor affected 25% of people with MS. Severe tremor affected 3% of that population.² Those affected by severe tremor were more disabled and more likely to be unemployed and unable to drive. The TACS focuses on the impact of tremor and incoordination on the person with MS, rather than on measuring the characteristics or severity of these symptoms. In our cohort, 75% of participants reported disability due to tremor or incoordination, with 22.7% being moderately or severely affected. Physical QOL was reduced in participants reporting tremor or incoordination. This highlights the negative impact of these symptoms and the importance of further study.

Increasingly, the importance of the patient perspective in assessing symptoms and treatment efficacy is being recognized by regulatory authorities.²⁰ Few validated measures of tremor or ataxia exist for MS⁷; thus we aimed to validate the brief TACS used by the NARCOMS Registry. In two studies, MS-related tremor was associated with disability, reduced grip strength, and dysarthria.^{1 2} Similarly, we found that higher TACS scores were associated with greater disability as measured by the EDSS and worse mobility, supporting the construct validity of the TACS. Although we cannot be certain that individuals using this questionnaire are accurately distinguishing limb ataxia from limb weakness as the etiology of their disability, the criterion validity of the TACS is well supported by its moderate correlations with the NHPT and the cerebellar FSS.

This study has some limitations. The sample size was small, but the study was adequately powered to detect correlations of 0.40 or more, after accounting for multiple comparisons. Although the study included patients with relapsing-remitting and progressive forms of MS, and a spectrum of disability, patients with EDSS scores greater than 6.5 were not represented. Because of the small number of participants reporting severe (score = 4) or total disability (score = 5) related to tremor and incoordination, we could not fully assess the criterion validity of the TACS. The relatively narrow range of disability in addition to the small sample size likely explains some of the broad confidence intervals for the correlations in this study. A larger, more diverse study population will be needed to fully evaluate the validity of the TACS. The cross-sectional nature of the study precluded assessment of reliability of the scale.

The TACS queries respondents about disability due to tremor and impaired coordination; thus the instrument cannot distinguish between the impact of tremor and the impact of ataxia. This reflects the limitations of a single-item instrument. However, the TACS was developed by the NARCOMS Registry for use in semi-annual questionnaires distributed to thousands of participants, and it has several features that are desirable in patient-reported outcome measures. The brevity and simplicity of the instrument facilitate longitudinal data collection in the large NARCOMS cohort while limiting response burden for participants. The instrument is acceptable to patients and easy to administer, interpret, and process.

The NARCOMS TACS appears to have adequate construct validity and criterion validity in a mildly to moderately disabled population with MS. Further evalu-Further evaluation is needed to establish the criterion and construct validity of the scale in a severely disabled population, and to assess the test-retest reliability of the scale and its responsiveness to change.

PracticePoints

Nearly 23% of MS patients report moderate or severe disability due to tremor or incoordination.

The NARCOMS Tremor and Coordination Scale is a single-item instrument with adequate criterion and construct validity for assessing tremor and incoordination in MS.

Test-retest reliability and responsiveness of the NARCOMS Tremor and Coordination Scale need to be established.