Reliability and Validity of a Danish Version of the Multiple Sclerosis Neuropsychological Screening Questionnaire

Cognitive impairment is a common problem in multiple sclerosis (MS), occurring in approximately 50% of patients. It typically involves memory, information-processing speed, learning, and executive function and, thereby, affects activities of daily living and quality of life.^{1 2} Moreover, cognitive impairment is often the primary cause of unemployment, reported in 70% to 80% of all patients with MS.^{1 3 4} Cognitive impairment may prove to be the only parameter in a few patients with MS and especially in patients with MS treated with immunomodulatory drugs that can induce progressive multifocal leukoencephalopathy. Cognitive screening is, thus, incredibly valuable as a means of assessment and as a guide for treatment.^{5 6} Considering the prevalence and morbidity, there is a need for cost-effective screening tools for cognitive impairment in MS.

The best-validated fast screening test in patients with MS is the Symbol Digit Modalities Test (SDMT). This test is easy to administer and correlates well with cognitive impairment as measured by other standardized neuropsychological batteries assessing multiple functions and brain atrophy.⁷ The SDMT, however, addresses only a fraction of the cognitive functions, such as process speed and visuospatial and working memory.^{8 9}

The Multiple Sclerosis Neuropsychological Screening Questionnaire (MSNQ) is a brief patient-reported outcome scale by either the patient (MSNQ-P) or the informant (MSNQ-I) and is easy to administer. It has previously been shown to have acceptable reproducibility and to provide valid assessment of cognitive dysfunction in American, Argentinean, and Dutch populations. In contrast to the SDMT, the MSNQ can provide information about self- and informant-perceived cognitive dysfunction.¹⁰–¹³

The objective of this study was to validate a Danish translation of the MSNQ compared with formal neuropsychological testing and with the SDMT and measures of depression and disability.

Methods

Ethics

All the procedures were performed according to the Declaration of Helsinki and with permission from the Danish Data Protection Agency (14/8330).

Translation of MSNQ

The MSNQ was translated by a bilingual translator from English to Danish and then was translated from Danish to English by another translator to correct for any linguistic corrections or oversights.

Study Population

We studied 126 patients diagnosed as having MS in the Department of Neurology, Odense University Hospital (Odense, Denmark), over 7 consecutive years (January 1, 2000, through December 31, 2006). The inclusion criteria were 1) a diagnosis of MS according to the McDonald criteria,¹⁴ 2) an informant with face-to-face contact with the patient three or more times a week, 3) age older than 18 years, 4) Danish as the first language, and 5) informed consent. The exclusion criteria were 1) neurologic deficits not related to MS, 2) a history of developmental disorders or other learning disabilities, 3) previous or present psychiatric disease that is unlikely to be part of the patient's MS, 4) alcohol or drug abuse, and 5) corticosteroid treatment in the 4 weeks before evaluation. Informants were selected by the following criteria: closest family or friend with whom the patient lives or has at least three weekly contacts, in the following prioritized rank: spouse, father/mother, daughter/son, friend/closest family.

Application of MSNQ, SDMT, and Neuropsychological Testing

Patients and their respective informants were carefully given standardized instructions on how to fill out the MSNQ¹⁰ and the Beck Depression Inventory (BDI)¹⁵ during each contact. Patients completed questionnaires in the department, and a neurologist (M.B. or T.S.) assessed clinical deficits by means of the MS Impairment Scale (MSIS),^{16 17} the Expanded Disability Status Scale (EDSS),¹⁸ and the SDMT.^{19 20}

Informants completed the MSNQ-I during the visit or within a few weeks. The first 77 patients examined were chosen for neuropsychological testing, which occurred a few weeks after contact with the neurologist. The test battery comprised the following tests: the Rey Auditory Verbal Learning Test, the Trail Making Test B, the Wisconsin Card Sorting Test, the Boston Naming Test, and Digit-Symbol Coding. A z score of the combined neuropsychological tests was calculated from the test results, and a z score less than −1.5 SD was used as the cutoff point to diagnose true cognitive impairment.

Based on earlier studies, a cutoff score of 26 points or greater on the MSNQ-I and the MSNQ-P was chosen as a sign of cognitive impairment.^{10 12 13 21 22} Forty-four patients were retested (Figure 1) to investigate the test-retest variability of the MSNQ-I and the MSNQ-P.

Study flowchart

Statistical Methods

Z scores were based on sex- and age-matched controls. Z scores could not be calculated for nine patients due to either missing age-matched normal values (patients >60 years old) or incomplete neuropsychological testing. Three informants did not return the MSNQ-I (Figure 1). Statistical analyses were performed using GraphPad PRISM 7 (GraphPad Software Inc, La Jolla, CA). Because data were not distributed normally, nonparametric statistics were used.

Results

Demographics and Comparison of Subgroups

Comparing patients who underwent neuropsychological testing (n = 77) with those who did not (n = 49), we found no differences in demographics in relation to age, disease duration, and disability as measured by the EDSS and the MSIS. There were also more men in the neuropsychological group (43%) compared with in the group that was not tested by a neuropsychologist (26%) (P < .05) (Table 1).

Baseline demographic characteristics of the 126 study participants

Test-Retest of MSNQ-P and Correlation Between MSNQ-I and MSNQ-P

The squared test-retest correlation of the MSNQ-P was R ² = 0.79 (P < .0001) (Figure 2A) and the squared correlation of the informants (MSNQ-I) and patients (MSNQ-P) was low but significant (R ² = 0.22, P < .0001) (Figure 2B).

Linear fit regression of Multiple Sclerosis Neuropsychological Screening Questionnaire–Patient (MSNQ-P) and MSNQ-Informant (MSNQ-I)

Squared Correlation Between MSNQ and Neuropsychological Tests and BDI

When correlating the MSNQ-I or the MSNQ-P (greater than the cutoff score of 26 points) with the z score of the complete neuropsychological test battery, we found no significant squared correlation with either scale (MSNQ-P: R ² = 0.0084, P = .457; MSNQ-I: R ² = 0.0345, P = .1388) (Figure 3). Neither did the MSNQ-I and the MSNQ-P correlate with the individual neuropsychological tests (Table 2). We found low sensitivity and specificity of the MSNQ-P (21.4% and 76%, respectively) and the MSNQ-I (33% and 65.5%, respectively). Both scales did, however, correlate to BDI scores (MSNQ-P: R ² = 0.25, P < .0001; MSNQ-I: R ² = 0.197, P < .0001 [data not shown]).

Linear fit regression of Multiple Sclerosis Neuropsychological Screening Questionnaire–Patient (MSNQ-P) or MSNQ-Informant (MSNQ-I) and neuropsychological test battery

Correlations between MSNQ-P, MSNQ-I, and SDMT and individual neuropsychological tests

Squared Correlation Between MSNQ and Disability Scores

To investigate whether there is a correlation between cognitive dysfunction measured by the MSNQ and disability, we compared MSNQ-I with EDSS and MSIS scores. We found no squared correlation to either objective disability measure (EDSS: R ² = 0.0036, P = .5; MSIS: R ² = 0.0001, P = .89 [data not shown]).

Correlation of SDMT and Neuropsychological Testing

As a positive control for bedside neuropsychological testing in patients with MS, we also tested the squared correlation between the well-established SDMT and the total neuropsychological test panel. As expected, this squared correlation was highly significant (R ² = 0.68, P < .0001) (Figure 4). When correlated to the individual neuropsychological tests, we found the best squared correlations to Digit-Symbol Coding and the Trail Making Test B. The SDMT also correlated to the Rey Auditory Verbal Learning Test and the Wisconsin Card Sorting Test but not to the Boston Naming Test (Table 2).

Linear fit regression of Symbol Digit Modalities Test (SDMT) and neuropsychological test battery

Discussion

The results of the present study show that a Danish translation of the MSNQ, despite good test-retest reliability, has no statistically significant correlation to cognitive impairment found on standard neuropsychological testing. This is in contrast to validation studies in other languages.^{12 13 21 22} When using the now well-established SDMT scale, we did, however, find a highly significant correlation with the neuropsychological test battery, demonstrating the superiority of this scale in establishing the presence of cognitive impairment in MS.²³–²⁶ Several factors may explain the low sensitivity and specificity of the MSNQ in the present study. First, we used an MSNQ cutoff score greater than 26, which is the cutoff score used in the original validation.¹⁰ Other studies have used different cutoff values to achieve higher sensitivity and specificity, and the optimal cutoff score seems to change from study to study.^{13 22 27} Using other cutoff points did not increase the sensitivity or specificity in the present population.

Second, compared with most of the earlier MSNQ validation studies performed,^{13 22 27} the present cohort demonstrated a shorter disease duration (mean, 7.8 years), a lower EDSS score (mean, 2.8), and fewer cognitively impaired patients (13 of 77). A screening tool for cognitive impairment should, however, be relevant also in younger MS cohorts with shorter disease durations, and, when using the SDMT in the present cohort, we did find very high sensitivity compared with neuropsychological testing.

Sonder et al²⁷ demonstrated lower loadings regarding how individual questions are weighted and affect the MSNQ score. Different study populations and cultural differences regarding the content of the questions should also be considered in future studies. Unfortunately, this study was not designed to test for such differences.

Finally, it should be considered whether the MSNQ actually does measure cognitive impairment. Strober et al²⁸ recently reported that self-assessed measures of cognition do not correlate with systematic neuropsychological testing but rather with quality of life and behavioral data. We found that the MSNQ-I and the MSNQ-P correlated significantly with BDI scores. Other studies have also suggested that the MSNQ is influenced by psychosocial variables, such as anxiety, rather than by objective status.^{13 29 30} The impact of psychosocial variables could be the explanation for the subpopulation of patients who reported high impact on self-experienced cognitive impairment but had normal results on standard neuropsychological testing.

An opposite subgroup with low MSNQ-I and MSNQ-P scores had severe cognitive impairment when tested with formal neuropsychological tests. This might be explained by a coping strategy of the patients and informants to undervalue cognitive impairment and, therefore, report low MSNQ-P and MSNQ-I scores in self-perceived cognitive impairment. Another explanation for low MSNQ-P scores and high cognitive impairment could also be that severely affected patients do not perceive the dementia. Further studies are needed to clarify these issues of self-assessed measures of cognition.

In conclusion, this study does not support use of the MSNQ-P or the MSNQ-I as a sensitive or valid screening tool for cognitive impairment in Danish patients with MS. In contrast, this study found the SDMT to be more reliable owing to higher sensitivity and specificity. Further studies are needed to assess the effect of differences in study populations, choice of cutoff values, and cultural and psychosocial impact.

PRACTICE POINTS

We studied the validity and reliability of the Multiple Sclerosis Neuropsychological Screening Questionnaire (MSNQ), a screening tool for cognitive impairment, in 126 Danish patients and their informants.

Test-retest reliability of the MSNQ-Patient (MSNQ-P) was assessed on a subsample of 44 patients and was found to be high. There was a high correlation between the MSNQ-P and the MSNQ-Informant (MSNQ-I).

The Danish versions of the MSNQ-P and the MSNQ-I showed no correlation with formal neuropsychological testing. In contrast, they demonstrated significant (albeit low) correlation with depression, suggesting correlation of the Danish MSNQ-P and MSNQ-I with behavioral outcomes rather than with neuropsychological measures.