Content

Issues

Research

Permissions

Community Content

Videos and Multimedia

IJMSC Awards

For Authors

Information for Authors

Manuscript Submission

Patient Consent Form

About IJMSC

Aim and Scope

Ethics

Editorial Board

Peer Review

Contact Us

Home

Advertising

IJMSC

1rem

Authors

Multiple Sclerosis Adult Day Programs and Health-Related Quality of Life of Persons with Multiple Sclerosis and Informal Caregivers

Restless legs syndrome (RLS) is a sleep disorder present in as many as 26% of persons with multiple sclerosis (MS) and can be associated with cognitive function. The present study examined the relationships between RLS symptoms (severity, frequency, occurrence) and cognitive function in adults with MS who have RLS.

Twenty-two participants attended 1 laboratory session and completed the International Restless Legs Syndrome Study Group Rating Scale (IRLS), the Restless Legs Syndrome-6 Scale, and then the Brief International Cognitive Assessment for Multiple Sclerosis battery consisting of the Symbol Digit Modalities Test; California Verbal Learning Test, Second Edition; and Brief Visuospatial Memory Test–Revised.

Nonparametric bivariate correlations indicated that worse IRLS total severity was associated with slower processing speed (ρ = −0.42), worse verbal memory (ρ = −0.63), and worse visual memory (ρ = −0.61); worse RLS severity at falling asleep was associated with worse verbal memory (ρ = −0.45) and worse visual memory (ρ = −0.55); and worse RLS severity during the day while active was associated with slower processing speed (ρ = −0.58), worse verbal memory (ρ = −0.52), and worse visual memory (ρ = −0.60).

These results suggest that those with more severe RLS, including worse symptoms at falling asleep and during the day while active, might experience worse cognitive function, particularly processing speed and memory. Future research should evaluate whether treatment of RLS symptoms can offer new opportunities for managing cognitive dysfunction in adults with MS.

From the Department of Physical Therapy, University of Alabama at Birmingham, USA (KLJC [now at Stanford University], BM, MLS, RWM). 

: Katie L.J. Cederberg, MS, Department of Psychiatry and Behavioral Sciences, Stanford School of Medicine, 401 Quarry Rd, Stanford, CA 94304, USA; 

 The authors declare no conflicts of interest.

 This work was supported, in part, by the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health (grant F31HD097903).

 The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funding sources had no involvement in the study design; data collection, analysis, or interpretation; writing of the report; or decision to submit the article for publication.

 Aspects of this study were presented in abstract form at the Virtual Annual Meeting of the Consortium of Multiple Sclerosis Centers; May-June 2020.

Restless legs syndrome (RLS) is a sensorimotor sleep disorder that occurs in an estimated 26% of persons with multiple sclerosis (MS),

 and it may exacerbate the symptoms and consequences of MS,

 including cognitive function. Indeed, there is evidence that sleep disorders and cognitive impairment are prominent and debilitating consequences of MS,

 and markers of sleep quality have been associated with cognitive impairment in MS. There is recent evidence supporting the association between the presence and severity of RLS with perceived cognitive impairment in adults with MS.

 In this recently published study, adults with MS and RLS perceived their cognitive impairment as significantly worse than adults with MS alone

; however, the relationship between RLS symptoms and neuropsychological measures of cognitive function has not been evaluated in adults with MS. This is important because there is a strong degree of uncertainty regarding the meaning of scores from measures of perceived cognitive function in MS whereby self-reported scores often correlate with psychological symptoms (ie, mood and emotion) more so than neuropsychological tests of cognitive function.

The present pilot study extends previous research

 and explores the relationships between RLS symptoms (ie, overall severity, frequency, and occurrence) and objective cognitive function in adults with MS and RLS. We use neuropsychological testing to assess different domains of cognitive function, including processing speed, working memory, verbal memory, and visual memory. We expect that worse overall RLS severity, greater frequency of RLS symptoms, and worse RLS symptoms when falling asleep and during the night will be associated with slower processing speed as well as worse working, verbal, and visual memory based on previous research in sleep impairment and cognitive function in adults with MS.

 Because RLS symptoms can be particularly distracting, we further expect that worse RLS severity during the day, while active and while resting, will be associated with worse cognitive function as outcomes of cognitive function are captured during the day.

The present study involves a secondary analysis of data from a subsample of participants who enrolled in a center-based study validating the Suggested Immobilization Test in MS; the data are not part of our previous mail-based study on patient-reported cognitive difficulty and RLS.

 To that end, no sample size calculations were performed because this secondary analysis was exploratory. Participants were recruited by sending letters and flyers to persons identified through a search of the University of Alabama at Birmingham’s i2b2 database for persons with MS residing in the Birmingham area. Interested persons were asked to contact research staff for a brief description of the study followed by screening for the following inclusion/exclusion criteria: age 18 years or older, diagnosis of MS, and a positive screen for the presence of RLS based on confirmatory responses to the 5 diagnostic criteria for RLS

 using the Cambridge-Hopkins Restless Legs Syndrome Short Form Diagnostic Questionnaire.

 Participants were excluded from the study based on mimics of RLS (eg, positional discomfort, muscle cramps) and diagnosis of radiculopathy, peripheral edema, peripheral neuropathy, anemia, renal disease, or diabetes.

The International Restless Legs Syndrome Study Group Rating Scale (IRLS) is a validated 10-question survey that provides a global score regarding the severity and frequency of a patient’s RLS symptoms during the previous week.

 The severity is determined by summing the individual item scores into a total score ranging from 0 to 40, with higher scores indicating greater severity. Severity scores are categorized as no symptoms (score of 0), mild (scores of 1–10), moderate (scores of 11–20), severe (scores of 21–30), and very severe (scores of 31–40). Of note, item 7 on the IRLS provides a measure of RLS frequency on a 5-point scale ranging from 0 (never) to 4 (very often, ie, 6 to 7 days per week).

The Restless Legs Syndrome-6 Scale (RLS-6) is a validated scale for assessing a patient’s sleep satisfaction (item 1), RLS severity during different periods of a 24-hour day (items 2–5), and daytime sleepiness (item 6) during the previous week.

 The RLS-6 assesses the severity of RLS symptoms on a scale from 0 (none) to 10 (very severe) in 4 domains: at falling asleep (item 2), at night (item 3), during the day while resting (item 4), and during the day while active (item 5). The RLS-6 also contains items for assessing sleep satisfaction (item 1) and daytime sleepiness (item 6) on a scale from 0 to 10, whereby higher scores reflect worse sleep satisfaction and daytime sleepiness. We further calculated a sleep composite score by taking the average of items 1 and 6.

The Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) battery was administered as a valid and reliable method for detecting cognitive impairment in the MS population

 and is composed of the Symbol Digit Modalities Test (SDMT); the California Verbal Learning Test, Second Edition (CVLT-II); and the Brief Visuospatial Memory Test–Revised (BVMT-R). The BICAMS offers a feasible, cost-effective method of assessing cognitive function in MS across cultures.

 The first author (K.L.J.C) administered all the neuropsychological outcomes as she is thoroughly trained and has 4 years of experience in administering the BICAMS battery in adults with MS. We included the raw scores for the analyses because this was a nonclinical application of BICAMS and not for making judgments regarding detection of cognitive impairment.

The SDMT provides a measure of visuospatial processing speed and working memory; it consists of 9 single-digit numbers paired with 9 particular abstract symbols. Participants were provided with a key that included the number and symbol pairings at the top of the page, followed by rows of pseudorandomly aligned symbols. Participants were asked to call out each number associated with the symbol according to the way they are paired in the key at the top of the page, completing as many as possible in 90 seconds. Scores are presented as the total number of correct pairings identified by the participant in 90 seconds, with lower scores indicative of worse visuospatial processing speed and working memory.

The learning trial of the CVLT-II is an assessment of immediate verbal memory and memory difficulties often used in adults with MS. The test is composed of a 16-item word list with 4 words associated with 4 categories (eg, animals, foods) that are randomly arranged. Participants were read the list of words at a rate of 1 word per second and then were asked to recall as many items as possible, in any order, across 5 learning trials. The total number of correct words in each trial was summed to provide a score ranging from 0 to 16, and the learning trial scores were summed for a total score ranging from 0 to 80, with lower scores indicative of worse recall.

The learning trial of the BVMT-R is an examination of immediate visuospatial memory. The test is composed of 3 recall trials wherein participants are presented with a display of 6 abstract geometric figures for 10 seconds; the display is hidden from view while participants attempt to draw each figure as accurately as possible and in the correct position on the page from memory. For each trial, the figures receive a score of 0, 1, or 2 based on the accuracy and position of the figure on the page. Thus, each trial results in a score ranging from 0 to 12, and the trial scores are summed for a total score ranging from 0 to 36, with lower scores indicative of worse visual memory and visuospatial memory.

Participants completed a questionnaire for age, sex, race, years of education, employment status, MS type, MS disease duration (ie, time since diagnosis), time since MS symptom onset, previous diagnosis of RLS, time since RLS symptom onset, and family history of RLS. Participants provided a list of all current medications, including over-the-counter medications, for assessment of what might exacerbate or mitigate RLS symptoms. Participants further underwent a brief neurologic assessment for scoring the Expanded Disability Status Scale (EDSS) as a valid measure of disability status in persons with MS

; the first author (K.L.J.C.) is Neurostatus-certified for EDSS administration and scoring.

The institutional review board of the University of Alabama at Birmingham approved the study procedures, and all participants provided written informed consent. Participants with MS were screened for RLS diagnosis over the phone and were scheduled for a single, 90-minute session in the laboratory. The present study was part of a larger testing session wherein a rater performed a brief neurologic examination for scoring the EDSS. Participants then completed the questionnaires, followed by the BICAMS battery (ie, SDMT, CVLT-II, and BVMT-R). Participants and researchers were not blinded regarding the aims of the larger study, and participants were remunerated $25 for completing all the measures.

All statistical analyses were conducted using SPSS Statistics for Windows, version 26.0 (IBM Corp), and descriptive data are presented as means (SDs) or as frequency (percentage) of individuals, unless otherwise specified. Skewness and kurtosis values along with assessment of frequency distributions were inspected for establishing normality of the variables.

 Based on the distribution and varying measurement characteristics (ie, continuous and categorical) of the outcome variables, we adopted nonparametric analyses and a 

 < .05 for interpreting statistical significance with the inferential analyses. We provide raw scores for BICAMS outcomes alongside calculated 

 scores using regression-based norms adjusting for age, sex, and years of education.

 The associations among IRLS total score, RLS frequency (ie, IRLS item 7), RLS occurrence (ie, RLS-6 items 2 through 5), and cognitive function (ie, SDMT, CVLT-II, and BVMT-R raw and 

 scores) were examined using bivariate Spearman rho (ρ) correlation coefficients. Values of 0.1, 0.3, and 0.5 were interpreted as small, moderate, and large, respectively.

 We further evaluated the relationship between outcomes of RLS symptom severity, sleep, and neuropsychological impairment that were significantly associated using multivariate linear regressions (α = 0.05). We regressed BVMT-R (raw and norm-corrected) scores on significantly associated outcomes of RLS severity (ie, IRLS total score and RLS-6 items 2 and 5 for raw scores and IRLS total score and RLS-6 items 2, 4, and 5 for norm-corrected scores) in step 1, and RLS severity plus the sleep component score, which was significantly correlated with both BVMT-R and RLS severity scores in bivariate correlation analyses, in step 2. We examined the change in the standardized beta coefficient for RLS severity predicting BVMT-R scores between steps 1 and 2 for judging the variables that might be intermediary factors in the associations between RLS and neuropsychological function in persons with MS and RLS.

Characteristics of the 22 adults with MS and RLS in the study sample are presented in 

. The sample had a mean (SD) age of 51.6 (12.0) years and was primarily White (55%) and female (77%) with relapsing-remitting MS (73%), a mean (SD) disease duration of 15.4 (9.1) years, and moderate disability (median [interquartile range] EDSS score = 3.5 [2.6]). The sample had RLS symptoms for a mean (SD) of 16.8 (13.7) years, and only 46% of participants had received a previous diagnosis of RLS, with 27% reporting an immediate family member with RLS. Approximately 73% of individuals were receiving disease-modifying therapy for managing MS, including 41% of participants taking 1 or more medications that could improve RLS symptoms (eg, pramipexole, gabapentin, and rotigotine

) and 32% taking 1 or more medications that could worsen RLS symptoms (eg, antidepressants and first-generation antihistamines

). Regarding cognitive impairment, mean (SD) scores were 47.2 (13.0) for the SDMT, 41.7 (10.7) for the CVLT-II, and 20.5 (6.1) for the BVMT-R, and all were comparable with a recent systematic review of validation studies for the BICAMS battery.

 Scores for the BICAMS battery were slightly above the suggested cutoff scores of 44 for the SDMT, 39 for the CVLT-II, and 17 for the BVMT-R for identifying impaired cognitive performance for adults with MS.

 Mean (SD) regression-based norm-corrected 

 scores adjusting for age, sex, and years of education were −1.15 (1.1) for the SDMT, −1.54 (1.0) for the CVLT-II, and −0.75 (1.2) for the BVMT-R.

A summary of RLS symptoms, including severity, frequency, and occurrence, is presented in 

, which is published in the online version of this article at IJMSC.org. The sample reported having moderate-to-severe RLS symptoms (mean [SD] IRLS total score = 20.4 [6.1]) with symptoms at least 2 days per week and most participants (63.6%) reporting symptoms on 4 or more days per week. Participants reported that mean (SD) RLS severity was worst at the time of falling asleep (5.0 [2.4]), followed by during the day while resting (4.6 [2.0]), during the night (4.5 [2.5]), and during the day while active (3.9 [2.7]).

Relationship Between RLS Symptoms and Cognitive Function

The summary of bivariate Spearman rho correlation analysis for RLS severity, frequency, and occurrence with cognitive function in 22 participants with RLS and MS is presented in 

. Higher IRLS total scores were associated with slower processing speed and worse working memory (ρ = −0.42 and −0.50) as well as worse immediate visuospatial memory based on raw scores (ρ = −0.61) and norm-corrected scores (ρ = −0.69), and worse immediate verbal memory based on raw scores only (ρ = −0.63). Worse RLS severity at falling asleep (ie, RLS-6 item 2) was associated with worse immediate verbal memory difficulties based on raw scores (ρ = −0.45) and worse immediate visuospatial memory based on raw scores (ρ = −0.55) and norm-corrected scores (ρ = −0.47). There was no association between RLS severity at falling asleep and processing speed and working memory. There were no significant associations between RLS severity during the night (ie, RLS-6 item 3) and neuropsychological outcomes. Worse RLS severity during the day while resting (ie, RLS-6 item 4) was associated with norm-corrected scores for immediate visuospatial memory (ρ = −0.44). Worse RLS severity during the day while active (ie, RLS-6 item 5) was associated with slower processing speed and worse working memory (ρ = −0.58 and −0.67) as well as worse immediate visuospatial memory (ρ = −0.60 and −0.63) based on raw and norm-corrected scores and worse immediate verbal memory based on raw scores only (ρ = −0.52). Associations between cognitive function outcomes and RLS symptoms in adults with MS and RLS are presented in 

There were no significant associations between sleep satisfaction (ie, RLS-6 item 1) or daytime sleepiness (ie, RLS-6 item 6) and scores from the SDMT, CVLT-II, or BVMT-R. Worse sleep quality, based on the sleep composite score, was associated with worse immediate visuospatial memory (ρ = −0.43 and −0.48), based on raw and norm-corrected scores. The summary of linear regression analyses for further evaluating the relationships among RLS, sleep, and immediate visuospatial recall performance is presented in 

. Severity of RLS explained a significant amount of the variance in step 1 (raw scores: F = 6.17, 

 = 0.414), and the inclusion of the sleep composite score did not affect this relationship in step 2 (raw scores: F = 4.37, 

The present pilot study was the first, to our knowledge, to evaluate RLS symptoms and performance on neuropsychological measures of cognition in adults with MS. These results indicated that (1) worse IRLS total severity was associated with slower processing speed and working memory, worse verbal memory, and worse visuospatial memory; (2) worse RLS severity at falling asleep was associated with worse verbal memory and worse visuospatial memory; and (3) worse RLS severity during the day while active was associated with slower visuospatial processing speed and working memory, worse verbal memory, and worse visuospatial memory. When adjusting neuropsychological outcomes for age, sex, and education, associations between RLS severity and processing speed, working memory, and immediate visuospatial memory strengthened, whereas associations between RLS severity and immediate verbal memory weakened. The cross-sectional design of this pilot study precludes any inferences regarding the directionality of the relationship between RLS and performance on neuropsychological measures of cognition, and these relationships could be associated with other structural or functional disease-related outcomes in MS. Nevertheless, the possible interpretation that worse RLS symptoms yield reduced cognitive function, particularly memory and processing speed in adults with MS who have RLS requires confirmation using longitudinal designs that control for other variables as confounders.

The present results suggest that worse RLS severity, specifically higher IRLS total scores and worse severity at falling asleep and during the day while active, is associated with slower processing speed and worse visual and verbal memory in adults with MS who have RLS. When adjusting for age, sex, and education, the associations between RLS severity and processing speed and working memory and immediate visuospatial memory strengthened, whereas associations between RLS severity and immediate verbal memory weakened and were no longer significant. These results suggest that age, sex, and education might be accounting for the relationship between immediate verbal memory and IRLS total scores, RLS severity at falling asleep, and RLS severity during the day while active. In addition, when adjusting for age, sex, and education, there was a significant association between RLS severity during the day while resting and immediate visuospatial memory. Although this seems to be the first study to examine the association between RLS symptoms and neuropsychological outcomes in adults with MS, studies outside of MS have demonstrated similar relationships between RLS severity and cognitive function. One study reported that worse RLS severity was associated with worse attention and verbal fluency in 23 adults with untreated RLS in the general population,

 and another study suggested that severe RLS can result in more impaired working memory in 13 adults with untreated RLS in the general population.

 One potential explanation for this relationship could be related to the effect of RLS symptoms on concentration, as 1 study in the general population suggested that sensory symptoms related to RLS may lead to a lack of concentration.

 This suggests that participants who experience greater severity of RLS symptoms during the day while active might experience a reduced capacity to concentrate, therefore decreasing performance on neuropsychological outcomes, including learning and memory as well as processing speed.

Another possible explanation for the relationship between greater RLS severity and impaired neuropsychological outcomes may be related to chronic sleep loss due to RLS rather than a direct effect of RLS pathology. Outside of RLS, a study in adults with MS observed a reduction in objective cognitive function as a result of sleep disturbances, namely, visual memory, verbal memory, executive function, attention, processing speed, and working memory.

 This suggests that chronic sleep loss as a result of RLS symptoms might affect neuropsychological function in adults with MS. Chronic sleep loss from RLS symptoms at night combined with the aforementioned reduced concentration due to worse RLS severity during the day while active could yield reduced performance on neuropsychological outcomes. The present study demonstrated no significant associations between sleep satisfaction or daytime sleepiness with outcomes of neuropsychological function; however, there was a significant association between worse immediate visuospatial memory and worse sleep based on the sleep composite score. These results suggest that reduced performance on neuropsychological outcomes may not be a result of individual aspects of poor sleep but rather a function of overall sleep quality; however, linear regression analyses suggest that the sleep composite score did not account for the relationship between RLS and immediate visuospatial recall. Importantly, this pilot study might not be adequately powered to estimate such relationships with precision, and future research should evaluate the relationships among RLS symptoms, sleep dysfunction, and neuropsychological outcomes in a larger sample of adults with MS who have RLS.

There are important limitations to consider when interpreting these results. The small sample limited the power for detecting smaller correlations and the generalizability of the results. The cross-sectional design precludes any inferences of causality between RLS and cognitive function. The provision of longitudinal analyses of these relationships may provide additional insight into causality. There were no adjustments for multiple comparisons in this exploratory study, which may increase the likelihood of false alarms given the high number of correlations. A large percentage of the sample was taking medications that could mitigate (41%) or exacerbate (32%) RLS symptoms, and this could influence the associations between RLS and neuropsychological outcomes. Adverse effects of these medications on cognitive function are not well understood. Future research should consider comparing groups based on medication use as well as those who are untreated because medication could have secondary effects on cognition. The outcomes related to RLS symptoms were self-reported; thus, future research should evaluate objective measures of RLS symptoms, such as electromyography or actigraphy, in the relationship between RLS and cognitive function. Outcome measures of other symptoms and consequences of MS that may be related to both RLS and cognition (eg, sleep, fatigue, depression

) were not included. Neither participants nor researchers were blinded regarding the study design and outcomes, although the main study was designed to examine the validity of the Suggested Immobilization Test rather than to focus on RLS severity and cognition.

This pilot study was the first study to examine the relationship between RLS symptoms and neuropsychological outcomes in persons with MS. These results suggest that those with more severe RLS, including worse IRLS total severity and worse symptoms at falling asleep and during the day while active, may experience worse cognitive function, particularly slower processing speeds and more memory difficulties. Future research should examine (1) whether these relationships are present over time in a larger sample of adults with MS and RLS and (2) whether managing symptoms of RLS improves neuropsychological function, as treatment of RLS might offer new opportunities for managing cognitive dysfunction in adults with MS.

In this study examining the relationships between restless legs syndrome (RLS) symptoms and cognitive function in adults with multiple sclerosis and RLS, worse overall RLS severity was associated with slower processing speed and worse memory.

Restless Legs Syndrome Severity and Cognitive Function in Adults With Multiple Sclerosis: An Exploratory Pilot Study

The role of conformity to masculine gender norms in health behaviors in men with multiple sclerosis (MS) has not received attention. This cross-sectional study explores these issues and their relationship to coping and health behaviors.

Eighty-one men with MS completed the Conformity to Masculine Norms Inventory-46 and the Ways of Coping Questionnaire and provided demographic and clinical variables. These results were used to predict subscale scores of the Health Behavior Inventory-20 in multivariable regression models.

Models for the Preventive Self-care and Avoiding Anger and Stress subscales were successfully fit. For the former, respondents endorsing lower levels of masculine conformity related to Emotional Control and higher levels of Heterosexual Self-presentation predicted greater self-care, as did higher use of Positive Reappraisal as a coping strategy. For men reporting low levels of Positive Reappraisal as a coping strategy, increasing Heterosexual Self-presentation was associated with higher levels of self-care. For those with high levels of coping with Positive Reappraisals, increased Heterosexual Self-presentation was associated with modest declines in self-care. For the Avoiding Anger and Stress subscale score, men endorsing Violence or Heterosexual Self-presentation as important aspects of masculinity also reported less efforts in controlling stress and anger.

Masculinity adherence to traditional gender norms was a significant predictor of how men engaged in health behaviors and, in the case of Preventive Self-care, was found to interact with Positive Reappraisal as a coping strategy. Such information is novel and important to providers serving male patients with MS and can improve provider awareness/conceptualization of male patient needs.

From the Mellen Center for Multiple Sclerosis (BD [now at Norton Healthcare], ABS) and Department of Quantitative Health Sciences (RH), Cleveland Clinic, Cleveland, OH, USA. 

 Bryan Davis, PsyD, 3991 Dutchmans Lane Suite 310, Louisville, KY 40207; 

 Dr Davis has been a consultant for Novartis. Dr Sullivan has been a consultant and speaker for Novartis, Biogen, Bristol Myers Squibb, and Medscape. Dr Honomichl declares no conflicts of interest.

 This study was supported by a Cleveland Clinic Neurological Institute Research Project Pilot Funding Award.

Although men have lower prevalence rates of multiple sclerosis (MS) than women,

 it is still important to understand their experience with an MS diagnosis. Previous research has demonstrated sex differences in a variety of relevant psychological domains: higher self-efficacy in women,

 higher levels of depression in men with MS,

 typically worse prognosis in men with MS, and more of an effect of disability on health-related quality of life in men.

Given these findings, some data cite evidence that men differ from women in how they experience MS, but this difference is not overwhelming. Male socialized gender norms, such as Western culture gender roles, may be a contributor to such discrepancies. These socialized gender norms are established at a young age and guide young men in how to be masculine, which is socially reinforced by peers, caregivers, and society at large.

 Although such gender norms do not seem to be inherently detrimental and can also teach helpful behaviors (eg, independence, toughness, provision), some behaviors may contribute to differences in health behaviors and disease management.

Previous research suggests that the difficulty men have utilizing health care services is an effect of gender norm socialization.

 Traditionally dominant masculine ideals often include toughness, resiliency, independency, and stoicism.

 Specifically, young men may not practice help-seeking behaviors due to the contradictions to gender norms (vulnerability, weakness, and dependence).

 It has been found that men also allocate less time towards their personal health and practice less preventative health behaviors when compared to women (eg, increased substance/nicotine use and poorer diets).

 Furthermore, Robertson and Williams found that men within the age range of 16 to 44 years perceive the medical setting as more feminine and therefore underutilize health services.

 examined the relationship of male gender norms and male health behaviors in a sample of college- and community-dwelling men. They found that certain subscale scores of the Conformity to Masculine Norms Inventory-46 (CMNI-46)

 seemed to act as protective buffers of health-promoting behaviors, whereas others were risk factors. Specifically, men who self-reported higher levels of masculinity related to Emotional Control were also better at Avoiding Anger and Stress and Avoiding Substance Use. However, men who reported a stronger belief in a drive to win in life as an important component of masculinity showed a poorer ability to avoid stress and anxiety. These men also reported lower use of health care resources and less focus on diet as a health-promoting strategy. Men who endorse risk-taking and being a “playboy” as important components of masculinity were also less likely to avoid substance abuse. Risk-taking was also associated with lower health care use. However, men viewing success in life as an important aspect of masculinity were more likely to use health care properly and to see diet as an important component of health. Finally, men who viewed violence as being key to masculinity, as well as men who emphasized self-reliance, were less likely to report diet as a focus.

Although findings such as these have contributed to additional research on men’s health and other chronic conditions have been highlighted, no studies have focused exclusively on MS.

 Given this, the relationship between masculinity and health behaviors in men with MS may also be nuanced and deserves exploration. If socialized gender norms are particularly important in health behaviors and support seeking, they warrant focus to understand the relationship between how men with MS conceptualize their own masculinity and the behaviors they engage in to promote their health.

The present study explored the relationship between socialized gender norms and health behaviors in a cross-sectional survey study of men with MS receiving treatment at a large academic hospital. Specifically, the association between conformity to masculinity and health-promoting behaviors was examined. We predicted that higher levels of conformity to masculinity would be related to more negative health behaviors (anger/stress, lack of self-care, improper use of health care, poor diet, increased substance use).

In addition to exploring masculinity and health behaviors, we were interested in measuring the effect that such relationships may have on the coping styles of men with MS. Previous research suggests that differences may exist between men and women in terms of adjustment and ways of coping.

 Specifically, men with MS were found to be less likely than women with MS to seek support, practice coping skills/acceptance, or attend a support goup.

 We predicted that higher levels of conformity to masculinity would also be related to limited/rigid coping styles (nonsupport seeking, distancing, confrontive, avoidant, etc).

The institutional review board–approved study took place from January 1 through May 31, 2020, at the Cleveland Clinic Mellen Center for MS. The study protocol involved patients completing a series of survey measures at a single time point. Potential participants were contacted based on their completion of a previous survey regarding their research interest. Inclusion criteria for the study included being a man older than 18 years, being diagnosed as having MS, and receiving treatment for MS at the Mellen Center. Approximately 500 men qualified as potential participants and were sent a message via a web-based patient portal (MyChart; Epic Systems Corp) inviting them to participate in the study. Interested patients were sent an email to access the electronic surveys in a REDCap (Research Electronic Data Capture) database. Completion of the surveys qualified them for a $25 stipend.

 was used to assess conformity to 9 masculine norms: Emotional Control (6 items) (ie, emotional restriction and suppression), Winning (6 items) (ie, drive to win), Playboy (4 items) (ie, desire for multiple non-committed sexual relationships), Violence (6 items) (ie, penchant for violence), Self-reliance (5 items) (ie, desire to solve problems on one’s own), Risk-Taking (5 items) (ie, inclination toward risky behaviors), Power Over Women (4 items) (ie, perceived control over women), Primacy of Work (4 items) (ie, viewing work as the primary focus in life), and Heterosexual Self-presentation (6 items) (ie, importance of presenting oneself as heterosexual). Items are indicated on a 4-point Likert scale (1 = strongly disagree to 4 = strongly agree). Higher scores reflect greater conformity to masculine norms.

In the Ways of Coping Questionnaire (WCQ),

 respondents are asked to rate how much they used a particular strategy in response to a stressor. Items are scored on a 4-point Likert scale (0 = does not apply or not used, 1 = used somewhat, 2 = used quite a bit, 3 = used a great deal) and measure 8 coping styles: Confrontive Coping (6 items) (ie, aggressive efforts to alter the situation, suggesting some degree of hostility and risk-taking), Distancing (6 items) (ie, cognitive efforts to detach oneself and to minimize the significance of the situation), Self-controlling (7 items) (ie, efforts to regulate one’s feelings and actions), Seeking Social Support (6 items) (ie, efforts to seek informational support, tangible support, and emotional support), Accepting Responsibility (4 items) (ie, acknowledges one’s own role in the problem with a concomitant theme of trying to put things right), Escape-Avoidance (8 items) (ie, wishful thinking and behavioral efforts to escape or avoid the problem), Planful Problem-Solving (6 items) (ie, deliberate problem-focused efforts to alter the situation, coupled with an analytic approach to solving the problem), and Positive Reappraisal (7 items) (ie, efforts to create positive meaning by focusing on personal growth, includes a religious dimension). Higher scores reflect a greater reliance on a coping style.

The Health Behavior Inventory-20 (HBI-20)

 was used to measure health behaviors and consists of 20 items and 5 subscales. Three subscales reflect health-promoting behaviors: Diet (5 items), Preventive Self-care (7 items), and Proper Use of Health Care Resources (2 items). Two subscales reflect health risk behaviors: Avoiding Anger and Stress (3 items) and Avoiding Substance Use (3 items). Participants were asked to rate the extent to which each item describes themselves using a 7-point Likert scale (1 = always to 7 = never). Higher scores reflect greater efforts at health promotion.

Subscale scores are obtained by averaging response scores. A total scale score is obtained by averaging the scores of all the items. Levant et al

 reported Cronbach α values for the subscales and the full scale ranging from 0.68 to 0.79.

Disease impact was evaluated using the Patient-Determined Disease Steps (PDDS) scale,

 a self-assessment that subjectively scales personal MS disease status. Higher scores reflect higher perceived disability related to MS (normal = 0, mild disability = 1, moderate disability = 2, gait disability = 3, early cane = 4, late cane = 5, bilateral support = 6, wheelchair/scooter = 7, bedridden = 8).

The following patient demographic and clinical characteristics were collected: age, race, marital status, educational status, and years since MS diagnosis.

Descriptive statistics are used to summarize demographic and clinical characteristics (including the PDDS scale score) and subscale scores on the CMNI-46, WCQ, and HBI-20. Frequency counts with percentages are used to present categorical data, and mean ± SD is used to present continuous variables.

To examine the relationships among demographic and clinical characteristics, conformity to masculinity, and coping, Pearson correlation coefficients were generated for all variables. The associations between these variables and each of the HBI-20 subscale scores were determined using univariate linear regression models. Conformity to masculinity and coping sub-scale scores that were associated with the outcome at least at the 

 < .10 level were added to multivariate linear regression models, controlling for age, race, marital status, educational status, PDDS scale score, and years since MS diagnosis. Additional interaction effects were also evaluated between masculinity conformity and coping variables in the model.

Given the exploratory nature of the study, there was no correction for multiple testing. Missing data were handled using complete case analysis. Computations were performed in R, version 3.6.3.

Eighty-one patients completed the study protocol. Descriptive statistics are provided in 

. Race lacked variability and was, therefore, eliminated from future analyses. Also, PDDS scale scores were highly variable, with some categories showing low frequency. Therefore, this variable was dichotomized to reflect whether a patient had a score higher than 3 (gait disability). Finally, 2 subscale scores, WCQ Self-controlling and HBI-20 Avoiding Substance Use, were not found to be internally consistent (Cronbach α < 0.60) and were eliminated from further analyses.

Correlation coefficients between demographic and clinical characteristics, as well as conformity to masculinity and coping subscale scores, are presented in 

, which is published in the online version of this article at IJMSC.org. Age was related to lower levels of Self-reliance, as well as to Distancing and Escape-Avoidance as coping strategies. Being married was related to lower scores on the Confrontive Coping, Distancing, Accepting Responsibility, and Escape-Avoidance subscales. Length of diagnosis, PDDS scale score, and educational level were not related to subscale scores for conformity to masculinity or coping. For adherence to masculine norms, respondents high on Emotional Control were lower on Seeking Social Support and Accepting Responsibility. Winning was correlated with more Confrontive Coping. Men reporting higher Self-reliance reported lower levels of Seeking Social Support, as did those with higher levels of Power Over Women.

Univariate linear regression models on Diet identified the Planful Problem-Solving and Positive Reappraisal coping styles for further analyses in multivariable models. For modeling on the Preventive Self-care subscale, Emotional Control, Heterosexual Self-presentation, Planful Problem-solving, and Positive Reappraisal were retained (

). For modeling on the Proper Use of Health Care Resources subscale, Primacy of Work, Seeking Social Support, Planful Problem-solving, and Positive Reappraisal met the threshold to remain in the multivariable model. Finally, for the Avoiding Anger and Stress subscale, conformity to the masculine norms of Winning, Violence, and Heterosexual Self-presentation as well as Confrontive Coping were retained as multivariable predictors (

For the multivariable linear regression model for each of the 4 previously mentioned HBI-20 subscale scores, the following patient demographic and clinical characteristics were controlled for: age, race, marital status, educational status, and years since MS diagnosis. In addition, all CMNI-46 and WCQ subscale scores identified as predictors in the univariate analyses were added to the models. Because Planful Problem-solving and Positive Reappraisal were strongly correlated, the potential for multicollinearity was evident. Therefore, after initial examination of the impact of each subscale score on the first 3 outcomes, Positive Reappraisal was retained.

No significant model could be fit for the Diet or Proper Use of Health Care Resources subscale scores. For Preventive Self-care, only Emotional Control and Heterosexual Self-presentation were significant predictors. However, after adding an interaction between Heterosexual Self-presentation and Positive Reappraisal, a main effect for the latter also emerged. The interaction effect was also significant (

). Men reporting higher levels of Emotional Control reported worse efforts at Preventive Self-care. In contrast, men endorsing stronger beliefs in Heterosexual Self-presentation reported more efforts. In addition, engaging in Positive Reappraisal as a coping strategy was also positively associated with self-care efforts., With high and low Positive Reappraisal levels split at the median, the interaction is presented in 

. For men with low levels of Positive Reappraisal, increasing Heterosexual Self-presentation was associated with higher levels of Preventive Self-care. For those with low levels of Positive Reappraisal, increased Heterosexual Self-presentation was associated with declines in Preventive Self-care.

For the Avoiding Anger and Stress subscale score, having a PDDS scale score greater than 3 was associated with greater efforts to control negative emotions (

). In addition, CMNI-46 subscale scores for Violence and Heterosexual Self-presentation were negatively associated with controlling stress and anger. No interaction effect was found.

The present study investigated the relationship between adherence to traditional gender norms in men with MS and their health behaviors. After examining correlation coefficients and univariate regression analyses, candidate predictors in the areas of coping and conformity to masculinity were identified for multivariable models of health behaviors. Of the 5 domains of the HBI-20, 3 could not be successfully modeled. Avoiding Substance Use exhibited very poor reliability (α = 0.50) and was, therefore, eliminated from further analyses. In addition, models for Diet and Proper Use of Health Care Resources were not significant.

For Preventive Self-care, controlling for demographic and clinical characteristics, aspects of masculinity conformity related to Emotional Control and Heterosexual Self-presentation were significant predictors, as was the Positive Reappraisal coping strategy. Men reporting stronger adherence to the masculine norm of Emotional Control reported poorer Preventive Self-care. Men endorsing Heterosexual Self-presentation as a key aspect of masculinity reported greater efforts in self-care. Positive Reappraisal was also associated with higher reported self-care. In addition, the interaction between Heterosexual Self-presentation and Positive Reappraisal was significant. For men reporting low levels of Positive Reappraisal as a coping strategy, increasing Heterosexual Self-presentation was associated with higher levels of Preventive Self-care. For those with high levels of coping with Positive Reappraisals, increased Heterosexual Self-presentation was associated with modest declines in self-care.

For the Avoiding Anger and Stress subscale, having a PDDS scale score greater than 3 positively predicted efforts to control negative emotions. Related to conformity to masculine gender norms, men endorsing Violence or Heterosexual Self-presentation as important aspects of masculinity also reported less effort in controlling stress and anger. No interaction was found between conformity to masculinity and coping.

 found that the coping strategies of women with MS may differ from the coping strategies of men with MS (eg, men are less likely to accept the condition, adopt productive coping strategies, and seek social support). The present study showed that men with MS are in fact coping but may be doing so in a way that is influenced by their gender norms. Specifically, men who endorsed the gender norm of Heterosexual Self-presentation (importance of presenting oneself as heterosexual) reported greater efforts in self-care. In contrast, men with MS who endorsed the gender norm of Emotional Control (emotional restriction and suppression) reported poorer Preventive Self-care. Specific to coping, men with MS who used the strategy of positive appraisal (create positive meaning by focusing on personal growth) also reported greater effort in self-care.

The present study showed commonalities with and differences to the findings of Levant and Wimer.

 For Avoiding Anger and Stress, they found that Emotional Control was a positive predictor, as did the present study. However, they also found that Winning and Risk-Taking were predictive. For Preventive Self-care, no aspects of conformity to masculine norms were significant predictors, whereas the present study identified Emotional Control and Heterosexual Self-presentation. They also found a variety of predictive relationships between aspects of conformity to masculine norms and the subscale scores of Avoiding Substance Use, Diet, and Proper Use of Health Care Resources. A variety of factors may have contributed to these differences. First, their sample consisted of both college students and community members, with an average age of 23 years. In contrast, the sample for the present study was significantly older at nearly 52 years on average. Issues related to masculinity may have differed greatly between these 2 groups due to this difference. Perhaps also related to this, more respondents in the present study reported being married (21%) compared with only 12% reporting being married, partnered, or engaged in the sample from Levant and Wimer. Finally, approximately 13% of respondents in the sample from Levant and Wimer reported having a college degree or higher compared with nearly 50% in the present study.

Overall, the results of the present study show that aspects of conformity to masculine gender norms are associated with health behaviors in men with MS. In the case of Preventive Self-care, there was also evidence of interaction between masculinity conformity and coping strategies. However, contrary to expectations, several demographic and clinical characteristics were not found to be predictive of health behaviors at a univariate or multivariable level (ie, age, educational level, marital status, PDDS scale score, and years since diagnosis). This may be due to a restricted range of patients in terms of age and disease duration. Having a larger portion of younger men with MS in a sample may allow examination of age differences in some of these issues.

 Furthermore, the present sample consisted largely of patients with long-term disease duration (mean ± SD time since diagnosis of 12.24 ± 8.47 years). It may be that the earlier years of disease duration are critical to the examination of these issues. In addition, we did not look at sexual orientation or gender identification, which would have enhanced the study. This study population identified as 93% White, and recent studies have focused on the underrepresentation of ethnic minorities in MS.

 It is well-known that racial and ethnic diversity can play a significant role in how people with chronic disease, specifically MS, cope with their health and how health care disparities play a role in disease and progression. Finally, the cross-sectional nature of this study limits the strength of inferences that can be made about the interrelationships among conformity to masculinity, coping strategies, and health behaviors. Longitudinal data analyses would allow a clearer examination of whether and how adherence to traditional male gender norms is related to health behaviors and the mediating role that coping strategies may play.

In future research, it may be helpful to look at how men with MS differ from men living with other chronic illnesses. Also of particular interest would be an assessment of gender norms and their correlation with predicting male-identifying patients and a study of treatment approaches.

The present study displayed the nuanced relationship between masculine gender norms and specific coping strategies and male health behaviors.

Effects of Conformity to Masculine Norms and Coping on Health Behaviors in Men With Multiple Sclerosis

The ability to reliably monitor disease progression in patients with multiple sclerosis (MS) is integral to patient care. The Expanded Disability Status Scale (EDSS) is a commonly used tool to assess the disability status of patients with MS; however, it has limited sensitivity in detecting subtle changes in disability levels and, as a result, does not consistently provide clinicians with accurate insight into disease progression. At the 2019 European Committee for Treatment and Research in Multiple Sclerosis meeting in Stockholm, Sweden, a panel of neurologists met to discuss the limitations of the EDSS as a short-term predictor of MS progression. Before this panel discussion, a targeted literature review was conducted to evaluate published evidence on prognostic measures such as fatigue, physical assessments, and measures that are more taxing for patients, all of which may be useful to clinicians at different stages of the course of MS. This article summarizes currently available evidence in support of these measures. In addition, this article highlights the current state of expert clinical consensus regarding the current approaches used to predict and monitor disease progression and offers insight for future studies to assist clinicians in accurately monitoring disease progression in patients with MS.

From the Department of Neurology, Jacobs School of Medicine and Biomedical Sciences, State University of New York at Buffalo, Buffalo, NY, USA (BW-G); Department of Health Sciences, University of Genoa, Genoa, Italy (MPS); IRCCS Ospedale Policlinico San Martino, Genoa, Italy (MPS); and Swedish Medical Center at Seattle, Seattle, WA, USA (PR). 

 Bianca Weinstock-Guttman, MD, Department of Neurology, Jacobs School of Medicine and Biomedical Sciences, State University of New York at Buffalo, 1010 Main St, Buffalo, NY 14202, USA; 

 Dr Weinstock-Guttman has received grant/research support from, participated in speakers’ bureaus for, and/or served as a consultant for Biogen, EMD Serono, Novartis, Genentech, Celgene, AbbVie, Sanofi Genzyme, and Mallinckrodt and serves on the editorial board for 

. Dr Sormani has received consulting fees from Biogen, Merck, Novartis, Roche, Sanofi, Celgene, MedDay, GeNeuro, and Mylan. Dr Repovic declares no conflicts of interest.

 Financial support for the preparation of this review was provided by Celgene.

Multiple sclerosis (MS) is a chronic neurologic disease wherein overactive immune cells cause inflammation, demyelination, and axonal damage in the central nervous system. The resulting degenerative disease course requires accurate, sensitive, and comprehensive measures for the purposes of monitoring progression and making patient care decisions.

The Expanded Disability Status Scale (EDSS), an ordinal, clinician-administered assessment scale with scores ranging from 0 to 10, is widely used to assess the status of patient disability in MS. Short-term sustained changes in disability status are often reported in terms of confirmed disability progression (CDP), the presence of which corresponds to an increase in EDSS score maintained on repeated evaluation at 3 to 6 months. However, several studies have criticized the EDSS and CDP for limitations related to their prognostic value, including lack of accuracy, limited sensitivity to change at certain disability levels of the disease, focus on physical ability, and nonstandardized interpretability.

 As a binary indicator, CDP in particular may offer limited value for prediction of disability progression. Although the EDSS does have some predictive value, certain aspects, such as intrarater and interrater variability, leave room for improvement.

 Given these limitations, there is a need for expansion in the tools used to gauge disease progression in clinical practice as well as clinical trials evaluating the efficacy of new MS treatments.

In this narrative review, we first summarize the available evidence of the predictive power of a select number of commonly used measures to assess MS progression. The goal of this review is not a systematic assessment but rather a targeted review of recent literature that focuses on specific end points related to MS progression. We then discuss available evidence on key limitations of the EDSS as a prognostic measure and highlight some measures that may assist in improving the sensitivity of this scale in the clinical management of MS. We highlight general viewpoints from a panel of 4 experts in MS research (the authors [B.W.-G., M.P.S., P.R.] and Dr Enrique Alvarez of the University of Colorado) convened at the 2019 European Committee for Treatment and Research in Multiple Sclerosis meeting in Stockholm, Sweden, held September 13 to 19, 2019. At this meeting, the panel discussed the utility of clinical methods currently used to predict and monitor MS progression and suggested general directions for future studies that might help clinicians effectively monitor MS disease worsening.

Targeted Literature Review of Prognostic Measures

Before the expert panel discussion, a targeted literature review using ProQuest and Google Scholar was conducted to identify evidence on prognostic measures other than the EDSS that may be predictive of long-term MS disease progression. The targeted literature review largely focused on articles published between 2014 and 2019, although articles published before 2014 were included to capture relevant evidence. Articles not published in English were excluded from consideration. All identified studies were screened, reviewed, and synthesized based on the following measures: no evidence of disease activity; EDSS and CDP; brain volume loss; gray matter; white matter; lesions; serum neurofilament light; cognitive impairment; pregnancy and demographics, specifically age, sex, and education; relapse and annualized relapse rate; disease duration; macular volume; and Montgomery-Åsberg Depression Rating Scale. A summary of key search terms associated with each measure is provided in 

The impact of each measure used to assess the progression of MS and the patient populations evaluated are summarized in 

, which is published in the online version of this article at IJMSC.org. The evidence varied regarding each measure’s utility in assessing disease progression. Among the studies that met the inclusion criteria, the evidence regarding the predictive power of no evidence of disease activity and minimal evidence of disease activity was generally inconsistent.

 Brain volume loss generally showed positive predictive power on long-term MS progression, particularly regarding increases in EDSS score

 and assessments that combine brain atrophy and retinal thinning.

 Although assessments of gray matter reductions were also predictive of MS progression, the evidence was partially based on associations and correlations.

 Similarly, there was some evidence supporting the predictive power of white matter on long-term disease progression, but it was based on associations.

 Limited evidence was available in favor of the predictive power of other measures of brain volume, such as cerebrospinal fluid, parenchyma,

 demographic characteristics (age, sex, education),

 did not consistently predict disease progression, particularly in the long-term. Although the evidence was mixed regarding the predictive power of annualized relapse rate,

 there was some evidence to support the predictive power of relapse on long-term disease progression.

 There was also some evidence to support the predictive power of pregnancy on long-term disease progression; however, the research was largely based on congress presentations.

 The Montgomery-Åsberg Depression Rating Scale was found to have no predictive power in long-term assessments of MS progression.

There is ample evidence supporting the limitations of the EDSS as a prognostic measure. Foremost, the EDSS is meant to function as a measure of irreversible disability in MS; however, the literature shows that it fails to serve this basic purpose.

 Recently, the placebo arms of 31 randomized controlled trials in relapsing-remitting MS and secondary progressive MS patient populations were analyzed, and the results showed significant rates of EDSS score improvement, sometimes as high as rates of EDSS score worsening.

 To further illustrate the inaccuracy and instability of this instrument, the EDSS has been shown to overestimate the accumulation of permanent disability by up to 30%.

 The EDSS also has plateau scores at 6.0 and 6.5, that, over time, has discouraged researchers from including patients with these scores in clinical trials, given the need to quantify change in progression.

Further evidence shows that increases in EDSS scores do not accurately identify patients with irreversible long-term disease progression. A study conducted in persons with relapsing-remitting MS assessed multiple definitions of sustained progression using the EDSS.

 Between 15.8% and 42.2% of these individuals had sustained progression across 3.7 years, but nearly 50% of them did not maintain progression for the duration of follow-up.

 Relapses or changes in provider could not explain the poor performance of the EDSS,

 suggesting that the use of the EDSS score and the CDP to measure outcomes for clinical trials or observational studies could lead to incorrect conclusions due to the potential instability of EDSS scores.

The EDSS has also been criticized for not being a comprehensive measure of all dimensions of MS.

 Whereas lower scale values are influenced by impairments detected by a neurologic examination, values higher than 4 are mainly based on walking ability, and values higher than 6 are based on patient handicaps.

 The EDSS does not adequately capture the dimensions of cognition, upper extremity function, or fatigue, which are believed to be relevant predictors of long-term disease progression in MS.

 Moreover, studies recommend separate consideration of the lower and upper value ranges of the EDSS because EDSS scores of 6 and higher are less sensitive to change in disease severity.

 The evidence on the prognostic limitations of the EDSS across several domains highlights the scope for improved prediction of long-term disease progression in MS.

Panel’s Viewpoints on Alternative Prognostic Measures to the EDSS

Because of the potential prognostic limitations of the EDSS, it is the panel’s opinion that alternative measures are needed to better classify disease severity and assess disease progression in patients with MS. Although there is a large and growing body of evidence on the prognostic value of factors beyond the EDSS, there is no widely held understanding of whether these factors improve predictions of long-term disability beyond the EDSS. To detect clinically meaningful changes in patients with MS, prognostic measures will need increased reliability and sensitivity. To these ends, current research suggests that approaches incorporating fatigue and lower limb function, as well as combination measures, either as a supplement or alternative to the EDSS, may be promising avenues to explore. In the following sections, we provide literature to support the panel’s viewpoints on alternative prognostic measures to the EDSS.

Incorporation of Fatigue and Lower Limb Function

The related matters of fatigue and lower extremity dysfunction, including patient-reported leg weakness as well as functional limitation identified via examination, may be of particular interest among alternative measures of disease progression with promising predictive value.

Using prognostic factors beyond the EDSS may improve patient care and inform future trials in MS. Specifically, supplementing the EDSS with patient-reported physical measures, assessing fatigue, and using simultaneous cognitive and motor function testing may help in gauging MS disease progression.

Nicholas LaRocca, PhD

Articles

Latest Updated Articles