Validity and Reliability of the Scale to Report Emotional Stress Signs–Multiple Sclerosis (STRESS-MS) in Assessing Abuse and Neglect of Adults With Multiple Sclerosis

Approximately 30% of people with multiple sclerosis (MS) experience sufficient disability to require caregiving,¹ most of which is provided by unpaid family members² and other untrained and uncompensated lay caregivers, defined herein as “informal caregivers.”³ Most informal caregiving benefits recipients and may allow them to remain at home.^2–4 It can, however, increase risk of mistreatment.⁵

Although, to our knowledge, no published studies to date have elucidated the nature or extent of abuse and neglect of people with MS, existing publications document mistreatment of adults with disabilities⁶ and seniors, an estimated 50% of whom undergo psychological abuse by caregivers.⁷ Adults of all ages with physically disabling conditions experience higher rates of physical and sexual abuse than the general population.^{5 , 8} Risk factors for mistreatment include social isolation, impaired mobility, and neuropsychiatric comorbidities such as depression⁹ and cognitive impairment.^{10 , 11} Despite the high prevalence of mistreatment, only a minority of people with disabilities have ever discussed personal safety with a health care provider.¹² Many adults with physical disabilities who are victimized by domestic violence also engage in risky health behaviors and experience poor health outcomes.¹³

We are aware of 1 published study that specifically addressed mistreatment of people with MS: a focus group study of abusive relationships between people with advanced MS and their informal caregivers by our research group¹⁴ suggested that people with MS may not disclose their own mistreatment because of shame, feeling they deserve mistreatment, or fearing loss of the caregiver. Because depressed, cognitively impaired, and substance-abusing adults with disabilities are at higher risk of being mistreated,^9–11 it is especially concerning that people with MS are more likely to experience cognitive impairment, alcohol abuse, anxiety, and depressive disorders.¹⁵ Laypeople providing uncompensated care for adults with advanced MS become psychologically burdened themselves,¹⁶ often experiencing mounting levels of fatigue, depression, and anger that can undermine empathy for the person with MS.¹⁷ Serious cognitive impairment in people with MS, for example, tends to increase their caregivers’ stress more than physical disability does.¹⁸

To better understand these complex issues, our research team at the University of California, Irvine (UCI), Program in Geriatrics undertook the first extensive study of caregiver mistreatment of American adults with MS. The objectives were to develop, validate, and field-test a self-report measure of mistreatment, the Scale to Report Emotional Stress Signs– Multiple Sclerosis (STRESS-MS), and to explore risk factors for caregiver mistreatment. We hypothesized that mistreatment of people with MS would approach 50% prevalence and correlate with greater disability, limited social support, higher weekly caregiving hours, and caregiver substance abuse. The STRESS-MS questionnaire also served as the primary outcome measure for our nationwide survey of adults with advanced MS reported elsewhere.¹⁹

Methods

Participants

The UCI’s institutional review board approved the study, and the US Department of Health and Human Services granted a certificate of confidentiality. All study participants provided written informed consent. Inclusion criteria for people with MS included being at least 18 years old, speaking English, being diagnosed as having MS, having a primary informal caregiver who was a family member or friend, and having sufficient MS-related disability to require assistance with at least 1 activity of daily living (bathing, dressing, toileting, transferring, continence, and feeding)²⁰ or instrumental activity of daily living (using a telephone, shopping, preparing food, housekeeping, doing laundry, traveling, or taking medications).²¹ Inclusion criteria for caregivers included being at least 18 years old, speaking English, and serving as the primary informal caregiver for a person with advanced MS, following the definition by Buchanan and Huang²² of “the person who provides the majority of informal or unpaid care to you to help you cope with the effects of MS on your daily life.” The study excluded those who did not require caregiving or who lacked decision-making capacity to participate in research as measured by the Short Portable Mental Status Questionnaire.²³ We recruited adults with advanced MS and patient-caregiver dyads from local MS centers at UCI (Orange, CA), University of California, Los Angeles, and Eisenhower Medical Center (Rancho Mirage, CA), as well as from notices by the National Multiple Sclerosis Society (NMSS). We mailed 1000 recruitment letters to UCI Comprehensive MS Care Center patients and their family caregivers. We also placed recruitment flyers in Orange County neurologists’ offices. The local NMSS chapter included an electronic version of the recruitment flyer on their website and in electronic newsletters.

Developing and Validating the STRESS-MS Questionnaire

The measure development team was a 10-member multidisciplinary advisory board composed of MS specialists, experts in mistreatment of adults with disabilities, a caregiver, and a person with MS. The board used 5 major categories of mistreatment²⁴ of vulnerable adults (physical abuse, psychological abuse, neglect, financial abuse, and sexual abuse) to adapt the Revised Conflict Tactics Scales (CTS2)²⁵ into a new instrument, culling a list of CTS2 items most relevant to MS. Based on the validated CTS used as an outcome measure in hundreds of published studies of abusive relationships,²⁶ the CTS2 is a refined measure of interpersonal abusiveness that includes 39 items in 5 subscales (negotiation, psychological aggression, physical violence, sexual coercion, and injury). We selected the CTS2 to adapt because its factor structure fit the study population and because a published study describing its development and psychometric data²⁷ showed acceptable construct validity, discriminant validity, and reliability ranges (0.79–0.95). The board ranked each of the selected CTS2 items and offered detailed feedback. To assess face validity, we recruited 5 people with MS and 5 informal caregivers to pilot-test the draft questionnaire over a 2-week period through semistructured interviews, then used their feedback to finalize the STRESS-MS instrument and adapt it into 3 versions: 1 for adults with MS (Table 1; available online at IJMSC.org), another for caregivers, and a dyad version. The caregiver and dyad versions use the same item stems as the person-with-MS version. For example, the “My caregiver misused my possessions” item for people with MS was adapted to “I misused possessions of the person with MS.”

Over a 10-month period, 2 study personnel—a gerontologist specializing in elder abuse and a master’s level clinical research associate—administered the STRESS-MS questionnaire and all other study measures in person in the homes of people with advanced MS and additional informants living in Southern California, including patient-caregiver dyads. They collected demographic information and self-reported substance use history from all study participants. To examine risk factors for mistreatment, they administered additional measures to the participants with MS, including the Beck Depression Inventory²⁸ and the Modified Multiple Sclerosis Functional Composite,²⁹ a 3-part standardized measure of MS-related disability that includes the Timed 25-Foot Walk test, the Nine-Hole Peg Test, and the Symbol Digit Modalities Test, which served as this study’s measure of MS-related cognitive impairment. Both the gerontologist and the research associate received training in administering the Modified Multiple Sclerosis Functional Composite, including multiple practice tests. During the home visits, participants with MS also completed the CTS2 negotiation and psychological aggression subscales²⁵ (which include items analogous to those listed in Table 1 for the psychological abuse section of the STRESS-MS, eg, “Insulted or swore at my caregiver”), the Marlowe-Crowne Social Desirability Scale,^{30 , 31} the Medical Outcomes Study–Social Support Survey³² as adapted by Acierno and colleagues,³³ the Modified Fatigue Impact Scale,³⁴ the Multiple Sclerosis Self-Efficacy Scale,³⁵ and the Quality of Life Scale.³⁶ Additional measures collected for caregivers included the Caregiving Tasks in MS Scale³⁷ (which included whether the caregiver was assisting the person with MS with bladder or bowel care because of incontinence), questions about the caregiving relationship (nature of the relationship, duration, exclusivity, frequency of contact, and average number of caregiving hours per week), and whether the caregivers had any mental health diagnoses. Study visits lasted approximately 2 hours, typically scheduled during morning hours with breaks as needed to limit fatigue of participants with MS.

A Longitudinal, Expert, All Data (LEAD) panel followed methods described by members of our research team^{7 , 38} to serve as a criterion standard for validating STRESS-MS. The LEAD panel included an MS specialist physician, a gerontologist specializing in elder abuse, an Adult Protective Services manager for dependent adults, and a social worker specializing in domestic violence. The panel reviewed all data from the face-to-face interviews, determining for each caregiving dyad whether mistreatment had occurred, then documenting severity and the panel’s certainty of the mistreatment. To assess for any direct evidence of mistreatment, the study personnel collected and presented to the LEAD panel data from the Abuse Assessment Screen–Disability,³⁹ the modified Elder Abuse Instrument,⁴⁰ a financial abuse interview,⁴¹ the Safety of the Environment section of the Self-neglect Severity Scale,⁴² and the caregiver’s STRESS-MS responses. To corroborate self-reports by study participants, the research team independently assessed every home for direct evidence of neglect or safety hazards. If indicated by interview responses of participants with MS or of informants, they asked to examine the participant with MS for physical evidence of abuse or neglect, including poor hygiene, malnutrition, suspicious pressure ulcers, lacerations, contusions, burns, or other injuries. The study team privately provided anticipatory guidance to these study participants about how to seek help if they were mistreated again.

After reviewing all the data described in the preceding paragraph, LEAD panel members assessed whether any of the 5 types of caregiver mistreatment had occurred since the participant’s MS-related disability began. We blinded panelists to the identity of all study participants. Each panelist independently completed an outcomes form for each mistreatment category: presence of the type of mistreatment (yes or no), then if present, its severity (using a 1–4 Likert-type scale for mild through severe with an option of “unable to assess severity”), and finally, certainty that the mistreatment type had occurred (low, medium, or high). The LEAD panel then used round-robin polling by concealed written ballot with discussion of disagreements and repolling as needed, after a predetermined consensus process with voting if needed to reach the panel’s final verdict. As a last validation step, the advisory board used a consensus approach⁴³ to re-rank each item for the final version of the STRESS-MS.

Statistical Analysis

Item Analysis and Subscales

A biostatistician (D.S.) conducted an item analysis⁴⁴ to identify and eliminate items not supported by the advisory board and LEAD panel analyses. Within the STRESS-MS instrument, we developed 3 subscales for each of the 5 subtypes of mistreatment: incidence (whether each mistreatment event had occurred within the preceding 12 months), chronicity (how many times each event had occurred within those 12 months), and prevalence (whether each event had occurred since the person with MS first required caregiving). The final version of STRESS-MS included only those items that enhanced internal consistency reliability, as indicated by a corrected item-total correlation of 0.30 or greater. We estimated internal consistency reliability of each subscale using Cronbach α.

Validity

We calculated Spearman rank correlations and receiver operating characteristic curves to examine relationships between each STRESS-MS subscale and LEAD panel scores and to help establish cutoff points for how many occurrences of behaviors constituted abuse. For concurrent and discriminant validity we administered the Abuse Assessment Screen–Disability³⁹ and the negotiation subscale of the CTS2,²⁵ respectively, counting a single endorsement as positive for all items except the psychological abuse items, for which we required a stricter cutoff point of the abusive behavior having occurred at least twice.

Results

Study Participants

A total of 347 people with MS responded to the recruitment letters and notices and underwent telephone screening with study personnel, including 102 people with MS who met the inclusion criteria and consented to enroll in the study (84 from UCI, 11 from the NMSS website recruitment, and 7 from other local neurology offices). Of these 102 study enrollees with MS, 47 enrolled within patient-caregiver dyads, that is, their informal caregivers also consented to participate in the study. The other 55 participants with MS had informal caregivers who declined enrolling in the study themselves. We telephone screened 245 additional people with MS who responded to recruitment materials but did not enroll in the study: 183 did not meet the inclusion criteria because they did not need caregivers and 62 declined enrollment after hearing what study participation entailed. Two patient-caregiver dyads who initially enrolled were later removed from the study: the LEAD panel deemed 1 person with MS to have insufficient disability to require a caregiver. The panel found the home of a second person with MS unsafe for research personnel to visit and reported the household to Adult Protective Services.

Table 2 describes the 102 participants with MS and 97 of their informal primary caregivers, 47 of whom opted to participate directly in the dyad component of the study. The typical study participant with MS resided in Orange County with a spouse or other relative as an unpaid primary caregiver, was unemployed yet had a relatively high median household income of $50,000 to $74,999 per year, and had been diagnosed as having MS for an average of 18 years. More than three-quarters of participants with MS were female.

Characteristics of Study Participants

Mistreatment of Participants with MS

The LEAD panel found that 57 study participants with MS (55.9%) had definite evidence of having been abused or neglected since they had started to require a caregiver. Another 4 participants (3.9%) may have been abused or neglected but the LEAD panel lacked conclusive evidence. Among the 57 participants whom the panel determined to have undergone obvious mistreatment, 46 (45.1%) had been psychologically abused, 23 (22.5%) neglected, 18 (17.6%) financially abused, 13 (12.7%) physically abused, and 5 (4.9%) sexually abused. As mandated reporters for suspected mistreatment, the study team reported 28 households (27.5% of the entire study population) to Adult Protective Services agencies in their home counties.

The most commonly reported category of mistreatment was psychological abuse (Table 1). Nearly 53% of participants reported that a caregiver had shouted in anger at the person with MS. Nearly 10% of respondents with MS reported that a caregiver had used their credit or debit cards without permission. A substantial minority reported physical abuse by caregivers, with a 6.9% prevalence of being grabbed, pushed, or shoved in anger. Participants endorsed a few items from the neglect subscale, with approximately 5% endorsing that caregivers had failed to provide necessary food or to pay utility bills. Nearly 4% of respondents indicated that they had been touched sexually in a way that they did not want.

Validation of STRESS-MS Questionnaire

Reliability

Although the mean ± SD of STRESS-MS subscale scores varied widely from 1 subscale to the next, most showed adequate internal consistency when evaluated using the prevalence data, with a Cronbach α of 0.75 for psychological abuse, 0.80 for financial abuse, 0.71 for physical abuse, 0.57 for neglect, and 0.53 for sexual abuse (Table S1, which is published in the online version of this article at ijmsc.org).

Validity

Receiver operating characteristic curve data for each of the STRESS-MS subscales showed sensitivity and specificity, respectively, as follows: 0.81 and 0.96 for psychological abuse, 0.75 and 0.98 for physical abuse, 0.74 and 0.91 for financial abuse, 0.67 and 1.00 for sexual abuse, and 0.46 and 0.97 for neglect. Some subscales for the person-with-MS version of the questionnaire correlated better than others with criterion-standard LEAD panel ratings, with Spearman rank correlations for prevalence data (Table S2) ranging between 0.52 for neglect and 0.76 for sexual abuse (P < .001). The caregiver version had lower validity overall, correlating poorly with LEAD panel data (Table S3). Caregivers reported lower levels of most categories of mistreatment except psychological abuse and neglect, which they tended to report more frequently than did participants with MS (Table S4).

To measure the concurrent and discriminant validity of STRESS-MS, we scored and correlated the Abuse Assessment Screen–Disability with the overall STRESS-MS prevalence scale and also with each of its subscales. Certain variables for participants with MS, including older age and higher social support scores, conferred a protective effect against mistreatment. For every year of increase in age, people with MS were 13% less likely to be mistreated, and for every unit increase in social support they were 32% less likely. The logistic regression found other variables to be risk factors for mistreatment: the higher participants with MS scored on the Beck Depression Inventory or on the psychological aggression subscale of the CTS2, the more likely their caregivers were to abuse or neglect them. A high number of weekly caregiving hours was the greatest risk factor for mistreatment. The LEAD panel was considerably more likely to categorize as abusive or neglectful those caregivers who spent 20 hours per week or more taking care of the person with MS.

Discussion

The STRESS-MS questionnaire was reasonably reliable and valid for detecting caregiver abuse and neglect in this study population of 102 adults with advanced MS and 47 patient-caregiver dyads. The study participants largely reflected the typical demographics of adults living with severe MS-related disability: older, unemployed, and limited in mobility and resources and, therefore, reliant on caregivers for basic needs. Through their responses on the STRESS-MS questionnaire, participants reported worrisome rates of every category of mistreatment, including psychological abuse (52.9%), financial exploitation (9.8%), physical abuse (6.9%), neglect (4.9%), and sexual abuse (3.9%). The investigation team filed reports with Adult Protective Services and other agencies on behalf of 27.5% of the total study population. Although we were not mandated to report psychological abuse, we were nonetheless concerned to learn that more than half of the participants had been victimized by various forms of emotional abuse, including being shouted or cursed at, separated from loved ones, or threatened with abandonment or physical abuse.

Most STRESS-MS subscales showed adequate internal consistency. All subscales except neglect correlated highly with LEAD panel ratings, the de facto criterion standard for validity. The caregiver version of STRESS-MS had lower validity than did the person-with-MS version, presumably because caregivers were reluctant to report abusiveness. Yet caregivers were more likely to report their own psychological abusiveness and neglectfulness than were the participants with MS themselves. When evaluating the concurrent and discriminant validity of the STRESS-MS measure, we noted protective factors against mistreatment that included better social support and—perhaps surprisingly—older age. Additional risk factors for mistreatment included depression and psychological aggressiveness. Neither disability level nor history of substance abuse correlated with mistreatment risk. The single greatest risk factor for mistreatment was for the informal caregiver to spend 20 or more hours per week taking care of the person with MS. To these investigators, this latter finding is relatively encouraging because it may be modifiable by reducing caregivers’ work hours, a measurable objective if not an easy one to achieve.

This study was limited by a small sample size and a restricted catchment area in southern California. Because recruitment drew most study participants from UCI Medical Center, they represented a more affluent and less ethnically diverse group than the general population, which may limit how well the results may generalize to other populations. This assessment of participants’ disability level was limited to Multiple Sclerosis Functional Composite scores. This study did not address formal caregiving by clinicians and other trained individuals who are compensated for providing care. We found the neglect and sexual abuse STRESS-MS subscales less reliable than we had hoped, but because these were the least commonly reported categories of mistreatment, and because the other 3 subscales showed substantially higher reliability, we believe that the STRESS-MS questionnaire still offers utility. Future research needs to explore the prevalence and characteristics of mistreatment of people with MS by formal caregivers, including paid staff of skilled nursing facilities.

This validation study adds to the literature a new survey measure that we hope MS professionals will find useful for screening people with advanced MS at risk of abuse and neglect. In this preliminary study, STRESS-MS uncovered concerning rates of mistreatment of people with MS in southern California. In the context of the published literature documenting high mistreatment rates in adults with a variety of disabling conditions,⁶ this study’s results were not entirely surprising. It was nonetheless disconcerting to find that more than 50% of participants with MS in this pilot study disclosed—in the presence of their caregivers—that they had survived various forms of abuse or neglect. We wish to acknowledge the majority of informal caregivers who are not abusive and who provide excellent care under circumstances that can at times be extraordinarily challenging. We hope, however, that the preliminary data described in this report will highlight the vulnerability of people with advanced MS and their caregivers to the underrecognized public health issues of abuse and neglect.

PRACTICE POINTS

The Scale to Report Emotional Stress Signs–Multiple Sclerosis (STRESS-MS) was a reasonably reliable and valid questionnaire measure for detecting abuse and neglect of people with MS by their informal caregivers.

More than 50% of participants with MS reported mistreatment, including psychological abuse (52.9%), financial abuse (9.8%), physical abuse (6.9%), neglect (4.9%), and/or sexual abuse (3.9%).

The greatest risk factor was an informal caregiver spending more than 20 hours per week caring for the person with MS.