Home-Based Computer-Assisted Cognitive Training

Within the last 20 years, cognitive problems experienced by people with multiple sclerosis (MS) have become the subject of systematic research. It is clear that a large proportion (50–75%) of people with MS experience some type of cognitive impairment and that these impairments have a substantial impact on functioning, employment, and quality of life.^1–4 Research has documented the negative effects of MS on attention, information processing speed, new learning, and memory⁵ as well as aspects of executive functioning.² Although an expert review panel convened by the National Multiple Sclerosis Society in 2006 recognized the strong need for cognitive rehabilitation interventions for people with MS, work on empirically based treatment of cognitive deficits is in its infancy.^{5 6}

Although not extensive, there is some evidence to suggest that individual computer-assisted cognitive rehabilitation results in improved performance on neuropsychological tests for those with subjective concerns about or objective evidence of cognitive impairment. Initial evidence for the effectiveness of current cognitive interventions and functional magnetic resonance imaging studies⁷ suggest the presence of neuroplasticity in the brains of individuals with MS and provide support for the idea that restorative exercises using computer training may result in improvement in level of impairment. To date, most studies have explored the use of “in-office” computer programs. The feasibility and acceptability of computer training programs used at home rather than in the researcher's or clinician's office are important considerations related to the transferability of empirically based research programs to general practice.

A search of MEDLINE and review of references of published articles identified ten studies published in English (1994–2010) of computer-assisted cognitive rehabilitation programs for people with MS (Table 1).^7–16 All studies took place outside of the United States. Interventions focused on different cognitive domains, had participants with varying types of MS, and used a variety of computer programs. In the majority of the studies, the computer training occurred in a clinical or professional setting, often with one-on-one supervision by a psychologist or research assistant. Three of the studies^{7 14 16} reported interventions in which participants were able to use the computer program at home. Only one of the studies¹⁴ reported any participants' perceptions of the computer training. The authors indicated that there was a high degree of compliance with the training program and that participants reported that they enjoyed the game-like training.

Studies of computer-assisted cognitive rehabilitation programs

Computer-based training has several advantages over other types of training. The training can be self-paced and tailored to particular deficits or preferences. Programs typically can provide immediate feedback regarding success with the program, although the meaning of “success” may vary and may not be clear to participants. Although a home-based approach would seem to have considerable advantages related to cost, accessibility, and transportation, it does require that participants interact “alone” with the demands of the computer training and negotiate the program with limited outside assistance. If improvement in cognitive functioning requires practice and repeated training with the computer program, then users must perceive the program positively. The purpose of this study was to explore the feasibility of home-based computer-assisted cognitive training and systematically examine the perceptions of individuals with MS regarding a computer program (Neuropsychonline) as part of a randomized controlled trial of a cognitive rehabilitation intervention. Data reported here are qualitative in nature and were collected during the home training and during postintervention focus groups.

Methods

This exploratory, qualitative, descriptive study of perceptions of computer-assisted cognitive rehabilitation includes data collected as part of the weekly homework assignments during the 8 weeks in which participants were using the training program at home and a focus group conducted when each of three cohorts of participants had completed the intervention and follow-up data collection (3 months after the computer training). The protocol for the overall randomized controlled trial (including all procedures used in this analysis) of the cognitive rehabilitation intervention was approved by the institutional review board at The University of Texas at Austin. Potential participants were screened for study eligibility (see details below), consented to the study, and then completed baseline measures of neuropsychological performance as well as a series of questionnaires on perceived functioning and quality of life. Following baseline data collection, participants were randomly assigned to either the active intervention group or a wait-list control group. Data reported here are from the 34 participants who were assigned to the intervention group, attended some of the eight 90-minute classes, and provided responses to the weekly homework assignments.

Participants

Potential participants were recruited for the study through physician referral and notices in MS newsletters and publications. To participate, individuals had to be aged 18 to 60 years, be capable of understanding and complying with the study protocol, be able to read and write in English, and have visual acuity of at least 20/70 with correction in order to work on the computer screen. All participants had a clinically definite diagnosis of MS that was documented by their physician and stable disease at the time of entry into the study (relapse-free for at least 90 days). The Perceived Deficits Questionnaire¹⁷ was administered by phone, and those indicating that they experienced at least five problems “sometimes” or more often were eligible to participate. A total of 36 individuals were randomly assigned to the intervention group following the completion of baseline testing. Thirty-four of these individuals were able to participate in the cognitive rehabilitation intervention and provided data about the computer program on their weekly homework assignments; 21 of these participated in poststudy focus groups. Table 2 provides a summary of the characteristics of the participants. Most were female (85%), married (59%), and unemployed (68%). Almost all (97%) participants had completed high school, and over half (52%) had bachelor's or graduate degrees. Participants had an average (SD) age of 48.94 (8.37) years, and the majority (59%) had been diagnosed with MS at least 10 years previously. Participants completed the Self-Administered Expanded Disability Status Scale (SA-EDSS)¹⁸ at the baseline testing. Total scores on the SA-EDSS ranged from 2.5 to 8.5, with a mean (SD) score of 5.32 (1.23).

Characteristics of participants (N = 34)

Intervention

The overall intervention, MAPSS-MS (Memory, Attention & Problem Solving Skills for persons with MS), assessed in the larger feasibility study was intended to help participants build self-efficacy for skills to improve cognitive functioning and activities of daily living. The MAPSS-MS has two components: 1) weekly group sessions focused on building efficacy for use of cognitive strategies, and 2) a computer-assisted cognitive rehabilitation (CACR) program. The group component of the intervention was held in an accessible central location. It provided information specific to cognitive difficulties experienced by people with MS and helped participants assess their difficulties and identify strategies to manage those difficulties and improve functional performance. The home-based computer component enabled the participants to engage in intensive practice sessions to improve selected cognitive skills without leaving their homes.

Computer-Assisted Cognitive Rehabilitation (CACR) Protocol

The CACR methodology used in this study was developed by Bracy and colleagues at The Neuroscience Center of Indianapolis. The Neuroscience Center maintains a subscription-based website (www.neuropsychonline.com) providing web-based clinical applications. The website is secure and provides working, interactive programs and applications that run directly in standard web browsers. Individual users do not have to purchase or install software. The principal investigator (AS) subscribed to the website service for a fee for each participant and received unlimited access for each enrolled participant. The delivery of the CACR exercises over the Internet allows the participant to log in and complete the exercises from any computer with Internet access. As part of the intervention, the participants were prescribed practice with four of the six tracks of the Neuropsychonline cognitive rehabilitation therapy program. The four tracks (described below) address the most common problems experienced by individuals with MS. Each task in each track is set up in a game-like format.

Track 1: Attention Skills. This track is designed to provide training in focusing, initiating responses, attention shifting, and divided and multiple simultaneous shifting. Exercises include simple auditory and visual reactions to fixed and random points and divided auditory and visual attention.

Track 2: Executive Skills. Executive skills play a major role in information processing, abstract thinking, problem solving, initiation, and inhibition. Exercises in this track address organizing information, attributes and groups, sequences, serial addition, and simultaneous multiple addition.

Track 3: Memory Skills. Memory skills are essential to store, retain, and recall information, events, and procedures. Working memory—the memory that one uses at a particular moment of engagement in an activity—requires joint effort involving attention and executive skills. Exercises in the memory skills track include sequenced recall (visual, auditory), reversed sequenced recall, color match, trail trace, sequenced blocks, and recall for objects and locations.

Track 4: Problem Solving Skills. Problem solving skills allow one to analyze the facts of a situation to come to a conclusion about the situation. Exercises included in this track include puzzles requiring deductive reasoning and other puzzles that require the participant to organize and analyze facts.

During the first of the eight weekly group sessions of the MAPSS-MS, the facilitator had laptops and wireless Internet available so that participants could log on and work through a sample exercise (approximately 30–45 minutes). Additional research staff were available to help participants as needed with their first experience with the program. Participants were given detailed verbal and written instructions on how to log on at home, and they were asked to practice for a minimum of 45 minutes, three times per week. The length and frequency of practice time were selected based on findings from existing studies of computer training in individuals with MS (Table 1). Participants were told that the “games” would be difficult and that they should simply try their best and practice for at least the minimum amount of time prescribed. At subsequent group sessions, as we realized the significant challenges that participants were experiencing with the program, we emphasized that the practice exercises were not “games to win” but rather difficult practice. We also provided wireless laptops before and after class so that participants could practice (if desired) with others. Participation was limited, but those who took this opportunity seemed less frustrated. The facilitator of the intervention (a master's-prepared nurse with specific training in MS and the intervention) was available by e-mail and phone to assist participants who had trouble accessing the program. For the first 4 weeks of the intervention, participants had access to only one prescribed module each week (attention, memory, executive function, and problem solving). During the last 4 weeks they were allowed to practice on any of the tracks, but were asked to maintain their practice level at a minimum of three times per week and 135 minutes per week.

Data Collection

During each week of the 8-week MAPSS-MS intervention, participants were asked to complete a “homework sheet” and return it at the following week's class. The homework sheet repeated instructions for accessing the program and the minimum amount of practice required. Participants were asked to record their sessions on a log sheet provided (date, start and end time, total minutes) and were also asked to record comments about strategies they were learning in class—whether they were helpful, whether they tried them, and whether they would continue to use them. Participants also used this space to provide comments about their experiences with and perceptions of the computer program.

Additional data were collected as part of an “end of study” focus group conducted after each cohort completed the entire study protocol. The focus groups took place approximately 3 months after the computer training had ended and included a total of 21 participants, with each group consisting of five to nine participants. The purpose of the focus group was to assess participants' perceptions of the program after they were no longer actively engaged in the intervention. A co-investigator, who is a trained focus group moderator but did not participate in the delivery of the intervention, led each group. Each focus group session lasted approximately 1 hour and was audiotaped and transcribed verbatim. Other staff members who were also not involved in the intervention delivery took notes. Focus group participants were asked about their experiences and perceptions of the CACR program, including how it helped them build their cognitive skills and how they were able to incorporate strategies learned in class or from using the computer program into their lives. Additional probing questions were used to address issues and topics as they arose in the discussions.

Data Analysis

Descriptive statistics including frequencies, means, and standard deviations were used to assess participant characteristics and to summarize the number of homework assignments turned in and the frequency and duration of practice with the computer program. Responses on the homework sheet were transcribed verbatim into a text file. Comments were sorted by participant, cohort, weekly topic, and track of the intervention. Qualitative content-analysis techniques were used to obtain a comprehensive summary of the experience of using the CACR program at home and to convey it “in everyday language.”¹⁹ Qualitative content analysis is a form of analysis of verbal (written or spoken) data and is oriented primarily toward summarizing data collected. According to Sandelowski, qualitative content analysis is data-derived; that is, codes are generated from the data collected in the course of the study and then systematically applied.

Following each focus group, the recorders and moderator debriefed and identified preliminary themes for further analysis. The notes from the focus group recorders were then compared to the audiotapes of the focus groups. General themes from the focus groups were extracted, and a written report was produced for each focus group. The themes were then compared across the three focus groups. Themes that were consistent across the three groups received particular emphasis in the results presented here.

Results

Findings presented here include the amount and frequency of practice time as well as the participants' perceptions during training and perceptions 3 months after the training. As previously noted, participants were instructed to use the computer program for at least 45 minutes three times per week—a total of 135 minutes per week and 1080 minutes over the 8-week program. As shown in Table 3, the mean frequency of practice sessions per week exceeded three for all assigned modules. The reported mean amount of practice per week for the group also exceeded the minimum required (135 minutes) for each module. The relatively large standard deviations reflect the variability in the number of times and number of minutes the participants practiced with the program. More than two-thirds of the participants met the minimum number of times and number of minutes for each of the prescribed modules. However, only 55.9% met or exceeded the minimum total number of times of practice (24) for the entire 8-week period, and 67.6% met or exceeded the minimum total number of minutes (1080) over the 8 weeks.

Reported use of computer program (N = 34)

Perceptions During Training

The participants' comments expressed on the homework sheets completed during the 8-week intervention were grouped into three major categories and a series of subcategories. The three primary categories were as follows: 1) features of the computer program, 2) experiences using the program, and 3) strategy use. Each of these categories and the related subcategories are described below.

Features of the Computer Program

Many of the participants had comments regarding features of the computer program, including comments about 1) directions/getting started/unclear purpose, 2) lack of feedback or reinforcement, and 3) suggestions for improvement of the program. Some participants thought that many of the modules of the computer program were confusing. The most common complaint was a lack of direction. There were several comments questioning the purpose of sequencing and color changes during some of the exercises. Another complaint was the lack of indication of the participant's progress when working within a specific module. One participant wrote, “The highlighting and sequencing is deceiving,” and another participant expressed frustration in reporting that it was “not clear what this exercise is supposed to help. I couldn't tell how to do better.” Regarding the lack of positive feedback and reinforcement, another participant wrote, “I found the lack of positive reinforcement to be a negative aspect. I did not like having to go over the same exercises time and again from the start without getting credit for success.” One suggestion for improvement to the computer program was to make it more adaptable to the participants so they could participate more easily. One participant wrote that it “would be nice to have more hints or suggestions to help with the exercise. Perhaps a demonstration on correct methods of problem solving; an example could be given then [you] could go on to solve another problem on your own.” Another participant wrote, “better if I could have typed in the numbers rather than using the mouse.”

Experiences Using the Program

The majority of the participants described their experiences with the CACR program as difficult. The subcategory “physical and sensory challenges of MS that make it difficult” included participants' perceptions that MS-related limitations interfered with their ability to complete the computer exercise. The problems described included having difficulty visualizing the screen, tired fingers and joints, shaky hands, fatigue, and depression. One participant explained, “fingers and eyes are not working together . . . I get tired [and] my spastic nerves would click on a blank screen . . . my hand would move the mouse and I couldn't get back on the square.” Sometimes, even when the participant understood the exercise, he or she perceived that MS symptoms prevented successful completion of it: “I understand what to do [but] my fingers gets to clicking when it is time to stop…brain to finger just isn't fast enough.”

A second subcategory of experiences with the computer program was “recognizing cognitive weakness/limitations.” Many participants commented that they were able to identify their cognitive weaknesses and limitations while completing the exercises. Practicing on the four different tracks (attention, executive function, memory, and problem solving) helped participants realize that cognitive limitations were unique to each individual. For example, while one participant found the memory track to be frustrating and challenging in a negative way, another participant found the same track to be challenging in a positive way.

“Negative experiences” related to the computer program often referred to a noise or some visually distracting feature during the exercise. One participant described an experience in the memory skills track: “[I] tried remembering groups of 2 numbers at a time [and] had to close my eyes during the clock because it was too distracting.” Another participant wrote “organizing—the noise was distracting—by the time you looked for the commonality, the new sequence was up. So I stopped looking to see what the commonality was and focused on hitting the correct answer.” Several of the participants noted that the program would force them to go back and complete a prior level when they did not successfully complete the more advanced level. One participant described this as “I got depressed from being so frustrated . . . then the computer would glitch or something prevented me from going forward.” Another participant wrote, “Not sure why I passed level 2, then when I didn't pass level 3, I have to go back and complete level 2—very frustrating!”

There were also several “positive experiences” reported. Participants reported that the program helped them not only identify cognitive limitations but also improve their minds by staying mentally active. One participant described the program as “definitely worth the effort. I do believe this program has helped me . . . [I] have become more mentally active in solving problems that I had not worked on before, or for a long time, and helped me with a push to become intellectually active.” Another participant reported, “I found the program good because it did get me working on problems that I had not worked on in a long time. This reinforced my abilities to actually work on a problem normally out of my sphere. It did stimulate my working on problems that I normally don't see or work on. This reinforced my cognition abilities.” The participants also noted that they had different responses to particular exercises: “I found the CACR exercises that were in my personal mind set and way of thinking were the ones I enjoyed the most, not that I always did these quickly or correctly, but I was more likely to be able to work out any problems I ran into. I would more likely stay with the exercises that interested me and exercises that were more in my thinking.”

Strategy Use

The third category included comments regarding strategies participants used when working on the CACR program, as well as strategies they reported using to improve their lives with regard to physical and mental function and coping. The most important strategy related to use of the computer program described by participants was mental preparation before each exercise. Examples of mental preparation included being calm and patient. One participant wrote, “I do stretching exercises to warm up and tried to stay calm, a Zen attitude.” Another participant explained, “[I] had to spend twice as much time completing exercises as they increased in difficulty. I had anticipated this and planned to double my time spent due to the session.” Some participants referred to having the right attitude when performing the exercises and described determination and persistence as their mental foundation. One participant wrote, “I knew I already had 3 45-minute times [completed], but I wasn't satisfied with not finishing level 1. So I started again. This time I decided to stay left handed until I passed 3 back to back . . . then I had to decide what to do next—go right handed until 3 in a row or go to level 2. I did a little of both.” Participants also described successful strategies used to help master a level or skill, including repeating words or numbers out loud, using pen and paper, taking frequent breaks, and using association of problems to common tasks in life. For example, one participant described how association was used in the memory skills exercise: “I am beginning to make associations with the number—time on a clock—if the number is 730, then I can easily see 7:30 on a clock, if the group of numbers is 495, I think of $4.95.” Another participant described a similar experience: “In regards to the numbers, some strategies used were 5-digit zip codes, 7-digit phone numbers.”

Many of the strategies participants described using in everyday life were similar to the strategies used with the CACR computer program, including reading information out loud for improved understanding and comprehension, minimizing distractions to focus on important tasks, using verbal associations to remember names of people or numbers, and taking naps as needed to recoup energy. Additional strategies related to daily life included using a “to-do list” and a calendar to help remember daily tasks or to pay bills. One participant explained that being in this study “reinforced the importance of having a list of things to do written down and not relying on memory alone.” Another participant commented that “I recently started writing ‘to do's’ on small sticky notes as soon as I think of it and placing it on the corner of the bar between the kitchen and living room so I'll see them whenever I pass by.” Another participant explained a solution to a memory problem with television controls: “Strategy with three TV remote controls. I have labeled each one (1, 2, 3) to remind me which one to turn on 1st, 2nd, and 3rd. Then to shut the TV off, reverse the order 3, 2, 1. This has helped me a lot. No problems now.” Although these strategies were not specific to use of the computer program, participants linked their computer practice to everyday “successes,” and this encouraged them to engage in additional computer practice.

While participants had notable negative complaints about the program during the training experience, most of them acknowledged that the use of the CACR program helped them recognize cognitive limitations, create and practice strategies to enhance cognitive function, and improve the quality of their daily life.

Perceptions 3 Months After Training

In the 3-month follow-up focus groups, participants recalled finding the CACR program challenging. One participant stated that she had learned to “try, try again” if she didn't succeed initially, and others commented that working with the program had increased their perseverance. As one participant stated, the program encouraged her not to give up on herself too soon. Certain modules—such as number sequencing—were seen as more useful than others, perhaps because they were more easily understandable and translated to everyday activities. Some respondents attributed their improved ability to remember a sequence of numbers to the CACR exercises. Although participants were not specific about the particular track or module, exercises that incorporated flashing colored lights or buzzers were perceived as noxious. Some participants questioned whether that level of sensory stimulation was appropriate for people with MS. Others complained that the program penalized people with MS whose sensorimotor skills may be compromised.

The most consistent criticism centered around the program's format, particularly the introduction and initial directions for the program, which were described as confusing and not well formatted. For example, many participants had difficulty remembering the directions, which appeared only at the beginning of the module, and suggested that a “pop-up” directions screen would help them. Some of the participants who were former teachers were particularly critical about this aspect of the program and recommended that a better orientation or tutorial be developed. As one participant put it, “I wasn't getting it, and I don't understand why.” Some participants also expressed concern about the lack of corrective feedback—particularly positive reinforcement—about their performance, although others thought the program highlighted their cognitive strengths and weaknesses. However, on balance, most of the participants found the program beneficial, and some even expressed a desire to have continued access to it after the intervention was over. In addition, it was suggested that opportunities to practice the CACR program together be incorporated into the group sessions so that participants could share strategies they had learned with each other.

Discussion

To our knowledge, this is the first study to systematically examine participants' perceptions of home use of a computer-assisted cognitive training program. In addition, we included data collected through multiple methods—quantitative data from self-reports of the frequency and duration of practice, written qualitative data from the weekly homework assignments completed during training, and the verbal data collected as part of the poststudy focus groups. It should be noted that consistent with qualitative methodologies, the comments presented were selected to illustrate the various categories and are not necessarily representative of all participants. Although this study did not have a large sample and assessed participant responses to a home training experience with only one program, overall our findings suggest that home training is a feasible alternative to in-office training. Participants indicated that life events (eg, extra work demands, being sick, family demands) occasionally precluded practice during a specific week. In general, any interruptions in training were resolved the following week, and no trends of reduced use over time or excessive participant burden were evident. While most participants did practice at or above the recommended levels, the participants experienced a number of issues with use of the computer training that suggest several important considerations for future studies using any type of home-based computer training.

Participants pointed to multiple sources of frustration with the program, ranging from not understanding the purpose and directions to lack of feedback. Over time, the participants did seem to come to terms with the difficulty of the exercises and perceived that they were obtaining some benefit—although it is possible that those who were most frustrated or had the most difficulty did not wish to write about it or participate in the focus groups. In general, thorough advance preparation for the difficulties one might expect in home use of the computer program might lessen frustration and negative responses. It is important to emphasize to participants in advance that difficulties completing the exercises are expected and that they are unlikely to progress quickly through the multiple levels. Although we were not able to do so in our study, future studies might incorporate an initial one-on-one orientation and practice session with a professional trainer/coach (in the clinic or at home) to help participants overcome initial frustrations and understand the purpose of specific exercises.

Some participants also perceived that some of their difficulties were unique to the physical limitations or symptoms of MS. Although most researchers give careful consideration to screening for obvious difficulties (visual and fine motor impairment), there may be subtle difficulties that emerge only in interaction with the demands of the computer program. Thus MS researchers may want to beta-test cognitive training programs not developed specifically for MS populations in feasibility studies (as described here) prior to their use in larger trials.

Despite the difficulties and frustrations experienced with the computer training program, participants did identify several positive aspects and felt that training was a beneficial experience once they “persevered.” Participants in the focus groups suggested that even small successes accomplished through perseverance reinforced their engagement with the program and their self-efficacy in terms of their own cognitive skills. Participant comments regarding how they prepared themselves for the computer training (relaxation, adequate rest, planning time) are particularly useful and can be incorporated into directions for future trials. Data from the weekly homework logs indicated that most participants did practice the prescribed amount of time even in the home setting, supporting the feasibility of home training. The wide variations in practice time could be useful covariates in exploring posttreatment outcomes. Given the substantial feasibility benefits of home training, those designing programs should incorporate measures to electronically track the amount of time spent on different exercises. Such data would be useful to both clinicians and researchers monitoring home computer training protocols and assessing outcomes.

Cognitive rehabilitation training for individuals with MS—and particularly the use of computer-assisted training—is at an early stage in its scientific development. Given that producing and maintaining change may require training over a prolonged time period, computer training that can be accomplished at home has many advantages. Ongoing feedback systematically obtained from participants such as those in this study may be useful to those choosing or modifying existing programs. Additional work is necessary to more clearly understand the barriers and facilitators related to home use of computer-assisted cognitive training programs.

PracticePoints

Home-based computer-assisted cognitive training has potential advantages (eg, transportation, convenience) over clinic-based training.

People with MS were able to access and use a computer-assisted cognitive training program at home but initially found it frustrating and needed support and coaching regarding the purpose of the program and clarification of directions for specific exercises.

When prepared to expect the training to be challenging, people with MS persevered, practiced the prescribed amount, and perceived positive outcomes of the training.