Impact of Multiple Sclerosis Project ECHO (Extension for Community Healthcare Outcomes) on Provider Confidence and Clinical Practice

Caring for individuals with multiple sclerosis (MS) is a complex endeavor that requires knowledge ranging from understanding and managing the underlying disease to managing a multitude of associated physical, psychological, and cognitive symptoms. Advancements in treatment, such as new and emerging disease-modifying therapies (DMTs), create a dimension of constant evolution, necessitating ongoing education.¹ These challenges are present for all providers who care for individuals living with MS, but the task may be particularly daunting for non-MS specialists, such as general neurologists and primary care providers, who are expected to stay current on developments across a variety of medical and neurologic conditions.

Research on MS care in rural areas highlights the fact that the quality of care is influenced by geographic locale. Comprehensive MS specialty centers and MS specialist neurologists are often located in urban areas. In rural areas, where few MS specialists practice, difficulties related to MS care have been noted. Individuals with MS in rural settings describe poorer access to care,² less satisfaction with access to neurology care,³ less satisfaction with the care received,³ lower quality of mental health treatment,⁴ and lower health-related quality of life.⁵ In an effort to improve care for all individuals with MS, the National Multiple Sclerosis Society (NMSS) has taken special interest in addressing the disparity in care experienced by people with MS in rural and underserved areas through the Access to High Quality MS Healthcare Task Force.⁶

Geographic, financial, and health-related barriers limit the ability of patients in underserved areas to travel to MS specialty centers, thus creating the need for improvements in the care that individuals with MS receive in these underserved areas. In a previous publication,⁷ we described our successful adaptation of an approach to accelerating the transfer of knowledge from content experts to rural providers who then develop an enhanced level of clinical understanding and confidence, allowing them to serve as local experts in MS care. In this program, called MS Project ECHO (Extension for Community Healthcare Outcomes),⁸ providers who care for individuals with MS (the “participants”) in underserved areas join a weekly videoconference that includes a brief didactic focused on key information necessary to provide high-quality care, followed by case consultation and discussion. The intent of the program is to develop a community learning environment that includes both generalist providers and content experts, thus expanding the knowledge, skills, and self-efficacy of the participating providers.⁹–¹² These participants, in turn, become local content experts, thus expanding the availability of specialty care in underserved areas. In well-established Project ECHOs for HIV and hepatitis C, the effect of the program has been evident, as participants reported that they had developed the confidence to serve as local experts while reducing their perceived professional isolation, increasing their professional satisfaction, and subsequently increasing their individual capacity of care and retention in rural practices.¹⁰ Furthermore, in care related to hepatitis C, the Project ECHO model was shown to be as safe and effective for patients in rural communities as for those receiving their specialty care at an academic medical center.¹³ The present MS-focused effort was the first translation of Project ECHO for MS.

A primary tool of the Project ECHO model is the use of case consultations to facilitate guided mentoring of community providers. In previous research outside of Project ECHO, case-based learning has been demonstrated to be superior to seminar or lecture-based instruction¹⁴ and has been shown to be an effective method for changing provider behavior.¹⁵ Despite the common use of case-based learning in medical school and residency, such opportunities diminish once the formal training process ends, particularly in terms of training on real cases in a provider's clinical practice (as opposed to some continuing medical education training available using exemplar cases or situations).¹⁵ Uniquely, Project ECHO provides an efficient method to reintroduce case-based learning through this remotely delivered consultation program that uses actual cases from a participant's practice. This being said, the manner in which Project ECHO's case consultations drive meaningful change in clinical practice has not been clearly elucidated.

With this in mind, as part of our development and pilot testing of the first MS-focused Project ECHO, we conducted qualitative exit interviews with participants to understand how their participation in case consultations affected their approach to clinical practice. We were particularly interested in examining how the case consultations 1) directly affect the decision making of the participant provider on the specific case for which consultation was sought and 2) affect the participant provider's broader approach to delivering MS care in general to their patients with MS. Answering these questions would help us understand the extent to which the value of MS Project ECHO is limited to the cases targeted through consultation or is generalized to the clinician's broader MS practice.

Methods

Overview

The MS-focused version of Project ECHO (MS Project ECHO) was adapted, developed, and delivered by providers at the University of Washington (Seattle) in partnership with the NMSS. In a previous publication, we comprehensively described the Project ECHO concept, our adaptation to MS, objective data describing the providers who participated in the pilot study (clinical specialty, MS experience, geographic location), the topics covered during the program, and metrics of participation and satisfaction with the program.⁷ In contrast, the present article focuses exclusively on the case consultation portion of the program and summarizes qualitative feedback obtained via exit interviews.

Briefly, MS Project ECHO targeted neurologists, primary care providers, and other specialists (eg, rehabilitation medicine, naturopathic medicine) who care for individuals with MS outside of the specialty setting in the University of Washington's extended catchment area (Washington, Alaska, Montana, and Idaho). This was an ideal region to test this model because it consists of large geographic areas that are sparsely populated and primarily rural.¹⁶ Participating providers (“participants”) were initially engaged through a site visit by the project's medical director and subsequently participated in weekly videoconferences, as well as some variable between-session engagement with the MS Project ECHO team. The weekly videoconferences consisted of approximately 20 minutes of didactic education on a specific key aspect of MS management, followed by approximately 40 minutes of case consultation. This article reports specifically on the case consultation portion of the program, which was conducted as follows.

Participants were encouraged to submit cases for consultation in advance. Case submission was facilitated by the MS Project ECHO manager to obtain magnetic resonance images (MRIs), key details from the medical record, and specific questions for consultation. These elements were then formatted into a concise presentation that was reviewed in advance by the MS Project ECHO medical director and delivered by the participants during the live session. After the presentation, the MS Project ECHO team led a discussion of the case and consultation questions. At all times, the consultation was conducted as an open conversation between participants and experts, thus affording an opportunity to share ideas and ask follow-up questions. Although the Project ECHO model is designed to increase knowledge in participant providers, the case consultation process is best conducted in a collaborative manner. Specifically, the expert team may have the greatest knowledge of the current beliefs and expectations in the MS clinical and research communities surrounding management of the presenting concern, but the community providers have the greatest understanding of the resources available and the culture within their communities.

In most cases, discussion expanded beyond the original consultation question(s) to address the broader topics at hand; for example, if the primary consultation question was on switching between two DMTs, discussion would then be expanded to address the broader topic of deciding if, how, and when to adjust a DMT. In addition, the prepared cases often reminded participating providers of other similar or tangentially related cases. There is value in this spontaneity, so unprepared cases were encouraged and welcomed.

The MS Project ECHO team was available for follow-up consultation between sessions, by either e-mail or phone. This was encouraged for all aspects of participation (didactic or consultation), and specific follow-up was offered for cases that were presented in the sessions. Participants were also urged to bring cases back to future weekly sessions to provide updates or address new questions.

Data Collection Procedures

As detailed later herein, data used in this article were collected in three formats: provider self-report (collected before participation), program manager observations (collected on a weekly basis during the program), and exit interviews (conducted by the project's medical director in the immediate weeks after the close of the program). Individual approaches to measurement are detailed later herein. The University of Washington's Human Subjects Division determined that these data were exempt from institutional review board review owing to their focus on program development.

Measures

Demographic Features

At the time of enrollment, participants provided information on their discipline (ie, neurology, rehabilitation medicine, family practice) and number of patients with MS in their practice.

Participation

The MS Project ECHO team tracked attendance, engagement, and case presentations. Attendance was calculated as the number of sessions attended. Engagement was the rate of attendance: [number of sessions participant attended] / [number of sessions available after enrollment]. Basic descriptive information was collected for case presentations, such as who presented the case, the basic characteristics of the case, and the resulting recommendations of the consultations.

Impact of Case Consultations on Participants

At the conclusion of the MS Project ECHO pilot, the program's medical director invited each participant to an exit interview regarding the experience of participating in the program. Most participants (62.5%) opted to participate, with time constraints reported as the primary reason for opting not to participate in exit interviews. The interviews were divided into two parts. The first portion was a series of open-ended questions related to expectations of the program, quality of interaction between participants, opinion regarding the specialty mix of participants, barriers to weekly participation, impression of didactics, impression of case consultations, and desire to continue in the MS ECHO program. Also, a scripted question was answered on a 5-point Likert scale: “As compared to before your participation in ECHO, do you feel more confident in your ability to treat people with MS?” The second part of the interview was a series of individualized questions to assess the impact of the case consultation on the participating provider. Cases were reviewed before the exit interview to refamiliarize both the participant and the medical director with the case. The participant was asked to review any follow-up visits that had occurred since the case had been presented. Participants were then asked to comment on the extent to which the case consultation affected their confidence, the delivery of care to the patient whose case was presented, and care for patients with MS in general. Participants were encouraged to elaborate on each answer to allow the MS ECHO evaluation team to identify whether more specific themes about the impact of case consultations emerged.

Analyses

Participant descriptive data were summarized through traditional methods, including calculating means, percentages, and range scores, as applicable. The qualitative data collected regarding the impact of case consultations on clinical practice were analyzed using a phenomenological approach.¹⁷ This type of approach required two evaluators (herein, the lead investigators for MS Project ECHO's program evaluation component [K.N.A. and G.A.S.]) to independently review interview responses to identify and code themes that emerged from the responses. The evaluators then collaborated to reach consensus on their coding of responses, resulting in a final list of themes.

Results

Descriptive Data Regarding Case Consultations

The MS Project ECHO program included 24 providers, 18 of whom completed exit interviews. A total of 38 unique cases were presented over the course of the project. Of the 18 exit interview participants, 15 (83%) presented cases. These 15 individuals accounted for 34 of the 38 case presentations. Notably, the participants with more MS experience (those who cared for >50 patients with MS) were more likely to present more than one case. The three participants who did not present cases were also the least engaged ECHO participants in general—they had the fewest sessions attended and perceived the least impact of MS Project ECHO on their confidence in providing MS care.

Perceived Impact of Case Consultation on Provider Confidence and Clinical Care

Exit interview responses regarding the impact of case consultations generated four themes: 1) improved confidence among participants in the existing treatment decision, 2) direct change in the care of the patient provided by the participant, 3) changed practice habits for all of the participant's patients with MS, and 4) increased perception that patients had confidence in the participant as an MS care provider. These four domains were not mutually exclusive, such that some providers indicated that the case consultations affected them in multiple ways. A summary of the 34 case consultations presented by the 15 exit interviewees, including information about the presenters, the topics, and the outcomes, is available in Table 1.

Summary of MS Project ECHO case consultations by program participant

Improved Confidence in Existing Treatment Decision

Of the 15 participants who presented cases, 11 specifically cited an improvement in their confidence level regarding decisions made concerning the cases presented. One presenter (a less experienced neurologist) stated that participation in MS ECHO “improved my confidence in the case and my decisions.” Another participant (an experienced physiatrist) stated that it “absolutely influenced me and gave me so much confidence.” Another participant (an experienced physiatrist) described improved confidence related to the ambiguity at times in decisions surrounding DMT choice, stating that it “gave me confidence in supporting the patient's wishes.” Three of our generalist participants cited “improved confidence in discussing the case with the patient's neurologist,” “improved confidence in counseling my MS patient about DMTs,” and “more confidence in how I approach my MS patient.”

Directly Changed Care for Patients

Participants (with various levels of MS experience) presenting these cases reported direct changes in their care of their patients in 16 of the 34 cases (47%). There was a mixture of generalists, less experienced neurologists, and more experienced specialists. Changes in care included decisions related to DMTs (one case from an experienced physiatrist and one from an experienced neurologist), imaging/safety monitoring (two cases from less experienced neurologists and two from a less experienced physiatrist), symptom management (two cases from a less experienced physiatrist and one from an experienced physiatrist), and additional referrals (one case from a less experienced neurologist and one from an experienced neurologist). Examples of comments from participants included “I shortened the time gap between DMTs when switching” and “I was able to get a shower seat and walker for my patient after referring to the National MS Society.” Two participants reported that presenting the case eliminated the need for referral to an MS center, thus saving each patient from needing to travel more than 1000 miles for a second opinion. Another case resulted in a change in diagnosis (identifying a congenital abnormality on MRI) that eliminated the need for follow-up imaging.

Changed Participant's Practice Habit for All Patients with MS

Most participants who presented cases (14 of the 15 [93%]) reported that the feedback received during case consultations resulted in changes to their overall delivery of MS care. Five participants cited this as “general influence on their MS patient care.” Some of the specific examples of changes in practice habits included monitoring of vitamin D levels, more frequent baseline spine imaging, a greater likelihood of starting DMTs sooner in the disease course, and a better understanding of interpretation of changes on MRI as a measure of disease progression. Other more specific practice changes were noted as well, including changing their diagnostic work-up (one generalist) and changing how they monitor disease progression (a less experienced neurologist and an experienced specialist). In addition, one participant (a more experienced specialist) cited changes in monitoring disease progression as well as using NMSS resources, and another generalist specifically cited symptom management as the area of change in their general practice.

Increased Perception That Patients Had Confidence in Participant as MS Care Provider

A final theme that emerged from the case consultation was the perception that their patient became more confident in their care on hearing that the case was being reviewed through MS Project ECHO, regardless of the participant's MS experience and current number of patients with MS. Without exception, providers also reported that their patients were pleased to learn of their participation in MS Project ECHO. Specific examples included one participant citing that her patient was “excited to know she was increasing her knowledge of MS care,” and another expressed improved confidence “knowing he was part of a team” dedicated to MS care.

Discussion

The MS version of Project ECHO was developed and piloted as an approach to reducing disparities in MS care. In this report we described participants' experiences with the program's videoconference-based case consultations, a primary element of the program. Exit interviews yielded four themes: 1) improved confidence among participants in the existing treatment decision, 2) direct change in the care of the patient provided by the participant, 3) changed practice habits for all of the participant's patients with MS, and 4) increased perception that patients had confidence in the participant as an MS care provider. These responses support MS Project ECHO as a program that may directly and indirectly affect the way providers deliver MS care in underserved areas.

The perceived positive impact of MS Project ECHO provides a preliminary indication that this approach may be a viable strategy for effectively transferring knowledge to enhance MS specialty care in underserved areas. This is not surprising because previous research on case-based training has demonstrated greater engagement and better enhancement of critical thinking relative to lecture-based instruction¹⁴ and has been particularly effective for changing provider behavior.¹⁵ Furthermore, neuroscientists have previously rated it as their preferred method of learning.¹⁸ Finally, individuals who lead and facilitate Project ECHOs focus on building relationships and rapport with participants, to increase the engagement of participants in guided practice over the course of longitudinal mentorship. Thus, the program attends to the “internal and intrapersonal factors”¹⁹ that are uniquely targeted through case-based learning but serve as barriers to engaging fully with standard educational opportunities.

We noted the extent to which participants reported an increase in confidence as MS providers. Whereas some studies have demonstrated how case-based learning increases confidence,²⁰,²¹ we were also aware that case-based learning has the potential to decrease confidence by highlighting the extent to which an individual might lack important knowledge on the topic of interest. The Dunning-Kruger effect,²² for example, suggests that individuals in the lowest quartile of a skill are at risk for being unaware of what they do not know about that skill, and thus may be unintentionally overconfident. Gaining knowledge, therefore, may expose the individual to the breadth of information they do not know, which could initially result in reduced confidence. Once an individual in that subgroup experiences this “correction” in confidence, they would likely follow the more expected path of confidence increasing in line with the new knowledge they acquire. Although the sample size was insufficient for subgroup analyses, we did subjectively observe that participants with less MS-specific experience described less of a change in confidence, perhaps supporting the concept that the impact of case-based learning on confidence may be moderated by an individual's level of knowledge at baseline.

An important underlying goal of the Project ECHO concept is the development of a community of practice, where the collaborative participation in this program with providers of similar training and in similar situations (eg, in our case primarily community generalist neurologists in rural and underserved areas) reduces the sense of social isolation. When executed successfully, the model can contribute to a new level of local expertise that has the potential to reduce the health care disparity,⁸,¹⁰,¹²,²³ a perspective supported in the present study—at least in part—by the perceived increase in knowledge and confidence for providing MS care. The extent to which participants in MS Project ECHO become local leaders in MS care is something that could only be recognized over a longer pilot with continuation of the MS ECHO sessions over time. This is an important target to consider in a larger-scale examination of the effectiveness of the model.

Ultimately, the goal of MS Project ECHO is to improve patient outcomes, particularly for patients in rural and underserved areas. This pilot program supports enthusiasm for Project ECHO within MS, with participating providers detailing specific examples of how their participation resulted in improved treatment plans, reinforced existing plans, altered the way they care for patients with MS in general, and even optimized financial resources (eg, eliminated the need for a costly multistate trip for a specialty consultation in our center). Ultimately, there are a variety of important questions to consider. For example, building on previous research that suggests a disparity in care for rural patients, do we now see patients in MS Project ECHO participant practices begin to achieve outcomes akin to their specialty center peers? Such findings have been evident in other Project ECHOs with more robust histories, such as in the hepatitis C Project ECHO that demonstrated an elimination of the disparity in quality of care as a result of Project ECHO participation.¹³ Because this was the first MS-focused Project ECHO, a larger-scale trial is needed to explore objective changes in provider practice and patient-level outcomes. In addition, it is important to note that although Project ECHO takes a unique approach to spreading expert care by sharing knowledge among providers, there are other telehealth models that hold promise through the direct delivery of care through a patient-provider videoconference. Each of these models has pros and cons, such that Project ECHO is a time- and cost-efficient approach to enhancing care provided locally, whereas direct care models emphasize consolidating specialty care around the existing experts in a field. Ultimately, comparative effectiveness research that compares the cost, time, and effectiveness of these telehealth models head to head will be useful.

There were, of course, limitations to the pilot. Most notably, this was a time-limited developmental and feasibility pilot. Established Project ECHOs operate as ongoing programs. Attending to the development and feasibility goals, we focused our evaluation on basic metrics of success (eg, attendance, quantity of sessions, successful recruitment; reported in Johnson et al⁷). We evaluated the preliminary effect of the program based on participant self-report collected via exit interview. Should the program continue, evaluation should be improved to include patient-level data, although this would require a more robust research arm of the program. Relatedly, we opted to recruit providers based on a specific characteristic (provides MS care in a rural or underserved area) and allowed participation regardless of medical specialty. Thus, there was significant heterogeneity in the population in terms of clinical focus and expertise. Although this is important from a feasibility perspective, demonstrating that there is interest and perceived utility in this program regardless of medical specialty, it resulted in a very heterogeneous sample. We recommend that future research consider whether participation in this program affects providers with specific foci differently, which will require adequate sampling for subsequent analyses. Finally, from a methodology perspective, we note that program evaluation was conducted by content experts who also contributed to the expert team in the weekly sessions. In a more rigorous trial, we would recommend having independent implementation and evaluation teams to eliminate any concerns of interference or bias.

This first demonstration of the Project ECHO program in MS provides preliminary insights into a model that may be effective in directly and indirectly improving the care delivered by providers outside of MS specialty centers. Participant providers were enthusiastic in providing feedback about the way the model affected their clinical decision making, confidence, and approach to practice. We look forward to studying and optimizing this model further, including pursuing opportunities to objectively assess changes in provider behavior and changes in patient outcomes.

PRACTICE POINTS

Disparities exist in MS care, particularly in rural and underserved areas.

Opportunities exist for the use of technology to expedite the transfer of knowledge from MS specialty centers to community providers who care for individuals with MS.

Providers who participated in the MS Project ECHO (Extension for Community Healthcare Outcomes) program indicated perceived positive effects on their direct care of patients about whom they sought consultation, approach to MS patient care in general, and perception of their patient's confidence in them as an MS provider.