Feasibility of an International Multiple Sclerosis Rehabilitation Data Repository

Despite the growing number of investigations in multiple sclerosis (MS) rehabilitation, evidence of efficacy and effectiveness is still markedly limited due to several methodological factors. These factors include generally small sample sizes¹ that do not allow distinguishing of effects by type of MS or disability level, studies that do not systematically describe the detailed content and dosage of rehabilitation interventions (the so-called black box of rehabilitation),² shortcomings in uniform use of a core set of (multidimensional) outcome measures,³ and insufficient established psychometric properties (eg, minimal clinically important difference) of key outcome measures.³ These issues need to be addressed to move MS rehabilitation science and practice forward.

The International Progressive MS Alliance was founded to stimulate collaborative international projects to expedite the development of treatment options for progressive MS by removing scientific and technology barriers.⁴ In 2014, this alliance funded an infrastructure multipart pilot project to assess the feasibility of a potential international MS rehabilitation data repository. Although there are similar initiatives in MS (eg, MSBase,⁵ the European Register for Multiple Sclerosis [EUReMS],⁶ the North American Research Committee on Multiple Sclerosis [NARCOMS] Registry,⁷ and the North American Registry for Care and Research in Multiple Sclerosis [NARCRMS]⁸) and in other populations, such as traumatic brain injury,⁹ these do not adequately address rehabilitation domains¹⁰ or sufficiently detail rehabilitation research and practice needs.¹¹

The feasibility and benefit of collaborative networks in MS have been demonstrated in recent years by prospective data collection¹²–¹⁴ in the European Rehabilitation in Multiple Sclerosis (RIMS) network (www.eurims.org). These collaborations have led to documentation of the psychometric properties of walking capacity tests¹² and have affected standardization in clinical practice in this domain.¹³ The large sample size achieved allowed analyses to be performed in subgroups with different disabilities and MS types.¹⁴ However, these ad hoc collaborations with a narrow focus cannot be repeated endlessly. It is hypothesized that the benefits of collaboration could be maximized with a larger framework. Another approach in accelerating the insights in effectiveness of rehabilitation interventions is the publishing of systematic reviews and meta-analyses.¹⁵ Although critically important, they are often hampered by the use of different outcome measures, limited intervention description, and reporting of only aggregate data in the individual studies. A framework for sharing of original data could help address these limitations.

An international MS rehabilitation data repository such as the MS Rehabilitation Repository (or shortened, MSRehabRep) could provide this framework. The MSRehabRep was conceptualized as a data repository¹⁶ for the storage, retrieval, and sharing of rehabilitation data in MS and a platform for communication and collaboration between MS clinicians and researchers. The aims of the multipart MSRehabRep pilot project were to 1) gather information on repository legal regulation issues and transfer knowledge from existing initiatives, 2) consult key stakeholders (clinicians and researchers) on the challenges of and motivators for sharing data, interest in participation in, and key features of a repository and 3) conduct a proof-of-concept study of the feasibility of data sharing by researchers in MS. This article reports the results of the consultation of clinicians and researchers in MS rehabilitation on the perceived challenges of, motivators for, interest in, and potential key features of such an international rehabilitation data repository in MS.

Methods

A sequential exploratory investigation with multiple methods was used for the stakeholder consultancy of clinicians and researchers. Two focus groups¹⁷ using nominal group technique^{18 19} were conducted in 2014. They, along with study aims, informed the development of an online questionnaire that was distributed to a wider international audience in 2015.

Focus Groups

The focus groups used nominal group technique to identify and reach consensus on the perceived major challenges to and motivators for sharing data with an MS rehabilitation data repository. The first focus group was held in Europe at the RIMS Mobility Special Interest Group meeting and the second in the United States at the Fourth International Symposium on Gait and Balance in MS. Twelve to 14 participants from each conference were purposefully selected based on clinical or research experience and were invited to participate, with a goal of a 2:1 ratio of researchers to expert clinicians.

To enhance consistency between the focus groups, a manual was written to standardize procedures,¹⁹ one research team member was present at both groups, and audio recordings were made and reviewed. Ethical approval was obtained from the University of Hasselt (Diepenbeek, Belgium) for both groups. All the participants provided verbal consent.

In the focus groups, participants were first asked to independently and separately record on a worksheet their perceived challenges to and motivators for sharing data with an MS rehabilitation data repository. Participants were told it was envisioned that the proposed repository could include both retrospective and prospective data from researchers and clinicians. Further definitions were not given. Next, lists of challenges and motivators were generated through facilitated group discussion.¹⁹ Participants then selected and sequentially ranked their top five most important items¹⁹ from these lists. They could select items from the categories labeled as challenges, motivators, or both, depending on which they perceived to be the most important. Specific criteria for determining which item was most important were not given. The highest ranked item received the top score of 5; the least important item, the lowest score of 1. Participant votes were then summed, with the highest scored item across participants now receiving a top rank of 1.¹⁹

After completion of both focus groups, the results were combined following Van Brenda procedures.²⁰ In brief, individual statements (on perceived challenges and motivators), votes (number of times the challenge or motivator item was selected as a top five important item), and rankings (order of importance based on summed votes per individual focus group) from both groups were listed in a single Excel spreadsheet (Microsoft Corp, Redmond, WA). Content analysis²¹ was used to group similar statements on perceived challenges and motivators into core conceptual categories by two research team members (E.H.B. and I.B.), with the final listing and conceptual definitions approved by the full research team. The core categories were evaluated and given a combined rank based on the frequency of statements, the number of votes received, and the individual focus group ranking.²⁰ The final rank was calculated by summing the combined ranks, sorting in descending order, and then ranking by order listed.²⁰ These core categories maintained their classification within the original framework of challenges and motivators.

Online Questionnaire

An online questionnaire (Appendix S1, which is published in the online version of this article at ijmsc.org) was constructed after completion of the focus groups. The aim of the questionnaire was multifactorial: 1) to assess agreement with the challenges and motivators identified in the focus groups, 2) to detect potential additional challenges of and motivators for sharing data, 3) to investigate interest in participation in an MS rehabilitation data repository, and 4) to assess potential key features for the development of a successful repository. The questionnaire was distributed to a professional and international audience with expertise in MS rehabilitation through snowball sampling.²² Clinicians and researchers were identified and contacted by e-mail through professional contacts in the field of MS and RIMS registration (focus group participants were included). The questionnaire link was also posted on the MSRehabRep website (www.msrehabrep.org). Participants had to be older than 18 years and have a minimum of 1 year of experience in MS rehabilitation per self-report. No exclusions were made based on discipline, race, ethnicity, or sex. Ethical approval was obtained from Saint Louis University (St. Louis, MO). Participation in the questionnaire was taken as consent.

Formative feedback was provided from five rehabilitation professionals, outside of the research team (whose primary or secondary language was English), on questionnaire clarity, content, and readability. The final questionnaire consisted of three open-ended questions and 19 statements organized conceptually by the categories of challenges, motivators, interest in participation, and key features to create conditions for repository success. Questionnaire statements were derived from the core categories of challenges and motivators from the combined focus group results and study aims. For example, in the combined focus group results, “data uniformity within the database” was identified as a major challenge to sharing data. This information, along with study aims, contributed to the development of three statements on the online questionnaire. The first statement related to challenges, “Please indicate your level of agreement or disagreement that these areas represent major challenges... creating data uniformity within the database (eg, terminology, standardization of measures, descriptions of interventions, minimum criteria for reporting).” The second statement related to key features, “Standardized procedures for a core set of measures are critical to the success of the repository.” The third statement related to interest, “Interest in retrieving and using data on descriptions of interventions.” Respondents selected their level of agreement with these categorized statements on a 0 to 5 Likert scale, with higher scores representing higher levels of participant agreement with the statements. Data were collected over 3 weeks through the Qualtrics Experience Management survey program (Qualtrics, Provo, UT) via anonymous responses.

Analyses

Descriptive statistics were used to describe the participants in both the focus groups and the questionnaire. The top ten challenges and motivators to sharing data were listed based on the combined focus group results. For the questionnaire, counts and percentages for each statement were calculated, as well as the median and either the interquartile range (IQR) or the range for each category (challenges, motivators, interest, and key features). To compare the perceptions between different groups (clinicians vs. researchers/professors and participants in North America vs. those in Europe), subgroup analyses via Mann-Whitney U tests were made. Data were analyzed using a statistical software program (IBM SPSS Statistics for Windows, version 19.0; IBM Corp, Armonk, NY), with significance set at P < .05.

The narrative open-text questionnaire data were analyzed via content analysis.²¹ Results from the focus groups and the questionnaire were compared (by two team members [E.H.B. and I.B.]) to ensure that the categories were appropriate. Finally, thick descriptions,²¹ or full quotations, were used to represent the key findings from this stepwise combined analysis, organized by the four larger categories of motivators, challenges, interest, and key features.

Results

Participants

Twenty-one individuals participated in the focus groups, 12 of the 14 invited to the first focus group held in Europe and nine of the 12 invited to the second focus group held in the United States. Most focus group participants self-identified as a researcher or a professor (57.1%) and worked primarily in MS (80.9%) and as a physiotherapist (66.7%).

A total of 767 unique e-mail invitations to complete the online questionnaire were sent. Two hundred fifty questionnaires were viewed (estimated viewing rate of 32.6% based on the number of known e-mailed invitations). Of those, 182 respondents initiated and completed at least 10% of the questions (72.8% participation rate), and 166 submitted a completed questionnaire (≥75.0% of questions answered) and were included in the analysis (91.2% completeness rate).²³ Of these 166 respondents, most self-identified as a clinician (55.4%) and worked primarily in MS (59.1%) and as a physiotherapist (68.7%). Less than a fifth of the questionnaire participants had previously either contributed (16.3%) or retrieved (12.7%) data from an existing data repository, whereas almost half (42.8%) had shared data with other researchers outside of their affiliated university. See Table 1 for a summary of participant characteristics.

Focus group and online questionnaire participant characteristics

Focus Groups

The top ten challenges (C) and motivators (M) from the combined focus group analysis were the following in descending order: 1) database control and management (C); 2) ethical and legal concerns (C); 3) data quality standards (C); 4) time, effort, and cost to contributors and users (C); 5) clinical decision making and best practice (M); 6) data uniformity within the database (C); 7) sustainability (C); 8) comprehensive and deeper analysis (M); 9) collaboration (M); and 10) identifying research needs (M). See Table 2 for a full list of focus group results, including the final rank, conceptual definitions, and associated questionnaire statements.

Focus group core categories, rank, conceptual definitions, and associated online questionnaire statements

Online Questionnaire

Challenges and Motivators

Percentage Agreement. Overall, median percentage agreement (categories “agree”/“strongly agree”) with questionnaire statements was 84.8% (IQR, 6.35%) across identified challenges and 93.4% (IQR, 5.75%) across identified motivators (see Table 3 for individual counts and percentages). Median disagreement (“disagree”/“strongly disagree”) was 4.2% (IQR, 3.6%) for challenges and 1.2% (IQR, 0.6%) for motivators. Median uncertainty or neutral position (“neither agree nor disagree”) was 10.8% (IQR, 3.5%) for challenges and 5.4% (IQR, 5.1%) for motivators.

Online questionnaire results showing participant count and percentage agreement-disagreement with statements regarding challenges, motivators, interest, and key features

Percentage Agreement Group Comparisons. Subgroup analysis of stakeholder agreement revealed that researchers/professors (n = 51 of 143 [35.7%]) more strongly identified with ethics/legal concerns and high-quality standards for data sharing as challenges (P < .05) and that clinicians (n = 92 of 143 [64.3%]) more strongly identified with best practice as a motivator (P < .01). There were no statistically significant differences between participants in North America and those in Europe.

Narrative Data. Content analysis supported the focus group categories. The importance of access to knowledge about current research and practice standards was evident and went beyond simply an increased number of publications. Thus, the publication category may be better relabeled as “dissemination” to reflect this broader context (Table 2). Table 4 displays selected quotations from the questionnaire open-text responses.

Interest in Participation and Key Features

Percentage Agreement. Questionnaire agreement with interest in sharing prospective and retrospective data was 79.5% (132 of 166) and 72.1% (119 of 165), respectively. Median agreement was 75.3% (range, 12.6%) with interest in using data and 92.8% (range, 13.9%) with the three identified key features (a user-friendly and secure interface, a scientific advisory board, and standardized procedures for a core set of measures) (see Table 3 for individual counts and percentages). The 86.0% majority (142 of 165) agreed that creation of the repository was needed to advance MS rehabilitation science and clinical practice; 11.5% neither agreed nor disagreed. Median disagreement was 4.5% (range, 0.6%) with sharing data, 3.6% (range, 3.6%) with use of data, and 0.6% (range, 0.6%) with key features. Median uncertainty or neutral position was 19.7% (range, 6.7%) with sharing data, 21.1% (range, 9.0%) with use of data, and 6.6% (range, 13.3%) with key features.

Percentage Agreement Group Comparisons. Subgroup analysis revealed that researchers/professors (n = 51 of 143 [35.7%]) more strongly identified with need for a scientific advisory board and interest in use of de-identified patient data than did clinicians (n = 92 of 143 [64.3%], P < .05). Participants in North America (n = 52 of 159 [32.7%]) more strongly identified with interest in sharing retrospective data, use of de-identified patient data, and use of descriptions of interventions than did participants in Europe (n = 107 of 159 [67.3%], P < .05).

Narrative Data. Content analysis (see Table 4 for selected quotations) revealed strong support for the identified key features. Many responders expressed that large and more comprehensive data sets were needed to move rehabilitation forward in MS. However, interest in participating in the proposed repository was hesitant due to expressed concerns about the feasibility of addressing challenges and the uncertainty of policies and procedures at the time of the questionnaire.

Selected quotations from online questionnaire representing findings on challenges, motivators, interest, and key features

Discussion

An international MS rehabilitation data repository has the potential to strengthen rehabilitation evidence through international collaboration and the assembly of large data sets to advance best practice in MS. This article documented the challenges of, motivators for, interest in participation in, and potential key features regarding the feasibility of such a data repository from the perspectives of clinicians and researchers in the field of MS rehabilitation.

Internationally, participants were interested in an MS rehabilitation data repository, but several concerns and uncertainty about feasibility and benefits existed. Based on study findings and knowledge of current database and repository successes and limitations,⁵–^{10 24 25} the identified priorities are 1) ensuring that high-quality data standards are met by having a scientific advisory board⁵–⁸ advise on policy and procedures and oversee data use requests and implementing a strong data validation system; 2) creating data uniformity with common data elements,^{9 10} a core set of measures,^{10 24} and standardized collection protocols for prospective data sharing; 3) using a secure and user-friendly interface that minimizes the time commitment, such as iMED (Merck Serono SA, Geneva, Switzerland)⁵ or REDCap (Research Electronic Data Capture)²⁵; 4) following local and international ethical and legal guidelines, including those of the Council of Europe²⁶ and the US Department of Health and Human Services Office of Human Research Protection²⁷; and 5) addressing ownership and acknowledgment rights for data contributors and rules for authorship of aggregated analyses through the use of data-sharing and use agreements^{5 9} consistent with the International Committee of Medical Journal Editors authorship guidelines.²⁸

The importance of security, ease of use, and data validation checks for data quality have been reported previously and have been addressed successfully in existing electronic data-sharing systems.^{5 9 25} Regarding the importance of uniformity, the common data element project in traumatic brain injury research¹⁰ has established core, basic, and supplemental common data elements, as well as core and basic outcome measures. However, data elements established are limited in the rehabilitation domain by lack of a coherent rehabilitation treatment taxonomy. Despite this limitation, the common data element project remains a strong model to build from in MS. The Federal Interagency Traumatic Brain Injury Research Informatics System,⁹ which is a platform for data sharing and collaboration that uses but is not fully restricted to the established common data elements, offers a promising example of a collaborative platform to structure type and policies on data sharing and use in MS.

This report on stakeholder consultancy of clinicians and researchers in MS advances current knowledge by 1) documenting challenges of and motivators for data sharing in MS rehabilitation, 2) assessing perceptual differences between researchers and clinicians and between those in the United States and those in Europe on data-sharing issues, and 3) raising the importance of ownership and authorship rights to maximize engagement of data contributors. The findings revealed many similarities between the groups examined but also a few differences that should be considered in future planning for a data repository and similar projects in data sharing and use.

Similarities revealed that, overall, challenges were ranked as higher priorities than motivators in the focus groups and existed throughout data sharing, management, and use. All motivators were related to data use, either with searching within the data repository to identify research gaps or retrieving and using data for analysis and practice improvement. The ability to access detailed intervention descriptions and study design characteristics, even without patient data, were strong interests of the questionnaire participants. This highlights the need to define the content of the so-called black box of rehabilitation by agreement on a documentation system of rehabilitation content and dosage as well as better dissemination of research study design characteristics. The important work of the Rehabilitation Treatment Taxonomy² and the Template for Intervention Description and Replication guidelines²⁹ provides a solid foundation toward meeting this need.

Regarding differences, in the questionnaire, clinicians and researchers varied on agreement with motivation to improve best practice and with challenges of ethical/legal concerns and data quality issues to data sharing. This may be explained by the differences in roles, highlighting the potential importance of tailoring involvement opportunities and messaging.³⁰ There was greater interest by participants in North America compared with those in Europe in sharing retrospective data and in use of de-identified patient data and descriptions of interventions. This may be due to the stricter regulations in the United States and Canada, as well as previous European data-sharing involvement in the RIMS joint data collection studies, which has led to standardization of outcome measures in the domain of walking throughout the RIMS network.¹³

A proof-of-concept data-sharing study based on data of published articles, as part of the MSRehabRep project, is currently being conducted to examine the feasibility of data sharing for a specific research question. It will assist in shaping the long-term vision of an international MS rehabilitation data repository. To date, a literature search was conducted to identify peer-reviewed literature on the effect of exercise therapy on walking for persons with MS. The research question is whether the effects are different when analyzed by disability level and type of MS.

Study limitations exist and are discussed here. Within the focus groups, criteria for prioritizing challenges and motivators were not defined a priori, which could have led to potential variance within individual responses. Challenges outranked motivators, indicating that there may be greater perceived barriers than facilitators. Ranking of challenges and motivators separately could have provided more in-depth assessment within each category, but overall prioritization would have been lost.

Within the questionnaire, the relatively small non-probability sample limits credibility and generalizability. Subgroup analysis based on participant type as either researcher/professor or clinician does not adequately capture those with multiple roles (eg, researcher and clinician), and neither does it distinguish between professors who may be lecturers and not academic researchers. More precise categorization and analysis through a larger sample would have enhanced precision in the results. Approximately one-fifth of the questionnaire participants labeled their interest in contributing or using data as uncertain or neutral. This may represent true uncertainty toward participation in a repository or may reflect the ambiguity and, so far, exploratory nature of the current project.

One aspect that may have contributed to the ambiguity was use of the term retrospective with both the focus groups and the questionnaire. Clarification was not given whether this was intended as data collected retrospectively based on medical records or prospectively collected within the framework of a specific study and then used for a secondary analysis. The latter was the intention. Envisioned data sources and structures were described in general terms only. This lack of specifics may have contributed to potential variance in participants' responses and uncertainty.

In addition, persons with MS, database specialists, and other supporting decision makers, such as insurance providers, who may have additional insights to maximize feasibility and impact were not consulted. Clinicians and researchers were targeted in this 1-year feasibility project because of their roles in data collection, analysis, and reports. Overall, these findings highlight the need to further debate the scope of an international MS rehabilitation data repository, appoint working groups for specific topics, and publicly disseminate current proposed policies and procedures.

Next steps, as envisioned by the MSRehabRep project, are creation of 1) an international steering and scientific advisory board with representatives from international rehabilitation organizations and other major databases and registries for refining the scope and related policies and procedures, as well as potential collaboration with an established data-sharing initiative; 2) international work groups to set common data elements and outcome measures across MS rehabilitation that can be implemented in a research and clinical setting; and 3) international work groups for a treatment taxonomy in MS rehabilitation. Major rehabilitation and care organizations, such as the Consortium of Multiple Sclerosis Centers (CMSC) and the RIMS network, may be best suited to guide this process and convene the expert knowledge within the national and global network initiatives that have worked on similar projects and related components.^{2 4}–^{9 24}

In conclusion, the MSRehabRep stakeholder consultation's findings support clinician and researcher interest in and potential for success of an international MS rehabilitation data repository if prioritized challenges and motivators are addressed and key features are included. Creation of a large data set in MS rehabilitation and a platform for collaboration between researchers and clinicians can provide the tools and depth of information needed to enhance decision making and improve best practice for all types of MS.

PRACTICE POINTS

Large data sets have the potential to improve best practice in rehabilitation for persons with MS.

Many clinicians and researchers are interested in participating in an international MS rehabilitation data repository.

The major challenges of sharing data are ensuring that there are high-quality data standards and transparent operational procedures for database control and management, while minimizing time and costs of contributors. Addressing these challenges will be necessary for successful initiation and functioning of a data repository in MS rehabilitation.