Skip to main content

Lived experience-centred word clouds may improve research uncertainty gathering in priority setting partnerships

Abstract

Introduction

AO Spine RECODE-DCM was a multi-stakeholder priority setting partnership (PSP) to define the top ten research priorities for degenerative cervical myelopathy (DCM). Priorities were generated and iteratively refined using a series of surveys administered to surgeons, other healthcare professionals (oHCP) and people with DCM (PwDCM). The aim of this work was to utilise word clouds to enable the perspectives of people with the condition to be heard earlier in the PSP process than is traditionally the case. The objective was to evaluate the added value of word clouds in the process of defining research uncertainties in National Institute for Health Research (NIHR) James Lind Alliance (JLA) Priority Setting Partnerships.

Methods

Patient-generated word clouds were created for the four survey subsections of the AO Spine RECODE-DCM PSP: diagnosis, treatment, long-term management and other issues. These were then evaluated as a nested methodological study. Word-clouds were created and iteratively refined by an online support group of people with DCM, before being curated by the RECODE-DCM management committee and expert healthcare professional representatives. The final word clouds were embedded within the surveys administered at random to 50% of participants. DCM research uncertainties suggested by participants were compared pre- and post-word cloud presentation.

Results

A total of 215 (50.9%) participants were randomised to the word cloud stream, including 118 (55%) spinal surgeons, 52 (24%) PwDCM and 45 (21%) oHCP. Participants submitted 434 additional uncertainties after word cloud review: word count was lower and more uniform across each survey subsections compared to pre-word cloud uncertainties. Twenty-three (32%) of the final 74 PSP summary questions did not have a post-word cloud contribution and no summary question was formed exclusively on post-word cloud uncertainties. There were differences in mapping of pre- and post-word cloud uncertainties to summary questions, with greater mapping of post-word cloud uncertainties to the number 1 research question priority: raising awareness. Five of the final summary questions were more likely to map to the research uncertainties suggested by participants after having reviewed the word clouds.

Conclusions

Word clouds may increase the perspective of underrepresented stakeholders in the research question gathering stage of priority setting partnerships. This may help steer the process towards research questions that are of highest priority for people with the condition.

Peer Review reports

Introduction

Degenerative cervical myelopathy (DCM) is the most common cause of spinal cord impairment worldwide [1]. It arises when degenerative pathology in the cervical spine precipitates a subacute compression injury to the cervical spinal cord. Despite the best current management [2], many people with DCM are left with life-changing disabilities [3] and amongst the worst quality of life of all chronic diseases [4].

AO Spine RECODE-DCM (REsearch objectives and Common Data Elements for Degenerative Cervical Myelopathy) was a multi-stakeholder, international consensus process aiming to accelerate DCM research through recommendations that improve research efficiency [5]. It combined several consensus initiatives, including establishing the top 10 DCM research uncertainties [6,7,8,9,10,11,12,13,14,15,16,17,18,19].

The process of establishing research priorities is supported by organisations such as the National Institute for Health Research (NIHR) James Lind Alliance (JLA) [20]. The JLA define research uncertainties as (1) areas with no up-to-date, reliable systematic reviews of research evidence addressing the uncertainty, or (2) up-to-date systemic reviews of research evidence showing that uncertainties exist [21]. JLA methodology starts with an information gathering process, by seeking research suggestions from patients, family, caregivers, and healthcare professionals. Commonly this is conducted using web-based surveys, with survey design tailored to the specific condition under consideration. For AO Spine RECODE-DCM, survey sub-sections included diagnosis, treatment, long-term management, and other issues [5]. The process is designed to build consensus and encompasses several rounds of surveys. After each round participants are presented with an anonymised summary of all responses and can amend their individual responses [22].

A major driver of research inefficiency is the underrepresentation of end-users, including people living with degenerative cervical myelopathy (PwDCM), within research design [21, 22]. Involvement of end users, such as via surveys, focus groups and online crowdsourcing [23], has been shown to be essential for research results to be of value [24,25,26] While this is recognized in PSP methodology by participation of end users in the surveys and as steering committee members, the perspectives of the whole end-user population is often not clear to the wider group until the final consensus meeting. Moreover, studies of JLA PSP methodology have identified that current PSP surveys may underrepresent end-user perceptions in favour of the views of healthcare professionals [27]. In particular, end users with physical, language and communication disabilities may face the greatest challenges in accessibility, participation and engagement within current PSPs [28, 29]. We therefore hypothesized that the end user perspective could be integrated earlier in the process by including end-user generated word clouds in the information gathering phase to help stimulate research uncertainty suggestions from other stakeholders.

We also hypothesised that word clouds would help mitigate against a unique challenge for the DCM PSP, that a diversity of healthcare professionals is involved in DCM care, but generally, the practice of each healthcare professional group is confined to a limited stage of the disease, such as initial diagnosis [30, 31]. This narrow focus of each practitioner’s role has been proposed to limit clinical research creativity [32].

Word clouds are a tool which enable qualitative data to be displayed, with the relative importance or frequency of each idea is proportional to the size of the word in the cloud. In the medical literature, word clouds have mainly been used to report qualitative patient interview data [33], although further afield they have been utilised to stimulate creativity [26, 27, 34].

The methodology for the generation of the word clouds used in the AO Spine RECODE-DCM PSP has been described in detail elsewhere [35]. Here we evaluate their impact on the responses in the PSP process. The aim was to provide insight into the role and value of word clouds in stimulating a creative, end-user focused research agenda in PSPs. We hypothesised that the incorporation of lived-experience-centred word clouds may more effectively identify research uncertainties and priorities.

Methods

A detailed description of the methodology used to design and create the AO Spine RECODE-DCM word clouds has been published [35].

In brief, word clouds were generated in collaboration with Myelopathy Support, an arm of Myelopathy.org, which is an international DCM charity. Myelopathy Support is an online community hosted on Facebook (Meta, California, USA). The group is closed, and access is moderated by Myelopathy.org volunteers. Individuals wishing to join the group are required to confirm they have myelopathy and will adhere to community guidelines. Prior experience has demonstrated that demographics and disease characteristics of this group, aside from a female gender predominance, are broadly representative of DCM clinical trials [36,37,38,39,40]. Ethical approval was not required for the involvement of the Myelopathy Support members as their role was to help to develop the word clouds for the survey and they were not research participants [41]. The lead moderator of Myelopathy Support (IS) is a member of the AO Spine RECODE-DCM management group and steering committee.

Over a 2-week period, 4 posts were pinned to the top of the Myelopathy Support group: (1) “What words do you associate with the diagnosis phase of myelopathy?” (2) “What words do you associate with the management phase of myelopathy?” (3) “What words do you associate with the long-term care/living with myelopathy?” (4) “Any other relevant words?”. The posts were accompanied by covering information outlining the background to the exercise, piloted and approved by IS. Group members were encouraged to add their suggestions as comments on the posts and to vote for the words with which they agreed. No data was collected on individual participants. There was no limit on how many times a group member could contribute; each member could vote for each word only once.

Following the 2-week period, the submitted words and their respective votes were reviewed by the AO Spine RECODE-DCM Management Group [ODM, ES, DK, IS, BMD, OH] and duplicated words were removed. Words considered to be out of scope were removed and words felt to be better reflected in a different section were moved. Through consultation with representatives of the other healthcare professional (oHCP) stakeholder group (a general practitioner (EKS) and a neurologist (SRLS) and discussion amongst the management group, some additional suggestions were added. The aim of this was to include the perspective of another under-represented stake-holder group [42] ensuring the process remained patient-centred by presenting suggestions back to the Myelopathy Support group for a further round of voting. Full details on the process of creating the word clouds have been published elsewhere [35].

The final lists were re-posted to the group as a series of polls organised to allow members to review the list and vote for the words with which they most agreed. Members could continue to submit additional suggestions, which were then available to be polled. Again, members could not vote more than once for each word. This second-round exercise ran for a further 2 weeks. Final suggestions and polling were reviewed by the management group, with removal of duplicates and out of scope suggestions. A flow chart of the processes used is shown in Fig. 1.

Fig. 1
figure 1

Word Cloud Development. A 3-stage process was used to collect unique words relating to DCM and survey their popularity to create word clouds. Words were initially submitted by Myelopathy Support, a community of PwDCM and their supporters. These were then processed by the AO Spine RECODE-DCM Management Group and supplemented with the perspectives of a general practitioner and a neurologist; arrows denote added and removed words. The final list of words was re-reviewed by the Myelopathy Support community. Voting was conducted to capture the popularity of each word and further suggestions were also put forward. The four-column tables represent the number of words per subsection, ordered from left to right as: diagnosis, treatment, long-term management and other

Word clouds were generated using WordArt.com (California, USA). The size of the word was proportional to the number of votes it received (Fig. 2). Full polling results have been reported elsewhere [35]. In summary, the maximum number of votes received by a word was 49. Words without a vote were not included in the clouds. The first iteration of the final output was used, without modification.

Fig. 2
figure 2

Word clouds generated from the Myelopathy Support group polling. Words were associated with (a) DCM diagnosis, (b) treatment, (c) long-term management and (d) other. The size of the word is proportional to the number of votes it received during polling

Evaluation of word cloud utility in generating PSP summary questions

AO Spine RECODE-DCM contained several parallel consensus processes and included three main stakeholder groups: (1) spinal surgeons, (2) PwDCM and their supporters, (3) other healthcare professionals. Consent was obtained from all participants. For the PSP arm of RECODE-DCM, participants were randomised by stakeholder group using a 1:1 computerised block randomisation protocol, to see the traditional PSP survey alone, or the traditional PSP survey followed by the word clouds (Fig. 3). This approach was taken to ensure adherence to the standard JLA methodology, whilst simultaneously allowing evaluation of the word clouds. Participants were able to move freely within the survey up until the point they submitted their responses. Consequently, it was theoretically possible for participants in the word cloud stream to edit their pre-word cloud responses having seen the word clouds. Whilst this cannot be tracked by the survey platform, by retaining an arm who do not have access to the word clouds, suspicious activity could be evaluated and was not noted. The wider methodology and governance of RECODE-DCM has been reported in detail elsewhere [7].

Fig. 3
figure 3

Incorporation of word clouds in the PSP. Participants allocated to the AO Spine RECODE-DCM PSP were categorised by stakeholder group (spinal surgeons, PwDCM and their supporters, other healthcare professionals). Participants within each stakeholder group were randomly allocated to word cloud or no word cloud streams using a computerised 1:1 block randomisation protocol. Participants in the word cloud stream were presented with the word clouds after the standard survey seen by the non-word cloud group and then given the opportunity to submit further uncertainties

Data analysis

All suggested research uncertainties were then processed as is standard for a JLA PSP [8]. In short, uncertainties were reviewed by an information specialist (LT), who was specifically recruited to AO Spine RECODE-DCM to manage the data. Her role was to review the responses to the survey, organise and categorise the submitted uncertainties into themes, remove out of scope uncertainties, generate clearly formatted summary questions, check existing literature to verify that each summary question was a true research uncertainty and present summary questions to the steering committee for discussion. As per JLA protocol, an audit trail was maintained from submitted uncertainties to the final list of top ten research priorities and this process was overseen by the steering committee. This methodology has been reported in detail elsewhere [8].

In addition, submitted research uncertainties were coded by stream (word cloud vs. no word cloud). For those within the word cloud stream, the uncertainties were coded as arising either before or after seeing the word clouds. The number of suggested research uncertainties that were grouped under a particular summary question in the final analysis were recorded for each stage of the survey (without word clouds, before/after word clouds). To evaluate the impact of word clouds on uncertainties-gathering, the following results were compared before and after reviewing the word-clouds:

  1. 1)

    Median word count of suggested uncertainties

  2. 2)

    Mapping of uncertainties to summary questions

  3. 3)

    Number of submitted uncertainties informing summary questions

Statistical analysis

Data analysis was conducted in Microsoft Excel (Microsoft 365, Washington, USA). As an exploratory analysis, data is presented with summary statistics only.

Results

The AO Spine RECODE-DCM PSP recruited a total of 422 participants: 215 (50.9%) participants were randomised to the word cloud stream. Participants in the word cloud stream consisted of 118 (55%) spinal surgeons, 52 (24%) PwDCM and 45 (21%) oHCP. A total of 157 (73%) participants in the word cloud stream submitted additional uncertainties following word cloud presentation, including 86 (55%) spinal surgeons, 28 (18%) oHCP and 43 (27%) PwDCM. In total, participants submitted 434 additional research uncertainties after having reviewed the word clouds.

The average word count of uncertainties per survey subsection (diagnosis, treatment, long-term management and other) were comparable between the non-word cloud stream and the word-cloud stream before word cloud presentation. However, after word cloud review word count of uncertainties was lower and more uniform across each subsection (Fig. 4).

Fig. 4
figure 4

Median number of words submitted per survey subsection (diagnosis, treatment, long-term management and other). Each group of bars represents subsections at different stages: without word clouds (grey), before word clouds (pale orange) and after word clouds (dark orange). Ordered from left to right, the bars represent diagnosis, treatment, long-term management and other. The vertical black line separates the two streams. The number and distribution of submissions across subsections was similar for no words clouds and pre-word clouds. The number of words submitted post word clouds was fewer and consistent across each subsection

Of the 74 summary questions generated, 23 (32%) summary questions did not have a post-word-cloud contribution, and no summary question was formed based purely on an uncertainty generated after review of the word clouds. However, of the 434 research uncertainties made following word cloud review, only 108 (24.9%) were already represented in a participant’s submission prior to reviewing the word clouds and these uncertainties mapped to summary questions differently compared to those submitted prior to review of the word clouds (Fig. 5).

Fig. 5
figure 5

Radar plot of research uncertainties and how they map to the final 74 summary questions (black numbers) and top 26 priorities (green numbers), from uncertainties without (grey) or pre-word cloud (blue) compared to post-word cloud (orange). Post-word cloud uncertainties mapped to different research uncertainties than pre-word cloud uncertainties, including the number 1 research priority: raising awareness

In addition, 5 summary questions appeared more likely to map to a participant’s research uncertainties following word cloud review. Two of these summary questions were subsequently ranked in the top 26 priorities, including the top research priority: raising awareness (Table 1).

Table 1 Summary questions to which more uncertainties mapped after word cloud review. Numbers represent the number of responses

Discussion

The inclusion of PwDCM-generated word clouds in a JLA PSP enabled stakeholders to generate more research uncertainties. Whilst this did not lead to the formation of additional research questions, research uncertainties submitted in response to the word clouds were more likely to map to the number one research priority: raising awareness.

The JLA, following some initial iterations, has continued to use a single and proven methodology [43]. The incorporation of word clouds in AO Spine RECODE-DCM was therefore the first example of a prospective, nested methodological study for the JLA. We have shown that such methodology can capture and highlight inputs from under-represented stakeholder groups like PwDCM. Word clouds provided other stakeholder groups a view on the interaction of PwDCM with healthcare services, which can be effectively incorporated into a protocol to influence research uncertainty identification.

Unlike many other consensus processes [24,25,26, 44, 45] the JLA does not incorporate shared feedback until the final consensus meeting [43]. The consequence of this is that input from certain stakeholder groups, such as patients, is not represented until the final stages of a study. In AO Spine RECODE-DCM, our ambition for the word clouds was to ensure that the perspectives of under-represented stakeholders were available for all, especially during the early information gathering phase of the process.

As only one research uncertainty is required to generate a research question, and uncertainties based on the word clouds alone did not generate a unique research question, it may appear that word clouds were unnecessary and lead to inefficiency. However, the fact that word clouds stimulated individual participants to generate new ideas reflects a positive impact on creativity. Furthermore, the identification of themes for summary questions will invariably be partly informed by the frequency of their representation [46]. Finally, it was of note that word cloud submissions were more likely to map to the number one research priority.

It is too early to conclude definitively whether word clouds had a positive impact, but these are all promising indications. Further research is required, but it is of note that the JLA has now decided to conduct a consultation looking at a role for developing a research methodology stream including how different research methodologies could affect consensus-building. This is particularly welcome in the context of an expanding literature demonstrating the challenges of end-user participation in the traditional PSP [23, 27]. In particular, strategies to promote participation of people with disabilities are most acutely needed [29, 47].

Limitations

The present study represents an early-phase methodological innovation and is perhaps best conceptualised as a feasibility study. As such, lessons can be learnt from our experience for future studies. Firstly, a more granular analysis of the impact of word clouds on each of the 3 participants groups would be of future value. Secondly, future work should define the most appropriate outcome measures to assess the impact of word clouds and other methodological innovations in PSPs, particularly given the qualitative nature of this field, and the challenge of drawing statements of causation. It may well be that end-user perceptions of accessibility and inclusion are the most appropriate metrics, instead of more quantitative measures such as word count.

Future implications

One of the challenges for implementation strategies in healthcare is the provision of early feedback [29, 48]. For example, a commonly used metric to evaluate strategies to accelerate evidence transfer into practice is its adoption within clinical guidelines. This will, at best, be measured in years [49]. The simple success here of feeding back current perspectives on end-user perspective using word clouds may present additional opportunities for implementation science, including monitoring the impact of research prioritisation and evaluating its need for an update.

Word clouds may also provide a practical solution for gathering perspectives in PSP projects with an end-user population that faces challenges engaging with traditional JLA methodology. For example, in a study aiming to make involvement in research more inclusive for people with complex speech and motor disorders, participants reported that, researchers and funders need to be more realistic about time, funding and accessibility to facilities to enable people with disabilities to participate [50]. Word clouds may improve participation of such groups through ease of online accessibly. Other protocols have already proven to be successful in promoting end-user involvement both online [47] and in person [27, 29]. Our approach seeks to develop this accessibility further given that word clouds: (1) do not require advanced language skills such as grammar or syntax, (2) do not require specialist healthcare professional training or input, (3) may be flexibly accessed online and respect fatigue demands of certain end-user groups, and (4) reduce in-person contact, which may be particularly pertinent in light of the recent COVID-19 pandemic.

Conclusion

The inclusion of word clouds in a JLA PSP increased the number of research uncertainties suggested, including the frequency of uncertainties which aligned with prioritised research questions. In AO Spine RECODE-DCM, this included the theme of raising awareness, which was the number research priority.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Davies BM, Mowforth OD, Smith EK, Kotter MR. Degenerative cervical myelopathy. BMJ. 2018;360:k186. https://doi.org/10.1136/bmj.k186.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Fehlings MG. et al. ‘A Clinical Practice Guideline for the Management of Patients With Degenerative Cervical Myelopathy: Recommendations for Patients With Mild, Moderate and Severe Disease and Nonmyelopathic Patients With Evidence of Cord Compression’. Global Spine J. 2017;7(3_suppl):70S83S.  https://doi.org/10.1177/2192568217701914.

  3. Fehlings MG, et al. A Global Perspective on the Outcomes of Surgical Decompression in Patients With Cervical Spondylotic Myelopathy: Results From the Prospective Multicenter AOSpine International Study on 479 Patients. Spine. 2015;40(17):1322–8. https://doi.org/10.1097/BRS.0000000000000988.

    Article  PubMed  Google Scholar 

  4. Oh T, et al. Comparing Quality of Life in Cervical Spondylotic Myelopathy with Other Chronic Debilitating Diseases Using the Short Form Survey 36-Health Survey. World Neurosurgery. 2017;106:699–706. https://doi.org/10.1016/j.wneu.2016.12.124.

    Article  PubMed  Google Scholar 

  5. Davies BM, et al. ‘RE-CODE DCM (REsearch Objectives and Common Data Elements for Degenerative Cervical Myelopathy): A Consensus Process to Improve Research Efficiency in DCM. Through Establishment of a Standardized Dataset for Clinical Research and the Definition of the Research Priorities’. Global Spine J. 2019;9(1_suppl):65S-76S. https://doi.org/10.1177/2192568219832855.

  6. Davies BM, Kwon BK, Fehlings MG, Kotter MRN. ‘AO Spine RECODE-DCM: Why Prioritize Research in Degenerative Cervical Myelopathy?’. Global Spine J. 2022;12(1_suppl)5S-7S. https://doi.org/10.1177/21925682211035379.

  7. Mowforth OD, et alGathering Global Perspectives to Establish the Research Priorities and Minimum Data Sets for Degenerative Cervical Myelopathy: Sampling Strategy of the First Round Consensus Surveys of AO Spine RECODE-DCM. Global Spine J. 2022;12(1_suppl):8S-18S. https://doi.org/10.1177/21925682211047546.

  8. Tetreault L,  et al. James Lind Alliance Priority Setting Partnership for Degenerative Cervical Myelopathy [AO Spine RECODE-DCM]: An Overview of the Methodology Used to Process and Short-List Research Uncertainties. Global Spine J. 2022;12(1_suppl):19S-27S. https://doi.org/10.1177/21925682211062501.

  9. Davies BM, et al. Improving Awareness Could Transform Outcomes in Degenerative Cervical Myelopathy [AO Spine RECODE-DCM Research Priority Number 1]. Global Spine Journal. 2022;12:(1_suppl):28S-38S. https://doi.org/10.1177/21925682211050927.

  10. Nouri A. et alDegenerative Cervical Myelopathy: Development and Natural History [AO Spine RECODE-DCM Research Priority Number 2]. Global Spine J. 2022:12(1_suppl):39S-54S. https://doi.org/10.1177/21925682211036071.

  11. Hilton B. et alEstablishing Diagnostic Criteria for Degenerative Cervical Myelopathy [AO Spine RECODE-DCM Research Priority Number 3]. Global Spine J. 2022;12(1_suppl):55S-63S. https://doi.org/10.1177/21925682211030871.

  12. Tetreault L. et alImproving Assessment of Disease Severity and Strategies for Monitoring Progression in Degenerative Cervical Myelopathy [AO Spine RECODE-DCM Research Priority Number 4]. Global Spine J. 2022;12(1_suppl):64S-77S. https://doi.org/10.1177/21925682211063854.

  13. Davies B. M. et alA New Framework for Investigating the Biological Basis of Degenerative Cervical Myelopathy [AO Spine RECODE-DCM Research Priority Number 5]: Mechanical Stress, Vulnerability and Time. Global Spine J. 2022;12(1_suppl)78S-96S. https://doi.org/10.1177/21925682211057546.

  14. Boerger TF. et alDeveloping Peri-Operative Rehabilitation in Degenerative Cervical Myelopathy [AO Spine RECODE-DCM Research Priority Number 6]: An Unexplored Opportunity?. Global Spine J. 2022;12(1_suppl):97S-108S. https://doi.org/10.1177/21925682211050925.

  15. Gharooni AA. et alDeveloping Novel Therapies for Degenerative Cervical Myelopathy [AO Spine RECODE-DCM Research Priority Number 7]: Opportunities From Restorative Neurobiology. Global Spine J. 2022;12(1_suppl):109S-121S. https://doi.org/10.1177/21925682211052920.

  16. Davies BM. et alEstablishing the Socio-Economic Impact of Degenerative Cervical Myelopathy Is Fundamental to Improving Outcomes [AO Spine RECODE-DCM Research Priority Number 8]. Global Spine J. 2022;12(1_suppl):122S-129S. https://doi.org/10.1177/21925682211039835.

  17. Martin AR, et alImaging and Electrophysiology for Degenerative Cervical Myelopathy [AO Spine RECODE-DCM Research Priority Number 9]. Global Spine J. 2022;12(1_suppl):130S-146S. https://doi.org/10.1177/21925682211057484.

  18. Rodrigues-Pinto R, et alOptimizing the Application of Surgery for Degenerative Cervical Myelopathy [AO Spine RECODE-DCM Research Priority Number 10]. Global Spine J. 2022;12(1_suppl):147S-158S. https://doi.org/10.1177/21925682211062494.

  19. Khan DZ et alThe AO Spine RECODE-DCM International Collaborative—Establishing the Foundations for Accelerated and Patient-Centered Innovation. Global Spine J. 2022;12(1_suppl)159S-171S. https://doi.org/10.1177/21925682211035381.

  20. Petit-Zeman S, Firkins L, Scadding JW. The James Lind Alliance: tackling research mismatches. Lancet. 2010;376(9742):667–9. https://doi.org/10.1016/S0140-6736(10)60712-X.

    Article  PubMed  Google Scholar 

  21. ‘What are evidence uncertainties? | James Lind Alliance’. https://www.jla.nihr.ac.uk/jla-guidebook/chapter-2/what-are-evidence-uncertainties.htm (Accessed 17 Dec 2022).

  22. Barrett D, Heale R. What are Delphi studies? Evid Based Nurs. 2020;23(3):68–9. https://doi.org/10.1136/ebnurs-2020-103303.

    Article  PubMed  Google Scholar 

  23. Lavallee DC, et al. Comparing three approaches for involving patients in research prioritization: a qualitative study of participant experiences. Res Involv Engagem. 2020;6:18. https://doi.org/10.1186/s40900-020-00196-4.

    Article  PubMed  PubMed Central  Google Scholar 

  24. J. C. Crocker et al., ‘Impact of patient and public involvement on enrolment and retention in clinical trials: systematic review and meta-analysis’, BMJ, vol. 363, p. k4738, Nov. 2018, https://doi.org/10.1136/bmj.k4738.

  25. Crocker JC, et al. Patient and public involvement (PPI) in UK surgical trials: a survey and focus groups with stakeholders to identify practices, views, and experiences. Trials. 2019;20(1):119. https://doi.org/10.1186/s13063-019-3183-0.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Brett J, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect. 2014;17(5):637–50. https://doi.org/10.1111/j.1369-7625.2012.00795.x.

    Article  PubMed  Google Scholar 

  27. A. Pollock, B. St George, M. Fenton, S. Crowe, and L. Firkins, ‘Development of a new model to engage patients and clinicians in setting research priorities’, J Health Serv Res Policy, vol. 19, no. 1, pp. 12–18, Jan. 2014,https://doi.org/10.1177/1355819613500665.

  28. Madden M, Morley R. Exploring the challenge of health research priority setting in partnership: reflections on the methodology used by the James Lind Alliance Pressure Ulcer Priority Setting Partnership. Res Involv Engagem. 2016;2:12. https://doi.org/10.1186/s40900-016-0026-y.

    Article  PubMed  PubMed Central  Google Scholar 

  29. K. E. Chadd, A. A. Kulkarni, and L. M. Longhurst, ‘Involving Individuals with Developmental Language Disorder and their Parents/Carers in Research Priority Setting’, J Vis Exp, no. 160, Jun. 2020, https://doi.org/10.3791/61267.

  30. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9. https://doi.org/10.1016/S0140-6736(09)60329-9.

    Article  PubMed  Google Scholar 

  31. Chalmers I, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65. https://doi.org/10.1016/S0140-6736(13)62229-1.

    Article  PubMed  Google Scholar 

  32. D. A. Shaywitz and D. A. Ausiello, ‘Preserving Creativity in Medicine’, PLOS Medicine, vol. 1, no. 3, p. e34, Dec. 2004, https://doi.org/10.1371/journal.pmed.0010034.

  33. Hearst MA, Pedersen E, Patil L, Lee E, Laskowski P, Franconeri S. An Evaluation of Semantically Grouped Word Cloud Designs. IEEE Trans Visual Comput Graphics. 2020;26(9):2748–61. https://doi.org/10.1109/TVCG.2019.2904683.

    Article  Google Scholar 

  34. Viegas FB, Wattenberg M, Feinberg J. Participatory Visualization with Wordle. IEEE Trans Visual Comput Graphics. 2009;15(6):1137–44. https://doi.org/10.1109/TVCG.2009.171.

    Article  Google Scholar 

  35. Davies BM, et al. The development of lived experience-centered word clouds to support research uncertainty gathering in degenerative cervical myelopathy: results from an engagement process and protocol for their evaluation, via a nested randomized controlled trial. Trials. 2021;22(1):415. https://doi.org/10.1186/s13063-021-05349-8.

    Article  PubMed  PubMed Central  Google Scholar 

  36. B. Davies et al., ‘Recovery priorities in degenerative cervical myelopathy: a cross-sectional survey of an international, online community of patients’, BMJ Open, vol. 9, no. 10, p. e031486, Oct. 2019, https://doi.org/10.1136/bmjopen-2019-031486.

  37. T. Boerger et al., ‘Moving Beyond the Neck and Arm: The Pain Experience of People With Degenerative Cervical Myelopathy Who Have Pain’, Global Spine Journal, p. 2192568220986143, Feb. 2021, https://doi.org/10.1177/2192568220986143.

  38. B. M. Davies et al., ‘Outcomes of Degenerative Cervical Myelopathy From The Perspective of Persons Living With the Condition: Findings of a Semistructured Interview Process With Partnered Internet Survey’, Global Spine Journal, p. 2192568220953811, Nov. 2020, https://doi.org/10.1177/2192568220953811.

  39. Gharooni A-A, Grodzinski B, Davies BM, Kotter MRN. How common is repeat surgery and multi-level treatment in Degenerative Cervical Myelopathy? Findings from a patient perspective survey. J Clin Neurosci. 2020;77:181–4. https://doi.org/10.1016/j.jocn.2020.04.109.

    Article  PubMed  Google Scholar 

  40. M. B. Butler et al., ‘Provision and Perception of Physiotherapy in the Nonoperative Management of Degenerative Cervical Myelopathy (DCM): A Cross-Sectional Questionnaire of People Living With DCM’, Global Spine Journal, p. 2192568220961357, Oct. 2020, https://doi.org/10.1177/2192568220961357.

  41. ‘Briefing notes for researchers - public involvement in NHS, health and social care research’, National Institute for Health Research, Apr. 05, 2021. https://www.nihr.ac.uk/documents/briefing-notes-for-researchers-public-involvement-in-nhs-health-and-social-care-research/27371 (Accessed 11 Feb 2022).

  42. Mowforth OD, Davies BM, Goh S, O’Neill CP, Kotter MRN. Research Inefficiency in Degenerative Cervical Myelopathy: Findings of a Systematic Review on Research Activity Over the Past 20 Years. Global Spine Journal. 2020;10(4):476–85. https://doi.org/10.1177/2192568219847439.

    Article  PubMed  Google Scholar 

  43. ‘JLA Guidebook’. Available from: https://www.jla.nihr.ac.uk/jla-guidebook/ (Accessed 11 Jan 2022).

  44. P. R. Williamson et al., ‘The COMET Handbook: version 1.0’, Trials, vol. 18, no. 3, p. 280, Jun. 2017, https://doi.org/10.1186/s13063-017-1978-4.

  45. S. Parks, C. d’Angelo, and S. Gunashekar, ‘Citizen science: generating ideas and exploring consensus’, THIS.Institute, p. 16.

  46. Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nurs Health Sci. 2013;15(3):398–405. https://doi.org/10.1111/nhs.12048.

    Article  PubMed  Google Scholar 

  47. Rowbotham NJ, et al. Adapting the James Lind Alliance priority setting process to better support patient participation: an example from cystic fibrosis. Res Involv Engagem. 2019;5(1):24. https://doi.org/10.1186/s40900-019-0159-x.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. J. Tempest-Mitchell et al., ‘A comparison of radiological descriptions of spinal cord compression with quantitative measures, and their role in non-specialist clinical management’, Plos One, vol. 14, no. 7, p. e0219380, Jul. 2019, https://doi.org/10.1371/journal.pone.0219380.

  49. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510–20. https://doi.org/10.1258/jrsm.2011.110180.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Jayes M, et al. Making Public Involvement in Research More Inclusive of People With Complex Speech and Motor Disorders: The I-ASC Project. Qual Health Res. 2021;31(7):1260–74. https://doi.org/10.1177/1049732321994791.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors acknowledge the support of the AO Spine RECODE-DCM community and Myelopathy.org in the completion of this work. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health. AO Spine RECODE-DCM and its investigators would like to acknowledge the funding and support of AO Spine, without whom this initiative would not be possible.

Funding

ODM is supported by an Academic Clinical Fellowship at the University of Cambridge. AM is funded by a Clinician Scientist Fellowship (NIHR-CS-2017–17-010) from the UK National Institute for Health Research (NIHR) and supported by the NIHR Biomedical Research Centre at the University Hospitals Bristol NHS Foundation Trust and the University of Bristol. Research in the MRNK’s laboratory is supported by a core support grant from the Wellcome Trust and MRC to the Wellcome Trust-Medical Research Council Cambridge Stem Cell Institute. MRNK is supported by a NIHR Clinician Scientist Award, CS-2015–15-023. BMD is supported by an NIHR Clinical Doctoral Fellowship.

Author information

Authors and Affiliations

Authors

Contributions

LB—manuscript review and preparation, ODM – conceptualisation, data analysis, manuscript review and preparation, DZK- data analysis, manuscript review and preparation, XY- manuscript review and preparation, TG—data analysis, manuscript review and preparation, SRLS—data analysis, manuscript review and preparation, OH—data analysis, manuscript review and preparation, LT—manuscript review and preparation, SK—data analysis, manuscript review and preparation, MS—manuscript review and preparation, IS – data collection, manuscript review and preparation, ES—data analysis, manuscript review and preparation, DH—manuscript review and preparation, JC—manuscript review and preparation, PH—manuscript review and preparation, VR—manuscript review and preparation, JDG—manuscript review and preparation, BA—manuscript review and preparation, BKK—manuscript review and preparation, SNK—manuscript review and preparation, JH—manuscript review and preparation, JW—manuscript review and preparation, RG—manuscript review and preparation, EKS—manuscript review and preparation, AM—manuscript review and preparation, MGF—manuscript review and preparation, MRNK—manuscript review and preparation, BMD—conceptualization, data analysis, manuscript review and preparation. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Benjamin M. Davies.

Ethics declarations

Ethics approval and consent to participate

Ethical approval for the study was granted by the Human Biology Research Ethics Committee, University of Cambridge (HBREC.2019.14). Informed consent was obtained from all study participants before survey completion.

Word clouds were generated in collaboration with Myelopathy Support, an arm of Myelopathy.org, which is an international DCM charity. Ethical approval was not required for the involvement of the Myelopathy Support members as their role was to help to develop the word clouds for the survey and they were not research participants. The AO Spine RECODE-DCM Management group has members who are people with DCM. These members had significant input into the design of this study, the refinement of online poll data and the creation of a protocol for implementation into the AO Spine RECODE-DCM study.

All methods were carried out in accordance with relevant guidelines and regulations. All RECODE-DCM protocols were approved by AO Spine. Informed consent was obtained from all RECODE-DCM participants before survey completion.

Consent for publication

The authors give their consent for publication of this work.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mowforth, O.D., Burn, L., Khan, D.Z. et al. Lived experience-centred word clouds may improve research uncertainty gathering in priority setting partnerships. BMC Med Res Methodol 23, 100 (2023). https://doi.org/10.1186/s12874-023-01897-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12874-023-01897-6

Keywords