A review and synthesis of frameworks for engagement in health research to identify concepts of knowledge user engagement
BMC Medical Research Methodology volume 19, Article number: 211 (2019)
Engaging those who influence, administer and/or who are active users (“knowledge users”) of health care systems, as co-producers of health research, can help to ensure that research products will better address real world needs. Our aim was to identify and review frameworks of knowledge user engagement in health research in a systematic manner, and to describe the concepts comprising these frameworks.
An international team sharing a common interest in knowledge user engagement in health research used a consensus-building process to: 1) agree upon criteria to identify articles, 2) screen articles to identify existing frameworks, 3) extract, analyze data, and 4) synthesize and report the concepts of knowledge user engagement described in health research frameworks. We utilized the Patient Centered Outcomes Research Institute Engagement in Health Research Literature Explorer (PCORI Explorer) as a source of articles related to engagement in health research. The search includes articles from May 1995 to December 2017.
We identified 54 articles about frameworks for knowledge user engagement in health research and report on 15 concepts. The average number of concepts reported in the 54 articles is n = 7, and ranges from n = 1 to n = 13 concepts. The most commonly reported concepts are: knowledge user - prepare, support (n = 44), relational process (n = 39), research agenda (n = 38). The least commonly reported concepts are: methodology (n = 8), methods (n = 10) and analysis (n = 18). In a comparison of articles that report how research was done (n = 26) versus how research should be done (n = 28), articles about how research was done report concepts more often and have a higher average number of concepts (n = 8 of 15) in comparison to articles about how research should be done (n = 6 of 15). The exception is the concept “evaluate” and that is more often reported in articles that describe how research should be done.
We propose that research teams 1) consider engagement with the 15 concepts as fluid, and 2) consider a form of partnered negotiation that takes place through all phases of research to identify and use concepts appropriate to their team needs. There is a need for further work to understand concepts for knowledge user engagement.
In health research, there is growing support for the engagement of patients, caregivers, clinicians, managers, decision makers and other healthcare systems users as co-producers of evidence. The participation of researchers with those who influence, administer and/or who are active users of healthcare systems create opportunities for processes of co-production of research evidence and that is a form of knowledge. Collaboration between researcher and health care systems users can take many forms, and is often referred to using different terminology, for example, integrated knowledge translation , collaborative research, co-production of knowledge , and engaged scholarship . In all cases, however, there is a common aim: to produce and apply knowledge to address real-world needs. The resulting knowledge is more likely to be useful and useable and therefore more likely to be applied in policy and practice [3, 4]. We define "engagement" in research as an arrangement in the governance of the research process with those who influence, administer and/or who are active users of healthcare systems, and that leads to co-production of knowledge (beyond being a research participant) [5, 6].
Engagement in research disrupts the notion of research as the sole domain of academic researchers. The expansion of research roles invite many diverse forms of expertise into an involved and intentional process of knowledge production. We adopt and adapt the definitions developed by Parry et al. (2015) to describe the roles of those engaged in research that leads to knowledge production. The aim of engagement in research is to integrate the views and values of "knowledge users", meaning those who are actively involved in the knowledge production process of a study and who may benefit or be otherwise affected by the research . We expand the definition of knowledge user, and include those who may not be members of the mainstream academic research team or community but who will contribute expertise to knowledge production processes and/or who will influence, administer or be an active user of the research results to support their decision-making. Knowledge users are often members of groups for which the research holds significance. These groups may act on (or may be affected by) the research. Knowledge users might include people who will receive care that is based on the research findings. They may occupy a range of positions, such as research funders, patients or members of the public, health system and policy decision-makers, health care providers, instructors or students at training institutions, et cetera. For example, the inclusion of patient-, healthcare delivery systems- and research funding agency-representation was identified as valuable in the process of selection and refining of pilot research and quality improvement projects in diabetes care . Knowledge users are the recipients of research impacts and for this reason are the key partners and contributors of expertise in the co-production of research evidence; they are distinct from broader groups in which the research endeavour is situated.
The term "stakeholder" is used in many different ways: we define stakeholders to include those members of the groups, in which a research partnership of knowledge user(s) and researcher(s) is situated. Stakeholders are people or organizations who may be indirectly affected by research . They may have an interest in the research and use of findings but are not anticipated to directly influence, administer and/or actively use the research results in their own decision-making. For example, in studies that investigate the primary care based management of frailty with older people members of the general population would be considered stakeholders, as those who are not anticipated to influence, administer or utilize the healthcare system for services related to frailty, . Stakeholders represent individuals, groups or organizations in which a knowledge user-researcher team is situated (Fig. 1).
International work is underway to promote knowledge user engagement in health research. For example, the United States-based Patient Centred Outcomes Research Institute (PCORI) has identified the engagement of knowledge users throughout the research process to be more patient centred, useful, and trustworthy and ultimately to lead to greater use and uptake of research results by the patient and broader healthcare community . Beyond identifying engagement as valuable, PCORI requires engagement in much of its funded research. PCORI works to shape the field and provides thought leadership and sets standards for engagement in research for others to reference as a model, and summarizes and synthesizes findings that relate to the field of engagement in health research – to create accessible information and opportunities to further the learning of researchers . In Canada, there have been recent, similar initiatives, with the National Strategy for Patient-Oriented Research (SPOR) that aims to foster evidence-informed health care through innovation – to improve quality, accountability and accessibility . Like PCORI, SPOR aims to integrate the views and values of patients into health policy and practice through research. There are many other examples of work underway to define patient and other stakeholder engagement in health research, and that also aim to promote knowledge user engagement in health research [13,14,15,16].
There is a need to conceptualize how to do health research in ways that support the engagement of those who influence, administer and/or who are active health care system users (knowledge users). That is, it is important to gain insight on the planning, implementation, and dissemination processes of research conducted from within partnerships of knowledge users and researchers to co-produce knowledge that addresses real-world needs . Understanding the ways to support engagement of knowledge users in health research can lead to greater clarity in communication about and evaluation of health research conducted with knowledge user-researcher partnerships. Identification of the models and frameworks that organize concepts relevant to knowledge user engagement in health research is a valuable endeavor.
The organization of concepts in health research models and frameworks is a way to describe how to conduct research inclusive of knowledge users that leads to the co-production of knowledge. We use Walker and Avant’s (2005) definition of "concept" as a mental representation of a phenomenon, idea, action or thing, that represents observed occurrences . Concepts categorize information into meaningful constructs in the form of models and frameworks to explain phenomena. There are many ways to define the terms “models” and “frameworks” and often these terms may be conceptually different and inconsistently applied. Both terms relate to the organization of concepts; for the purposes of the work presented in this paper, we use the term “frameworks” to refer to the actual, intentional organization of concepts.
Research practices that promote knowledge user engagement in health research should be guided by conceptual frameworks to focus on and aid implementation [19, 20]; currently there exists little consensus on the essential concepts of knowledge user engagement and guidance for the effective conduct of health research . Our approach was to produce an evidence synthesis that combined the strengths of the comprehensive search process conducted by PCORI, with a process of critical review . Our aim was to identify and then review frameworks of knowledge user engagement in health research in a systematic manner, and to describe the concepts comprising these frameworks.
An international team, consisting of United States- and Canadian-based members, formed a collaboration and are identified here as “the team”. The team consists of an information scientist (LD); policy and practice leads (RD, SR, KW); and researchers (JJ, TN, IDG). All members of the team shared a common interest in understanding best practices in knowledge user engagement in health research, and are either acknowledged or listed as co-authors on this paper. We used an integrated knowledge translation approach meaning that there was an interactive process of knowledge exchange among team members to produce knowledge more likely to be useful to health systems’ knowledge users . The team engaged in a consensus-building process that consisted of a series of regular meetings and online communications to: 1) agree upon how to identify the concepts of knowledge user engagement in health research, 2) identify existing frameworks that capture concepts of knowledge user engagement in health research, and 3) identify, synthesize and report the concepts of knowledge user engagement described in health research frameworks.
PCORI developed the Engagement in Health Research Literature Explorer (“PCORI Explorer”) as a searchable list of articles related to engagement in research . To develop the searchable list of articles, a PubMed/MEDLINE search strategy was developed and inclusion/exclusion criteria applied to confirm the final list of articles. Selected articles were then tagged across three engagement-related categories that include: Topic, Stakeholder Involvement, and Phases of Research Engagement . For more information about the search strategy used to create this open-access resource, see: https://www.pcori.org/literature/engagement-literature. The PCORI Explorer is comprised of peer-reviewed articles that have been classified by AcademyHealth researchers into four article topic types: 1) Example of Engagement in Health Research, 2) Detailed Description of Engagement in Health Research, 3) Framework, Editorial, Commentary, and 4) Evidence for Engagement . The search includes articles from May 1995 and for our review include those up to December 2017.
Given the combined the strengths of PCORI’s landscape review of engagement frameworks in the development of the PCORI Explorer to obtain and then classify articles related to engagement in research, we chose to focus on the article topic type Framework, Editorial, Commentary as the data source for our review. The Framework, Editorial, Commentary section includes “Manuscripts that express a theoretical view on engagement in health research, including scientific commentaries, opinion briefs, or conceptual pieces such as models or frameworks” .
Our study procedure was as follows: 1) determine and agree upon a set of screening criteria for articles to identify frameworks that describe and/or depict concepts of knowledge user engagement in health research; 2) screen the articles within the PCORI Explorer article topic type Framework, Editorial, Commentary to identify those that report on frameworks of knowledge user engagement in health research; 3) use a predetermined approach to abstract and conduct a content analysis of information about frameworks of knowledge user engagement in health research (demographics, concepts of knowledge user engagement reported in the framework) from the identified health research frameworks, and 4) analyze and synthesize the abstracted data and disseminate results (Fig. 2).
Screening criteria to identify frameworks
First, the team agreed upon how to identify the concepts of knowledge user engagement in health research, and these definitions are reported earlier: “engagement”, “knowledge user”, “stakeholder”, “concept”, and “framework”. Next, the team determined a set of screening criteria to identify articles that report on frameworks depicting concepts of knowledge user engagement in health research. There were a series of meetings to test the screening criteria with example articles, and to refine and confirm the inclusion criteria. The team reached consensus for inclusion criteria and that include: 1) the article had to present what the author of the article referred to as a framework or model of knowledge user engagement in health research; or, 2) in the absence of self-labeling as framework or model, the article had to explicitly describe and/or depict (use of figure, diagram) the organization of concepts for knowledge user engagement in health research; and 3) the framework had to be about collaborative research involving a researcher and at least one other team member who is a knowledge user (i.e., patient, caregivers, family members, healthcare providers, managers, policy makers et cetera). For screening articles, we agreed to an operational version of the definition for knowledge user engagement in health research: as an intentional arrangement that creates opportunities for knowledge users to co-lead in the processes associated with planning, conducting or disseminating research that results in co-production of knowledge [5, 6]. When an article reported on a framework that is cited in another article, the original article was obtained and screened for inclusion.
Next, we screened the abstracts and full text of the 310 articles, sourced from the PCORI Explorer topic section Framework, Editorial, Commentary – up to December 2017. One team member (JJ) screened articles for inclusion, with duplicate screening done independently by a second team member (TN). A third reviewer was available for discussion for conflict between the reviewers (IDG), and there were opportunities with regular meetings and shared documents for process review by the entire team (JJ, LD, RD, TN, KW, SR, IDG). As well, during the screening process, there were team meetings to discuss articles and ensure the application of the screening criteria in an agreed-upon manner. During screening an additional seven articles (external to the PCORI Explorer) were identified because they had been reported as a source of frameworks in the reviewed articles; these additional articles were also obtained and screened (Fig. 3).
Abstract and conduct a content analysis
Content analysis is a useful way to describe ideas and to generate or extend ways of considering organization of concepts . With directed content analysis, pre-existing theory or research is used to develop the initial coding scheme . During the analysis, additional codes can be developed and the initial coding scheme can be revised or refined; in this way, existing approaches to theory or research can be extended and/or refined . We describe the use of a directed content analytic approach, and that involves the examination of the reported aims and concepts in articles that were identified to report the use of particular words and/or depict (with the use of figure, diagram) concepts for knowledge user engagement in health research [24, 25]:
Preparation, in which we decided upon the unit of analysis and the conduct of a review of each article for data. For our review, we agreed to use an iterative approach to collect data on pre-defined concepts of knowledge user engagement in health research and to inductively expand on and refine these concepts. We agreed upon and used the definition for co-production of knowledge . Each time a new concept relating to knowledge user engagement was identified, data was extracted and previously identified articles reviewed for presence of the newly identified concepts;
Analysis of each article to identify and code data by a first reviewer (JJ) and with verification of a second reviewer (TN or IDG); the entire team (JJ, LD, RD, TN, KW, SR, IDG) was engaged in regular and ongoing discussion and review. For each article there was demographic data extracted. As well, we started with a core set of concepts based on an agreed-upon definition for co-production of knowledge to extract knowledge-user engagement concepts. For each framework we sought evidence of concepts that relate to plans for or actual conduct of knowledge user engagement that describe one or more of the following: i) development of research questions, ii) development of methodology, iii) data collection and/or tool development, iv) interpretation of study findings, v) crafting messages around the results and moving the results into practice . During the data abstraction we refined definitions and identified additional concepts that related to engagement; and,
Reporting, to describe the analysis and findings. For our review, the aim was to identify and review frameworks of knowledge user engagement in health research in a systematic manner, and to describe the concepts comprising these frameworks. Following discussion, the entire team agreed upon final findings.
We identified 54 articles about frameworks for knowledge user engagement in health research and report on 15 concepts. A full list of the 54 articles is included (Additional file 1).
The included articles depicting frameworks were published between 2000 and 2017, with none published prior to 2000 and with the majority released in 2016 (n = 14, 26% of the articles). The first authors of the articles were reported to be located across eight countries (Netherlands, Switzerland, Spain, Canada, Australia, United Kingdom, United States) with the highest number of first authors located in the United States (n = 29, 50%). Knowledge users were explicitly identified in the articles as coauthors (n = 8, 15%) acknowledged on the publication (n = 24, 44%) or not identified on the publication (n = 30, 55%). The number (range) of knowledge users who were reported as participating with researchers ranged from none  to well-over over 200 .
In the articles, there were many and different combinations of individual and group terms used to describe/represent knowledge users in the articles such as: patients, members of the public, stakeholders, service users, consumers, expert participants, survivors, clinicians/healthcare providers, peer support workers, mentees, decision and policy makers, et cetera. As well, terms for members of group and institutional teams were used, such as: institutional review boards, sponsors, community (includes different versions, such as rural community, community stakeholders et cetera), healthcare team (includes terms such as palliative care team), community based organizations, et cetera. The range of descriptions for knowledge users indicates a breadth of consideration for who to consider in research partnerships and how they are labeled.
Concepts for knowledge user engagement
During the content analysis of frameworks, we identified 15 concepts for knowledge user engagement reported across the 54 articles. We report these concepts with a definition and an example (Table 1). The 15 concepts are reported across four general research phases: Prepare (as a precursor to the study), Plan (the design of the study), Conduct (the study), Apply (findings of the study) and do not mean to imply an order or denote the importance of the concepts relative to one another. For example, the concept of “ethics - principles values” was most often reported in articles when describing preparation for research; however, the concept is also evident throughout other research phases in multiple articles. The average number of concepts reported in the 54 articles is n = 7, and range from n = 1 to n = 13 concepts. The three most commonly reported concepts include: knowledge user - prepare, support (n = 44), relational process (n = 39), research agenda (n = 38). The three least commonly reported concepts include: methodology (n = 8), methods (n = 10) and analysis (n = 18) (Fig. 4).
Further, articles can be classified in one of two categories as they either report on frameworks that describe how research was done or report on frameworks that are the result of considering how research should be done. That is, the articles were found to either 1) describe how research was done and to have reported on what had occurred in a study about knowledge user engagement in health research (n = 26); or, 2) describe how research should be done to foster knowledge user engagement in health research, that is, to report on a literature review, expert opinion, analysis of theory about knowledge user engagement in health research et cetera (versus actual health research conduct) (n = 28). The articles are organized into two framework categories and with reporting across concepts: articles describing how research was done to foster knowledge user engagement in health research (Fig. 5), and; articles describing how research should be done to foster knowledge user engagement in health research (Fig. 6). We discuss the similarities and differences between the two groups of articles.
A comparison of frameworks that describe “How research was done” with “How research should be done”
In a comparison of the articles that report on frameworks that describe how research was done with how research should be done, those that report how research was done have a higher average number of concepts (n = 8) in comparison to articles that reported on how research should be done (n = 6). For frameworks that describe how research was done, the range of concepts reported was between n = 13 and n = 3: an article that describes collaborative research with an Indigenous community (n = 13)  and an article that describes priority setting with patients (n = 3) . For frameworks that describe how research should be done (n = 28) the range of concepts reported was between n = 13 and n = 1: an article that reports on a systematic review and environmental scan of the literature about patient and service user engagement in biomedical and health services research (n = 13)  and articles reporting on patient and public involvement in translational science (n = 1) , and on the advantages and disadvantages of public involvement in health services research agenda setting (n = 1) .
Overall, the articles that report on frameworks that describe how research was done reported concepts more often. One exception was the “evaluate” concept that was reported less often in frameworks that describe how research was done (38%) relative to frameworks that describe how research should be done (57%). The greatest differences in frequency of concept reporting was found for the concept of “resources” (how research was done = 58% versus how research should be done =18%) and the smallest difference was found for the concept “sustain” (how research was done = 38% versus how research should be done = 36%) (Table 2).
The aim of our work was to identify and review frameworks of knowledge user engagement in health research in a systematic manner, and to describe the concepts comprising these frameworks. To do this work, we identified, reviewed and synthesized 54 articles that reported on frameworks meeting our criteria for describing and/or depicting knowledge user engagement in health research. Our analysis identified 15 concepts related to knowledge user engagement in health research. The variation in the reported concepts between the frameworks means that it is difficult to conclude what and in which order might be best for knowledge user engagement in health research. Research teams should think about engagement with the concepts as being fluid rather than strictly required, as there is a lack of evidence at this time for the benefits of engaging knowledge users across each of the 15 identified concepts.
Understanding the concepts of knowledge user engagement in health research may help researchers and knowledge user partners to identify and operationalize engagement in health research, and in ways that account for the nuanced characteristics and preferences of particular group(s) of people. For knowledge users there are potentially many opportunities and ways to engage or be engaged in areas of research that may typically be the domain of researchers alone. There is a need to understand best practice in research engagement : encouraging dialogue and debate between knowledge users and researchers about the concepts of knowledge user engagement in health research on a situational basis may contribute to processes of engagement in health research.
We propose that it is reasonable to suggest that knowledge users and researchers discuss and negotiate the research process to determine who should be meaningfully engaged and in what ways. It is important to maximize the potential to advance research project objectives to reflect the needs and aims of knowledge user partners to ensure that the outcomes are useful and impactful. As well, we propose that a form of partnered negotiation takes place at the start and throughout the research study. A process of partnered negotiation is important for considering and possibly utilizing concepts throughout the phases of research, in order to foster successful knowledge user engagement. We depict these relationships in a met-framework (Fig. 7).
In our work, we have identified the need to think about knowledge user engagement in health research as an arrangement that creates ongoing opportunities for engagement by knowledge users throughout the research process. While the promotion of full engagement of knowledge users throughout the research process is described in the literature as an ideal [5, 6], dictating when and how knowledge users and researchers need to be involved is not helpful nor is it realistic. The equal involvement of knowledge users and researchers throughout the research process may vary and a focus on equity may instead be more realistic for knowledge user engagement. That is, while the contributions by knowledge users and researchers may differ or vary at times, the effort and impact is similar – sharing equally is not as important as considering the quality of engagement in research processes. In our work, studies were found to report on many types of individual and collective knowledge users, and we propose that engagement with knowledge users is a form of partnered negotiation. Every research partnership is unique, as it consists of individuals (knowledge users and researchers) who, themselves, understand their unique expertise, needs and context.
The use of frameworks can guide the exploration of how to operationalize measures of impact of knowledge user engagement. The findings of our study are reinforced by the literature; there are a broad range of ways in which knowledge users might be engaged in health research and little evidence for the impact of this engagement . As well, there has been an important - and dominant - focus on patient and lay involvement in health research . Our work shows that engagement of knowledge users in health research is more complex than bringing in one form of knowledge user: the pursuit of knowledge user engagement in health research means that there are teams consisting of researchers and knowledge users that span health care systems. The evaluation of knowledge user engagement in health research is identified as an area in need of development [17, 44,45,46,47]. To build understandings and unravel the complexity of knowledge user engagement in health research, it is imperative that research studies utilize guiding frameworks. For example, PCORI (a major research funder in the United States) encourages that those conducting funded projects evaluate the engagement of knowledge users (defined as patients and other stakeholders, such as clinicians, purchasers, payers et cetera) with the PCORI Engagement Rubric [5, 48, 49]. The aim of the evaluation is to develop knowledge about engagement practices with knowledge users. Our finding of lower reporting of the “evaluate” concept suggests that there may be some barriers to the actual conduct of evaluation in frameworks of knowledge user engagement in health research. To advance the field of knowledge user engagement in health research, future efforts should focus on evaluation.
In addition to the evaluation of engagement practices with knowledge users, we propose that studies need to go beyond reporting on the framework used for knowledge user engagement in health research, and present analysis about the use of frameworks that guide research. Many studies report on the use of frameworks to guide knowledge user engagement in health research; however, there are few examples of studies that systematically evaluate these guiding frameworks for their use in guiding teams consisting of knowledge users and researchers [11, 50]. Studies that evaluate the guiding frameworks for knowledge user engagement in health research are important for understanding how to engage people at different points in the study, and in what ways. Here we have reported on a range of included studies that show the complexity and variability of knowledge user engagement, across research stages and concepts. To advance the field of knowledge user engagement in health research, it is imperative that there is evaluation of the frameworks guiding knowledge user engagement.
Limitations and strengths
A key limitation of our study is that we may have included or excluded articles inappropriately due to the conceptually different and inconsistently used range of terms and descriptions for models and frameworks. For the purposes of this review, and in an attempt to mitigate issues with terminology, we agreed upon an operational definition to aid in the identification of models and frameworks. In addition, our study used a source of articles designed to capture the general literature about engagement in health research, rather than a search designed and conducted specifically for our purposes.
Strengths of our review include the use of the PCORI Explorer to identify articles, and the involvement of an international team with the use of a consensus-building process. The PCORI Explorer is a repository of peer-reviewed articles focused on guidance in the field of collaborative research conduct and that includes frameworks, conceptual models, and editorials to express a theoretical view on engagement in health research. Those who oversee the PCORI Explorer repository also included articles that are recommended by PCORI staff, or journals identified as highly relevant despite not yet being indexed in PubMed . Additionally, an international team with experience in conceptualizing the engagement of knowledge users in collaborative research across a range of settings included those with expertise in the use of the PCORI Explorer (LA, RD, SR, KW) and use and/or development of knowledge translation frameworks (JJ, TN, IDJ).
The aim of our work was to identify and review frameworks of knowledge user engagement in health research in a systematic manner, and to describe the concepts comprising these frameworks. We identify 15 concepts of knowledge user engagement in health research, and suggest that to advance the field of knowledge user engagement in health research there is a need to focus on evaluation. Given the variation in the number of concepts reported across the included frameworks it is difficult to say what and in which order concepts are best for collaborative research. Further work to develop understandings of knowledge user engagement in health research is encouraged.
Availability of data and materials
Supporting data may be found at the PCORI Explorer https://www.pcori.org/literature/engagement-literature repository, and the datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Graham ID, Tetroe J. Knowledge translation research group. Some theoretical underpinnings of knowledge translation. Acad Emerg Med. 2007;14(11):936–41.
Tetroe J. Knowledge Translation at the Canadian Institutes of Health Research: a primer. Focus Technical Brief. 2007;18:1-8.
Van de Ven D, Johnson P. Knowledge for theory and practice. Acad Manag Rev. 2006;31(4):802–21.
Bowen S, Graham I. Integrated knowledge translation. In: Straus SE, Tetroe J, Graham ID, editors. Knowledge translation in healthcare: moving evidence to practice. West Sussex: Wiley; 2013. p. 14–23.
Sheridan S, Schrandt S, Forsythe L, Hilliard TS, Paez KA. Advisory panel on patient engagement (inaugurel panel 2013) . The PCORI engagement rubric: promising practices for partnering in research. Ann Fam Med. 2017;15(2):165–70.
Canadian Institutes of Health Research (CIHR). Patient engagement: What is patient engagement? Government of Canada 2018 Available from: http://www.cihr-irsc.gc.ca/e/45851.html.
Parry D, Salsberg J, Macauley AC. Guide to researcher and knowledge-user collaboration in health research 2015. Available from: http://www.cihr-irsc.gc.ca/e/44954.html#s2.
Schmittdiel JA, Desai J, Schroeder EB, Paolino AR, Nichols GA, Lawrence JM, et al. Methods for engaging stakeholders in comparative effectiveness research: a patient-centered approach to improving diabetes care. Healthc (Amst). 2015;3(2):80–8.
Center for Commununity Health and Development at the University of Kansas. Community Toolbox. Section 8: Identifying and analyzing stakeholders and their interests. 2016 Available from: https://ctb.ku.edu/en/table-of-contents/participation/encouraging-involvement/identify-stakeholders/main.
Heaven A, Brown L, Foster M, Clegg A. Keeping it credible in cohort multiple randomised controlled trials: the community ageing research 75+ (CARE 75+) study model of patient and public involvement and engagement. Res Involv Engagem. 2016;2:30.
Forsythe L, Heckert A, Margolis MK, Schrandt S, Frank L. Methods and impact of engagement in research, from theory to practice and back again: early findings from the Patient-Centered outcomes research institute. Qual Life Res. 2018;27:17–31.
Canadian Institutes of Health Research (CIHR). Strategy for Patient-Oriented Research 2018 Available from: http://www.cihr-irsc.gc.ca/e/41204.html.
n.a. PaCER - Patient and Community Engagement Research. Welcome to PaCER: PaCER Innovates; 2018. Available from: https://pacerinnovates.ca. Accessed 27 Jan 2019.
Graham ID, Kothari A, McCutcheon C, on behalf of the Integrated Knowledge Translation Research Network Program Leads. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation. Implement Sci. 2018;13(1):22.
Hayes H, Buckland S, Tarpey M. INVOLVE briefing notes for researchers: involving the public in NHS. Eastleigh: INVOLVE: Public Health and Social Care Research; 2012.
Australian Government National Health and Medical Research Council. Partnership Projects: Commonwealth of Australia 2018; 2018 Available from: https://nhmrc.gov.au/funding/find-funding/partnership-projects.
Manafo E, Petermann L, Mason-Lei P, Vandall-Walker V. Patient engagement in Canada: a scoping review of the ‘how’ and ‘what’ of patient engagement in health research. Health Res Policy Syst. 2018;16:5.
Walker L, Avant K. Strategies for theory construction in nursing. 4th ed. Upper Saddle River: Pearson Prentice Hall; 2005.
Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Heal Prof. 2006;26(1):13–24.
Rycroft-Malone J. Theory and knowledge translation: setting some coordinates. Nurs Res. 2007;56(4 Suppl):S78–85.
Grant MJ, Booth A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.
Patient Centered Outcomes Research Institute (PCORI). Engagement in health research literature explorer: patient centered research outcomes research institute; 2018 Available from: https://www.pcori.org/literature/engagement-literature.
Patient Centred Outcomes Research Institute (PCORI). Engagement in Health Research Literature Supplemental Methods Information n.d. Available from: https://www.pcori.org/literature/engagement-literature. Accessed 23 Nov 2018.
Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15.
Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.
Oliver SR, Rees RW, Clarke-Jones L, Milne R, Oakley AR, Gabbay J, et al. A multidimensional conceptual framework for analysing public involvement in health services research. Health Expect. 2008;11(1):72–84.
Abma TA, Broerse JE. Patient participation as dialogue: setting research agendas. Health Expect. 2010;13(2):160–73.
James S, Arniella G, Bickell NA, Walker W, Robinson V, Taylor B, et al. Community ACTION boards: an innovative model for effective community-academic research partnerships. Prog Community Health Partnersh. 2011;5(4):399–404.
Johnson DS, Bush MT, Brandzel S, Wernli KJ. The patient voice in research-evolution of a role. Res Involv Engagem. 2016;2:6.
Jinks C, Carter P, Rhodes C, Taylor R, Beech R, Dziedzic K, et al. Patient and public involvement in primary care research - an example of ensuring its sustainability. Res Involv Engagem. 2016;2:1.
Dickert N, Sugarman J. Ethical goals of community consultation in research. Am J Public Health. 2005;95(7):1123–7.
Hunt JB, Curran G, Kramer T, Mouden S, Ward-Jones S, Owen R, et al. Partnership for implementation of evidence-based mental health practices in rural federally qualified health centers: theory and methods. Prog Community Health Partnersh. 2012;6(3):389–98.
Research. NIfH. Going the Extra Mile: improving the nation’s health and wellbeing through public involvement in research. The final report and recommendations to the Director General Research and Development/Chief Medical Officer Department of Health of the ‘Breaking Boundaries’ strategic review of public involvement in the National Institute for Health Research.: INVOLVE, NIHR; 2015.
Graham PW, Kim MM, Clinton-Sherrod AM, Yaros A, Richmond AN, Jackson M, et al. What is the role of culture, diversity, and community engagement in transdisciplinary translational science? Transl Behav Med. 2016;6(1):115–24.
Jull J, Giles A, Boyer Y, Stacey D. Minwaashin Lodge - The Aboriginal Women's Support Centre. Building and enacting ethical research processes: the example of culturally adapting a decision making strategy. ACME Int J Critical Geographies. 2016; (pre-print). Available from: https://acme-journal.org/index.php/acme/article/view/1317. Accessed 16 Sept 2018.
Fagan MB, Morrison CR, Wong C, Carnie MB, Gabbai-Saldate P. Implementing a pragmatic framework for authentic patient-researcher partnerships in clinical research. J Comp Eff Res. 2016;5(3):297–308.
Sauers-Ford HS, Simmons JM, Shah SS, Team HOS. Strategies to engage stakeholders in research to improve acute care delivery. J Hosp Med. 2016;11(2):123–5.
Shelef DQ, Rand C, Streisand R, Horn IB, Yadav K, Stewart L, et al. Using stakeholder engagement to develop a patient-centered pediatric asthma intervention. J Allergy Clin Immunol. 2016;138(6):1512–7.
Woolf SH, Purnell JQ, Simon SM, Zimmerman EB, Camberos GJ, Haley A, et al. Translating evidence into population health improvement: strategies and barriers. Annu Rev Public Health. 2015;36:463–82.
Deverka PA, Lavallee DC, Desai PJ, Esmail LC, Ramsey SD, Veenstra DL, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J Comp Eff Res. 2012;1(2):181–94.
Jones L, Wells K. Strategies for academic and clinician engagement in community-participatory partnered research. JAMA. 2007;297(4):407–10.
Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18(5):1151–66.
Savory C. Patient and public involvement in translative healthcare research. Clin Gov Int J. 2010;15(3):191–9.
Fergusson D, Monfaredi Z, Pussegoda K, Garritty C, Lyddiatt A, Shea B, et al. The prevalence of patient engagement in published trials: a systematic review. Res Involv Engagem. 2018;4:17.
Boaz A, Hanney S, Borst R, O'Shea A, Kok M. How to engage stakeholders in research: design principles to support improvement. Health Res Policy Syst. 2018;16(1):60.
Greenhalgh T, Hinton L, Finalay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect. 2019;22(4):785-801. https://doi.org/10.1111/hex.12888.
Boivin A, L'Espérance A, Gauvin FP, Dumez V, Macaulay AC, Lehoux P, et al. Patient and public engagement in research and health system decision making: a systematic review of evaluation tools. Health Expect. 2018;21(6):1075–84.
Frank L, Forsythe L, Ellis L, Schrandt S, Sheridan S, Gerson J, et al. Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research institute. Qual Life Res. 2015;24(5):1033–41.
Forsythe L, Carman K, Fayish L, Davidson L, Hickam D, Hall C, et al. Patient engagement in research: early findings from the patient-centered outcomes research institute. Health Aff. 2019;38(3):359–67.
Jull J, Graham ID, Kristjansson E, Moher D, Petkovic J, Yoganathan M, et al. Taking an integrated knowledge translation approach in research to develop the CONSORT-equity 2017 reporting guideline: an observational study. BMJ Open. 2019;9(7):e026866.
We thank Samantha Repka for her partnership and important contributions.
The views presented in this article are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors, or its Methodology Committee.
This paper is part of the integrated knowledge translation (IKT) concept paper series. The IKTRN brings together knowledge users and researchers to advance the science and practice of Integrated KT and train the next generation of Integrated KT researchers. Open access fees to publish the paper were covered by the IKTRN. The IKTRN is funded by a Canadian Institutes of Health Research Foundation Grant that is held by IDG (FDN #143247). TN holds a Postdoctoral Fellowship from the Canadian Institutes of Health Research (2016–2019). JJ held a Canadian Institutes of Health Research (CIHR) Postdoctoral Fellowship while conducting a portion of the work.
Ethics approval and consent to participate
Not applicable. As the study is a literature review of publically available information, it did not require ethical review prior to its conduct (Panel on Research Ethics, 2018 http://pre.ethics.gc.ca/eng/tcps2-eptc2_2018_chapter2-chapitre2.html).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Jull, J.E., Davidson, L., Dungan, R. et al. A review and synthesis of frameworks for engagement in health research to identify concepts of knowledge user engagement. BMC Med Res Methodol 19, 211 (2019). https://doi.org/10.1186/s12874-019-0838-1
- Health system
- Integrated knowledge translation