Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

A review and synthesis of frameworks for engagement in health research to identify concepts of knowledge user engagement

BMC Medical Research Methodology volume 19, Article number: 211 (2019) Cite this article

Abstract

Background

Engaging those who influence, administer and/or who are active users (“knowledge users”) of health care systems, as co-producers of health research, can help to ensure that research products will better address real world needs. Our aim was to identify and review frameworks of knowledge user engagement in health research in a systematic manner, and to describe the concepts comprising these frameworks.

Methods

An international team sharing a common interest in knowledge user engagement in health research used a consensus-building process to: 1) agree upon criteria to identify articles, 2) screen articles to identify existing frameworks, 3) extract, analyze data, and 4) synthesize and report the concepts of knowledge user engagement described in health research frameworks. We utilized the Patient Centered Outcomes Research Institute Engagement in Health Research Literature Explorer (PCORI Explorer) as a source of articles related to engagement in health research. The search includes articles from May 1995 to December 2017.

Results

We identified 54 articles about frameworks for knowledge user engagement in health research and report on 15 concepts. The average number of concepts reported in the 54 articles is n = 7, and ranges from n = 1 to n = 13 concepts. The most commonly reported concepts are: knowledge user - prepare, support (n = 44), relational process (n = 39), research agenda (n = 38). The least commonly reported concepts are: methodology (n = 8), methods (n = 10) and analysis (n = 18). In a comparison of articles that report how research was done (n = 26) versus how research should be done (n = 28), articles about how research was done report concepts more often and have a higher average number of concepts (n = 8 of 15) in comparison to articles about how research should be done (n = 6 of 15). The exception is the concept “evaluate” and that is more often reported in articles that describe how research should be done.

Conclusions

We propose that research teams 1) consider engagement with the 15 concepts as fluid, and 2) consider a form of partnered negotiation that takes place through all phases of research to identify and use concepts appropriate to their team needs. There is a need for further work to understand concepts for knowledge user engagement.

Peer Review reports

Background

In health research, there is growing support for the engagement of patients, caregivers, clinicians, managers, decision makers and other healthcare systems users as co-producers of evidence. The participation of researchers with those who influence, administer and/or who are active users of healthcare systems create opportunities for processes of co-production of research evidence and that is a form of knowledge. Collaboration between researcher and health care systems users can take many forms, and is often referred to using different terminology, for example, integrated knowledge translation [1], collaborative research, co-production of knowledge [2], and engaged scholarship [3]. In all cases, however, there is a common aim: to produce and apply knowledge to address real-world needs. The resulting knowledge is more likely to be useful and useable and therefore more likely to be applied in policy and practice [3, 4]. We define "engagement" in research as an arrangement in the governance of the research process with those who influence, administer and/or who are active users of healthcare systems, and that leads to co-production of knowledge (beyond being a research participant) [5, 6].

Engagement in research disrupts the notion of research as the sole domain of academic researchers. The expansion of research roles invite many diverse forms of expertise into an involved and intentional process of knowledge production. We adopt and adapt the definitions developed by Parry et al. (2015) to describe the roles of those engaged in research that leads to knowledge production. The aim of engagement in research is to integrate the views and values of "knowledge users", meaning those who are actively involved in the knowledge production process of a study and who may benefit or be otherwise affected by the research [7]. We expand the definition of knowledge user, and include those who may not be members of the mainstream academic research team or community but who will contribute expertise to knowledge production processes and/or who will influence, administer or be an active user of the research results to support their decision-making. Knowledge users are often members of groups for which the research holds significance. These groups may act on (or may be affected by) the research. Knowledge users might include people who will receive care that is based on the research findings. They may occupy a range of positions, such as research funders, patients or members of the public, health system and policy decision-makers, health care providers, instructors or students at training institutions, et cetera. For example, the inclusion of patient-, healthcare delivery systems- and research funding agency-representation was identified as valuable in the process of selection and refining of pilot research and quality improvement projects in diabetes care [8]. Knowledge users are the recipients of research impacts and for this reason are the key partners and contributors of expertise in the co-production of research evidence; they are distinct from broader groups in which the research endeavour is situated.

The term "stakeholder" is used in many different ways: we define stakeholders to include those members of the groups, in which a research partnership of knowledge user(s) and researcher(s) is situated. Stakeholders are people or organizations who may be indirectly affected by research [9]. They may have an interest in the research and use of findings but are not anticipated to directly influence, administer and/or actively use the research results in their own decision-making. For example, in studies that investigate the primary care based management of frailty with older people members of the general population would be considered stakeholders, as those who are not anticipated to influence, administer or utilize the healthcare system for services related to frailty, [10]. Stakeholders represent individuals, groups or organizations in which a knowledge user-researcher team is situated (Fig. 1).

Fig. 1
figure 1

Defining interests in co-production of knowledge

Full size image

International work is underway to promote knowledge user engagement in health research. For example, the United States-based Patient Centred Outcomes Research Institute (PCORI) has identified the engagement of knowledge users throughout the research process to be more patient centred, useful, and trustworthy and ultimately to lead to greater use and uptake of research results by the patient and broader healthcare community [5]. Beyond identifying engagement as valuable, PCORI requires engagement in much of its funded research. PCORI works to shape the field and provides thought leadership and sets standards for engagement in research for others to reference as a model, and summarizes and synthesizes findings that relate to the field of engagement in health research – to create accessible information and opportunities to further the learning of researchers [11]. In Canada, there have been recent, similar initiatives, with the National Strategy for Patient-Oriented Research (SPOR) that aims to foster evidence-informed health care through innovation – to improve quality, accountability and accessibility [12]. Like PCORI, SPOR aims to integrate the views and values of patients into health policy and practice through research. There are many other examples of work underway to define patient and other stakeholder engagement in health research, and that also aim to promote knowledge user engagement in health research [13,14,15,16].

There is a need to conceptualize how to do health research in ways that support the engagement of those who influence, administer and/or who are active health care system users (knowledge users). That is, it is important to gain insight on the planning, implementation, and dissemination processes of research conducted from within partnerships of knowledge users and researchers to co-produce knowledge that addresses real-world needs [17]. Understanding the ways to support engagement of knowledge users in health research can lead to greater clarity in communication about and evaluation of health research conducted with knowledge user-researcher partnerships. Identification of the models and frameworks that organize concepts relevant to knowledge user engagement in health research is a valuable endeavor.

The organization of concepts in health research models and frameworks is a way to describe how to conduct research inclusive of knowledge users that leads to the co-production of knowledge. We use Walker and Avant’s (2005) definition of "concept" as a mental representation of a phenomenon, idea, action or thing, that represents observed occurrences [18]. Concepts categorize information into meaningful constructs in the form of models and frameworks to explain phenomena. There are many ways to define the terms “models” and “frameworks” and often these terms may be conceptually different and inconsistently applied. Both terms relate to the organization of concepts; for the purposes of the work presented in this paper, we use the term “frameworks” to refer to the actual, intentional organization of concepts.

Research practices that promote knowledge user engagement in health research should be guided by conceptual frameworks to focus on and aid implementation [19, 20]; currently there exists little consensus on the essential concepts of knowledge user engagement and guidance for the effective conduct of health research [14]. Our approach was to produce an evidence synthesis that combined the strengths of the comprehensive search process conducted by PCORI, with a process of critical review [21]. Our aim was to identify and then review frameworks of knowledge user engagement in health research in a systematic manner, and to describe the concepts comprising these frameworks.

Methods

An international team, consisting of United States- and Canadian-based members, formed a collaboration and are identified here as “the team”. The team consists of an information scientist (LD); policy and practice leads (RD, SR, KW); and researchers (JJ, TN, IDG). All members of the team shared a common interest in understanding best practices in knowledge user engagement in health research, and are either acknowledged or listed as co-authors on this paper. We used an integrated knowledge translation approach meaning that there was an interactive process of knowledge exchange among team members to produce knowledge more likely to be useful to health systems’ knowledge users [1]. The team engaged in a consensus-building process that consisted of a series of regular meetings and online communications to: 1) agree upon how to identify the concepts of knowledge user engagement in health research, 2) identify existing frameworks that capture concepts of knowledge user engagement in health research, and 3) identify, synthesize and report the concepts of knowledge user engagement described in health research frameworks.

Data source

PCORI developed the Engagement in Health Research Literature Explorer (“PCORI Explorer”) as a searchable list of articles related to engagement in research [22]. To develop the searchable list of articles, a PubMed/MEDLINE search strategy was developed and inclusion/exclusion criteria applied to confirm the final list of articles. Selected articles were then tagged across three engagement-related categories that include: Topic, Stakeholder Involvement, and Phases of Research Engagement [23]. For more information about the search strategy used to create this open-access resource, see: https://www.pcori.org/literature/engagement-literature. The PCORI Explorer is comprised of peer-reviewed articles that have been classified by AcademyHealth researchers into four article topic types: 1) Example of Engagement in Health Research, 2) Detailed Description of Engagement in Health Research, 3) Framework, Editorial, Commentary, and 4) Evidence for Engagement [22]. The search includes articles from May 1995 and for our review include those up to December 2017.

Given the combined the strengths of PCORI’s landscape review of engagement frameworks in the development of the PCORI Explorer to obtain and then classify articles related to engagement in research, we chose to focus on the article topic type Framework, Editorial, Commentary as the data source for our review. The Framework, Editorial, Commentary section includes “Manuscripts that express a theoretical view on engagement in health research, including scientific commentaries, opinion briefs, or conceptual pieces such as models or frameworks” [23].

Procedure

Our study procedure was as follows: 1) determine and agree upon a set of screening criteria for articles to identify frameworks that describe and/or depict concepts of knowledge user engagement in health research; 2) screen the articles within the PCORI Explorer article topic type Framework, Editorial, Commentary to identify those that report on frameworks of knowledge user engagement in health research; 3) use a predetermined approach to abstract and conduct a content analysis of information about frameworks of knowledge user engagement in health research (demographics, concepts of knowledge user engagement reported in the framework) from the identified health research frameworks, and 4) analyze and synthesize the abstracted data and disseminate results (Fig. 2).

Fig. 2
figure 2

Procedure

Full size image

Screening criteria to identify frameworks

First, the team agreed upon how to identify the concepts of knowledge user engagement in health research, and these definitions are reported earlier: “engagement”, “knowledge user”, “stakeholder”, “concept”, and “framework”. Next, the team determined a set of screening criteria to identify articles that report on frameworks depicting concepts of knowledge user engagement in health research. There were a series of meetings to test the screening criteria with example articles, and to refine and confirm the inclusion criteria. The team reached consensus for inclusion criteria and that include: 1) the article had to present what the author of the article referred to as a framework or model of knowledge user engagement in health research; or, 2) in the absence of self-labeling as framework or model, the article had to explicitly describe and/or depict (use of figure, diagram) the organization of concepts for knowledge user engagement in health research; and 3) the framework had to be about collaborative research involving a researcher and at least one other team member who is a knowledge user (i.e., patient, caregivers, family members, healthcare providers, managers, policy makers et cetera). For screening articles, we agreed to an operational version of the definition for knowledge user engagement in health research: as an intentional arrangement that creates opportunities for knowledge users to co-lead in the processes associated with planning, conducting or disseminating research that results in co-production of knowledge [5, 6]. When an article reported on a framework that is cited in another article, the original article was obtained and screened for inclusion.

Screening articles

Next, we screened the abstracts and full text of the 310 articles, sourced from the PCORI Explorer topic section Framework, Editorial, Commentary – up to December 2017. One team member (JJ) screened articles for inclusion, with duplicate screening done independently by a second team member (TN). A third reviewer was available for discussion for conflict between the reviewers (IDG), and there were opportunities with regular meetings and shared documents for process review by the entire team (JJ, LD, RD, TN, KW, SR, IDG). As well, during the screening process, there were team meetings to discuss articles and ensure the application of the screening criteria in an agreed-upon manner. During screening an additional seven articles (external to the PCORI Explorer) were identified because they had been reported as a source of frameworks in the reviewed articles; these additional articles were also obtained and screened (Fig. 3).

Fig. 3
figure 3

Article selection flow diagram

Full size image

Abstract and conduct a content analysis

Content analysis is a useful way to describe ideas and to generate or extend ways of considering organization of concepts [24]. With directed content analysis, pre-existing theory or research is used to develop the initial coding scheme [24]. During the analysis, additional codes can be developed and the initial coding scheme can be revised or refined; in this way, existing approaches to theory or research can be extended and/or refined [25]. We describe the use of a directed content analytic approach, and that involves the examination of the reported aims and concepts in articles that were identified to report the use of particular words and/or depict (with the use of figure, diagram) concepts for knowledge user engagement in health research [24, 25]:

  1. 1)

    Preparation, in which we decided upon the unit of analysis and the conduct of a review of each article for data. For our review, we agreed to use an iterative approach to collect data on pre-defined concepts of knowledge user engagement in health research and to inductively expand on and refine these concepts. We agreed upon and used the definition for co-production of knowledge [1]. Each time a new concept relating to knowledge user engagement was identified, data was extracted and previously identified articles reviewed for presence of the newly identified concepts;

  2. 2)

    Analysis of each article to identify and code data by a first reviewer (JJ) and with verification of a second reviewer (TN or IDG); the entire team (JJ, LD, RD, TN, KW, SR, IDG) was engaged in regular and ongoing discussion and review. For each article there was demographic data extracted. As well, we started with a core set of concepts based on an agreed-upon definition for co-production of knowledge to extract knowledge-user engagement concepts. For each framework we sought evidence of concepts that relate to plans for or actual conduct of knowledge user engagement that describe one or more of the following: i) development of research questions, ii) development of methodology, iii) data collection and/or tool development, iv) interpretation of study findings, v) crafting messages around the results and moving the results into practice [1]. During the data abstraction we refined definitions and identified additional concepts that related to engagement; and,

  3. 3)

    Reporting, to describe the analysis and findings. For our review, the aim was to identify and review frameworks of knowledge user engagement in health research in a systematic manner, and to describe the concepts comprising these frameworks. Following discussion, the entire team agreed upon final findings.

Results

We identified 54 articles about frameworks for knowledge user engagement in health research and report on 15 concepts. A full list of the 54 articles is included (Additional file 1).

Demographic data

The included articles depicting frameworks were published between 2000 and 2017, with none published prior to 2000 and with the majority released in 2016 (n = 14, 26% of the articles). The first authors of the articles were reported to be located across eight countries (Netherlands, Switzerland, Spain, Canada, Australia, United Kingdom, United States) with the highest number of first authors located in the United States (n = 29, 50%). Knowledge users were explicitly identified in the articles as coauthors (n = 8, 15%) acknowledged on the publication (n = 24, 44%) or not identified on the publication (n = 30, 55%). The number (range) of knowledge users who were reported as participating with researchers ranged from none [26] to well-over over 200 [27].

In the articles, there were many and different combinations of individual and group terms used to describe/represent knowledge users in the articles such as: patients, members of the public, stakeholders, service users, consumers, expert participants, survivors, clinicians/healthcare providers, peer support workers, mentees, decision and policy makers, et cetera. As well, terms for members of group and institutional teams were used, such as: institutional review boards, sponsors, community (includes different versions, such as rural community, community stakeholders et cetera), healthcare team (includes terms such as palliative care team), community based organizations, et cetera. The range of descriptions for knowledge users indicates a breadth of consideration for who to consider in research partnerships and how they are labeled.

Concepts for knowledge user engagement

During the content analysis of frameworks, we identified 15 concepts for knowledge user engagement reported across the 54 articles. We report these concepts with a definition and an example (Table 1). The 15 concepts are reported across four general research phases: Prepare (as a precursor to the study), Plan (the design of the study), Conduct (the study), Apply (findings of the study) and do not mean to imply an order or denote the importance of the concepts relative to one another. For example, the concept of “ethics - principles values” was most often reported in articles when describing preparation for research; however, the concept is also evident throughout other research phases in multiple articles. The average number of concepts reported in the 54 articles is n = 7, and range from n = 1 to n = 13 concepts. The three most commonly reported concepts include: knowledge user - prepare, support (n = 44), relational process (n = 39), research agenda (n = 38). The three least commonly reported concepts include: methodology (n = 8), methods (n = 10) and analysis (n = 18) (Fig. 4).

Table 1 Concepts for knowledge user engagement in health research
Full size table
Fig. 4
figure 4

Reporting of concepts

Full size image

Further, articles can be classified in one of two categories as they either report on frameworks that describe how research was done or report on frameworks that are the result of considering how research should be done. That is, the articles were found to either 1) describe how research was done and to have reported on what had occurred in a study about knowledge user engagement in health research (n = 26); or, 2) describe how research should be done to foster knowledge user engagement in health research, that is, to report on a literature review, expert opinion, analysis of theory about knowledge user engagement in health research et cetera (versus actual health research conduct) (n = 28). The articles are organized into two framework categories and with reporting across concepts: articles describing how research was done to foster knowledge user engagement in health research (Fig. 5), and; articles describing how research should be done to foster knowledge user engagement in health research (Fig. 6). We discuss the similarities and differences between the two groups of articles.

Fig. 5
figure 5

Articles describing how research was done to foster knowledge user engagement in health research: reporting across concepts (n=26)

Full size image
Fig. 6
figure 6

Articles describing how research should be done to foster knowledge user engagement in health research: reporting across concepts (n=28)

Full size image

A comparison of frameworks that describe “How research was done” with “How research should be done”

In a comparison of the articles that report on frameworks that describe how research was done with how research should be done, those that report how research was done have a higher average number of concepts (n = 8) in comparison to articles that reported on how research should be done (n = 6). For frameworks that describe how research was done, the range of concepts reported was between n = 13 and n = 3: an article that describes collaborative research with an Indigenous community (n = 13) [35] and an article that describes priority setting with patients (n = 3) [8]. For frameworks that describe how research should be done (n = 28) the range of concepts reported was between n = 13 and n = 1: an article that reports on a systematic review and environmental scan of the literature about patient and service user engagement in biomedical and health services research (n = 13) [42] and articles reporting on patient and public involvement in translational science (n = 1) [43], and on the advantages and disadvantages of public involvement in health services research agenda setting (n = 1) [26].

Overall, the articles that report on frameworks that describe how research was done reported concepts more often. One exception was the “evaluate” concept that was reported less often in frameworks that describe how research was done (38%) relative to frameworks that describe how research should be done (57%). The greatest differences in frequency of concept reporting was found for the concept of “resources” (how research was done = 58% versus how research should be done =18%) and the smallest difference was found for the concept “sustain” (how research was done = 38% versus how research should be done = 36%) (Table 2).

Table 2 Frequency of concept reporting for articles that report how research was done in relation to how research should be done
Full size table

Discussion

The aim of our work was to identify and review frameworks of knowledge user engagement in health research in a systematic manner, and to describe the concepts comprising these frameworks. To do this work, we identified, reviewed and synthesized 54 articles that reported on frameworks meeting our criteria for describing and/or depicting knowledge user engagement in health research. Our analysis identified 15 concepts related to knowledge user engagement in health research. The variation in the reported concepts between the frameworks means that it is difficult to conclude what and in which order might be best for knowledge user engagement in health research. Research teams should think about engagement with the concepts as being fluid rather than strictly required, as there is a lack of evidence at this time for the benefits of engaging knowledge users across each of the 15 identified concepts.

Understanding the concepts of knowledge user engagement in health research may help researchers and knowledge user partners to identify and operationalize engagement in health research, and in ways that account for the nuanced characteristics and preferences of particular group(s) of people. For knowledge users there are potentially many opportunities and ways to engage or be engaged in areas of research that may typically be the domain of researchers alone. There is a need to understand best practice in research engagement [44]: encouraging dialogue and debate between knowledge users and researchers about the concepts of knowledge user engagement in health research on a situational basis may contribute to processes of engagement in health research.

We propose that it is reasonable to suggest that knowledge users and researchers discuss and negotiate the research process to determine who should be meaningfully engaged and in what ways. It is important to maximize the potential to advance research project objectives to reflect the needs and aims of knowledge user partners to ensure that the outcomes are useful and impactful. As well, we propose that a form of partnered negotiation takes place at the start and throughout the research study. A process of partnered negotiation is important for considering and possibly utilizing concepts throughout the phases of research, in order to foster successful knowledge user engagement. We depict these relationships in a met-framework (Fig. 7).

Fig. 7
figure 7

Meta-framework, knowledge user engagement in collaborative research

Full size image

In our work, we have identified the need to think about knowledge user engagement in health research as an arrangement that creates ongoing opportunities for engagement by knowledge users throughout the research process. While the promotion of full engagement of knowledge users throughout the research process is described in the literature as an ideal [5, 6], dictating when and how knowledge users and researchers need to be involved is not helpful nor is it realistic. The equal involvement of knowledge users and researchers throughout the research process may vary and a focus on equity may instead be more realistic for knowledge user engagement. That is, while the contributions by knowledge users and researchers may differ or vary at times, the effort and impact is similar – sharing equally is not as important as considering the quality of engagement in research processes. In our work, studies were found to report on many types of individual and collective knowledge users, and we propose that engagement with knowledge users is a form of partnered negotiation. Every research partnership is unique, as it consists of individuals (knowledge users and researchers) who, themselves, understand their unique expertise, needs and context.

The use of frameworks can guide the exploration of how to operationalize measures of impact of knowledge user engagement. The findings of our study are reinforced by the literature; there are a broad range of ways in which knowledge users might be engaged in health research and little evidence for the impact of this engagement [45]. As well, there has been an important - and dominant - focus on patient and lay involvement in health research [46]. Our work shows that engagement of knowledge users in health research is more complex than bringing in one form of knowledge user: the pursuit of knowledge user engagement in health research means that there are teams consisting of researchers and knowledge users that span health care systems. The evaluation of knowledge user engagement in health research is identified as an area in need of development [17, 44,45,46,47]. To build understandings and unravel the complexity of knowledge user engagement in health research, it is imperative that research studies utilize guiding frameworks. For example, PCORI (a major research funder in the United States) encourages that those conducting funded projects evaluate the engagement of knowledge users (defined as patients and other stakeholders, such as clinicians, purchasers, payers et cetera) with the PCORI Engagement Rubric [5, 48, 49]. The aim of the evaluation is to develop knowledge about engagement practices with knowledge users. Our finding of lower reporting of the “evaluate” concept suggests that there may be some barriers to the actual conduct of evaluation in frameworks of knowledge user engagement in health research. To advance the field of knowledge user engagement in health research, future efforts should focus on evaluation.

In addition to the evaluation of engagement practices with knowledge users, we propose that studies need to go beyond reporting on the framework used for knowledge user engagement in health research, and present analysis about the use of frameworks that guide research. Many studies report on the use of frameworks to guide knowledge user engagement in health research; however, there are few examples of studies that systematically evaluate these guiding frameworks for their use in guiding teams consisting of knowledge users and researchers [11, 50]. Studies that evaluate the guiding frameworks for knowledge user engagement in health research are important for understanding how to engage people at different points in the study, and in what ways. Here we have reported on a range of included studies that show the complexity and variability of knowledge user engagement, across research stages and concepts. To advance the field of knowledge user engagement in health research, it is imperative that there is evaluation of the frameworks guiding knowledge user engagement.

Limitations and strengths

A key limitation of our study is that we may have included or excluded articles inappropriately due to the conceptually different and inconsistently used range of terms and descriptions for models and frameworks. For the purposes of this review, and in an attempt to mitigate issues with terminology, we agreed upon an operational definition to aid in the identification of models and frameworks. In addition, our study used a source of articles designed to capture the general literature about engagement in health research, rather than a search designed and conducted specifically for our purposes.

Strengths of our review include the use of the PCORI Explorer to identify articles, and the involvement of an international team with the use of a consensus-building process. The PCORI Explorer is a repository of peer-reviewed articles focused on guidance in the field of collaborative research conduct and that includes frameworks, conceptual models, and editorials to express a theoretical view on engagement in health research. Those who oversee the PCORI Explorer repository also included articles that are recommended by PCORI staff, or journals identified as highly relevant despite not yet being indexed in PubMed [23]. Additionally, an international team with experience in conceptualizing the engagement of knowledge users in collaborative research across a range of settings included those with expertise in the use of the PCORI Explorer (LA, RD, SR, KW) and use and/or development of knowledge translation frameworks (JJ, TN, IDJ).

Conclusions

The aim of our work was to identify and review frameworks of knowledge user engagement in health research in a systematic manner, and to describe the concepts comprising these frameworks. We identify 15 concepts of knowledge user engagement in health research, and suggest that to advance the field of knowledge user engagement in health research there is a need to focus on evaluation. Given the variation in the number of concepts reported across the included frameworks it is difficult to say what and in which order concepts are best for collaborative research. Further work to develop understandings of knowledge user engagement in health research is encouraged.

Availability of data and materials

Supporting data may be found at the PCORI Explorer https://www.pcori.org/literature/engagement-literature repository, and the datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Graham ID, Tetroe J. Knowledge translation research group. Some theoretical underpinnings of knowledge translation. Acad Emerg Med. 2007;14(11):936–41.

    Article  PubMed  Google Scholar 

  2. Tetroe J. Knowledge Translation at the Canadian Institutes of Health Research: a primer. Focus Technical Brief. 2007;18:1-8.

  3. Van de Ven D, Johnson P. Knowledge for theory and practice. Acad Manag Rev. 2006;31(4):802–21.

    Article  Google Scholar 

  4. Bowen S, Graham I. Integrated knowledge translation. In: Straus SE, Tetroe J, Graham ID, editors. Knowledge translation in healthcare: moving evidence to practice. West Sussex: Wiley; 2013. p. 14–23.

    Chapter  Google Scholar 

  5. Sheridan S, Schrandt S, Forsythe L, Hilliard TS, Paez KA. Advisory panel on patient engagement (inaugurel panel 2013) . The PCORI engagement rubric: promising practices for partnering in research. Ann Fam Med. 2017;15(2):165–70.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Canadian Institutes of Health Research (CIHR). Patient engagement: What is patient engagement? Government of Canada 2018 Available from: http://www.cihr-irsc.gc.ca/e/45851.html.

    Google Scholar 

  7. Parry D, Salsberg J, Macauley AC. Guide to researcher and knowledge-user collaboration in health research 2015. Available from: http://www.cihr-irsc.gc.ca/e/44954.html#s2.

    Google Scholar 

  8. Schmittdiel JA, Desai J, Schroeder EB, Paolino AR, Nichols GA, Lawrence JM, et al. Methods for engaging stakeholders in comparative effectiveness research: a patient-centered approach to improving diabetes care. Healthc (Amst). 2015;3(2):80–8.

    Article  Google Scholar 

  9. Center for Commununity Health and Development at the University of Kansas. Community Toolbox. Section 8: Identifying and analyzing stakeholders and their interests. 2016 Available from: https://ctb.ku.edu/en/table-of-contents/participation/encouraging-involvement/identify-stakeholders/main.

    Google Scholar 

  10. Heaven A, Brown L, Foster M, Clegg A. Keeping it credible in cohort multiple randomised controlled trials: the community ageing research 75+ (CARE 75+) study model of patient and public involvement and engagement. Res Involv Engagem. 2016;2:30.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Forsythe L, Heckert A, Margolis MK, Schrandt S, Frank L. Methods and impact of engagement in research, from theory to practice and back again: early findings from the Patient-Centered outcomes research institute. Qual Life Res. 2018;27:17–31.

    Article  PubMed  Google Scholar 

  12. Canadian Institutes of Health Research (CIHR). Strategy for Patient-Oriented Research 2018 Available from: http://www.cihr-irsc.gc.ca/e/41204.html.

    Google Scholar 

  13. n.a. PaCER - Patient and Community Engagement Research. Welcome to PaCER: PaCER Innovates; 2018. Available from: https://pacerinnovates.ca. Accessed 27 Jan 2019.

  14. Graham ID, Kothari A, McCutcheon C, on behalf of the Integrated Knowledge Translation Research Network Program Leads. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation. Implement Sci. 2018;13(1):22.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Hayes H, Buckland S, Tarpey M. INVOLVE briefing notes for researchers: involving the public in NHS. Eastleigh: INVOLVE: Public Health and Social Care Research; 2012.

    Google Scholar 

  16. Australian Government National Health and Medical Research Council. Partnership Projects: Commonwealth of Australia 2018; 2018 Available from: https://nhmrc.gov.au/funding/find-funding/partnership-projects.

    Google Scholar 

  17. Manafo E, Petermann L, Mason-Lei P, Vandall-Walker V. Patient engagement in Canada: a scoping review of the ‘how’ and ‘what’ of patient engagement in health research. Health Res Policy Syst. 2018;16:5.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Walker L, Avant K. Strategies for theory construction in nursing. 4th ed. Upper Saddle River: Pearson Prentice Hall; 2005.

    Google Scholar 

  19. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Heal Prof. 2006;26(1):13–24.

    Article  Google Scholar 

  20. Rycroft-Malone J. Theory and knowledge translation: setting some coordinates. Nurs Res. 2007;56(4 Suppl):S78–85.

    Article  PubMed  Google Scholar 

  21. Grant MJ, Booth A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

    Article  Google Scholar 

  22. Patient Centered Outcomes Research Institute (PCORI). Engagement in health research literature explorer: patient centered research outcomes research institute; 2018 Available from: https://www.pcori.org/literature/engagement-literature.

    Google Scholar 

  23. Patient Centred Outcomes Research Institute (PCORI). Engagement in Health Research Literature Supplemental Methods Information n.d. Available from: https://www.pcori.org/literature/engagement-literature. Accessed 23 Nov 2018.

  24. Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15.

    Article  PubMed  Google Scholar 

  25. Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

    Article  PubMed  Google Scholar 

  26. Oliver SR, Rees RW, Clarke-Jones L, Milne R, Oakley AR, Gabbay J, et al. A multidimensional conceptual framework for analysing public involvement in health services research. Health Expect. 2008;11(1):72–84.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Abma TA, Broerse JE. Patient participation as dialogue: setting research agendas. Health Expect. 2010;13(2):160–73.

    Article  PubMed  PubMed Central  Google Scholar 

  28. James S, Arniella G, Bickell NA, Walker W, Robinson V, Taylor B, et al. Community ACTION boards: an innovative model for effective community-academic research partnerships. Prog Community Health Partnersh. 2011;5(4):399–404.

    PubMed  PubMed Central  Google Scholar 

  29. Johnson DS, Bush MT, Brandzel S, Wernli KJ. The patient voice in research-evolution of a role. Res Involv Engagem. 2016;2:6.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Jinks C, Carter P, Rhodes C, Taylor R, Beech R, Dziedzic K, et al. Patient and public involvement in primary care research - an example of ensuring its sustainability. Res Involv Engagem. 2016;2:1.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Dickert N, Sugarman J. Ethical goals of community consultation in research. Am J Public Health. 2005;95(7):1123–7.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Hunt JB, Curran G, Kramer T, Mouden S, Ward-Jones S, Owen R, et al. Partnership for implementation of evidence-based mental health practices in rural federally qualified health centers: theory and methods. Prog Community Health Partnersh. 2012;6(3):389–98.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Research. NIfH. Going the Extra Mile: improving the nation’s health and wellbeing through public involvement in research. The final report and recommendations to the Director General Research and Development/Chief Medical Officer Department of Health of the ‘Breaking Boundaries’ strategic review of public involvement in the National Institute for Health Research.: INVOLVE, NIHR; 2015.

  34. Graham PW, Kim MM, Clinton-Sherrod AM, Yaros A, Richmond AN, Jackson M, et al. What is the role of culture, diversity, and community engagement in transdisciplinary translational science? Transl Behav Med. 2016;6(1):115–24.

    Article  PubMed  Google Scholar 

  35. Jull J, Giles A, Boyer Y, Stacey D. Minwaashin Lodge - The Aboriginal Women's Support Centre. Building and enacting ethical research processes: the example of culturally adapting a decision making strategy. ACME Int J Critical Geographies. 2016; (pre-print). Available from: https://acme-journal.org/index.php/acme/article/view/1317. Accessed 16 Sept 2018.

  36. Fagan MB, Morrison CR, Wong C, Carnie MB, Gabbai-Saldate P. Implementing a pragmatic framework for authentic patient-researcher partnerships in clinical research. J Comp Eff Res. 2016;5(3):297–308.

    Article  PubMed  Google Scholar 

  37. Sauers-Ford HS, Simmons JM, Shah SS, Team HOS. Strategies to engage stakeholders in research to improve acute care delivery. J Hosp Med. 2016;11(2):123–5.

    Article  PubMed  Google Scholar 

  38. Shelef DQ, Rand C, Streisand R, Horn IB, Yadav K, Stewart L, et al. Using stakeholder engagement to develop a patient-centered pediatric asthma intervention. J Allergy Clin Immunol. 2016;138(6):1512–7.

    Article  PubMed  Google Scholar 

  39. Woolf SH, Purnell JQ, Simon SM, Zimmerman EB, Camberos GJ, Haley A, et al. Translating evidence into population health improvement: strategies and barriers. Annu Rev Public Health. 2015;36:463–82.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Deverka PA, Lavallee DC, Desai PJ, Esmail LC, Ramsey SD, Veenstra DL, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J Comp Eff Res. 2012;1(2):181–94.

    Article  PubMed  Google Scholar 

  41. Jones L, Wells K. Strategies for academic and clinician engagement in community-participatory partnered research. JAMA. 2007;297(4):407–10.

    Article  CAS  PubMed  Google Scholar 

  42. Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18(5):1151–66.

    Article  PubMed  Google Scholar 

  43. Savory C. Patient and public involvement in translative healthcare research. Clin Gov Int J. 2010;15(3):191–9.

    Article  Google Scholar 

  44. Fergusson D, Monfaredi Z, Pussegoda K, Garritty C, Lyddiatt A, Shea B, et al. The prevalence of patient engagement in published trials: a systematic review. Res Involv Engagem. 2018;4:17.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Boaz A, Hanney S, Borst R, O'Shea A, Kok M. How to engage stakeholders in research: design principles to support improvement. Health Res Policy Syst. 2018;16(1):60.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Greenhalgh T, Hinton L, Finalay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect. 2019;22(4):785-801. https://doi.org/10.1111/hex.12888.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Boivin A, L'Espérance A, Gauvin FP, Dumez V, Macaulay AC, Lehoux P, et al. Patient and public engagement in research and health system decision making: a systematic review of evaluation tools. Health Expect. 2018;21(6):1075–84.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Frank L, Forsythe L, Ellis L, Schrandt S, Sheridan S, Gerson J, et al. Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research institute. Qual Life Res. 2015;24(5):1033–41.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Forsythe L, Carman K, Fayish L, Davidson L, Hickam D, Hall C, et al. Patient engagement in research: early findings from the patient-centered outcomes research institute. Health Aff. 2019;38(3):359–67.

    Article  Google Scholar 

  50. Jull J, Graham ID, Kristjansson E, Moher D, Petkovic J, Yoganathan M, et al. Taking an integrated knowledge translation approach in research to develop the CONSORT-equity 2017 reporting guideline: an observational study. BMJ Open. 2019;9(7):e026866.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank Samantha Repka for her partnership and important contributions.

Disclaimer

The views presented in this article are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors, or its Methodology Committee.

Funding

This paper is part of the integrated knowledge translation (IKT) concept paper series. The IKTRN brings together knowledge users and researchers to advance the science and practice of Integrated KT and train the next generation of Integrated KT researchers. Open access fees to publish the paper were covered by the IKTRN. The IKTRN is funded by a Canadian Institutes of Health Research Foundation Grant that is held by IDG (FDN #143247). TN holds a Postdoctoral Fellowship from the Canadian Institutes of Health Research (2016–2019). JJ held a Canadian Institutes of Health Research (CIHR) Postdoctoral Fellowship while conducting a portion of the work.

Author information

Authors and Affiliations

  1. School of Rehabilitation Therapy, Faculty of Health Sciences, Queen’s University, 31 George Street, Louise D. Acton Building, Kingston, Ontario, Canada

    Janet E. Jull

  2. Ottawa Hospital Research Institute, 501 Smyth Road, Ottawa, Ontario, Canada

    Janet E. Jull

  3. Patient-Centered Outcomes Research Institute (PCORI), 1828 L Street, NW, Washington D.C., 20008, USA

    Laurie Davidson & Rachel Dungan

  4. Clinical Epidemiology Program, Ottawa Hospital Research Institute, 501 Smyth Road, Ottawa, Ontario, Canada

    Tram Nguyen & Ian D. Graham

  5. CanChild Centre for Childhood Disability Research, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada

    Tram Nguyen

  6. Public and Patient Engagement Department, Patient-Centered Outcomes Research Institute (PCORI), 1828 L Street, NW, Washington D.C., 20008, USA

    Krista P. Woodward

  7. Department of Epidemiology and Community Medicine, University of Ottawa, Ottawa, Ontario, Canada

    Ian D. Graham

Authors
  1. Janet E. Jull
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laurie Davidson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rachel Dungan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tram Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Krista P. Woodward
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ian D. Graham
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JJ and IDG conceptualized the study with input from LD, RD, TN, KW. JJ, TN and IDG developed and piloted the data collection and analysis methods, helped by LD, RD, KW. JJ and TN conducted the data analysis. JJ wrote the paper with input from LD, RD, TN, KW, IDG. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Janet E. Jull.

Ethics declarations

Ethics approval and consent to participate

Not applicable. As the study is a literature review of publically available information, it did not require ethical review prior to its conduct (Panel on Research Ethics, 2018 http://pre.ethics.gc.ca/eng/tcps2-eptc2_2018_chapter2-chapitre2.html).

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

List of included articles.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jull, J.E., Davidson, L., Dungan, R. et al. A review and synthesis of frameworks for engagement in health research to identify concepts of knowledge user engagement. BMC Med Res Methodol 19, 211 (2019). https://doi.org/10.1186/s12874-019-0838-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12874-019-0838-1

Keywords

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us