Table 3 Suggested approach to organize, synthesize, validate and make sense of complex findings

1. Selection of analysis method

• Which method is the most appropriate to answer research questions?

• We searched the literature for various synthesis methods of complex evidence

• Potentially more valid if the method matches the question

• There was no single synthesis method that best fit our questions

• Need to adopt a flexible approach to match appropriate methods to answer research questions

• Consider selecting a primary analysis method supplemented by other or modified methods to address all questions

2. Organization and analysis of data

• How will the data be organized?

• We sorted and organized our data (1736 guideline attributes) in an Excel database

• Sorting of concepts and themes on multiple levels (e.g., across attributes, categories, disciplines)

• Difficult to keep track of changes from multiple reviewers

• We used a modified duplicate review process that involved a group of second reviewers “auditing” the analysis of primary reviewers

• Ensure that document tracking is transparent and efficient (e.g., track and document changes and include detailed notes from all reviewers)

• Duplicate review is time consuming and resource intensive

• Analysis process was done in duplicate

• Duplicate analysis minimizes bias

• Will also depend on selected analysis method

3. Validity measures

• How are you going to verify findings and minimize bias?

• Sought expert consensus on findings using survey methodology

• Survey methodology is quick and efficient

• Survey methodology has inherent biases

• Depending on resources, other consensus methods may increase validity such as the Delphi method

• Transparency (i.e., document what was planned, what was done and why)

4. Representation of data

• How will the results and data be used?

• We developed a conceptual map of guideline implementability for guideline developers and end-users

• The conceptual map contributes to the understanding of guideline implementability

• There may be other factors not captured in the map that may influence guideline implementability

• The conceptual framework needs to be refined according to the codebook of definitions

• The conceptual framework needs to be rigorously evaluated to determine the feasibility of its use by guideline developers, and its potential to influence guideline uptake by family physicians

• The process advances the knowledge about analysis methods for complex evidence

• Who are the target knowledge end users?

5. Dissemination of data

• To what extent should the data be disseminated?

• The map will inform a guideline implementability framework for guideline developers, users and policy makers

• The framework will inform end-users about attributes that facilitate guideline uptake; and may also inform policy around guideline development

• There may be other factors influencing guideline implementability

• Prior to dissemination, the framework will need to undergo rigorous evaluation (including quantitative and qualitative studies) to test its potential to influence guideline uptake by family physicians who are the primary end-users of clinical practice guidelines

• Will the work inform practice, system, policy?