Table 3 Recommendations regarding the methods of choosing or identifying clinical covariates for investigation and interpretation of the findings

When to identify covariates in the review process

A priori (e.g., in protocol)

17

76, 92,93,95, 100, 98, 18, 26, 39, 40, 30, 59, 29, 31, 46, 94, 114

How to find important clinical covariates from trial information

Looking at forest plots (variation in point estimates/CI overlap/ adding a vertical line for levels of some clinical variable)

6

92, 98, 93, 97, 98, 94

Proceed regardless of formal testing of statistical heterogeneity

5

35, 92, 97, 98, 29

Looking at L’Abbe plots

4

98, 45, 93, 98

Influence plot

3

98, 54, 85

Looking at summary tables

2

92, 24

Looking at funnel plots

2

49, 98

Use conceptual frameworks to facilitate choice of covariates (i.e., using taxonomies for active ingredients)

2

98, 112

I² (look at the change in statistical heterogeneity by adding subgroups)

2

87, 100

Plot of effect size against individual covariates

1

48

Using an adaptation of multidimensional scaling (CoPlot)

1

55

Plot of normalized z-scores

1

93

Radial/Galbraith plot

1

93

Frequency distributions

1

98

Dose-response graph

1

3?

Use P.I.C.O. model to guide choice of characteristics

1

115

Use causal mediating processes

1

113

Treat strata within trials as separate studies; these subgroups if similar across studies can be combined

1

46

Rationale for choice of covariate

Scientific (e.g., pathophysiological, pharmacologic argument)

10

7,76,92,93, 100, 18, 26, 59, 31, 115

Previous research (e.g., large RCT)

3

76, 68, 100

Clinical grounds

2

96, 100

Indirect evidence

1

59

Personnel

Use of clinical experts

2

21, 115

Blind to results of trials

1

35

Number of covariates/trials needed

Small number of covariates

7

92, 95, 100, 18, 26, 31, 94

Each covariate investigation should be based on an adequate number of studies (e.g., 10 for every moderator)

6

100, 59, 50, 94, 115

Investigators must report actual number of covariates investigated for reader to determine the potential for false-positives

1

115

Number of outcomes to investigate

Restrict investigations to small number of outcomes (e.g., primary)

1

26

Limit to central question in the analysis

1

94

Interpretation of results of investigations

Use caution (4 resources note especially with post hoc testing)

12

100, 18, 29, 31, 85, 16, 20, 23, 25, 61, 32, 35

Observational only

6

59, 23, 94, 98, 100, 114

Exploratory or hypothesis generating only

4

32, 100, 40, 94

Consider confounding between covariates

4

100, 50, 115, 59

Consider artifactual causes of between-study variation

2

6, 98

Consider biases (e.g., misclassification, dilution, selection)

2

93, 115

Look at magnitude of the effect and the 95% CI; not just effect and p-value; consider precision of the subgroup effects (e.g., sample sizes in the studies dictate precision of the subgroup effects)

2

100, 115

Seek evidence to justify claims of subgroup findings

1

26

Identify knowledge gaps in the investigations

1

24

Consider effect of variability within studies

1

19

Consider if the magnitude is clinically important (i.e., differences in effect between subgroups)

1

100

Think through causal relationships, especially directionality

1

113

Use caution with variables grouped after randomization

1

23

Consider parabolic relationships (i.e., beyond linear regression)

1

115

Be cautious not to say there is a consistency of effect if no subgroup effects are found

1

115

Descriptive methods

Perform a narrative synthesis of these investigations

4

115, 98, 27, 100

Other: 1. idea webbing, 2. qualitative case descriptions, 3. investigator/methodological/conceptual triangulation

1

98

Use of types of data

Aggregate patient data for trial level covariates

4

23, 25, 118, 46

Only group characteristics derived prior to randomization (e.g., stratifying)

2

23, 46

Individual patient data for participant level covariates

1

59

Individual patient data only for all covariates where possible

1

59