This was a meta-epidemiological study, a study design used to compare intervention effect estimates between trials with and without a characteristic of interest [12]. In this study, we focused on the setting, with the aim of comparing PC-RCTs to ST-RCTs. A meta-epidemiological study is performed by generally using a two-step approach and, contrary to traditional epidemiological studies, units of analysis are studies rather than patients [13]. First, for each selected MA, we assessed the difference in intervention effects between studies with the characteristics of interest (i.e., primary care setting) and without the characteristics of interest. This step involved using a meta-regression for each selected MA (with the intervention effect considered the outcome and the characteristic of interest the independent variable). Second, results from the meta-regression were meta-analysed over the different MAs.
Search strategy
On July 9, 2020, we searched the Cochrane Database of Systematic Reviews to retrieve all systematic reviews with meta-analyses (MAs) that were published, with no restriction on time, by using the following text words in the full text: “primary care” OR “primary healthcare” OR “general practice” OR “family practice”. We only used primary care keywords to ensure that we had PC-RCTs in each MA.
Selection of relevant MAs and trials
We screened the full text of potentially eligible systematic reviews to select MAs of binary outcomes that had at least 3 trials (3 studies is the minimum to perform a meta-regression) and at least one PC-RCT and one ST-RCT. PC-RCTs were defined as trials recruiting patients in general practices, primary care practices, family practices, community centers or community pharmacies according to the definition by Afonso et al. [14]. Trials in nursing homes or at home were not considered PC-RCTs and were excluded. ST-RCTs were defined as trials recruiting in hospitals, including hospitalized patients, hospital outpatients and patients from emergency departments. Hospital-at-home trials were not considered ST-RCTs and were excluded. Trials with both primary and secondary or tertiary care settings were excluded, as were trials with an unclear setting or trials including patients in other than primary or secondary or tertiary care settings, such as schools. For this study, we selected only RCTs and excluded non-randomised or quasi-randomised trials.
If more than one MA was eligible within the same review, we selected the MA for the primary efficacy outcome, then the MA with the highest number of trials. MAs of adverse events were not included because of the uncertainty in the direction of bias. We also discarded MAs when it was impossible to determine which group was the experimental and control group.
When a trial was included several times within the same MA, we kept only the duplicate with the largest sample size. When a trial was included in several selected MAs, we kept the one in the most recent systematic review.
All this selection process was performed independently by two reviewers (A.D., A.H.), with disagreements resolved by discussion, referring to a third opinion (C.D.D.) when necessary.
Data collection
Two independent reviewers (A.D., A.H.) collected data from all selected MAs and trials by using a standardised data collection form. Disagreements were resolved by discussion, referring to a third opinion (C.D.D.) when necessary. The following characteristics were extracted:
For each meta-analysis:
-
General information: year of publication, first author name
-
Medical field
-
Type of intervention (pharmacological, non-pharmacological), type of comparator (active control or inactive control, with inactive control defined as placebo or no added intervention to usual care, sham or other)
-
Outcome and whether it was subjective or objective following the classification provided by Savović et al. [15]
-
All-cause mortality
-
Objectively assessed (e.g., laboratory results)
-
Objectively assessed but potentially influenced by a clinician or patient (e.g., smoking cessation)
-
Subjectively assessed (e.g., pain)
For each eligible RCT included in the MA:
-
Author
-
Year of publication
-
First author name
-
Trial type (PC-RCT or ST-RCT)
-
Number of centers (for primary care, a center was defined as a practice, and for secondary and tertiary care, it was a hospital)
-
Sample size
-
For each group, the number of events and the number of patients analysed
-
Domains of the Cochrane Risk of Bias tool [16] as was reported by the authors of the systematic reviews
For PC- RCTs, we extracted the following additional information if mentioned in the MA:
-
Type of primary care setting
-
Healthcare professionals who recruited patients (when mentioned)
-
Whether the settings where patients are seen during the study for recruitment, follow-up and primary outcome assessment were the same or not
-
Country(ies) where the study was performed because of existing differences in healthcare systems between countries in primary-care
All data were collected from the systematic review report except for the number of events, which was collected from the trial reports when missing in the MA report.
Statistical analysis
We estimated intervention effects as odds ratios (OR). Outcome events were re-coded so that an OR less than 1 indicated a beneficial effect of the experimental intervention. Randomised controlled trials with no event in both groups did not contribute to the analysis.
Meta-epidemiological analysis
To compare PC-RCTs and ST-RCTs, we used the two-step approach described by Sterne et al. [12]. First for each meta-analysis, we estimated a ratio of odds ratio (ROR) by using random-effects meta-regression. In our study, the ROR was the ratio of the OR for PC-RCTs to the OR for ST-RCTs, An ROR less than 1 indicates lower intervention effects for PC-RCTs. Second, we estimated a combined ROR across meta-analyses and the 95% confidence interval (CI) by using a random-effects meta-analysis model. The heterogeneity across MAs was assessed with the I2 statistic and its 95% CI and the between–meta-analysis variance τ2.
Subgroup and sensitivity analyses
Subgroup analyses were planned according to the objectivity of outcomes, type of intervention in the experimental group (pharmacological vs non-pharmacological), and control group (active vs inactive). We used an interaction test to assess whether the combined ROR varied between subgroups. We also performed sensitivity analyses by adjusting meta-regression models on each item of the Risk of Bias tool [16] (high or unclear risk vs low risk).
