- Research
- Open access
- Published:
Multivariate meta-analysis of critical care meta-analyses: a meta-epidemiological study
BMC Medical Research Methodology volume 21, Article number: 148 (2021)
Abstract
Background
Meta-analyses typically consider multiple outcomes and report univariate effect sizes considered as independent. Multivariate meta-analysis (MVMA) incorporates outcome correlation and synthesises direct evidence and related outcome estimates within a single analysis. In a series of meta-analyses from the critically ill literature, the current study contrasts multiple univariate effect estimates and their precision with those derived from MVMA.
Methods
A previous meta-epidemiological study was used to identify meta-analyses with either one or two secondary outcomes providing sufficient detail to structure bivariate or tri-variate MVMA, with mortality as primary outcome. Analysis was performed using a random effects model for both odds ratio (OR) and risk ratio (RR); borrowing of strength (BoS) between multivariate outcome estimates was reported. Estimate comparisons, β coefficients, standard errors (SE) and confidence interval (CI) width, univariate versus multivariate, were performed using Lin’s concordance correlation coefficient (CCC).
Results
In bivariate meta-analyses, for OR (n = 49) and RR (n = 48), there was substantial concordance (≥ 0.69) between estimates; but this was less so for tri-variate meta-analyses for both OR (n = 25; ≥ 0.38) and RR (≥ -0.10; n = 22). A variable change in the multivariate precision of primary mortality outcome estimates compared with univariate was present for both bivariate and tri-variate meta-analyses and for metrics. For second outcomes, precision tended to decrease and CI width increase for bivariate meta-analyses, but was variable in the tri-variate. For third outcomes, precision increased and CI width decreased. In bivariate meta-analyses, OR coefficient significance reversal, univariate versus MVMA, occurred once for mortality and 6 cases for second outcomes. RR coefficient significance reversal occurred in 4 cases; 2 were discordant with OR. For tri-variate OR meta-analyses reversal of coefficient estimate significance occurred in two cases for mortality, nine cases for second and 7 cases for third outcomes. In RR meta-analyses significance reversals occurred for mortality in 2 cases, 6 cases for second and 3 cases for third; there were 7 discordances with OR. BoS was greater in trivariate MVMAs compared with bivariate and for OR versus RR.
Conclusions
MVMA would appear to be the preferred solution to multiple univariate analyses; parameter significance changes may occur. Analytic metric appears to be a determinant.
Background
Meta-analyses typically consider more than one outcome, and the conventional approach is to report multiple univariate effect size estimates of these separate outcomes. Such an approach has two attendant consequences; it ignores the effect of outcome correlation upon individual estimates, assuming that they are independent [1], and engenders multiplicity of the Type I error rate [2]. Confounding such effects is the selective reporting of outcomes, or outcome reporting bias (ORB), whereby secondary outcomes are selectively reported based upon outcome results [3, 4]. Multivariate meta-analysis (MVMA), whereby direct evidence and results from related outcomes are synthesised to yield a summary outcome result [5,6,7], is an elegant solution to the above problems.
In meta-analyses of interventions in the critically ill, where mortality is a common primary outcome, it would be expected that secondary outcomes such as intensive care unit (ICU) and hospital length of stay, infections and the requirement for mechanical ventilation would demonstrate substantial correlation [6], and with the primary mortality event. MVMA in such meta-analyses would allow joint inference upon multiple outcomes and be of relevance from a methodological and clinical viewpoint. Price et al. suggested that where multiple outcomes routinely occur, MVMA would be “…more likely to have an impact” [8]. From a previous study which reported mortality outcome of a series of meta-analyses in the critically ill [9] utilising only randomised controlled trials, a meta-analytic cohort was identified where secondary outcomes were reported in such detail as to yield bivariate or tri-variate data structures. Tri-variate data structures have been rarely subjected to MVMA; in the Price et al. analysis [8], only one such MVMA was reported. Univariate and multivariate analyses were undertaken and compared with respect to differences between estimated outcome variable coefficients, their standard errors (SE) and 95% confidence interval (CI) width and statistical significance, with no selection of meta-analyses based upon the number of RCTs per meta-analysis. As a by-product of MVMA coefficient estimation, variable correlations, direct information and borrowing of strength (BoS) were determined. Whereas direct information describes the contribution of data from the same outcome, BoS represents the contribution of data from all other outcomes [10, 11]. One problematic requirement of MVMA is the provision of with-study correlations which are rarely reported, although methods based upon individual patient [12] or aggregated data [13] and within the Bayesian framework [14] have been undertaken. Any recommendation for the practical application of MVMA must be accompanied by appropriate software. As such, the “alternative” MVMA model of Riley [15] was employed, whereby an overall correlation, the total marginal correlation between outcomes, was modelled, enabling seamless application to all meta-analyses considered. As results based upon indices of risk, odds ratio (OR) and risk ratio (RR), are not generally inter-translatable [16], both OR and RR estimates were compared.
Methods
Ethics
The data for this study was abstracted from published studies and an Ethics clearance was not appropriate.
Data management
A previous study [9] was used to identify meta-analyses with either one or two secondary outcomes that provided sufficient detail to generate a bivariate or tri-variate MVMA data structure, with mortality as the primary outcome; all meta-analyses were of randomised controlled trials (RCT). Usable second and third outcomes were identified as presented in the original meta-analysis.
Statistical analysis
-
1.
To facilitate rapid data processing over a large number of models, initial univariate meta-analytic point estimates and standard errors (SE) were computed within Stata™ V17 [17] using the “meta” suite of commands [18]; default estimation used restricted maximum likelihood (REML [19]).
-
2.
Subsequently, both univariate and multivariate outcomes were estimated using the user written Stata command “mvmeta” ([20], Version 3.2.0 6apr2018) in a random effects (RE) formulation. Estimation employed REML with an unstructured covariance and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm for likelihood maximisation or the Davidon-Fletcher-Powell (DFP) algorithm if there were convergence difficulties. Maximisation employed the “difficult” option (use a different stepping algorithm in nonconcave regions) provided by Stata™.
-
(i)
The two sets of univariate estimates were subsequently compared.
-
(ii)
Under persistent convergence difficulties of “mvmeta”, the model was refit assuming the overall correlation matrix was fixed and known, with values set equal to the estimates from either the BGFS or DFP algorithm using the “bscovariance” option of “mvmeta” ([8], Appendix 4).
-
(iii)
In the MVMA note was taken of very small β coefficient standard errors (SE) with consequent large z values for coefficient significance and very small p-values and CI width, such that the estimates were implausible.
-
(iv)
To avoid the requirement for specific within study correlations [1], the “alternative” model of Riley was used [15], whereby an overall correlation, the total marginal correlation between outcomes [21], was modelled; that is an amalgam of the within and between-study correlations [6, 8]. The reported correlation(s) in this paper were these overall correlation(s) [22].
-
(v)
Direct information and BoS between estimates were also reported [8, 10, 11], using the default (“sd”) method of “mvmeta”. BoS may be conceptualised as a comparison of variances of the estimated rth component of β under the uni- and multivariate models \(BoS_{r}^{{RV}} = 1 - \frac{{{\text{var}}\left( {\hat{\beta }_{{mv,r}} } \right)}}{{{\text{var}}\left( {\hat{\beta }_{{uv,r}} } \right)}}\), where RV refers to relative variance [11]. This ratio has also been described as the efficiency, “E” [10, 23]. An equivalent but alternative method, decomposition of the score function for β, has advantage in that it defines appropriate study weights within an MVMA [11]. In a univariate RE meta-analysis study weights are inversely proportional to the sum of the within- and between- study variances. In an MVMA analysis, as undertaken by “mvmeta”, weights were derived using the score decompensation method, where the score function \(S\left( \theta \right)\) is the first derivative of the log-likelihood function \(l\left( \theta \right);\)\(S\left( \theta \right) = \frac{{dl\left( \theta \right)}}{{d\theta }}\) and \(l(\theta )\) is the likelihood. The weights were broken down into direct information, the contribution of data from the same outcome, and BoS, the contribution of data for all other outcomes. For a univariate analysis, the weights sum to 1, or when expressed as a percentage, 100, as in the “mvmeta” output. In a MVMA, a simple tabulation of direct information and BoS will sum to 100 for each outcome. In particular, the methodology takes the variance components as fixed and the precisions of the point estimates from a MVMA have an expectation of being greater than or equal to those from separate univariate meta-analyses [24], albeit the latter study employed the methods of Van Houwelingen et al. [25] using “Proc Mixed’’ with SAS statistical software, not the “Riley” method [15].
-
a.
The use of the “Riley” method [15] excluded the computation of multivariate I2 for each outcome.
-
a.
-
(i)
-
3.
The reported confounding effect of small study effects upon changes of statistical significance between univariate and MVMA [5, 8] was explored by inspection of contour-enhanced funnel plots [26] and formal regression based tests, in particular, the Harbord (for binary outcomes) and Egger (for continuous outcomes) tests [27]. Small study effects were reported for all meta-analyses, as a matter of complete reporting, but suffer from the problem of multiple testing. The power and interpretation of the tests are problematic for small RCT number (< 10) meta-analyses and in the presence of moderate (see 4., below) heterogeneity [28, 29]. More importantly, univariate tests may be underpowered compared with the recently described multivariate small study effect test (MSSET), a multivariate extension of Egger’s regression test [30].
-
4.
Meta-analytic heterogeneity was reported as the I2 index and adjudged as medium and high if I2 ≥ 50 and 75% respectively. The I2 index was preferred, compared with τ2, as it is comparable across different metrics and number of RCTs [31].
-
5.
The analyses, using frequentist methods, were performed for bivariate and tri-variate models with both OR and RR metrics.
-
6.
Agreement or otherwise between univariate and multivariate estimation results was undertaken using Lin’s concordance correlation coefficient (CCC) via the user written Stata command “concord” [32]. The CCC combines measures of both precision and accuracy to determine how far the observed data deviate from the line of perfect concordance (that is, the line at 45 degrees on a square scatterplot). Other measure to characterise the comparison were:
-
(i)
Estimate differences average and standard deviation (SD); univariate versus MVMA.
-
(ii)
95% (Bland and Altman) limits of agreement (LOA)
-
(iii)
An F test (Bradley-Blackwood) of equality of means and variances; non-significance implies concordance.
-
(i)
-
7.
Boxplots [33] were used to visualise the density distribution of BOS and total marginal correlations of both OR and RR for bivariate and tri-variate models.
Statistical significance was ascribed at p < 0.05.
Results
The cohort was composed of forty-nine meta-analyses, 18% nutritional therapeutic, 18% non-pharmaceutical therapeutic and 64% pharmaceutical therapeutic, published between 2002 and 2018. The primary outcome in all was mortality; forty-nine were bivariate in outcome data composition and 30 were tri-variate. Details of the mortality, second and third outcome meta-analyses are shown in Tables 1, 2 and 3, respectively. Heterogeneity, as the I2 index, of ≥ 50 and ≥ 75% was found in mortality, second and third outcomes in 12, 31 and 50%, and 0, 16 and 23%, respectively. Of the 49 mortality meta-analyses, five [38, 43, 53, 60, 66] demonstrated evidence of small study effects on formal testing (p < 0.05); for the second outcome, five [37, 53, 59,60,61]; and for the third, five [37, 49, 67, 68, 78]. The disparity between the formal test of small study effects (p < 0.05) and the increased frequency of “query” for contour-enhanced funnel plot assessment in second and third outcomes versus mortality outcome (37, 40 and 4%, respectively) was noted and may be a function of the power of the test (see Methods, 3.). There was uniform agreement (to the second or third decimal point) between univariate estimates of “mvmeta” and “meta” in Stata™.
Bivariate model: OR
For the 49 meta-analyses, median (minimum, p25, p75, maximum) number of RCT per meta-analysis for the primary mortality outcome was 13(4, 10, 17, 31); for the second outcomes, 8(4, 6, 10, 31); see Tables 1 and 2. In only 11 meta-analyses was there equality between the reported primary and secondary outcome study numbers. In the MVMA the “bscovariance” option was used once only and there were no instances of “large” Z values. Second outcomes were binary in 39 and continuous in 10 (Tables 1 and 2). Estimate analysis is given in Table 4. Across all outcomes and estimates, the concordance, univariate versus multivariate, was substantial, with a general relative increment, albeit uneven, in the magnitude of multivariate estimates. Means and variances demonstrated little concordance. Reversal of coefficient estimate significance, univariate versus MVMA, occurred no cases for mortality and 6 cases for second outcomes (significant to non-significant in five [36, 43, 70, 71, 79], one meta-analysis exhibiting small study effects [43]; non-significant to significant in in one [59]).
Bivariate model: RR
In the MVMA, 49 meta-analyses were considered and there were no instances of “large” Z values. Concordance estimate analysis is given in Table 5. Substantial concordance was seen between uni- and multivariate estimates, with a variable relative increment of multivariate estimates (SE and CI width) across outcomes. Multivariate β estimates were variable with respect to univariate and means and variances lacked concordance. Reversal of coefficient estimate significance, univariate versus MVMA, occurred in one case for mortality outcome (significant to non-significant, [52]) and 3 cases for second outcomes (significant to non-significant [36, 61, 70]; one instance [61] was discordant with the OR metric and one instance exhibiting small study effects [61]).
The bivariate distributions of BoS are displayed in Fig. 1, where an increment of BoS for RR compared with OR, for both mortality and the second outcome is evident.
The bivariate total marginal correlations, mortality vs second outcome, are shown in Fig. 2; both metrics displayed similar distribution.
Tri-variate model: OR
For the 30 meta-analyses, the median (minimum, p25, p75, maximum) number of studies per meta-analysis for the primary mortality outcome was 13(4, 9, 16, 24); for the second outcome 8(4, 6, 10, 20); and the third 7(3, 5, 10, 15). In only 2 meta-analyses [37, 49] was there equality between the reported primary, second and third outcome study numbers. In the MVMA the “bscovariance” option was used on 13 occasions [37, 38, 46, 54, 57, 62, 66, 72, 73, 77, 78] and there were 5 instances of “large” Z values [37, 54, 72, 77, 78] which were sufficient to render estimates implausible and they were not further considered (median number of RCT per meta-analysis for primary, second and third outcomes 12, 6 and 5 respectively). The outcome data set was thus 25 meta-analyses. Second outcomes were binary in 18 and continuous in 7.; third outcomes were binary in 6 and continuous in 19; the “bscovariance” option being used in eight cases.
Concordance estimate analysis is given in Table 6. Variable concordance between uni- and multivariate estimates was observed. Multivariate estimate precision (SE) increased, and confidence interval width tended to decrease compared with univariate, across and within outcomes. A tendency for concordance between means and variances was apparent. Reversal of coefficient estimate significance, univariate versus MVMA, occurred in two cases for mortality ([38, 73] non-significant to significant, one meta-analysis exhibiting small study effects [38]); nine cases for second outcomes (significant to non-significant in 3 [67, 69, 79], one meta-analysis exhibiting small study effects [67]; non-significant to significant in 6 [37, 57, 59, 62, 66, 73]) and 7 cases for third outcomes (significant to non-significant in 3 [40, 46, 67], one meta-analysis exhibiting small study effects [67]; non-significant to significant in 4 [37, 38, 69, 73] with one demonstrating small study effects [37]).
Tri-variate model: RR
Of the 30 tri-variate meta-analyses, there was one instance of complete convergence failure [46] and seven instances of “large” Z values [37, 38, 56, 66, 72, 73, 78] which were sufficient to render estimates implausible (median number of RCT per meta-analysis for primary, second and third outcomes 10, 6 and 5 respectively); the outcome data set was thus 22 meta-analyses [37, 40, 47, 49, 50, 53,54,55, 57, 59, 60, 62, 67,68,69, 71, 76, 77, 79, 80]. The median (minimum, maximum) number of studies per meta-analysis for the primary mortality outcome was 13(4, 24); for the second outcome, 8(4, 20); and the third 7(4, 15). Second outcomes were binary in 15 and continuous in 7; third outcomes were binary in 7 and continuous in 15. In the MVMA the “bscovariance” option was used on 3 occasions [57, 62, 67]. Concordance estimate analysis is given in Table 7. Concordance between uni- and multivariate estimates was uneven, with no consistent relative change in multivariate estimates, compared with univariate, across or within outcomes. A tendency for concordance between means and variances in second and third outcomes was apparent. Reversal of coefficient estimate significance, univariate versus MVMA, occurred in two case for mortality ([37, 62] with no small study effects, non-significant to significant, not concordant with the OR cases); five cases for second outcomes with no small study effects (significant to non-significant in 2 [69, 79], concordant with OR cases; non-significant to significant in 3 [54, 57, 77]; concordant with one OR cases only [57]); and 3 cases ([54, 60, 67] non-significant to significant, one exhibiting small study effects [67]) for third outcomes with no concordance with OR cases.
The tri-variate distributions of BoS are displayed using boxplots in Fig. 3. The increment of BoS for OR compared with RR for mortality and the third outcome is evident. In the panel (right top) showing BoS mortality RR there were points of large BoS for two MVMA meta-analyses, 99.3 and 93.6 [57, 62]. Both these MVMA utilized the “bscovariance” option of “mvmeta” as there was initial unresolved convergence. The estimated between study mortality variance was minimal for both (5.24e-06 and 0.005, respectively) and the status of these estimates may be circumspect.
The tri-variate total marginal correlations for both OR (left) and RR (right)are shown via boxplots in Fig. 4; with progressive movement to positive correlations from mortality-second outcome through second-third outcome. Positive correlations appeared more frequent with the RR metric.
Discussion
It is easy to forget that the MVMA approach has a long history dating back to at least 1993 [81] and has subsequently been formally implemented in popular statistical software packages [82,83,84,85]. This being said, MVMA still appears rarely used by practitioners, a decade after a 2009 review by Riley [1]. From within the social science paradigm Becker, in 2000, pointed out that ignoring outcome dependence in meta-analysis will affect Type I error rates and precision and bias of estimates: “No reviewer should ever ignore dependence among study outcomes” [82]. In the current study the total marginal correlations for both bi- and tri-variate analyses was sizeable overall and, depending upon the composition of the non-primary outcomes, more positive than negative and more so for the tri-variate case.
One of the principal attractions of MVMA is estimation of the BoS between parameters, well demonstrated in Fig. 3. Most of the BoS would appear to derive from studies which are more “atypical” in design. In particular, the BoS of secondary outcomes of the ith study is a function of the within-study variance matrix \(\left( {V_{i} } \right)\) and the harmonic average \(\bar{V}\) of all the \(V_{i} {\text{s}}\). BoS can only arise if there are differences between the \(V_{i} {\text{s}}\); which would entail studies of”..substantive difference[s] in background and research methods…”, not simply different sample sizes [10]. The magnitude of outcome BoS would appear to be bounded by percentage of missing data for that outcome [6, 24], which in the current study was substantial (see Results). A percentage missingness of 30–35% of studies informing an outcome was found to result in a “more pronounced” BoS in one empirical study [14]. Any nexus between BoS and missingness requires a missing at random (MAR) assumption, as opposed to missing completely at random (MCAR) for univariate meta-analysis [21]. The notions of MAR and MCAR are well recognised in the bio-medical literature [86], albeit inconsistency of usage has been documented [87]; in particular, the conflation of (non)”ignorable” and MAR [22]. Perhaps not surprisingly, within the domain of outcome reporting bias (ORB) [4], MVMA has been a method of choice to investigate the impact of ORB upon meta-analytic conclusions [22, 88].
Computationally, MVMA requires both within- and between-study correlations and the former are typically not known and are likely not to be available, especially in higher order (trivariate) models [24]. Riley provided four alternate methods to overcome these problems [1]; the most straightforward, yet laborious, being a sensitivity analysis by correlation imputation over the entire range (-1 to + 1). Riley’s alternate model [15] has been found to have good asymptotic statistical properties compared with a fully hierarchical REML model, with known within-study correlations, and with separate univariate meta-analyses. The performance may be problematic when the overall correlation \(\left( {\hat{\rho }} \right)\) is very close to 1 or -1. In the current study, only two instances were found; in the bivariate RR MVMA, \(\hat{\rho }\) = 0.999 [75], and the trivariate OR MVMA, \(\hat{\rho }\) = -0.986 ([57], second versus third outcome); both MVMA utilised the “bscov” option. As the Riley model is a “working” model when the true data generating mechanism is a RE model, the standard variance estimates may not provide confidence interval coverage at the nominal level [21]. Complete failure of convergence in the current study was rare, occurring in one instance [46], but problematic SE estimation was exhibited in the trivariate series, 5 instances in OR metric and 7 in RR. This may relate to the small number of RCT in second and third outcomes (see Table 8), but these numbers were not substantially different compared with meta-analyses not demonstrating this feature, as shown in Table 8.
Frequentist and Bayesian empirical comparisons between univariate meta-analyses and MVMA have appeared in the literature [8, 14, 89,90,91,92,93] with results demonstrating similar (pooled) parameter estimates between the two analytic forms. However, papers by Riley and co-workers [1, 15, 24, 94], which included formal simulation studies, found advantage; a smaller standard error and mean-square error of pooled estimates, predicated upon the presence of missing data; again, assuming missing at random. That is, in the presence of complete data a bivariate analysis would not be expected to produce a gain in statistical efficiency. The extension to trivariate and higher order outcome data and the inability to provide within study correlations was thus identified as a “pressing research issue”; to wit, the “alternative” model of Riley [15]. Price et al. suggested that estimates of clinical and /or statistical conclusions from MVMA may occasionally differ from those from univariate analyses and observed, somewhat wryly, that any claimed discrepancy “…says more about the dangers of using concepts of statistical significance than it does the use of MVMA” [8]. The results from the current analysis were somewhat at odds with these sentiments and with the general results of bivariate studies, both empirical and simulation (see below), albeit the caution about the variance estimates of the Riley model, above, are noted. A variable change in the multivariate precision of primary mortality outcome estimates compared with univariate analysis was present for both bivariate and tri-variate meta-analyses and for metric. For second outcomes, precision tended to decrease, and CI width increase for bivariate meta-analyses; for third outcomes, precision increased, and CI width decreased. The latter finding appears not to have been previously reported although analytic reports of the tri-variate structure are rare; one case only reported by Price et al. study [8] and two by Trikalinos et al. [14]. With respect to the observed relative changes (univariate versus multivariate) across four concordance analyses, the magnitude of the difference was rather small and accompanied by a more substantial SD, suggesting a heterogeneity of the MVMA effect, grounded in the individual meta-analyses and dependent upon the nature of the outcome, binary or continuous. As MVMA allows for correlation between outcomes, CIs may be wider on the basis of increased between-study variance [8], but this was observed only in the bivariate case in the current analysis. The experience of Price et al. that “MVMA methods can be applied only in a minority of reviews of interventions in pregnancy and childbirth” [8] was not consistent with the current study.
A reviewer pointed to the wide LOA of the β estimates for the second continuous outcome (days) in Table 6 (trivariate OR MVMA), this being -10.894, 8.122. Of the seven meta-analyses considered, two had stand-out differences between univariate and MVMA estimates; the study of Wan et al. ([57], intra-meta-analytic study number = 4), -11.31, -18.17 and Chen et al. ([69],intra-meta-analytic study number = 10), -11.27 and -4.26. The former study used the “bscov” option, recording a BOS for the second outcome of 54% and correlation between second and third outcomes of 0.986; the latter had normal convergence but record a BOS for the second outcome of 92.4% with a correlation between second and third outcomes of 0.995. This may be indicative of problematic estimation, which has been mentioned above and further addressed in “Limitations”, below. Trikalinos et al. ([14], point 4.1), using Bayesian methods, observed that “Generally, point estimates are comparable”; Price et al. ([8], Table 2) using “mvmeta” recorded differences in β between univariate and MVMA, but did not focus attention on such; and in the bivariate simulation study of Riley et al. ([94], Table 4), bias of the mean for \(\beta _{1} \,{\text{and}}\,\beta _{2}\) was comparable with coverage for both between 93–98%; similar results were also observed when considering the “alternate” model of Riley ([15], Table 1).
The differences between the results of the current study and those referenced above [8, 14, 89,90,91,92,93] needs some further explication with regard to data structure. The bi- and tri-variate meta-analyses under consideration were relatively conventional; a primary mortality outcome and second and third recorded outcomes which were not direct extensions of the primary outcome. For second and third outcomes, both categorical (binary) and continuous outcomes were considered, unlike Trikalinos et al. [14] where outcomes were categorical. No repeated measures of a primary outcome, such as different mortality time-points or different types of mortality (all cause or disease specific) were considered; the latter structure featured in the studies of Trikalinos et al. [14, 91], Arends [93] and also in an empirical example Riley et al. [15]. Within the critical care domain the use of MVMA analysis with different mortality time-points has been recently presented [95].The current study did not focus on the impact of different meta-analytic estimators as in Berkey et al. [92], generalized least squares and multivariate maximum likelihood, nor adopt the Bayesian framework of Trikalinos [14]. That bivariate models have been used in systematic reviews of diagnostic test studies for some years, was noted in 2009 by Riley and both Simel et al. [90] and Dahabreh et al. [89] found little advantage for bivariate approaches when considering estimates of sensitivity and specificity. With respect to the change of estimate significance reported here, univariate versus MVMA, the use of the MVMA “bscov” option may have been consequential. For the OR metric, where 24 significance changes occurred, there were seven instances [37, 46, 57, 62, 66, 69, 73], all in trivariate MVMA. For the RR metric, again with the trivariate data structure, there were three [57, 62, 67].
These changes of statistical significance are shown in forest plots as couplets, univariate versus MVMA, for binary (null line unity, Fig. 5) and continuous (null line zero, Fig. 6) outcomes. A majority of the CI width changes that achieved a change of significance about the null appear substantial; the clinical import of such changes would require case by case determination [96].
Disparities between the OR and RR occurred over a range of indices and may be a function of the current cohort. However, OR and RR are not merely interchangeable metrics and there is no monotone relationship between them [16]. Recent papers have drawn attention to potential estimation problems with the RR. First, the RR effect magnitude is dependent upon the underlying baseline prevalence, shifting toward 1 as prevalence increases, and is a ratio of two conditional probabilities, whereas the OR is a likelihood ratio whose magnitude reflects the fold increase in odds, baseline to intervention, independent of prevalence [97]. Second, under both the DerSimonian-Laird [98] and REML formulations, the requirements of log(RR) estimation to be compatible with study level event rates in the [0,1] interval \((\pi _{j} {\text{treat}}\,{\text{ < }}\,{\text{1}}\,{\text{and 0}}\, < \,\pi _{j} {\text{control}}\,{\text{ < }}\,{\text{1)}}\) demand restriction on the parameter space with ensuant bias in estimates of both \(\tau ^{2}\) and log(RR). Thus risk relativism may be an “illusion “ [97] and the OR “appears to be a safer option” [99]. This being said, Xiao and colleagues argued that interpretability issues of the OR, lack of collapsibility and a dependence on the baseline risk, negates any in-principle recommendation for the OR [100].
Limitations
The current study utilised a single meta-analytic cohort from the critical care domain and had a modest number of bivariate meta-analyses, but less so in the trivariate series. The preference for the alternate model of Riley was a potential limitation, but when reviewing a number of bivariate and tri-variate studies in two metrics the use of sensitivity analysis by specifying within study correlations (via the “wscor” option of “mvmeta”) would be unwieldy and potentially uninterpretable. This being said, the recommendation of Riley et al. in the landmark 2008 paper [15], was that in the presence of overall correlations > 0.9 in absolute value, practitioners “..should assess the robustness of pooled results to small changes in \(\hat{\rho }\) as a sensitivity analysis”. In the MVMA where large z values were found and subsequently not considered, for the OR studies [37, 54, 72, 77, 78] and for the RR studies [37, 38, 56, 66, 72, 73, 78], all meta-analyses had \(\hat{\rho }\) > 0.9 in at least one of the correlations. Whether such a modus operandi would yield credible z values and pooled estimates has not been explored.
The current study has adopted a workable and practical solution to the particular requirements of MVMA. Future studies should replicate or otherwise the findings in this paper using the “alternate” meta-analytic model of Riley and consider meta-analyses from specific disciplines, moving beyond the bivariate data structure to encompass “…three or more end points…” [1], albeit such estimation may be challenging.
Conclusions
MVMA elucidates the structure and correlation between multiple reported outcomes in univariate meta-analyses and resolves outcome reporting bias. Change in estimate precision and CI width with MVMA appeared context dependent. The BoS entailed in this technique may be quantified and change of parameter significance may be a consequence. MVMA is a feasible solution to the meta-analytic estimation of multiple univariate effects.
Availability of data and materials
The collection of the original data set (current reference [9]) from published papers was the joint work of the author (John L Moran) and Dr Petra Graham (see below) and ownership of the data resides with both persons and is thus not available in the public domain, nor in repository.
References
Riley RD. Multivariate meta-analysis: the effect of ignoring within-study correlation. J R Stat Soc Ser A Stat Soc. 2009;172(4):789–811.
Lopez-Lopez JA, Page MJ, Lipsey MW, Higgins JPT. Dealing with effect size multiplicity in systematic reviews and meta-analyses. Res Synth Methods. 2018;9(3):336–51.
Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias G. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS One. 2013;8(7):e66844.
Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340(feb15_1):c365.
Jackson D, Riley R, White IR. Multivariate meta-analysis: Potential and promise. Stat Med. 2011;30(20):2481–98.
Riley RD, Jackson D, Salanti G, Burke DL, Price M, Kirkham J, White IR. Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples. BMJ. 2017;358:j3932.
Jackson D, White IR, Riley RD. Multivariate meta-analysis. In: Schmid CH, Stijnen T, White IR, editors. Handbook of meta-analysis. Boca Raton: CRC Press; 2021. p. 163–86.
Price MJ, Blake HA, Kenyon S, White IR, Jackson D, Kirkham JJ, Neilson JP, Deeks JJ, Riley RD. Empirical comparison of univariate and multivariate meta-analyses in Cochrane Pregnancy and Childbirth reviews with multiple binary outcomes. Res Synth Methods. 2019;10(3):440–51.
Moran JL, Graham PL. Risk related therapy in meta-analyses of critical care interventions: Bayesian meta-regression analysis. J Crit Care. 2019;53:114–9.
Copas JB, Jackson D, White IR, Riley RD. The role of secondary outcomes in multivariate meta‐analysis. J R Stat Soc Ser C Appl Stat. 2018;67(5):1177–205.
Jackson D, White IR, Price M, Copas J, Riley RD. Borrowing of strength and study weights in multivariate and network meta-analysis. Stat Methods Med Res. 2017;26(6):2853–68.
Riley RD, Price MJ, Jackson D, Wardle M, Gueyffier F, Wang J, Staessen JA, White IR. Multivariate meta-analysis using individual participant data. Res Synth Methods. 2015;6(2):157–74.
Wei Y, Higgins JP. Estimating within-study covariances in multivariate meta-analysis with multiple outcomes. Stat Med. 2013;32(7):1191–205.
Trikalinos TA, Hoaglin DC, Schmid CH. An empirical comparison of univariate and multivariate meta-analyses for categorical outcomes. Stat Med. 2014;33(9):1441–59.
Riley RD, Thompson JR, Abrams KR. An alternative model for bivariate random-effects meta-analysis when the within-study correlations are unknown. Biostatistics. 2008;9(1):172–86.
Feng C, Wang B, Wang H. The relations among three popular indices of risks. Stat Med. 2019;38(23):4772–87.
StataCorp. Stata release 17. 2021. https://www.stata.com/products/.
StataCorp. Stata meta-analysis reference manual: release 17. 2021. https://www.stata-press.com/manuals/meta-analysis-reference-manual/.
Partlett C, Riley RD. Random effects meta-analysis: coverage performance of 95% confidence and prediction intervals following REML estimation. Stat Med. 2017;36(2):301–17.
White IR. Multivariate random-effects meta-regression: updates to mvmeta. Stata J. 2011;11(2):255–70.
Hong C, Riley RD, Chen Y. An improved method for bivariate meta-analysis when within-study correlations are unknown. Res Synth Methods. 2018;9(1):73–88.
Kirkham JJ, Riley RD, Williamson PR. A multivariate meta-analysis approach for reducing the impact of outcome reporting bias in systematic reviews. Stat Med. 2012;31(20):2179–95.
Lin L, Xing A, Kofler MJ, Murad MH. Borrowing of strength from indirect evidence in 40 network meta-analyses. J Clin Epidemiol. 2018;106:41–9.
Riley RD, Abrams KR, Lambert PC, Thompson JR. An evaluation of bivariate random-effects meta-analysis for the joint synthesis of two correlated outcomes. Stat Med. 2007;26(1):78–97.
Van Houwelingen HC, Arends LR, Stijnen T. Advanced methods in meta-analysis: multivariate approach and meta-regression. Stat Med. 2002;21(4):589–624.
Palmer TM, Peters JL, Sutton AJ, Moreno SG. Contour-enhanced funnel plots for meta-analysis. Stata J. 2008;8(2):242–54.
StataCorp: meta bias. Stata V17 documentation. 2021. https://www.stata.com/manuals/metametabias.pdf. Accessed 4 May 2021.
Ioannidis JP, Trikalinos TA. The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey. Can Med Assoc J. 2007;176(8):1091–6.
Sterne JA, Gavaghan D, Egger M. Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol. 2000;53(11):1119–29.
Hong C, Salanti G, Morton SC, Riley RD, Chu H, Kimmel SE, Chen Y. Testing small study effects in multivariate meta-analysis. Biometrics. 2020;76(4):1240–50.
Huedo-Medina TB, Sánchez-Meca J, Marin-Martinez F, Botella J. Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychol Methods. 2006;11(2):193–206.
Steichen TJ, Cox NJ. A note on the concordance correlation coefficient. Stata J. 2002;2(2):183–9.
Cleveland WS. Graphical methods. In: The elements of graphing data. edn. Summit: Hobart Press; 1994. p. 139–143.
Griesdale DEG, de Souza RJ, van Dam RM, Heyland DK, Cook DJ, Malhotra A, Dhaliwal R, Henderson WR, Chittock DR, Finfer S, et al. Intensive insulin therapy and mortality among critically ill patients: a meta-analysis including NICE-SUGAR study data. Can Med Assoc J. 2009;180(8):821–7.
Annane D, Bellissant E, Bollaert PE, Briegel J, Confalonieri M, De Gaudio R, Keh D, Kupfer Y, Oppert M, Meduri GU. Corticosteroids in the treatment of severe sepsis and septic shock in adults: a systematic review. JAMA. 2009;301(22):2362–75.
Bangalore S, Wetterslev J, Pranesh S, Sawhney S, Gluud C, Messerli FH. Perioperative β blockers in patients having non-cardiac surgery: a meta-analysis. Lancet. 2008;372(9654):1962–76.
Marik P, Zaloga G. Immunonutrition in critically ill patients: a systematic review and analysis of the literature. Intensive Care Med. 2008;34(11):1980–90.
Chan EY, Ruest A, Meade MO, Cook DJ. Oral decontamination for prevention of pneumonia in mechanically ventilated adults: systematic review and meta-analysis. Br Med J. 2007;334(7599):889.
Gonzalez R, Zamora J, Gomez-Camarero J, Molinero LM, Banares R, Albillos A. Meta-analysis: combination endoscopic and drug therapy to prevent variceal rebleeding in cirrhosis. Ann Intern Med. 2008;149(2):109–22.
Ho KM, Dobb GJ, Webb SAR. A comparison of early gastric and post-pyloric feeding in critically ill patients: a meta-analysis. Intensive Care Med. 2006;32(5):639–49.
Ho KM, Ng JY. The use of propofol for medium and long-term sedation in critically ill adult patients: a meta-analysis. Intensive Care Med. 2008;34(11):1969–79.
Siempos II, Ntaidou TK, Falagas ME. Impact of the administration of probiotics on the incidence of ventilator-associated pneumonia: a meta-analysis of randomized controlled trials. Crit Care Med. 2010;38(3):954–62.
Singh S, Amin AV, Loke YK. Long-term use of inhaled corticosteroids and the risk of pneumonia in chronic obstructive pulmonary disease a meta-analysis. Arch Intern Med. 2010;169(3):219–29.
Peterson K, Carson S, Carney N. Hypothermia treatment for traumatic brain injury: a systematic review and meta-analysis. J Neurotrauma. 2008;25(1):62–71.
Silvestri L, van Saene HKF, Milanese M, Gregori D, Gullo A. Selective decontamination of the digestive tract reduces bacterial bloodstream infection and mortality in critically ill patients. Systematic review of randomized, controlled trials. J Hosp Infect. 2007;65(3):187–203.
Whitlock RP, Chan S, Devereaux PJ, Sun J, Rubens FD, Thorlund K, Teoh KHT. Clinical benefit of steroid use in patients undergoing cardiopulmonary bypass: a meta-analysis of randomized trials. Eur Heart J. 2008;29(21):2592–600.
Piccini JP, Berger JS, O’Connor CM. Amiodarone for the prevention of sudden cardiac death: a meta-analysis of randomized controlled trials. Eur Heart J. 2009;30(10):1245–53.
Landoni G, Mizzi A, Biondi-Zoccai G, Bignami E, Prati P, Ajello V, Marino G, Guarracino F, Zangrillo A. Levosimendan reduces mortality in critically ill patients. A meta-analysis of randomized controlled studies. Minerva Anestesiol. 2010;76(4):276–86.
Brar SS, Leon MB, Stone GW, Mehran R, Moses JW, Brar SK, Dangas G. Use of drug-eluting stents in acute myocardial infarction a systematic review and meta-analysis. J Am Coll Cardiol. 2009;53(18):1677–89.
Landoni G, Biondi-Zoccai GGL, Tumlin JA, Bove T, De Luca M, Calabro MG, Ranucci M, Zangrillo A. Beneficial impact of Fenoldopam in critically ill patients with or at risk for acute renal failure: a meta-analysis of randomized clinical trials. Am J Kidney Dis. 2007;49(1):56–68.
Mazaki T, Ebisawa K. Enteral versus parenteral nutrition after gastrointestinal surgery: a systematic review and meta-analysis of randomized controlled trials in the English literature. J Gastrointest Surg. 2008;12(4):739–55.
Masip J, Roque M, Sanchez B, Fernandez R, Subirana M, Exposito JA. Noninvasive ventilation in acute cardiogenic pulmonary edema - systematic review and meta-analysis. JAMA. 2005;294(24):3124–30.
Oldani M, Sandini M, Nespoli L, Coppola S, Bernasconi DP, Gianotti L. Glutamine supplementation in intensive care patients a meta-analysis of randomized clinical trials. Medicine. 2015;94(31):e1319.
Szakmany T, Russell P, Wilkes AR, Hall JE. Effect of early tracheostomy on resource utilization and clinical outcomes in critically ill patients: meta-analysis of randomized controlled trials. Br J Anaesth. 2015;114(3):396–405.
Alkhawaja S, Martin C, Butler RJ, Gwadry-Sridhar F. Post-pyloric versus gastric tube feeding for preventing pneumonia and improving nutritional outcomes in critically ill adults. Cochrane Database Syst Rev. 2015;2015(8):CD008875.
van Zanten ARH, Dhaliwal R, Garrel D, Heyland DK. Enteral glutamine supplementation in critically ill patients: a systematic review and meta-analysis. Crit Care. 2015;19(1):294.
Wan X, Gao XJ, Bi JC, Tian F, Wang XY. Use of n-3 PUFAs can decrease the mortality in patients with systemic inflammatory response syndrome: a systematic review and meta-analysis. Lipids Health Dis. 2015;14(1):1–9.
Teo J, Liew Y, Lee W, Kwa AL-H. Prolonged infusion versus intermittent boluses of beta-lactam antibiotics for treatment of acute infections: a meta-analysis. Int J Antimicrob Agents. 2014;43(5):403–11.
Manzanares W, Dhaliwal R, Jiang XR, Murch L, Heyland DK. Antioxidant micronutrients in the critically ill: a systematic review and meta-analysis. Crit Care. 2012;16(2):1–3.
Tian FMD, Heighes PTMPS, Allingstrup MJP, Doig GSP. Early enteral nutrition provided within 24 hours of ICU admission: a meta-analysis of randomized controlled trials*. Crit Care Med. 2018;46(7):1049–56.
Rhodes NJ, Liu J, O’Donnell JN, Dulhunty JM, Abdul-Aziz MH, Berko PY, Nadler B, Lipman J, Roberts JA. Prolonged infusion piperacillin-tazobactam decreases mortality and improves outcomes in severely ill patients: results of a systematic review and meta-analysis. Crit Care Med. 2018;46(2):236–43.
Nunez-Patino RA, Zorrilla-Vaca A, Rivera-Lara L. Comparison of intensive versus conventional insulin therapy in traumatic brain injury: a meta-analysis of randomized controlled trials. Brain Inj. 2018;32(6):693–703.
Kawano-Dourado L, Zampieri FG, Azevedo LCP, Correa TD, Figueiro M, Semler MW, Kellum JA, Cavalcanti AB. Low-versus high-chloride content intravenous solutions for critically ill and perioperative adult patients: a systematic review and meta-analysis. Anesth Analg. 2018;126(2):513–21.
Dallimore J, Ebmeier S, Thayabaran D, Bellomo R, Bernard G, Schortgen F, Saxena M, Beasley R, Weatherall M, Young P. Effect of active temperature management on mortality in intensive care unit patients. Crit Care Resusc. 2018;20(2):150–63.
Chong MA, Krishnan R, Cheng D, Martin J. Should transfusion trigger thresholds differ for critical care versus perioperative patients? A meta-analysis of randomized trials. Crit Care Med. 2018;46(2):252–63.
Yang X-M, Tu G-W, Zheng J-L, Shen B, Ma G-G, Hao G-W, Gao J, Luo Z. A comparison of early versus late initiation of renal replacement therapy for acute kidney injury in critically ill patients: an updated systematic review and meta-analysis of randomized controlled trials. BMC Nephrol. 2017;18(1):264.
Osadnik CR, Tee VS, Carson-Chahhoud KV, Picot J, Wedzicha JA, Smith BJ. Non-invasive ventilation for the management of acute hypercapnic respiratory failure due to exacerbation of chronic obstructive pulmonary disease. Cochrane Database Syst Rev. 2017;7(7):CD004104.
Lu C, Sharma S, McIntyre L, Rhodes A, Evans L, Almenawer S, Leduc L, Angus DC, Alhazzani W. Omega-3 supplementation in patients with sepsis: a systematic review and meta-analysis of randomized trials. Ann Intensive Care. 2017;7(1):58.
Chen QH, Zheng RQ, Lin H, Shao J, Yu JQ, Wang HL. Effect of levosimendan on prognosis in adult patients undergoing cardiac surgery: a meta-analysis of randomized controlled trials. Crit Care. 2017;21(1):253.
Qureshi SH, Rizvi SI, Patel NN, Murphy GJ. Meta-analysis of colloids versus crystalloids in critically ill, trauma and surgical patients. Br J Surg. 2016;103(1):14–26.
Elke G, van Zanten ARH, Lemieux M, McCall M, Jeejeebhoy KN, Kott M, Jiang X, Day AG, Heyland DK. Enteral versus parenteral nutrition in critically ill patients: an updated systematic review and meta-analysis of randomized controlled trials. Crit Care. 2016;20(1):117.
Parikh HG, Miller A, Chapman M, Moran JL, Peake SL. Calorie delivery and clinical outcomes in the critically ill: a systematic review and meta-analysis. Crit Care Resusc. 2016;18(1):17–24.
Davies ML, Chapple L-AS, Chapman MJ, Moran JL, Peake SL. Protein delivery and clinical outcomes in the critically ill: a systematic review and meta-analysis. Crit Care Resusc. 2017;19(2):117–27.
Abroug F, Ouanes I, Abroug S, Dachraoui F, Ben Abdallah S, Hammouda Z, Ouanes-Besbes L. Systemic corticosteroids in acute exacerbation of COPD: a meta-analysis of controlled studies with emphasis on ICU patients. Ann Intensive Care. 2014;4:32.
Manzanares W, Lemieux M, Langlois PL, Wischmeyer PE. Probiotic and synbiotic therapy in critical illness: a systematic review and meta-analysis. Crit Care. 2016:262.
Peter JV, Moran JL, Phillips-Hughes J, Warne D. Non-invasive ventilation (NIV) in acute respiratory failure: a meta-analysis update. Crit Care Med. 2002;30(3):555–62.
Wang CH, Ma MH-M, Chou HC, Yen ZS, Yang CW, Fang CC, Chen SC. High-dose vs non high-dose proton pump inhibitors after endoscopic treatment in patients with bleeding peptic ulcer a systematic review and meta-analysis of randomized controlled trials. Arch Intern Med. 2010;170(9):751–8.
Tao W, Li P-S, Shen Z, Shu Y-S, Liu S. Effects of omega-3 fatty acid nutrition on mortality in septic patients: a meta-analysis of randomized controlled trials. BMC Anesthesiol. 2016;16:39.
Tang H, Huang T, Jing J, Shen H, Cui W. Effect of procalcitonin-guided treatment in patients with infections: a systematic review and meta-analysis. Infection. 2009;37(6):497–507.
Muscedere J, Rewa O, McKechnie K, Jiang XR, Laporta D, Heyland DK. Subglottic secretion drainage for the prevention of ventilator-associated pneumonia: a systematic review and meta-analysis. Crit Care Med. 2011;39(8):1985–91.
Vanhouwelingen HC, Zwinderman KH, Stijnen T. A bivariate approach to meta-analysis. Stat Med. 1993;12(24):2273–84.
Becker BJ. Multivariate meta-analysis. In: Tinsley HEA, Brown SD, editors. Handbook of applied multivariate statistics and mathematical modeling. San Diego: Academic; 2000. p. 499–525.
Gasparrini A, Armstrong B, Kenward MG. Multivariate meta-analysis for non-linear and other multi-parameter associations. Stat Med. 2012;31(29):3821–39.
Luo S, Chen Y, Su X, Chu H. mmeta: an R package for multivariate meta-analysis. J Stat Softw. 2014;56(11):26.
White IR. Multivariate random-effects meta-analysis. Stata J. 2009;9(1):40–56.
Vesin A, Azoulay E, Ruckly S, Vignoud L, Rusinova K, Benoit D, Soares M, Azeivedo-Maia P, Abroug F, Benbenishty J, et al. Reporting and handling missing values in clinical studies in intensive care units. Intensive Care Med. 2013;39(8):1396–404.
Seaman S, Galati J, Jackson D, Carlin J. What is meant by “missing at random”? Stat Sci. 2013;28(2):257–68.
Frosi G, Riley RD, Williamson PR, Kirkham JJ. Multivariate meta-analysis helps examine the impact of outcome reporting bias in Cochrane rheumatoid arthritis reviews. J Clin Epidemiol. 2015;68(5):542–50.
Dahabreh IJ, Trikalinos TA, Lau J, Schmid CH. Univariate and bivariate likelihood-based meta-analysis methods performed comparably when marginal sensitivity and specificity were the targets of inference. J Clin Epidemiol. 2017;83:8–17.
Simel DL, Bossuyt PMM. Differences between univariate and bivariate models for summarizing diagnostic accuracy may not be large. J Clin Epidemiol. 2009;62(12):1292–300.
Trikalinos TA, Olkin I. Meta-analysis of effect sizes reported at multiple time points: a multivariate approach. Clin Trials. 2012;9(5):610–20.
Berkey CS, Hoaglin DC, Antczak-Bouckoms A, Mosteller F, Colditz GA. Meta-analysis of multiple outcomes by regression with random effects. Stat Med. 1998;17(22):2537–50.
Arends L. Multivariate meta-analysis: modelling the heterogeneity. PhD Disseration. The Netherlands: Erasmus University; 2006. https://core.acu.k/download/pdf/19187601.pdf, downloaded 4th June 2016.
Riley R, Abrams K, Sutton A, Lambert P, Thompson J. Bivariate random-effects meta-analysis and the estimation of between-study correlation. BMC Med Res Methodol. 2007;7(1):3.
Moran JL, Graham PL. Multivariate meta-analysis of the mortality effect of prone positioning in the acute respiratory distress syndrome. J Intensive Care Med. 2021:08850666211014479. https://doi.org/10.1177/08850666211014479.
Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, p values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31(4):337–50.
Doi SA, Furuya-Kanamori L, Xu C, Lin L, Chivese T, Thalib L. Questionable utility of the relative risk in clinical research: a call for change to practice. J Clin Epidemiol. 2020. https://www.jclinepi.com/article/S0895-4356(20)31171-9/fulltext, Downloaded 19th November 2020.
DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177–88.
Bakbergenuly I, Hoaglin DC, Kulinskaya E. Pitfalls of using the risk ratio in meta-analysis. Res Synth Methods. 2019;10(3):398–419.
Xiao M, Chen Y, Cole S, MacLehose R, Richardson D, Chu H. Is OR “portable” in meta-analysis? Time to consider bivariate generalized linear mixed model. medRxiv : the preprint server for health sciences 2020. https://doi.org/10.1101/2020.11.05.20226811.
Acknowledgements
Dr Petra Graham (Macquarie University, NSW Australia) for diligent help in assembling the original series of meta-analyses as reported in Reference [8].
Professor Ian White (Medical Research Council Clinical Trials Unit at University College London, UK) for advice on command syntax of his user-written Stata module “mvmeta”.
Funding
Local Intensive Care Unit funds only.
Author information
Authors and Affiliations
Contributions
JLM is the sole author of this paper. The author(s) read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The data for this study was abstracted from published studies and an Ethics clearance was not appropriate.
Competing interests
The author declares no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Moran, J.L. Multivariate meta-analysis of critical care meta-analyses: a meta-epidemiological study. BMC Med Res Methodol 21, 148 (2021). https://doi.org/10.1186/s12874-021-01336-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12874-021-01336-4