 Research
 Open Access
 Published:
Multivariate metaanalysis of critical care metaanalyses: a metaepidemiological study
BMC Medical Research Methodology volume 21, Article number: 148 (2021)
Abstract
Background
Metaanalyses typically consider multiple outcomes and report univariate effect sizes considered as independent. Multivariate metaanalysis (MVMA) incorporates outcome correlation and synthesises direct evidence and related outcome estimates within a single analysis. In a series of metaanalyses from the critically ill literature, the current study contrasts multiple univariate effect estimates and their precision with those derived from MVMA.
Methods
A previous metaepidemiological study was used to identify metaanalyses with either one or two secondary outcomes providing sufficient detail to structure bivariate or trivariate MVMA, with mortality as primary outcome. Analysis was performed using a random effects model for both odds ratio (OR) and risk ratio (RR); borrowing of strength (BoS) between multivariate outcome estimates was reported. Estimate comparisons, β coefficients, standard errors (SE) and confidence interval (CI) width, univariate versus multivariate, were performed using Lin’s concordance correlation coefficient (CCC).
Results
In bivariate metaanalyses, for OR (n = 49) and RR (n = 48), there was substantial concordance (≥ 0.69) between estimates; but this was less so for trivariate metaanalyses for both OR (n = 25; ≥ 0.38) and RR (≥ 0.10; n = 22). A variable change in the multivariate precision of primary mortality outcome estimates compared with univariate was present for both bivariate and trivariate metaanalyses and for metrics. For second outcomes, precision tended to decrease and CI width increase for bivariate metaanalyses, but was variable in the trivariate. For third outcomes, precision increased and CI width decreased. In bivariate metaanalyses, OR coefficient significance reversal, univariate versus MVMA, occurred once for mortality and 6 cases for second outcomes. RR coefficient significance reversal occurred in 4 cases; 2 were discordant with OR. For trivariate OR metaanalyses reversal of coefficient estimate significance occurred in two cases for mortality, nine cases for second and 7 cases for third outcomes. In RR metaanalyses significance reversals occurred for mortality in 2 cases, 6 cases for second and 3 cases for third; there were 7 discordances with OR. BoS was greater in trivariate MVMAs compared with bivariate and for OR versus RR.
Conclusions
MVMA would appear to be the preferred solution to multiple univariate analyses; parameter significance changes may occur. Analytic metric appears to be a determinant.
Background
Metaanalyses typically consider more than one outcome, and the conventional approach is to report multiple univariate effect size estimates of these separate outcomes. Such an approach has two attendant consequences; it ignores the effect of outcome correlation upon individual estimates, assuming that they are independent [1], and engenders multiplicity of the Type I error rate [2]. Confounding such effects is the selective reporting of outcomes, or outcome reporting bias (ORB), whereby secondary outcomes are selectively reported based upon outcome results [3, 4]. Multivariate metaanalysis (MVMA), whereby direct evidence and results from related outcomes are synthesised to yield a summary outcome result [5,6,7], is an elegant solution to the above problems.
In metaanalyses of interventions in the critically ill, where mortality is a common primary outcome, it would be expected that secondary outcomes such as intensive care unit (ICU) and hospital length of stay, infections and the requirement for mechanical ventilation would demonstrate substantial correlation [6], and with the primary mortality event. MVMA in such metaanalyses would allow joint inference upon multiple outcomes and be of relevance from a methodological and clinical viewpoint. Price et al. suggested that where multiple outcomes routinely occur, MVMA would be “…more likely to have an impact” [8]. From a previous study which reported mortality outcome of a series of metaanalyses in the critically ill [9] utilising only randomised controlled trials, a metaanalytic cohort was identified where secondary outcomes were reported in such detail as to yield bivariate or trivariate data structures. Trivariate data structures have been rarely subjected to MVMA; in the Price et al. analysis [8], only one such MVMA was reported. Univariate and multivariate analyses were undertaken and compared with respect to differences between estimated outcome variable coefficients, their standard errors (SE) and 95% confidence interval (CI) width and statistical significance, with no selection of metaanalyses based upon the number of RCTs per metaanalysis. As a byproduct of MVMA coefficient estimation, variable correlations, direct information and borrowing of strength (BoS) were determined. Whereas direct information describes the contribution of data from the same outcome, BoS represents the contribution of data from all other outcomes [10, 11]. One problematic requirement of MVMA is the provision of withstudy correlations which are rarely reported, although methods based upon individual patient [12] or aggregated data [13] and within the Bayesian framework [14] have been undertaken. Any recommendation for the practical application of MVMA must be accompanied by appropriate software. As such, the “alternative” MVMA model of Riley [15] was employed, whereby an overall correlation, the total marginal correlation between outcomes, was modelled, enabling seamless application to all metaanalyses considered. As results based upon indices of risk, odds ratio (OR) and risk ratio (RR), are not generally intertranslatable [16], both OR and RR estimates were compared.
Methods
Ethics
The data for this study was abstracted from published studies and an Ethics clearance was not appropriate.
Data management
A previous study [9] was used to identify metaanalyses with either one or two secondary outcomes that provided sufficient detail to generate a bivariate or trivariate MVMA data structure, with mortality as the primary outcome; all metaanalyses were of randomised controlled trials (RCT). Usable second and third outcomes were identified as presented in the original metaanalysis.
Statistical analysis

1.
To facilitate rapid data processing over a large number of models, initial univariate metaanalytic point estimates and standard errors (SE) were computed within Stata™ V17 [17] using the “meta” suite of commands [18]; default estimation used restricted maximum likelihood (REML [19]).

2.
Subsequently, both univariate and multivariate outcomes were estimated using the user written Stata command “mvmeta” ([20], Version 3.2.0 6apr2018) in a random effects (RE) formulation. Estimation employed REML with an unstructured covariance and the BroydenFletcherGoldfarbShanno (BFGS) algorithm for likelihood maximisation or the DavidonFletcherPowell (DFP) algorithm if there were convergence difficulties. Maximisation employed the “difficult” option (use a different stepping algorithm in nonconcave regions) provided by Stata™.

(i)
The two sets of univariate estimates were subsequently compared.

(ii)
Under persistent convergence difficulties of “mvmeta”, the model was refit assuming the overall correlation matrix was fixed and known, with values set equal to the estimates from either the BGFS or DFP algorithm using the “bscovariance” option of “mvmeta” ([8], Appendix 4).

(iii)
In the MVMA note was taken of very small β coefficient standard errors (SE) with consequent large z values for coefficient significance and very small pvalues and CI width, such that the estimates were implausible.

(iv)
To avoid the requirement for specific within study correlations [1], the “alternative” model of Riley was used [15], whereby an overall correlation, the total marginal correlation between outcomes [21], was modelled; that is an amalgam of the within and betweenstudy correlations [6, 8]. The reported correlation(s) in this paper were these overall correlation(s) [22].

(v)
Direct information and BoS between estimates were also reported [8, 10, 11], using the default (“sd”) method of “mvmeta”. BoS may be conceptualised as a comparison of variances of the estimated rth component of β under the uni and multivariate models \(BoS_{r}^{{RV}} = 1  \frac{{{\text{var}}\left( {\hat{\beta }_{{mv,r}} } \right)}}{{{\text{var}}\left( {\hat{\beta }_{{uv,r}} } \right)}}\), where RV refers to relative variance [11]. This ratio has also been described as the efficiency, “E” [10, 23]. An equivalent but alternative method, decomposition of the score function for β, has advantage in that it defines appropriate study weights within an MVMA [11]. In a univariate RE metaanalysis study weights are inversely proportional to the sum of the within and between study variances. In an MVMA analysis, as undertaken by “mvmeta”, weights were derived using the score decompensation method, where the score function \(S\left( \theta \right)\) is the first derivative of the loglikelihood function \(l\left( \theta \right);\)\(S\left( \theta \right) = \frac{{dl\left( \theta \right)}}{{d\theta }}\) and \(l(\theta )\) is the likelihood. The weights were broken down into direct information, the contribution of data from the same outcome, and BoS, the contribution of data for all other outcomes. For a univariate analysis, the weights sum to 1, or when expressed as a percentage, 100, as in the “mvmeta” output. In a MVMA, a simple tabulation of direct information and BoS will sum to 100 for each outcome. In particular, the methodology takes the variance components as fixed and the precisions of the point estimates from a MVMA have an expectation of being greater than or equal to those from separate univariate metaanalyses [24], albeit the latter study employed the methods of Van Houwelingen et al. [25] using “Proc Mixed’’ with SAS statistical software, not the “Riley” method [15].

a.
The use of the “Riley” method [15] excluded the computation of multivariate I^{2} for each outcome.

a.

(i)

3.
The reported confounding effect of small study effects upon changes of statistical significance between univariate and MVMA [5, 8] was explored by inspection of contourenhanced funnel plots [26] and formal regression based tests, in particular, the Harbord (for binary outcomes) and Egger (for continuous outcomes) tests [27]. Small study effects were reported for all metaanalyses, as a matter of complete reporting, but suffer from the problem of multiple testing. The power and interpretation of the tests are problematic for small RCT number (< 10) metaanalyses and in the presence of moderate (see 4., below) heterogeneity [28, 29]. More importantly, univariate tests may be underpowered compared with the recently described multivariate small study effect test (MSSET), a multivariate extension of Egger’s regression test [30].

4.
Metaanalytic heterogeneity was reported as the I^{2} index and adjudged as medium and high if I^{2} ≥ 50 and 75% respectively. The I^{2} index was preferred, compared with τ^{2}, as it is comparable across different metrics and number of RCTs [31].

5.
The analyses, using frequentist methods, were performed for bivariate and trivariate models with both OR and RR metrics.

6.
Agreement or otherwise between univariate and multivariate estimation results was undertaken using Lin’s concordance correlation coefficient (CCC) via the user written Stata command “concord” [32]. The CCC combines measures of both precision and accuracy to determine how far the observed data deviate from the line of perfect concordance (that is, the line at 45 degrees on a square scatterplot). Other measure to characterise the comparison were:

(i)
Estimate differences average and standard deviation (SD); univariate versus MVMA.

(ii)
95% (Bland and Altman) limits of agreement (LOA)

(iii)
An F test (BradleyBlackwood) of equality of means and variances; nonsignificance implies concordance.

(i)

7.
Boxplots [33] were used to visualise the density distribution of BOS and total marginal correlations of both OR and RR for bivariate and trivariate models.
Statistical significance was ascribed at p < 0.05.
Results
The cohort was composed of fortynine metaanalyses, 18% nutritional therapeutic, 18% nonpharmaceutical therapeutic and 64% pharmaceutical therapeutic, published between 2002 and 2018. The primary outcome in all was mortality; fortynine were bivariate in outcome data composition and 30 were trivariate. Details of the mortality, second and third outcome metaanalyses are shown in Tables 1, 2 and 3, respectively. Heterogeneity, as the I^{2} index, of ≥ 50 and ≥ 75% was found in mortality, second and third outcomes in 12, 31 and 50%, and 0, 16 and 23%, respectively. Of the 49 mortality metaanalyses, five [38, 43, 53, 60, 66] demonstrated evidence of small study effects on formal testing (p < 0.05); for the second outcome, five [37, 53, 59,60,61]; and for the third, five [37, 49, 67, 68, 78]. The disparity between the formal test of small study effects (p < 0.05) and the increased frequency of “query” for contourenhanced funnel plot assessment in second and third outcomes versus mortality outcome (37, 40 and 4%, respectively) was noted and may be a function of the power of the test (see Methods, 3.). There was uniform agreement (to the second or third decimal point) between univariate estimates of “mvmeta” and “meta” in Stata™.
Bivariate model: OR
For the 49 metaanalyses, median (minimum, p25, p75, maximum) number of RCT per metaanalysis for the primary mortality outcome was 13(4, 10, 17, 31); for the second outcomes, 8(4, 6, 10, 31); see Tables 1 and 2. In only 11 metaanalyses was there equality between the reported primary and secondary outcome study numbers. In the MVMA the “bscovariance” option was used once only and there were no instances of “large” Z values. Second outcomes were binary in 39 and continuous in 10 (Tables 1 and 2). Estimate analysis is given in Table 4. Across all outcomes and estimates, the concordance, univariate versus multivariate, was substantial, with a general relative increment, albeit uneven, in the magnitude of multivariate estimates. Means and variances demonstrated little concordance. Reversal of coefficient estimate significance, univariate versus MVMA, occurred no cases for mortality and 6 cases for second outcomes (significant to nonsignificant in five [36, 43, 70, 71, 79], one metaanalysis exhibiting small study effects [43]; nonsignificant to significant in in one [59]).
Bivariate model: RR
In the MVMA, 49 metaanalyses were considered and there were no instances of “large” Z values. Concordance estimate analysis is given in Table 5. Substantial concordance was seen between uni and multivariate estimates, with a variable relative increment of multivariate estimates (SE and CI width) across outcomes. Multivariate β estimates were variable with respect to univariate and means and variances lacked concordance. Reversal of coefficient estimate significance, univariate versus MVMA, occurred in one case for mortality outcome (significant to nonsignificant, [52]) and 3 cases for second outcomes (significant to nonsignificant [36, 61, 70]; one instance [61] was discordant with the OR metric and one instance exhibiting small study effects [61]).
The bivariate distributions of BoS are displayed in Fig. 1, where an increment of BoS for RR compared with OR, for both mortality and the second outcome is evident.
The bivariate total marginal correlations, mortality vs second outcome, are shown in Fig. 2; both metrics displayed similar distribution.
Trivariate model: OR
For the 30 metaanalyses, the median (minimum, p25, p75, maximum) number of studies per metaanalysis for the primary mortality outcome was 13(4, 9, 16, 24); for the second outcome 8(4, 6, 10, 20); and the third 7(3, 5, 10, 15). In only 2 metaanalyses [37, 49] was there equality between the reported primary, second and third outcome study numbers. In the MVMA the “bscovariance” option was used on 13 occasions [37, 38, 46, 54, 57, 62, 66, 72, 73, 77, 78] and there were 5 instances of “large” Z values [37, 54, 72, 77, 78] which were sufficient to render estimates implausible and they were not further considered (median number of RCT per metaanalysis for primary, second and third outcomes 12, 6 and 5 respectively). The outcome data set was thus 25 metaanalyses. Second outcomes were binary in 18 and continuous in 7.; third outcomes were binary in 6 and continuous in 19; the “bscovariance” option being used in eight cases.
Concordance estimate analysis is given in Table 6. Variable concordance between uni and multivariate estimates was observed. Multivariate estimate precision (SE) increased, and confidence interval width tended to decrease compared with univariate, across and within outcomes. A tendency for concordance between means and variances was apparent. Reversal of coefficient estimate significance, univariate versus MVMA, occurred in two cases for mortality ([38, 73] nonsignificant to significant, one metaanalysis exhibiting small study effects [38]); nine cases for second outcomes (significant to nonsignificant in 3 [67, 69, 79], one metaanalysis exhibiting small study effects [67]; nonsignificant to significant in 6 [37, 57, 59, 62, 66, 73]) and 7 cases for third outcomes (significant to nonsignificant in 3 [40, 46, 67], one metaanalysis exhibiting small study effects [67]; nonsignificant to significant in 4 [37, 38, 69, 73] with one demonstrating small study effects [37]).
Trivariate model: RR
Of the 30 trivariate metaanalyses, there was one instance of complete convergence failure [46] and seven instances of “large” Z values [37, 38, 56, 66, 72, 73, 78] which were sufficient to render estimates implausible (median number of RCT per metaanalysis for primary, second and third outcomes 10, 6 and 5 respectively); the outcome data set was thus 22 metaanalyses [37, 40, 47, 49, 50, 53,54,55, 57, 59, 60, 62, 67,68,69, 71, 76, 77, 79, 80]. The median (minimum, maximum) number of studies per metaanalysis for the primary mortality outcome was 13(4, 24); for the second outcome, 8(4, 20); and the third 7(4, 15). Second outcomes were binary in 15 and continuous in 7; third outcomes were binary in 7 and continuous in 15. In the MVMA the “bscovariance” option was used on 3 occasions [57, 62, 67]. Concordance estimate analysis is given in Table 7. Concordance between uni and multivariate estimates was uneven, with no consistent relative change in multivariate estimates, compared with univariate, across or within outcomes. A tendency for concordance between means and variances in second and third outcomes was apparent. Reversal of coefficient estimate significance, univariate versus MVMA, occurred in two case for mortality ([37, 62] with no small study effects, nonsignificant to significant, not concordant with the OR cases); five cases for second outcomes with no small study effects (significant to nonsignificant in 2 [69, 79], concordant with OR cases; nonsignificant to significant in 3 [54, 57, 77]; concordant with one OR cases only [57]); and 3 cases ([54, 60, 67] nonsignificant to significant, one exhibiting small study effects [67]) for third outcomes with no concordance with OR cases.
The trivariate distributions of BoS are displayed using boxplots in Fig. 3. The increment of BoS for OR compared with RR for mortality and the third outcome is evident. In the panel (right top) showing BoS mortality RR there were points of large BoS for two MVMA metaanalyses, 99.3 and 93.6 [57, 62]. Both these MVMA utilized the “bscovariance” option of “mvmeta” as there was initial unresolved convergence. The estimated between study mortality variance was minimal for both (5.24e06 and 0.005, respectively) and the status of these estimates may be circumspect.
The trivariate total marginal correlations for both OR (left) and RR (right)are shown via boxplots in Fig. 4; with progressive movement to positive correlations from mortalitysecond outcome through secondthird outcome. Positive correlations appeared more frequent with the RR metric.
Discussion
It is easy to forget that the MVMA approach has a long history dating back to at least 1993 [81] and has subsequently been formally implemented in popular statistical software packages [82,83,84,85]. This being said, MVMA still appears rarely used by practitioners, a decade after a 2009 review by Riley [1]. From within the social science paradigm Becker, in 2000, pointed out that ignoring outcome dependence in metaanalysis will affect Type I error rates and precision and bias of estimates: “No reviewer should ever ignore dependence among study outcomes” [82]. In the current study the total marginal correlations for both bi and trivariate analyses was sizeable overall and, depending upon the composition of the nonprimary outcomes, more positive than negative and more so for the trivariate case.
One of the principal attractions of MVMA is estimation of the BoS between parameters, well demonstrated in Fig. 3. Most of the BoS would appear to derive from studies which are more “atypical” in design. In particular, the BoS of secondary outcomes of the ith study is a function of the withinstudy variance matrix \(\left( {V_{i} } \right)\) and the harmonic average \(\bar{V}\) of all the \(V_{i} {\text{s}}\). BoS can only arise if there are differences between the \(V_{i} {\text{s}}\); which would entail studies of”..substantive difference[s] in background and research methods…”, not simply different sample sizes [10]. The magnitude of outcome BoS would appear to be bounded by percentage of missing data for that outcome [6, 24], which in the current study was substantial (see Results). A percentage missingness of 30–35% of studies informing an outcome was found to result in a “more pronounced” BoS in one empirical study [14]. Any nexus between BoS and missingness requires a missing at random (MAR) assumption, as opposed to missing completely at random (MCAR) for univariate metaanalysis [21]. The notions of MAR and MCAR are well recognised in the biomedical literature [86], albeit inconsistency of usage has been documented [87]; in particular, the conflation of (non)”ignorable” and MAR [22]. Perhaps not surprisingly, within the domain of outcome reporting bias (ORB) [4], MVMA has been a method of choice to investigate the impact of ORB upon metaanalytic conclusions [22, 88].
Computationally, MVMA requires both within and betweenstudy correlations and the former are typically not known and are likely not to be available, especially in higher order (trivariate) models [24]. Riley provided four alternate methods to overcome these problems [1]; the most straightforward, yet laborious, being a sensitivity analysis by correlation imputation over the entire range (1 to + 1). Riley’s alternate model [15] has been found to have good asymptotic statistical properties compared with a fully hierarchical REML model, with known withinstudy correlations, and with separate univariate metaanalyses. The performance may be problematic when the overall correlation \(\left( {\hat{\rho }} \right)\) is very close to 1 or 1. In the current study, only two instances were found; in the bivariate RR MVMA, \(\hat{\rho }\) = 0.999 [75], and the trivariate OR MVMA, \(\hat{\rho }\) = 0.986 ([57], second versus third outcome); both MVMA utilised the “bscov” option. As the Riley model is a “working” model when the true data generating mechanism is a RE model, the standard variance estimates may not provide confidence interval coverage at the nominal level [21]. Complete failure of convergence in the current study was rare, occurring in one instance [46], but problematic SE estimation was exhibited in the trivariate series, 5 instances in OR metric and 7 in RR. This may relate to the small number of RCT in second and third outcomes (see Table 8), but these numbers were not substantially different compared with metaanalyses not demonstrating this feature, as shown in Table 8.
Frequentist and Bayesian empirical comparisons between univariate metaanalyses and MVMA have appeared in the literature [8, 14, 89,90,91,92,93] with results demonstrating similar (pooled) parameter estimates between the two analytic forms. However, papers by Riley and coworkers [1, 15, 24, 94], which included formal simulation studies, found advantage; a smaller standard error and meansquare error of pooled estimates, predicated upon the presence of missing data; again, assuming missing at random. That is, in the presence of complete data a bivariate analysis would not be expected to produce a gain in statistical efficiency. The extension to trivariate and higher order outcome data and the inability to provide within study correlations was thus identified as a “pressing research issue”; to wit, the “alternative” model of Riley [15]. Price et al. suggested that estimates of clinical and /or statistical conclusions from MVMA may occasionally differ from those from univariate analyses and observed, somewhat wryly, that any claimed discrepancy “…says more about the dangers of using concepts of statistical significance than it does the use of MVMA” [8]. The results from the current analysis were somewhat at odds with these sentiments and with the general results of bivariate studies, both empirical and simulation (see below), albeit the caution about the variance estimates of the Riley model, above, are noted. A variable change in the multivariate precision of primary mortality outcome estimates compared with univariate analysis was present for both bivariate and trivariate metaanalyses and for metric. For second outcomes, precision tended to decrease, and CI width increase for bivariate metaanalyses; for third outcomes, precision increased, and CI width decreased. The latter finding appears not to have been previously reported although analytic reports of the trivariate structure are rare; one case only reported by Price et al. study [8] and two by Trikalinos et al. [14]. With respect to the observed relative changes (univariate versus multivariate) across four concordance analyses, the magnitude of the difference was rather small and accompanied by a more substantial SD, suggesting a heterogeneity of the MVMA effect, grounded in the individual metaanalyses and dependent upon the nature of the outcome, binary or continuous. As MVMA allows for correlation between outcomes, CIs may be wider on the basis of increased betweenstudy variance [8], but this was observed only in the bivariate case in the current analysis. The experience of Price et al. that “MVMA methods can be applied only in a minority of reviews of interventions in pregnancy and childbirth” [8] was not consistent with the current study.
A reviewer pointed to the wide LOA of the β estimates for the second continuous outcome (days) in Table 6 (trivariate OR MVMA), this being 10.894, 8.122. Of the seven metaanalyses considered, two had standout differences between univariate and MVMA estimates; the study of Wan et al. ([57], intrametaanalytic study number = 4), 11.31, 18.17 and Chen et al. ([69],intrametaanalytic study number = 10), 11.27 and 4.26. The former study used the “bscov” option, recording a BOS for the second outcome of 54% and correlation between second and third outcomes of 0.986; the latter had normal convergence but record a BOS for the second outcome of 92.4% with a correlation between second and third outcomes of 0.995. This may be indicative of problematic estimation, which has been mentioned above and further addressed in “Limitations”, below. Trikalinos et al. ([14], point 4.1), using Bayesian methods, observed that “Generally, point estimates are comparable”; Price et al. ([8], Table 2) using “mvmeta” recorded differences in β between univariate and MVMA, but did not focus attention on such; and in the bivariate simulation study of Riley et al. ([94], Table 4), bias of the mean for \(\beta _{1} \,{\text{and}}\,\beta _{2}\) was comparable with coverage for both between 93–98%; similar results were also observed when considering the “alternate” model of Riley ([15], Table 1).
The differences between the results of the current study and those referenced above [8, 14, 89,90,91,92,93] needs some further explication with regard to data structure. The bi and trivariate metaanalyses under consideration were relatively conventional; a primary mortality outcome and second and third recorded outcomes which were not direct extensions of the primary outcome. For second and third outcomes, both categorical (binary) and continuous outcomes were considered, unlike Trikalinos et al. [14] where outcomes were categorical. No repeated measures of a primary outcome, such as different mortality timepoints or different types of mortality (all cause or disease specific) were considered; the latter structure featured in the studies of Trikalinos et al. [14, 91], Arends [93] and also in an empirical example Riley et al. [15]. Within the critical care domain the use of MVMA analysis with different mortality timepoints has been recently presented [95].The current study did not focus on the impact of different metaanalytic estimators as in Berkey et al. [92], generalized least squares and multivariate maximum likelihood, nor adopt the Bayesian framework of Trikalinos [14]. That bivariate models have been used in systematic reviews of diagnostic test studies for some years, was noted in 2009 by Riley and both Simel et al. [90] and Dahabreh et al. [89] found little advantage for bivariate approaches when considering estimates of sensitivity and specificity. With respect to the change of estimate significance reported here, univariate versus MVMA, the use of the MVMA “bscov” option may have been consequential. For the OR metric, where 24 significance changes occurred, there were seven instances [37, 46, 57, 62, 66, 69, 73], all in trivariate MVMA. For the RR metric, again with the trivariate data structure, there were three [57, 62, 67].
These changes of statistical significance are shown in forest plots as couplets, univariate versus MVMA, for binary (null line unity, Fig. 5) and continuous (null line zero, Fig. 6) outcomes. A majority of the CI width changes that achieved a change of significance about the null appear substantial; the clinical import of such changes would require case by case determination [96].
Disparities between the OR and RR occurred over a range of indices and may be a function of the current cohort. However, OR and RR are not merely interchangeable metrics and there is no monotone relationship between them [16]. Recent papers have drawn attention to potential estimation problems with the RR. First, the RR effect magnitude is dependent upon the underlying baseline prevalence, shifting toward 1 as prevalence increases, and is a ratio of two conditional probabilities, whereas the OR is a likelihood ratio whose magnitude reflects the fold increase in odds, baseline to intervention, independent of prevalence [97]. Second, under both the DerSimonianLaird [98] and REML formulations, the requirements of log(RR) estimation to be compatible with study level event rates in the [0,1] interval \((\pi _{j} {\text{treat}}\,{\text{ < }}\,{\text{1}}\,{\text{and 0}}\, < \,\pi _{j} {\text{control}}\,{\text{ < }}\,{\text{1)}}\) demand restriction on the parameter space with ensuant bias in estimates of both \(\tau ^{2}\) and log(RR). Thus risk relativism may be an “illusion “ [97] and the OR “appears to be a safer option” [99]. This being said, Xiao and colleagues argued that interpretability issues of the OR, lack of collapsibility and a dependence on the baseline risk, negates any inprinciple recommendation for the OR [100].
Limitations
The current study utilised a single metaanalytic cohort from the critical care domain and had a modest number of bivariate metaanalyses, but less so in the trivariate series. The preference for the alternate model of Riley was a potential limitation, but when reviewing a number of bivariate and trivariate studies in two metrics the use of sensitivity analysis by specifying within study correlations (via the “wscor” option of “mvmeta”) would be unwieldy and potentially uninterpretable. This being said, the recommendation of Riley et al. in the landmark 2008 paper [15], was that in the presence of overall correlations > 0.9 in absolute value, practitioners “..should assess the robustness of pooled results to small changes in \(\hat{\rho }\) as a sensitivity analysis”. In the MVMA where large z values were found and subsequently not considered, for the OR studies [37, 54, 72, 77, 78] and for the RR studies [37, 38, 56, 66, 72, 73, 78], all metaanalyses had \(\hat{\rho }\) > 0.9 in at least one of the correlations. Whether such a modus operandi would yield credible z values and pooled estimates has not been explored.
The current study has adopted a workable and practical solution to the particular requirements of MVMA. Future studies should replicate or otherwise the findings in this paper using the “alternate” metaanalytic model of Riley and consider metaanalyses from specific disciplines, moving beyond the bivariate data structure to encompass “…three or more end points…” [1], albeit such estimation may be challenging.
Conclusions
MVMA elucidates the structure and correlation between multiple reported outcomes in univariate metaanalyses and resolves outcome reporting bias. Change in estimate precision and CI width with MVMA appeared context dependent. The BoS entailed in this technique may be quantified and change of parameter significance may be a consequence. MVMA is a feasible solution to the metaanalytic estimation of multiple univariate effects.
Availability of data and materials
The collection of the original data set (current reference [9]) from published papers was the joint work of the author (John L Moran) and Dr Petra Graham (see below) and ownership of the data resides with both persons and is thus not available in the public domain, nor in repository.
References
 1.
Riley RD. Multivariate metaanalysis: the effect of ignoring withinstudy correlation. J R Stat Soc Ser A Stat Soc. 2009;172(4):789–811.
 2.
LopezLopez JA, Page MJ, Lipsey MW, Higgins JPT. Dealing with effect size multiplicity in systematic reviews and metaanalyses. Res Synth Methods. 2018;9(3):336–51.
 3.
Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias G. Systematic review of the empirical evidence of study publication bias and outcome reporting bias  an updated review. PLoS One. 2013;8(7):e66844.
 4.
Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340(feb15_1):c365.
 5.
Jackson D, Riley R, White IR. Multivariate metaanalysis: Potential and promise. Stat Med. 2011;30(20):2481–98.
 6.
Riley RD, Jackson D, Salanti G, Burke DL, Price M, Kirkham J, White IR. Multivariate and network metaanalysis of multiple outcomes and multiple treatments: rationale, concepts, and examples. BMJ. 2017;358:j3932.
 7.
Jackson D, White IR, Riley RD. Multivariate metaanalysis. In: Schmid CH, Stijnen T, White IR, editors. Handbook of metaanalysis. Boca Raton: CRC Press; 2021. p. 163–86.
 8.
Price MJ, Blake HA, Kenyon S, White IR, Jackson D, Kirkham JJ, Neilson JP, Deeks JJ, Riley RD. Empirical comparison of univariate and multivariate metaanalyses in Cochrane Pregnancy and Childbirth reviews with multiple binary outcomes. Res Synth Methods. 2019;10(3):440–51.
 9.
Moran JL, Graham PL. Risk related therapy in metaanalyses of critical care interventions: Bayesian metaregression analysis. J Crit Care. 2019;53:114–9.
 10.
Copas JB, Jackson D, White IR, Riley RD. The role of secondary outcomes in multivariate meta‐analysis. J R Stat Soc Ser C Appl Stat. 2018;67(5):1177–205.
 11.
Jackson D, White IR, Price M, Copas J, Riley RD. Borrowing of strength and study weights in multivariate and network metaanalysis. Stat Methods Med Res. 2017;26(6):2853–68.
 12.
Riley RD, Price MJ, Jackson D, Wardle M, Gueyffier F, Wang J, Staessen JA, White IR. Multivariate metaanalysis using individual participant data. Res Synth Methods. 2015;6(2):157–74.
 13.
Wei Y, Higgins JP. Estimating withinstudy covariances in multivariate metaanalysis with multiple outcomes. Stat Med. 2013;32(7):1191–205.
 14.
Trikalinos TA, Hoaglin DC, Schmid CH. An empirical comparison of univariate and multivariate metaanalyses for categorical outcomes. Stat Med. 2014;33(9):1441–59.
 15.
Riley RD, Thompson JR, Abrams KR. An alternative model for bivariate randomeffects metaanalysis when the withinstudy correlations are unknown. Biostatistics. 2008;9(1):172–86.
 16.
Feng C, Wang B, Wang H. The relations among three popular indices of risks. Stat Med. 2019;38(23):4772–87.
 17.
StataCorp. Stata release 17. 2021. https://www.stata.com/products/.
 18.
StataCorp. Stata metaanalysis reference manual: release 17. 2021. https://www.statapress.com/manuals/metaanalysisreferencemanual/.
 19.
Partlett C, Riley RD. Random effects metaanalysis: coverage performance of 95% confidence and prediction intervals following REML estimation. Stat Med. 2017;36(2):301–17.
 20.
White IR. Multivariate randomeffects metaregression: updates to mvmeta. Stata J. 2011;11(2):255–70.
 21.
Hong C, Riley RD, Chen Y. An improved method for bivariate metaanalysis when withinstudy correlations are unknown. Res Synth Methods. 2018;9(1):73–88.
 22.
Kirkham JJ, Riley RD, Williamson PR. A multivariate metaanalysis approach for reducing the impact of outcome reporting bias in systematic reviews. Stat Med. 2012;31(20):2179–95.
 23.
Lin L, Xing A, Kofler MJ, Murad MH. Borrowing of strength from indirect evidence in 40 network metaanalyses. J Clin Epidemiol. 2018;106:41–9.
 24.
Riley RD, Abrams KR, Lambert PC, Thompson JR. An evaluation of bivariate randomeffects metaanalysis for the joint synthesis of two correlated outcomes. Stat Med. 2007;26(1):78–97.
 25.
Van Houwelingen HC, Arends LR, Stijnen T. Advanced methods in metaanalysis: multivariate approach and metaregression. Stat Med. 2002;21(4):589–624.
 26.
Palmer TM, Peters JL, Sutton AJ, Moreno SG. Contourenhanced funnel plots for metaanalysis. Stata J. 2008;8(2):242–54.
 27.
StataCorp: meta bias. Stata V17 documentation. 2021. https://www.stata.com/manuals/metametabias.pdf. Accessed 4 May 2021.
 28.
Ioannidis JP, Trikalinos TA. The appropriateness of asymmetry tests for publication bias in metaanalyses: a large survey. Can Med Assoc J. 2007;176(8):1091–6.
 29.
Sterne JA, Gavaghan D, Egger M. Publication and related bias in metaanalysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol. 2000;53(11):1119–29.
 30.
Hong C, Salanti G, Morton SC, Riley RD, Chu H, Kimmel SE, Chen Y. Testing small study effects in multivariate metaanalysis. Biometrics. 2020;76(4):1240–50.
 31.
HuedoMedina TB, SánchezMeca J, MarinMartinez F, Botella J. Assessing heterogeneity in metaanalysis: Q statistic or I^{2} index? Psychol Methods. 2006;11(2):193–206.
 32.
Steichen TJ, Cox NJ. A note on the concordance correlation coefficient. Stata J. 2002;2(2):183–9.
 33.
Cleveland WS. Graphical methods. In: The elements of graphing data. edn. Summit: Hobart Press; 1994. p. 139–143.
 34.
Griesdale DEG, de Souza RJ, van Dam RM, Heyland DK, Cook DJ, Malhotra A, Dhaliwal R, Henderson WR, Chittock DR, Finfer S, et al. Intensive insulin therapy and mortality among critically ill patients: a metaanalysis including NICESUGAR study data. Can Med Assoc J. 2009;180(8):821–7.
 35.
Annane D, Bellissant E, Bollaert PE, Briegel J, Confalonieri M, De Gaudio R, Keh D, Kupfer Y, Oppert M, Meduri GU. Corticosteroids in the treatment of severe sepsis and septic shock in adults: a systematic review. JAMA. 2009;301(22):2362–75.
 36.
Bangalore S, Wetterslev J, Pranesh S, Sawhney S, Gluud C, Messerli FH. Perioperative β blockers in patients having noncardiac surgery: a metaanalysis. Lancet. 2008;372(9654):1962–76.
 37.
Marik P, Zaloga G. Immunonutrition in critically ill patients: a systematic review and analysis of the literature. Intensive Care Med. 2008;34(11):1980–90.
 38.
Chan EY, Ruest A, Meade MO, Cook DJ. Oral decontamination for prevention of pneumonia in mechanically ventilated adults: systematic review and metaanalysis. Br Med J. 2007;334(7599):889.
 39.
Gonzalez R, Zamora J, GomezCamarero J, Molinero LM, Banares R, Albillos A. Metaanalysis: combination endoscopic and drug therapy to prevent variceal rebleeding in cirrhosis. Ann Intern Med. 2008;149(2):109–22.
 40.
Ho KM, Dobb GJ, Webb SAR. A comparison of early gastric and postpyloric feeding in critically ill patients: a metaanalysis. Intensive Care Med. 2006;32(5):639–49.
 41.
Ho KM, Ng JY. The use of propofol for medium and longterm sedation in critically ill adult patients: a metaanalysis. Intensive Care Med. 2008;34(11):1969–79.
 42.
Siempos II, Ntaidou TK, Falagas ME. Impact of the administration of probiotics on the incidence of ventilatorassociated pneumonia: a metaanalysis of randomized controlled trials. Crit Care Med. 2010;38(3):954–62.
 43.
Singh S, Amin AV, Loke YK. Longterm use of inhaled corticosteroids and the risk of pneumonia in chronic obstructive pulmonary disease a metaanalysis. Arch Intern Med. 2010;169(3):219–29.
 44.
Peterson K, Carson S, Carney N. Hypothermia treatment for traumatic brain injury: a systematic review and metaanalysis. J Neurotrauma. 2008;25(1):62–71.
 45.
Silvestri L, van Saene HKF, Milanese M, Gregori D, Gullo A. Selective decontamination of the digestive tract reduces bacterial bloodstream infection and mortality in critically ill patients. Systematic review of randomized, controlled trials. J Hosp Infect. 2007;65(3):187–203.
 46.
Whitlock RP, Chan S, Devereaux PJ, Sun J, Rubens FD, Thorlund K, Teoh KHT. Clinical benefit of steroid use in patients undergoing cardiopulmonary bypass: a metaanalysis of randomized trials. Eur Heart J. 2008;29(21):2592–600.
 47.
Piccini JP, Berger JS, O’Connor CM. Amiodarone for the prevention of sudden cardiac death: a metaanalysis of randomized controlled trials. Eur Heart J. 2009;30(10):1245–53.
 48.
Landoni G, Mizzi A, BiondiZoccai G, Bignami E, Prati P, Ajello V, Marino G, Guarracino F, Zangrillo A. Levosimendan reduces mortality in critically ill patients. A metaanalysis of randomized controlled studies. Minerva Anestesiol. 2010;76(4):276–86.
 49.
Brar SS, Leon MB, Stone GW, Mehran R, Moses JW, Brar SK, Dangas G. Use of drugeluting stents in acute myocardial infarction a systematic review and metaanalysis. J Am Coll Cardiol. 2009;53(18):1677–89.
 50.
Landoni G, BiondiZoccai GGL, Tumlin JA, Bove T, De Luca M, Calabro MG, Ranucci M, Zangrillo A. Beneficial impact of Fenoldopam in critically ill patients with or at risk for acute renal failure: a metaanalysis of randomized clinical trials. Am J Kidney Dis. 2007;49(1):56–68.
 51.
Mazaki T, Ebisawa K. Enteral versus parenteral nutrition after gastrointestinal surgery: a systematic review and metaanalysis of randomized controlled trials in the English literature. J Gastrointest Surg. 2008;12(4):739–55.
 52.
Masip J, Roque M, Sanchez B, Fernandez R, Subirana M, Exposito JA. Noninvasive ventilation in acute cardiogenic pulmonary edema  systematic review and metaanalysis. JAMA. 2005;294(24):3124–30.
 53.
Oldani M, Sandini M, Nespoli L, Coppola S, Bernasconi DP, Gianotti L. Glutamine supplementation in intensive care patients a metaanalysis of randomized clinical trials. Medicine. 2015;94(31):e1319.
 54.
Szakmany T, Russell P, Wilkes AR, Hall JE. Effect of early tracheostomy on resource utilization and clinical outcomes in critically ill patients: metaanalysis of randomized controlled trials. Br J Anaesth. 2015;114(3):396–405.
 55.
Alkhawaja S, Martin C, Butler RJ, GwadrySridhar F. Postpyloric versus gastric tube feeding for preventing pneumonia and improving nutritional outcomes in critically ill adults. Cochrane Database Syst Rev. 2015;2015(8):CD008875.
 56.
van Zanten ARH, Dhaliwal R, Garrel D, Heyland DK. Enteral glutamine supplementation in critically ill patients: a systematic review and metaanalysis. Crit Care. 2015;19(1):294.
 57.
Wan X, Gao XJ, Bi JC, Tian F, Wang XY. Use of n3 PUFAs can decrease the mortality in patients with systemic inflammatory response syndrome: a systematic review and metaanalysis. Lipids Health Dis. 2015;14(1):1–9.
 58.
Teo J, Liew Y, Lee W, Kwa ALH. Prolonged infusion versus intermittent boluses of betalactam antibiotics for treatment of acute infections: a metaanalysis. Int J Antimicrob Agents. 2014;43(5):403–11.
 59.
Manzanares W, Dhaliwal R, Jiang XR, Murch L, Heyland DK. Antioxidant micronutrients in the critically ill: a systematic review and metaanalysis. Crit Care. 2012;16(2):1–3.
 60.
Tian FMD, Heighes PTMPS, Allingstrup MJP, Doig GSP. Early enteral nutrition provided within 24 hours of ICU admission: a metaanalysis of randomized controlled trials*. Crit Care Med. 2018;46(7):1049–56.
 61.
Rhodes NJ, Liu J, O’Donnell JN, Dulhunty JM, AbdulAziz MH, Berko PY, Nadler B, Lipman J, Roberts JA. Prolonged infusion piperacillintazobactam decreases mortality and improves outcomes in severely ill patients: results of a systematic review and metaanalysis. Crit Care Med. 2018;46(2):236–43.
 62.
NunezPatino RA, ZorrillaVaca A, RiveraLara L. Comparison of intensive versus conventional insulin therapy in traumatic brain injury: a metaanalysis of randomized controlled trials. Brain Inj. 2018;32(6):693–703.
 63.
KawanoDourado L, Zampieri FG, Azevedo LCP, Correa TD, Figueiro M, Semler MW, Kellum JA, Cavalcanti AB. Lowversus highchloride content intravenous solutions for critically ill and perioperative adult patients: a systematic review and metaanalysis. Anesth Analg. 2018;126(2):513–21.
 64.
Dallimore J, Ebmeier S, Thayabaran D, Bellomo R, Bernard G, Schortgen F, Saxena M, Beasley R, Weatherall M, Young P. Effect of active temperature management on mortality in intensive care unit patients. Crit Care Resusc. 2018;20(2):150–63.
 65.
Chong MA, Krishnan R, Cheng D, Martin J. Should transfusion trigger thresholds differ for critical care versus perioperative patients? A metaanalysis of randomized trials. Crit Care Med. 2018;46(2):252–63.
 66.
Yang XM, Tu GW, Zheng JL, Shen B, Ma GG, Hao GW, Gao J, Luo Z. A comparison of early versus late initiation of renal replacement therapy for acute kidney injury in critically ill patients: an updated systematic review and metaanalysis of randomized controlled trials. BMC Nephrol. 2017;18(1):264.
 67.
Osadnik CR, Tee VS, CarsonChahhoud KV, Picot J, Wedzicha JA, Smith BJ. Noninvasive ventilation for the management of acute hypercapnic respiratory failure due to exacerbation of chronic obstructive pulmonary disease. Cochrane Database Syst Rev. 2017;7(7):CD004104.
 68.
Lu C, Sharma S, McIntyre L, Rhodes A, Evans L, Almenawer S, Leduc L, Angus DC, Alhazzani W. Omega3 supplementation in patients with sepsis: a systematic review and metaanalysis of randomized trials. Ann Intensive Care. 2017;7(1):58.
 69.
Chen QH, Zheng RQ, Lin H, Shao J, Yu JQ, Wang HL. Effect of levosimendan on prognosis in adult patients undergoing cardiac surgery: a metaanalysis of randomized controlled trials. Crit Care. 2017;21(1):253.
 70.
Qureshi SH, Rizvi SI, Patel NN, Murphy GJ. Metaanalysis of colloids versus crystalloids in critically ill, trauma and surgical patients. Br J Surg. 2016;103(1):14–26.
 71.
Elke G, van Zanten ARH, Lemieux M, McCall M, Jeejeebhoy KN, Kott M, Jiang X, Day AG, Heyland DK. Enteral versus parenteral nutrition in critically ill patients: an updated systematic review and metaanalysis of randomized controlled trials. Crit Care. 2016;20(1):117.
 72.
Parikh HG, Miller A, Chapman M, Moran JL, Peake SL. Calorie delivery and clinical outcomes in the critically ill: a systematic review and metaanalysis. Crit Care Resusc. 2016;18(1):17–24.
 73.
Davies ML, Chapple LAS, Chapman MJ, Moran JL, Peake SL. Protein delivery and clinical outcomes in the critically ill: a systematic review and metaanalysis. Crit Care Resusc. 2017;19(2):117–27.
 74.
Abroug F, Ouanes I, Abroug S, Dachraoui F, Ben Abdallah S, Hammouda Z, OuanesBesbes L. Systemic corticosteroids in acute exacerbation of COPD: a metaanalysis of controlled studies with emphasis on ICU patients. Ann Intensive Care. 2014;4:32.
 75.
Manzanares W, Lemieux M, Langlois PL, Wischmeyer PE. Probiotic and synbiotic therapy in critical illness: a systematic review and metaanalysis. Crit Care. 2016:262.
 76.
Peter JV, Moran JL, PhillipsHughes J, Warne D. Noninvasive ventilation (NIV) in acute respiratory failure: a metaanalysis update. Crit Care Med. 2002;30(3):555–62.
 77.
Wang CH, Ma MHM, Chou HC, Yen ZS, Yang CW, Fang CC, Chen SC. Highdose vs non highdose proton pump inhibitors after endoscopic treatment in patients with bleeding peptic ulcer a systematic review and metaanalysis of randomized controlled trials. Arch Intern Med. 2010;170(9):751–8.
 78.
Tao W, Li PS, Shen Z, Shu YS, Liu S. Effects of omega3 fatty acid nutrition on mortality in septic patients: a metaanalysis of randomized controlled trials. BMC Anesthesiol. 2016;16:39.
 79.
Tang H, Huang T, Jing J, Shen H, Cui W. Effect of procalcitoninguided treatment in patients with infections: a systematic review and metaanalysis. Infection. 2009;37(6):497–507.
 80.
Muscedere J, Rewa O, McKechnie K, Jiang XR, Laporta D, Heyland DK. Subglottic secretion drainage for the prevention of ventilatorassociated pneumonia: a systematic review and metaanalysis. Crit Care Med. 2011;39(8):1985–91.
 81.
Vanhouwelingen HC, Zwinderman KH, Stijnen T. A bivariate approach to metaanalysis. Stat Med. 1993;12(24):2273–84.
 82.
Becker BJ. Multivariate metaanalysis. In: Tinsley HEA, Brown SD, editors. Handbook of applied multivariate statistics and mathematical modeling. San Diego: Academic; 2000. p. 499–525.
 83.
Gasparrini A, Armstrong B, Kenward MG. Multivariate metaanalysis for nonlinear and other multiparameter associations. Stat Med. 2012;31(29):3821–39.
 84.
Luo S, Chen Y, Su X, Chu H. mmeta: an R package for multivariate metaanalysis. J Stat Softw. 2014;56(11):26.
 85.
White IR. Multivariate randomeffects metaanalysis. Stata J. 2009;9(1):40–56.
 86.
Vesin A, Azoulay E, Ruckly S, Vignoud L, Rusinova K, Benoit D, Soares M, AzeivedoMaia P, Abroug F, Benbenishty J, et al. Reporting and handling missing values in clinical studies in intensive care units. Intensive Care Med. 2013;39(8):1396–404.
 87.
Seaman S, Galati J, Jackson D, Carlin J. What is meant by “missing at random”? Stat Sci. 2013;28(2):257–68.
 88.
Frosi G, Riley RD, Williamson PR, Kirkham JJ. Multivariate metaanalysis helps examine the impact of outcome reporting bias in Cochrane rheumatoid arthritis reviews. J Clin Epidemiol. 2015;68(5):542–50.
 89.
Dahabreh IJ, Trikalinos TA, Lau J, Schmid CH. Univariate and bivariate likelihoodbased metaanalysis methods performed comparably when marginal sensitivity and specificity were the targets of inference. J Clin Epidemiol. 2017;83:8–17.
 90.
Simel DL, Bossuyt PMM. Differences between univariate and bivariate models for summarizing diagnostic accuracy may not be large. J Clin Epidemiol. 2009;62(12):1292–300.
 91.
Trikalinos TA, Olkin I. Metaanalysis of effect sizes reported at multiple time points: a multivariate approach. Clin Trials. 2012;9(5):610–20.
 92.
Berkey CS, Hoaglin DC, AntczakBouckoms A, Mosteller F, Colditz GA. Metaanalysis of multiple outcomes by regression with random effects. Stat Med. 1998;17(22):2537–50.
 93.
Arends L. Multivariate metaanalysis: modelling the heterogeneity. PhD Disseration. The Netherlands: Erasmus University; 2006. https://core.acu.k/download/pdf/19187601.pdf, downloaded 4th June 2016.
 94.
Riley R, Abrams K, Sutton A, Lambert P, Thompson J. Bivariate randomeffects metaanalysis and the estimation of betweenstudy correlation. BMC Med Res Methodol. 2007;7(1):3.
 95.
Moran JL, Graham PL. Multivariate metaanalysis of the mortality effect of prone positioning in the acute respiratory distress syndrome. J Intensive Care Med. 2021:08850666211014479. https://doi.org/10.1177/08850666211014479.
 96.
Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, p values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31(4):337–50.
 97.
Doi SA, FuruyaKanamori L, Xu C, Lin L, Chivese T, Thalib L. Questionable utility of the relative risk in clinical research: a call for change to practice. J Clin Epidemiol. 2020. https://www.jclinepi.com/article/S08954356(20)311719/fulltext, Downloaded 19th November 2020.
 98.
DerSimonian R, Laird N. Metaanalysis in clinical trials. Control Clin Trials. 1986;7(3):177–88.
 99.
Bakbergenuly I, Hoaglin DC, Kulinskaya E. Pitfalls of using the risk ratio in metaanalysis. Res Synth Methods. 2019;10(3):398–419.
 100.
Xiao M, Chen Y, Cole S, MacLehose R, Richardson D, Chu H. Is OR “portable” in metaanalysis? Time to consider bivariate generalized linear mixed model. medRxiv : the preprint server for health sciences 2020. https://doi.org/10.1101/2020.11.05.20226811.
Acknowledgements
Dr Petra Graham (Macquarie University, NSW Australia) for diligent help in assembling the original series of metaanalyses as reported in Reference [8].
Professor Ian White (Medical Research Council Clinical Trials Unit at University College London, UK) for advice on command syntax of his userwritten Stata module “mvmeta”.
Funding
Local Intensive Care Unit funds only.
Author information
Affiliations
Contributions
JLM is the sole author of this paper. The author(s) read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The data for this study was abstracted from published studies and an Ethics clearance was not appropriate.
Competing interests
The author declares no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Moran, J.L. Multivariate metaanalysis of critical care metaanalyses: a metaepidemiological study. BMC Med Res Methodol 21, 148 (2021). https://doi.org/10.1186/s12874021013364
Received:
Accepted:
Published:
Keywords
 Multivariate metaanalysis
 Critical care
 Random effects
 Borrowing of strength
 Metric