 Research
 Open access
 Published:
Connecting a disconnected trial network with a new trial: optimizing the estimation of a comparative effect in a network metaanalysis
BMC Medical Research Methodology volume 23, Article number: 79 (2023)
Abstract
Background
In network metaanalysis, estimation of a comparative effect can be performed for treatments that are connected either directly or indirectly. However, disconnected trial networks may arise, which poses a challenge to comparing all available treatments of interest. Several modeling approaches attempt to compare treatments from disconnected networks but not without strong assumptions and limitations. Conducting a new trial to connect a disconnected network can enable calculation of all treatment comparisons and help researchers maximize the value of the existing networks. Here, we develop an approach to finding the best connecting trial given a specific comparison of interest.
Methods
We present formulas to quantify the variation in the estimation of a particular comparative effect of interest for any possible connecting twoarm trial. We propose a procedure to identify the optimal connecting trial that minimizes this variation in effect estimation.
Results
We show that connecting two treatments indirectly might be preferred to direct connection through a new trial, by leveraging information from the existing disconnected networks. Using a real network of studies on the use of vaccines in the treatment of bovine respiratory disease (BRD), we illustrate a procedure to identify the best connecting trial and confirm our findings via simulation.
Conclusion
Researchers wishing to conduct a connecting twoarm study can use the procedure provided here to identify the best connecting trial. The choice of trial that minimizes the variance of a comparison of interest is network dependent and it is possible that connecting treatments indirectly may be preferred to direct connection.
Background
Network metaanalysis (NMA) enables estimation of comparative effects of treatments that are directly connected as well as those that are indirectly connected. Through such direct or indirect comparisons, researchers and clinicians are able to obtain comparisons of treatments available in an entire evidence network, where a network is defined as a collection of trials that compare some number of treatments for a given clinical outcome [1]. Networks are often referred to as graphs, which consist of nodes, e.g., treatments, and edges (or links) that represent direct comparisons between treatments. The studies included in a NMA of treatments are ideally randomized controlled trials identified as a result of a systematic review such that the included trials are consistent with the assumptions of an NMA. The transitivity assumption states that each subject in a trial must have been eligible for enrollment in all other trials. If this assumption is violated, then the estimates of the direct and indirect comparative effects may not be valid. Further, the consistency assumption states that pairwise comparisons between treatments must be able to be written as a function of the baseline treatment. Again, this assumption is vital for proper estimation of the direct and indirect comparative effects.
The evidence base for treatments is often driven by the interests of individual researchers and funding agencies that fail to consider how to maximize the value of the entire evidence base. As a result, networks of trials can be disconnected, in which there is neither a direct or indirect route to compare all treatments. Conversely, a connected network is one in which there is a path, or edge, “linking”, or connecting, every treatment to all others. Disconnected networks may arise when there is no common standard of treatment or when there are many available treatments [2]. Such networks pose difficulties when researchers wish to make comparisons between all possible treatments. Similarly, the issue of disconnected experimental design with respect to treatments has been studied [3], where the focus has been on checking and avoiding disconnected treatments in one single experiment.
There are several proposed approaches for dealing with disconnected networks in NMA. One approach relies on the use of nonrandomized evidence to connect the networks [4]. The use of nonrandomized evidence, often referred to as real world evidence (RWE), assumes that the expected response to a control treatment is constant between historical studies and the randomized studies [5]. This assumption is thought to be both unlikely and associated with bias [1]. Component network meta analysis (CNMA) has been proposed as an alternative [1]. While CNMA addresses the issues of using RWE, in terms of not relying on the aforementioned assumptions, the networks must be of a certain form. That is, the treatments must consist of common treatment components occurring in both networks (i.e., disconnected networks can be bridged together only if the networks are made up of multicomponent treatments that are common to each network). There are both additive and twoway interaction models for CNMA, but in order to connect a disconnected network, the authors note that at least some treatments must consist of components and the subnetworks need a “sufficient” number of common components. Another modeling approach to analyze disconnected networks is through random baseline effects [6]. This method has been found to be appropriate for two example data sets, but the authors note that there is a risk for the assumptions regarding the normality and exchangeability of the baseline treatments effects to be violated in other data sets. There are also several population adjustment methods for disconnected networks proposed [7] and in the case of doseresponse modeling, there are methods to make comparisons between treatments belonging to disconnected subnetworks [8]. All of the methods mentioned here show motivation for connecting disconnected networks, but all are limited in their own ways.
Given the strong reliance on assumptions in the aforementioned approaches, researchers may decide to design new studies that connect a disconnected network. While there are many methods that discuss how to design a trial within an existing connected network [9,10,11,12,13], we are unable to identify literature for designing a connecting trial for a disconnected network. In this paper, we formalise an approach to connecting two components of a disconnected network, which we refer to as subnetworks, with a new twoarm trial, based on an approach that minimizes the variance of a comparative effect estimation between two treatments of interest. The subnetworks themselves must meet the aforementioned assumptions of NMA. After deriving variance formulas for a particular effect size estimate under a connecting trial, we propose a straightforward computational procedure to guide researchers conducting a trial. We confirm these results through a simulation study. We conclude that given a comparison of treatments, the best connecting twoarm trial in terms of minimizing the estimation variation is network dependent and can be found through a straightforward computational procedure.
Methods
To consider how to best connect two subnetworks from a disconnected network, we focus on minimizing the variance of a specific comparative effect size between two treatments, which we call a comparison of interest (COI). Here, the term treatment is generic and could refer to any active intervention or a placebo. Our goal is to form a connected network, where a connected network is formally defined as one in which, for any two treatments (A,B), there exists an ordered sequence of treatments (\(t_1,t_2,\ldots ,t_k\)), k\(\le\)T2, where T is the total number of treatments, such that:

Treatments A and \(t_1\) are both included in one or more trials,

Treatments \(t_1\) and \(t_2\) are both included in one or more trials,

......,

Treatments \(t_k\) and B are both included in one or more trials.
For treatments (A,B) that are included in the same trial, the comparison is direct and the ordered sequence is empty. Within such a network, the difference in effects between any two treatments can be evaluated through the path “linking”, or connecting, the treatments. In this section, we establish formulas for the variation of a particular COI under several scenarios. The organization of the section is as follows. We review the variance estimation of a comparative effect size in a traditional twoarm trial. We then review general notation and variance estimation of a comparative effect under a fixed effects NMA model. Next, we establish the notation and properties of disconnected networks. Last, we derive variance formulas under the connection of two disconnected subnetworks using a twoarm trial.
Estimating the variance of a comparison of interest in a traditional twoarm trial
Suppose we conduct a twoarm trial with a binary outcome using treatments A and B, with a total fixed sample size n such that \(n_A + n_B = n\). Let \(r_i\) denote the number of events in the subjects belonging to treatment group i, \(i \in \{A,B\}\). Then the number of events \(r_i\) follows a binomial distribution; that is, \(r_i \sim \text {binomial}(n_i,p _i)\), where \(p _i\) is the probability of an event occurring in treatment i. Through a Bernoulli generalized linear model, we have
Here, \(\beta _0\) is the logodds of the probability of an event occurring in subjects in treatment group A, and \(\beta _1\) is the logodds ratio of treatment B to treatment A. We can estimate the coefficients \(\beta _0\) and \(\beta _1\) using a maximum likelihood approach and obtain the information matrix. The comparative effect of treatment B to A is of interest (\(\hat{\beta _1}\)), and it follows that
Let \(\text {Var}(\hat{\beta _1}) := \sigma ^2_{A,B}\) represent the estimated withintrial variance for the comparative effect size of treatment B to A. Note that in the context of NMA, \(\beta _1\) is written as \(\mu _{AB}\). We will utilize this derivation in our proposed procedure for connecting two disconnected subnetworks.
Fixed effects NMA model: notation and estimating the variance of a COI
Now, consider a network of T treatments with J studies, and \(n_j\) arms in the \(j^{th}\) study. Let \(\varvec{\mu }_b = (\mu _{AB}, \mu _{AC},\mu _{AD}, \ldots )'\) be the vector of comparative effect parameters of all treatments to the baseline treatment A. This is called the vector of basic parameters.
Let \(\varvec{y}_j\) denote the observed comparative effect size for the \(j^{th}\) study, \(\varvec{y}_j = (y_{j, 1}, \ldots , y_{j, n_{j 1}})^{\prime }\), and \(\varvec{y} = (\varvec{y}_1^{\prime }, \ldots , \varvec{y}_J^{\prime })^{\prime }.\) Let \(\varvec{\mu }_j\) be the vector of comparative effect sizes for the \(j^{th}\) study, \(\varvec{\mu }_j = (\mu _{j, 1}, \ldots , \mu _{j, n_{j 1}})^{\prime }\), and \(\varvec{\mu } = (\varvec{\mu }_1^{\prime }, \ldots , \varvec{\mu }_J^{\prime })^{\prime }.\) Then we have
where \(\varvec{\epsilon }_j\) is assumed to be normally distributed and independent across studies with covariance \(\varvec{S}_j\) corresponding to the estimated withintrial variances. The distribution of \(\varvec{y}\) is then MVN(\(\varvec{\mu }\), \(\varvec{S}\)), where \(\varvec{S}\) is a block diagonal matrix with each block \(\varvec{S}_j\). Since \(\varvec{\mu }\) is a linear combination of \(\varvec{\mu }_b\), it can be written as \(\varvec{\mu } = \varvec{X}\varvec{\mu }_b\), where \(\varvec{X}\) is the design matrix of size \(\sum _{j=1}^{J} n_j \times (T1)\). The distribution of \(\varvec{y}\) is then MVN(\(\varvec{X}\varvec{\mu }_b\), \(\varvec{S})\). The maximum likelihood estimate of \(\varvec{\mu _b}\) and its variance are
Similar notation and derivations are provided below for disconnected networks.
Notation and properties of disconnected networks
Suppose that the network of studies presented above is actually composed of K disconnected subnetworks. We can rewrite the vector of basic parameters as \(\varvec{\mu }_b = (\mu _{AB}, \mu _{AC}, \mu _{AD}, \ldots )' = (\varvec{\mu }_{D1}', \varvec{\mu }_{D2}', \ldots , \varvec{\mu }_{DK}')'\), where \(\varvec{\mu }_{Dk}'\) is the subvector corresponding to the basic parameters for treatments in the (disconnected) subnetwork k compared to the overall baseline treatment, A.
Since the K subnetworks are not connected, we can rewrite the design matrix \(\varvec{X}\) and variance matrix \(\varvec{S}\) as block diagonal matrices corresponding to the components from the K subnetworks, denoted by the subscripts Dk, for \(k = 1,\ldots ,K\):
where
and \(\varvec{y}_{Dk}\) are the observed comparative effects sizes for all studies in the kth subnetwork.
Then we have
With this notation, there are \(T1\) basic parameters. The \(\text {rank}(\varvec{X}) = T  K\), where K is the number of disconnected subnetworks. Thus, \(\hat{\varvec{\mu }}_b\) is not unique. Further, we propose the following lemma with regards to disconnected subnetworks.
Lemma 1
A set of K subnetworks are disconnected if and only if the design matrix for the entire network can be written in the form
with \(\varvec{\mu }_b = \left( \varvec{\mu }_{D1}, \varvec{\mu }_{D2}, \varvec{\mu }_{D3}, \ldots \varvec{\mu }_{DK}\right) '\).
Proof
\(\Leftarrow\) The design matrix is written so that each block corresponds to a subnetwork, Dk for \(k = 1, \ldots , K\). That is, estimates of comparative effect sizes from any given subnetwork Dk can be written as linear combinations of parameters unique to subnetwork Dk. Any subnetwork Dk then depends solely on its own parameters and not on parameters from any other network. Thus, it follows that the entire network must be disconnected.
\(\Rightarrow\) This follows directly from the setup above.
With the aforementioned general notation and properties of disconnected networks, we can now consider variance estimation of a comparative effect size of interest when connecting two disconnected subnetworks with a twoarm trial.
Estimating the variance of a comparison of interest under a new connecting twoarm trial
Consider a special case of the disconnected subnetworks above. That is, suppose that there are only \(K=2\) disconnected subnetworks. Researchers wish to connect these two disconnected subnetworks with the goal of estimating a specific comparative effect size, or COI, as precisely as possible.
We first consider the case when the connecting trial is also the COI. We define this as a direct trial. Intuitively, all of the information about the comparison should be captured by the new observed trial data, so the variance of the comparison is the new withintrial variance. That is, the connecting trial encompasses all of the evidence for the comparison as the variance of the comparison of interest is the variance of the estimate of the comparative effect from the connecting trial. We develop this idea more formally below.
Suppose that we have two existing, disconnected subnetworks, with network one consisting of treatments \(t_{11}, t_{12}, \ldots , t_{1m_1}\) and network two consisting of treatments \(t_{21}, t_{22},\ldots , t_{2m_2}\). Without loss of generality, consider \(t_{11}\) as the overall baseline treatment. We write the network model as:
where
and \(\varvec{\mu }_{D1}, \varvec{\mu }_{D2}\) correspond to the basic parameters with respect to the overall baseline treatment, \(t_{11}\). Connecting the two subnetworks with a study including the baseline treatment \(t_{11}\), say with treatment \(t_{21}\), gives us the model formulation:
where
Here, \(y_{t_{11},t_{21}}\) is the data from the new connecting trial and \(\sigma ^{2}_{t_{11}, t_{21}}\) is the new withintrial variance from the connecting trial. We partition \(\varvec{X}_{D2}\) columnwise into \(\varvec{X}_{D2_1}\) and \(\varvec{X}_{D2_2}\) and partition \(\varvec{\mu }_b\) such that the comparative effect size of interest is isolated. Then,
From this we see that finding \(\text {Var}(\hat{\mu }_{t_{11},t_{21}})\), which is the variance of our COI under this setup, relies on inverting the matrix
We now show that the variance of the COI, i.e., \(\text {Var}(\hat{\mu }_{t_{11},t_{21}})\), simplifies to \(\sigma ^2_{t_{11},t_{21}}\).
Proof
In order to show that the \(\text {Var}(\mu _{t_{11},t_{21}}) = \sigma ^2_{t_{11},t_{21}}\) when two disconnected subnetworks are connected using \(t_{11}\) and \(t_{21}\), we use the following facts:

1.
Given a block matrix \(\varvec{M} = \left[ \begin{array}{cc} \varvec{E} &{} \varvec{F}\\ \varvec{G} &{} \varvec{H} \end{array}\right]\), where \(\varvec{E},\varvec{F},\varvec{G},\varvec{H}\) are \(n \times n, n \times m, m \times n, m \times m\) matrices with \(\varvec{H}\) invertible,
$$\begin{aligned} \text {det}(\varvec{M}) = \text {det}(\varvec{E}\varvec{F}\varvec{H}^{1}\varvec{G})\text {det}(\varvec{H}). \end{aligned}$$and
$$\begin{aligned} \varvec{M}^{1} = \left[ \begin{array}{cc} (\varvec{E}\varvec{F}\varvec{H}^{1}\varvec{G})^{1} &{} (\varvec{E}\varvec{F}\varvec{H}^{1}\varvec{G})^{1}\varvec{F}\varvec{H}^{1}\\ \varvec{H}^{1}\varvec{G}(\varvec{E}\varvec{B}\varvec{H}^{1}\varvec{G})^{1} &{} \varvec{H}^{1}+\varvec{H}^{1}\varvec{G}(\varvec{E}\varvec{F}\varvec{H}^{1}\varvec{G})^{1}\varvec{F}\varvec{H}^{1} \end{array}\right] . \end{aligned}$$ 
2.
If the above \(\varvec{E}\) is scalar,
$$\begin{aligned} \text {det}(\varvec{M}) = (\varvec{E}\varvec{F}\varvec{H}^{1}\varvec{G})\text {det}(\varvec{H}). \end{aligned}$$ 
3.
If the rank of a square matrix \(\varvec{J}\) of size \(n \times n\) is less than n, \(\text {det}(\varvec{J}) = 0\).
Now, we can write
where \(\varvec{D}\) is invertible and \(\varvec{X}_{D2}^{\prime }\varvec{S}_{D2}^{1}\varvec{X}_{D2}\) is not full rank, which follows directly from the aforementioned model parameterization for disconnected networks. By facts 13, we have that the (1, 1)th element of the inverse is
which completes the proof.
The results here are limited to the case when the COI is exactly the connecting trial (i.e., a direct trial). We now present variance formulas for indirect trials. We define a partially indirect connecting trial as one that involves exactly one of the two treatments in the COI, and a completely indirect trial involves neither of the two treatments in the COI.
Consider again two existing, disconnected subnetworks, with subnetwork one consisting of treatments \(t_{11}. t_{12}, \ldots , t_{1m_1}\) and subnetwork two consisting of treatments \(t_{21}, t_{22},\ldots , t_{2m_2}\), with \(t_{11}\) as the overall baseline treatment. The two subnetworks are connected through a new trial including treatments \(t_{11}\) and \(t_{21}\) such that the withintrial variance is \(\sigma ^2_{t_{11},t_{21}}\). We have shown above that \(Var(\hat{\mu }_{t_{11},t_{21}}) = \sigma ^2_{t_{11},t_{21}}\). We will now show that for any COI \(\mu _{t_{1i},t_{2j}}\) for \(i = 2,\ldots ,m_1, j = 2,\ldots ,m_2\) the variance of the estimate of the COI can be written as the sum of \(\sigma ^2_{t_{11},t_{21}}\) and additional variance terms from subnetworks. Note that the COI in this case is not obtained from the connecting trial.
Proof
To start, suppose we are interested in a comparison between the overall baseline \(t_{11}\) and an arbitrary treatment from subnetwork two, \(t_{2j}, j = 2,\ldots ,m_2\). By independence and consistency assumptions [14], we have the following:
where \(\sigma ^2_{t_{21},t_{2j,pooled}}\) is a pooled variance from the NMA analysis on subnetwork 2. This is an example of a partially indirect connecting trial as defined earlier. If we are interested in a nonbaseline comparison between treatments \(t_{1i}\) and \(t_{2j}\) ,\(i = 2,\ldots ,m_1, j = 2,\ldots ,m_2\), it follows that
This is an example of a completely indirect connecting trial. Thus, we have shown that when connecting two arbitrary subnetworks with a single twoarm indirect trial, the variance of any comparative effect size of interest is the sum of the new withintrial variance \(\sigma ^2_{t_{11},t_{21}}\) and additional variance parameters that correspond to the individual subnetworks.
Under the assumption that \(\sigma ^2_{t_{11},t_{21}}\) is constant across all possible trials, the best connecting trial will always be direct (i.e., exactly the COI), which is clear from the variance structure presented above. However, in the case of binomial responses, the assumption of constant withintrial variance is not appropriate; instead, we can use information from the existing network to estimate the risk of all of the treatments. This will provide insight on the variance in the new trial. The choice of trial that minimizes a COI is then network dependent, and it may not always be the case that the best connection is direct. Connecting treatments indirectly may result in a lower variance estimate under certain conditions, which we will explore through our real data application.
Simple example
To illustrate a network of trials, we have included an example in Fig. 1 of a network consisting of two disconnected subnetworks, with subnetwork one having treatments A, B, and C and subnetwork two having treatments D, E, and F. In this case, a new trial has been conducted between treatments C and E to connect subnetworks one and two. If the researchers are originally interested in a comparison between C and E, this is a direct connecting trial, otherwise, it is an indirect connecting trial.
Proposed procedure
We propose a general selection procedure to find the optimal twoarm trial to connect a disconnected network when the COI is between two disconnected treatments of interest, \(c_1\) and \(c_2\), from subnetwork one and subnetwork two, respectively. The steps are as follows:

1.
Consider all possible connecting twoarm trials.

2.
For each connecting twoarm trial, do the following:

a
Create a new fully connected network consisting of the two previously separated networks and the new trial. Set the new design matrix \(\varvec{X}\) such that the overall baseline is the baseline from network one. Append a new row corresponding to the connecting trial.

b
Calculate the withintrial variance \(\sigma ^2_{t_1,t_2}\) (based on Eq. 2) as
$$\begin{aligned} \sigma ^2_{t_1,t_2} = \frac{1}{n_1p_1(1p_1)}+\frac{1}{n_2p_2(1p_2)} \end{aligned}$$for given treatments \(t_1\) and \(t_2\). Fix \(n_1 = n_2 = n/2\), where n is the total sample size in the connecting trial, and set \(p_1,p_2\) as the risks of treatments \(t_1,t_2\).

c
Use Eq. 11 (or equivalently, Eq. 4) to determine the variance of the COI between treatments \(c_1\) and \(c_2\) , \(\text {Var}(\hat{\mu }_{c_1,c_2})\).

a

3.
Find the optimal connecting twoarm trial that minimizes \(\text {Var}(\hat{\mu }_{c_1,c_2})\).
In practice, the risks used in the above selection procedure, \(p_1\) and \(p_2\), can be estimated from existing data in the following manner:

1.
Analyze each network separately using a frequentist based fixedeffects model for NMA.

2.
For each subnetwork, calculate the risk of all treatments. To do so, obtain estimates of the risks of the baseline treatment, \(p_{b}\), from the literature. Then, for any other treatment t the risk \(p_t\) is calculated as
$$\begin{aligned} p_{t}&= \frac{\frac{p_b}{1p_b}e^{LOR_{t,b}}}{1+\frac{p_b}{1p_b}e^{LOR_{t,b}}}\nonumber \\&= \frac{p_b e^{LOR_{t,b}}}{1+p_b e^{LOR_{t,b}}p_b}, \end{aligned}$$(12)where \(LOR_{t,b}\) is the estimated logodds ratio of treatment t to baseline b from the network metaanalysis conducted in step 1.
Application and simulation
In this section, we use a real data set to illustrate our procedure for identifying the best connecting twoarm trial based on the methods above. We then conduct a simulation study to confirm our findings and verify the variance formulas we presented. All analysis is performed using R version 4.1.1.
Real data application procedure
We apply our methods to data from a previously published network metaanalysis on the use of bacterial and viral vaccines for the prevention of bovine respiratory disease (BRD) in beef cattle [15]. A total of 53 studies reported morbidity due to BRD, with the full network shown in Fig. 2. The outcome of interest is an indicator for morbidity and all studies reported logodds ratios. When conducting a network metaanalysis, the authors focused on the largest subnetwork; that is, the authors did not use all of the information related to BRD due to the disconnected nature of the full network. Here, we focus on the two largest subnetworks, which are the two subnetworks with treatments labeled in Fig. 2.
The two subnetworks that are used throughout the remainder of this paper are shown in more detail in Figs. 3 and 4. These two subnetworks will be referred to from this point forward as networks one and two, respectively. Network one includes 17 vaccines from a total of 14 studies. Two of these studies were threearm trials, one was a fourarm trial, and the remainder were twoarm trials. Network two includes six vaccines from three studies, with one fourarm trial and two twoarm trials.
Suppose that the researchers would like to connect the two largest subnetworks with the goal of estimating a comparison between two vaccines as precisely as possible, for example, the comparison between the two chosen baselines (N and E). Note that these baseline treatments were chosen without loss of generality. Also, we refer to the vaccines as treatments to ensure the language is consistent with the above discussion. Researchers would like to know if it is better to connect the two treatments N and E directly, or if it would be better to leverage information from indirect comparisons and connect the subnetworks elsewhere. To determine the best connecting twoarm trial given a COI, we can apply the proposed selection procedure outlined in the Methods section. In this case, there are a total of 102 (17 x 6) possible twoarm trials.
The results from a network metaanalysis of subnetworks one and two are shown in Tables 1 and 2. We then consider several possibilities for the total sample size of the new connecting trial, n, and find the connecting trial such that the variance of \(\hat{\mu }_{N,E}\) is minimized (e.g., the COI is between treatments \(c_1=N\) and \(c_2=E\)), as shown in Table 3. This is an example of the COI being the baseline to baseline comparison.
Based on these results, we can see that there are conditions when the best connecting trial is not a direct trial of the two treatments involved in the comparison. Practically, however, there is not a large difference between the variance of the best trial and variance of the direct trial. As the sample size increases, the best connecting trial is the direct trial since as the sample size in the connecting trial increases, so does the precision in the comparative effect size estimate.
Now suppose that researchers are not interested in a baselinetobaseline comparison of the two subnetworks (this might occur if one of the baselines is no longer a feasible treatment). In the case that the COI is treatments LG, the best connecting trial is shown in Table 4. In this case, there is a much larger difference between the variance of the best connecting trial and the variance of the direct trial.
We further illustrate that it is possible for trials involving completely indirect connections to be better than a direct connection given a COI. Continuing with the COI as LG, results in Table 5 show that it is possible that a trial involving neither treatment L nor G may be better than a direct trial including both. For a fixed sample size of \(n=6\), the best trial that does not include L or G is the trial DJE, in which the variance of the COI, \(\text {Var}(\hat{\mu }_{L,G}) = 5.4843\). This is smaller than the case when the trial is direct between LG, in which \(\text {Var}(\hat{\mu }_{L,G}) = 11.7115\). Results for additional sample sizes up to \(n=20\) are shown in Table 5. These results confirm that it is possible for a trial involving neither of the treatments involved in a COI to be better than a direct connection. The best connecting trial for the the COI LG is direct with sample sizes \(n \ge 50\) and results are the same as those in Table 4.
As a concluding example, we consider the COI IEQ. Neither of the treatments involved in this comparison have extreme risks, and this illustrates a case where the difference between the best connecting trial and the direct trial is larger than in the case of NE. In fact, for \(n=6\) this is the COI that results in the largest difference between the best connecting trial and the direct trial (excluding comparisons involving treatment L). Results are shown in Table 6.
Simulation
We extend the ideas presented in the Real data application procedure section to include the simulation of data from new connecting trials to validate our proposed method. We use the following procedure to generate 1000 simulated trials and define the COI to be LG (that is, \(c_1\) = L and \(c_2\) = G):

1.
Estimate the risks of \(p_1\) and \(p_2\) as described in the Methods section; that is, estimate the risks of the trial treatments, t_{1 }and t_{2} ,using the existing network data.

2.
For each partially indirect connecting twoarm trial that was identified as best (found in Table 4) do the following:

a
Simulate data from the new connecting trial by drawing from both a binomial(\(n_1\),\(p_1\)) and binomial(\(n_2\),\(p_2\)), where \(n_1=n_2=n/2\) and \(p_1, p_2\) are the risks of treatments \(t_1\),\(t_2\) given by Tables 1 and 2. Now we have a simulated number of events for each of the two treatments involved in the new trial, \(r_1\) and \(r_2\).

b
Estimate \(\hat{p}_1 = \frac{r_1}{n_1}\) and \(\hat{p}_2 = \frac{r_2}{n_2}\) and use the adjustment to account for proportions of 0,1 in [16].

c
Create a new fully connected network consisting of the two previously separated networks and the new trial. Set the new design matrix \(\varvec{X}\) where the overall baseline is treatment N from network one. Append a new row corresponding to the connecting trial.

d
Append a new element to the vector \(\varvec{y}\) as \(\text {log}(\frac{\hat{p_2}/(1\hat{p_2})}{\hat{p_1}/(1\hat{p_1})})\).

e
Calculate the withintrial variance \(\sigma ^2_{t_1,t_2}\) as
$$\begin{aligned} \sigma ^2_{t_1,t_2} = \frac{1}{n_1\hat{p}_1(1\hat{p_1})}+\frac{1}{n_2\hat{p}_2(1\hat{p}_2)}, \end{aligned}$$for treatments \(t_1\),\(t_2\).

f
Use Eq. 4 to calculate the value of \(\hat{\mu }_{c_1,c_2}\).

a

3.
Repeat step 2 1000 times and record \(\hat{\mu }_{c_1,c_2}\) in each simulated trial.
Results from this simulation are shown in Table 7. Both the bias\((\hat{\mu }_{c_1,c_2})\) and root mean square error, RMSE\((\hat{\mu }_{c_1,c_2})\) are shown. By examining the bias and RMSE in the estimator, we can see that for small sample sizes, a partially indirect trial is preferred to a direct trial. This aligns with the results found in the real data application.
Verification of formulas
We further simulate data to verify the formulas for estimating the variance of a COI presented in the Methods section. As an example, we focus on the case where subnetworks one and two are connected through a twoarm trial involving treatments N and E. Let \(\sigma ^2_{t_1,t_2}\) denote the withinstudy variance of the comparison between treatments \(t_1\) and \(t_2\), and \(\sigma ^2_{t_1,t_2,pooled}\) be the estimate of the variance of the comparative effect size that arises from analysis of the network (i.e., using a fixed effects model for NMA). We verify that Eqs. 4 and 11 produce the same variance estimate. Table 8 shows the breakdown of the variance estimate, which is confirmed using Eq. 4. Table 9 shows a specific example using one data set with a fixed sample size for the connecting trial, \(n=1000\). Several comparisons of interest between subnetwork one and subnetwork two are considered. The results show that when the COI is exactly the connecting trial, the variance of the COI is simply the withintrial variance. Otherwise, the variance is the sum of the new withintrial variance and additional variance parameters that correspond to links in the entire network. These results are consistent with the formulas presented in the Methods section.
Discussion
Evidence from an existing NMA is commonly used to plan a new trial. When a group of trials are connected in a network, several methods have been proposed to identify trial(s) that achieve a desired power or precision for a COI. However, in the case of disconnected networks, there is no literature to guide researchers on how to design a trial to connect any subnetworks. In this paper, we address how to identify connecting trials that minimize the variance of a COI between two disconnected subnetworks. We derive variance formulas which lead to a straightforward computational procedure to identify the best connecting twoarm trial and confirm the results via simulation.
The formulas derived in the Methods section of this paper have several implications. Eq. 11 shows that under a completely indirect connecting trial, the variance of any COI can be written as the sum of three components: a pooled variance from the first subnetwork, the withintrial variance from the connecting trial, and a pooled variance from the second subnetwork. By writing the variance in this manner, it is clear that the best connecting trial may not always be direct, and is instead network dependent. Further, from Eq. 2, it holds that as \(n \xrightarrow {} \infty\), \(\sigma ^2_{t_1,t_2} \xrightarrow {} 0\), where \(\sigma ^2_{t_1,t_2}\) is the withintrial variance for a trial between treatments \(t_1\) and \(t_2\). Given a large enough sample size, this implies that the best trial will always be direct. However, the rate of convergence is network dependent, as \(\sigma ^2_{t_1,t_2}\) is a function of risks from each subnetwork.
Our proposed procedure is a straightforward way for researchers to apply the formulas presented throughout the paper and determine which trial minimizes the variance of a COI. Using a real disconnected network, we have shown that there are several cases when an indirect trial should be preferred to a direct trial. Practically, in the baselinetobaseline example of NE, the differences between a direct trial and a partially indirect trial are not very large. In this case, researchers may not be motivated to conduct an indirect trial. However, when the COI is LG, there is a notable difference between the direct and indirect trial. This difference is evident for both a partially indirect connecting trial and a completely indirect connecting trial. This is due to the extreme risk of treatment L, which makes a direct trial less ideal than an indirect trial in terms of the variance of the COI. Nonetheless, a key takeaway from this paper is exhibited here  in practice, conducting an indirect trial may be preferable to a direct trial when connecting two subnetworks. Further, it may not always be possible to design a direct trial. For example, a feedlot might have an existing contractual obligation to use certain products. Conducting a completely indirect trial would enable the feedlot to obtain an estimate of comparative efficacy of a rival company’s product to make longer term decisions. Similarly, perhaps an older standard of care is the baseline in one subnetwork but is expensive to use or has adverse side effects. For example, a long withdrawal for meat consumption will detract from including it as a treatment arm, yet because it is a commonly known treatment stakeholders may still find the comparison to that older standard of care meaningful. Using a partially or completely indirect connecting trial enables such a comparison.
By simulating realizations from both the best (partially indirect) connecting trial and a direct trial of LG, we have confirmed the conclusions of the real data analysis. Conducting an indirect trial leads to less bias, and a smaller RMSE, than a direct trial, adding more support to our conclusion that the best indirect trial is preferred to a direct trial for certain sample sizes. We note that the small sample size in the connecting trial is contributing to the bias, as discussed in other work [17]. Future simulations may be needed to further validate our proposed procedure.
Limitations
The methods presented in this paper are limited to use under the assumptions of fixed effects NMA. That is, they can be used under the assumptions of transitivity and consistency. When any of said assumptions are not met, the model needs to be modified and analogous formulas would need to be derived. Further, when designing a trial, researchers may prefer to use a random effects model. The methods presented here do not address that model, but similar results could be derived under a random effect NMA model formulation. In a random effects model, there will be an additional assumption of equal variances. If this assumption is violated then a more general model allowing heterogeneous betweentrial variances could be used to derive formulas [18]. The methods here also only consider a single comparison of interest, but in practice researchers may be interested in multiple treatment comparisons. This is a possible extension to our research.
Conclusion
The goal of this paper is to inform researchers that a direct trial may not always be the best trial to connect subnetworks, and to provide an approach to determine the best trial. In practice, researchers can simplify the procedure by only considering the connecting trials that are of interest to them, rather than all possible connections. With a large enough sample size, the best trial will be the direct trials. However, there are reasons why despite being interested in a particular comparison the feasibility of that comparison may be restricted due to cost, availability and adverse facts, in those situations researchers can use the procedure proposed to find the best connecting trial that does not involve said treatment.
The example used throughout this paper is based on livestock populations, however, the approach proposed is agnostic to this application. The foundation of the method is a valid comparison of NMA networks arising from a systematic review of trials that are reasonably considered to meet the same assumptions for NMA. As such, the approach proposed could be applied to any group trials related to interventions such as biological interventions, pharmaceutical interventions or medical devices. Overall, the purpose of this work is to make it clear that the best connecting trial is network dependent, and this idea is confirmed through both a real data application and simulation.
Availability of data and materials
We provide the R code and data we used in this paper at https://github.com/lamckeen/Connectingdisconnectednetworks.
Abbreviations
 NMA:

Network MetaAnalysis
 CNMA:

Component Network MetaAnalysis
 RWE:

Real World Evidence
 COI:

Comparison of Interest
 BRD:

Bovine Respiratory Disease
References
Rücker G, Schmitz S, Schwarzer G. Component network metaanalysis compared to a matching method in a disconnected network: A case study. Biom J. 2021;63(2):447–61.
Goring SM, Gustafson P, Liu Y, Saab S, Cline SK, Platt RW. Disconnected by design: analytic approach in treatment networks having no common comparator. Res Biom J. 2016;7(4):420–32.
Godolhin JD. Simple pilot procedures for the avoidance of disconnected experimental designs. JRSSC (Appl Statist). 2004;53(1):133–47.
Jenkins D, Martina R, Dequen P, Bujkiewicz S, Abrams K. The Added Value of RealWorld Evidence to Connect Disconnected Networks for Network MetaAnalysis: A Case Study in Rheumatoid Arthritis. Value Health. 2016;19(7):A393–A393.
Stevens JW, Fletcher C, Downey G, Sutton A. A review of methods for comparing treatments evaluated in studies that form disconnected networks of evidence. Res Synth Methods. 2018;9(2):148–62.
Béliveau A, Goring S, Platt RW, Gustafson P. Network metaanalysis of disconnected networks: how dangerous are random baseline treatment effects? Res Synth Methods. 2017;8(4):465–74.
Phillippo DM, Dias S, Elsada A, Ades AE, Welton NJ. Population Adjustment Methods for Indirect Comparisons: A Review of National Institute for Health and Care Excellence Technology Appraisals. Int J Technol Assess Health Care. 2019;35(3):221–8.
Pedder H, Dias S, Bennetts M, Boucher M, Welton NJ, Welton NJ. Joining the Dots: Linking Disconnected Networks of Evidence Using DoseResponse ModelBased Network MetaAnalysis. Med Decis Making. 2021;41(2):194–208.
Roloff V, Higgins JPT, Sutton AH. Planning future studies based on the conditional power of a metaanalysis. Stat Med. 2013;32:11–24.
Sutton AJ, Cooper NJ, Jones DR, Lambert PC, Thompson JR, Abrams KR. Evidencebased sample size calculations based upon updated metaanalysis. Stat Med. 2007;26:2479–500.
Nikolakopoulou A, Mavridis D, Salanti G. Planning future studies based on the precision of network metaanalysis results. Stat Med. 2015;35:978–1000.
Nikolakopoulou A, Mavridis D, Salanti G. Using conditional power of network metaanalysis (NMA) to inform the design of future clinical trials. Biom J. 2014;56:973–90.
Salanti G, Nikolakopoulou A, Sutton AJ, Reichenbach S, Trelle S, Naci H, et al. Planning a future randomized clinical trial based on a network of relevant past trials. Trials. 2018;19:365.
Dias S, Ades AE, Welton NJ, Jansen JP, Sutton AJ. Introduction to evidence synthesis. In: Network metaanalysis for decision making. 1st ed. New Jersey: Wiley; 2018. p. 1–16.
O’Connor AM, Hu D, Totton SC, Scott N, Winder CB, Wang B, et al. A systematic review and network metaanalysis of bacterial and viral vaccines, administered at or near arrival at the feedlot, for control of bovine respiratory disease in beef cattle. Animal Health Res Rev. 2020;20(2):143–62.
Agresti A, Coull BA. Approximate is better than exact for interval estimation of binomial proportions. Am Stat. 1998;52(2):119–26.
Lin L. Bias caused by sampling error in metaanalysis with small sample sizes. PLoS ONE. 2018;13(9):e0204056.
Hu D, Wang C, O’Connor AM. A likelihood ratio test for the homogeneity of betweenstudy variance in network metaanalysis. Syst Rev. 2021;10(1):310.
Acknowledgements
We would like to thank the Editor and Reviewers for their valuable comments and suggestions, which helped to improve the quality of the manuscript.
Funding
This study has been partially supported by FDA grant U01 FD00704901.
Author information
Authors and Affiliations
Contributions
LM proposed the method, wrote the code to conduct the data analysis, and wrote the code to perform the simulation. LM and PM developed notation and proofs, with input from CW. CW also coordinated the project team and assisted with the data analysis and interpretation of results. AOC provided the data and assisted with the data analysis. MM reviewed the manuscript and assisted with interpretation of the overall project. The manuscript was primarily prepared by LM, with secondary input from all other authors. The author(s) read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
McKeen, L., Morris, P., Wang, C. et al. Connecting a disconnected trial network with a new trial: optimizing the estimation of a comparative effect in a network metaanalysis. BMC Med Res Methodol 23, 79 (2023). https://doi.org/10.1186/s12874023018967
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12874023018967