Including non-concurrent control patients in the analysis of platform trials: is it worth it?

Background Platform trials allow adding new experimental treatments to an on-going trial. This feature is attractive to practitioners due to improved efficiency. Nevertheless, the operating characteristics of a trial that adds arms have not been well-studied. One controversy is whether just the concurrent control data (i.e. of patients who are recruited after a new arm is added) should be used in the analysis of the newly added treatment(s), or all control data (i.e. non-concurrent and concurrent). Methods We investigate the benefits and drawbacks of using non-concurrent control data within a two-stage setting. We perform simulation studies to explore the impact of a linear and a step trend on the inference of the trial. We compare several analysis approaches when one includes all the control data or only concurrent control data in the analysis of the newly added treatment. Results When there is a positive trend and all the control data are used, the marginal power of rejecting the corresponding hypothesis and the type one error rate can be higher than the nominal value. A model-based approach adjusting for a stage effect is equivalent to using concurrent control data; an adjustment with a linear term may not guarantee valid inference when there is a non-linear trend. Conclusions If strict error rate control is required then non-concurrent control data should not be used; otherwise it may be beneficial if the trend is sufficiently small. On the other hand, the root mean squared error of the estimated treatment effect can be improved through using non-concurrent control data.


Background
The implementation of innovative trial designs for drug development programmes has increased in recent years [1]. A platform trial is one such innovative design that helps improve efficiency. It has a multi-arm multi-stage design where a single control group is utilized to evaluate multiple experimental treatments for a single disease. Apart from the feature of dropping arms, a platform trial allows adding arms when new experimental treatments become available. This adds efficiency as adding arms takes less time and has a lower cost than setting up new trials from scratch.
A platform trial can be implemented through the specification of a master protocol [2][3][4]. Cohen et al [5] review the statistical aspects on the adding arm adaptation and present some trials that have adopted a platform approach. For pairwise comparisons between the effectiveness of newly added treatments and the control treatment, there are different views on utilising the control data [5]. To be more specific, the control data of a platform trial can be separated into non-concurrent and concurrent. The former (latter) corresponds to the control data of the patients who are recruited before (after) a new experimental treatment is added. This definition applies to each newly added arm independently; the control data can be concurrent to a newly added treatment, and be non-concurrent to another treatment that is added at a later time point.
To our knowledge, methodology and guidance for adding arms remain limited and the operating characteristics of platform trials have not been well-studied. The literature on other trial adaptations such as dropping arms and changing treatment allocation ratio are more comprehensive [6][7][8][9][10][11]. In particular, many have explored the presence of time trends when more complex randomization procedures are recommended/ explored [12][13][14][15][16][17][18][19][20][21]. A trend may be present in a trial when there is a learning curve among the study personnel; a shift in the population baseline characteristics; and/ or the effect of the control treatment may change due to other reasons (e.g. improvement of the standard of care/ practice) [7,22].
In order to minimise the impact caused by the presence of a trend, most researchers advocate the use of concurrent control data when making inference about the newly added treatment(s) [23][24][25]. Others stipulate that non-concurrent control data may be used when making inference about the newly added arm by adjusting for possible trends [26,27]. Here we investigate whether using non-concurrent control data is worthwhile in platform trials. Several analysis approaches are considered. Within a two-stage setting where a new treatment is added at the end of stage one [25,28], we explore the impact of i) the timing of adding a new arm ii) the sample size of the new arm and iii) the magnitude of a linear or a step trend [15] on the inference about the newly added experimental treatment. We provide recommendations about under which conditions non-concurrent control data should be used.

Method
Consider a two-stage multi-arm trial that initially has K treatments and a control treatment with a total sample size of N = N 1 + N 2 where N s is stage s sample size, s = 1, 2. Denote k = 0 for a control treatment, k = 1, ..., K for the initial experimental treatments. Let n ks be the sample size of treatment k in stage s with n k1 + n k2 = n k . At the end of stage one, in particular after N 1 = K k=0 n k1 patients have been randomized to the initial arms in stage one, a new treatment (denoted by K + 1) is added to the on-going trial with n K+1,2 patients. This increases the second stage sample size to N 2 +n K+1,2 and the overall sample size of the trial to N + n K+1,2 .
Denote n 01 /n 0 as the timing of adding a new arm. Small n 01 /n 0 indicates the newly available treatment is added to the trial after a small number of patients have been randomized to the initial arms in stage one. We consider the case that all arms finish recruitment simultaneously (although as long as controls continue to be recruited while treatment K + 1 is allocated patients, results apply more generally). For j = 1, ..., N + n K+1,2 , denoting the sequence of patient enrolment to the trial, let: be the response of treatment k of subject j; μ k be the true effect of treatment k; τ (j) be a trend that could be a function of the patient ordering j; j be the random errors that are identically and independently normally distributed with mean 0 and variance σ 2 ; and T j be the allocated treatment of subject j, with I(·) is an indicator function. LetX k.s andX k denote the stage s sample mean response of treatment k and the overall mean response of treatment k. When there is no trend in the trial, i.e. τ (j) = 0(j), the For illustration purpose, we consider the following when exploring the impact of a fixed trend on the inference: • linear trend: • stepwise trend: where c j is an indicator that subject j is enrolled in stage two, and λ is a positive fixed value. Both of these trends inflate the value of the outcome variable in a sequential manner; the responses of the patients who were enrolled later are larger than those who were enrolled earlier.

Simple approach: Z-test
We first consider a simple method that is analogous to ignoring the possibility of a trend or stage effect: a standard Z-test for testing each hypothesis H 0k : μ k = μ 0 against H 1k : μ k > μ 0 , using the sample means. When there is no trend, the (pairwise) marginal power of rejecting a null hypothesis to detect a difference in treatment effect, δ k = μ k − μ 0 , is where and −1 are the cumulative density function and the inverse of the cumulative density function of a standard normal distribution. For the newly added treatment, the power of rejecting H 0,K+1 depends on whether all the control data (of size n 0 ) or only the stage two control data (of size n 02 ) is used in the inference. Elm et al [28] describe a t-test for the case where σ 2 is unknown.

Model-based approach: weighted linear regression
One way to account for the changes of a trial design is to consider a model-based approach. Elm et al [28] consider the following weighted linear regression model. LetX (k) s = X k.s −X 0.s be the stage-wise difference in mean responses; fit a linear model to the samples of the stage-wise mean differences: s is normally distributed with mean zero and variance σ 2 /n ks + σ 2 /n 0s . The covariance between mean responses, cov(X 2 ) = 0 ∀l and k = 1, ..., K, as patient data are independent; whereas cov(X (l) s ,X (k) s ) = σ 2 /n 0s , ∀l = k, where l, k = 1, ..., K, as the data of a shared control group is used in all the pairwise comparisons. For the newly added treatment, cov(X 2 ) = σ 2 /n 02 for k = 1, ..., K, when only the concurrent control data is used.
Elm et al [28] have not considered using all the control data in estimating the difference in mean responses of treatment K + 1 and the control, i.e. defined asX s ) = σ 2 /n 0 for k = 1, ..., K, and s = 1, 2, asX 0 can be re-expressed as n 01 n 0X 0.1 + n 02 n 0X 0.2 . Using weighted least squares estimation, an estimate of the vector of treatment effects relative to control, δ = (δ 1 , ..., δ K , δ K+1 ) T , can be obtained. The estimate,δ, is unbiased and has a multivariate normal distribution with mean δ when there is no trend. This joint distribution can be used to test the hypotheses, {H 01 , ..., H 0,K+1 }, in a similar way to the Dunnett test [29]. The marginal power of rejecting a hypothesis can be computed by considering the corresponding marginal distribution. Note that when K > 0, the marginal power is different to the power obtained from a Z-test (or a t-test) as the joint distribution accounts for the fact that a common control group is used in the analysis. Note also that var(δ K+1 ) depends on the number of control patients.
We label these approaches as WLS all and WLS s2 respectively, when all the control data and when only the concurrent control data are used in the estimation of δ K+1 .

Model-based approach: linear regression
In the context of an adaptive randomisation procedure, Villar et al. [19] considered fitting a linear regression model to adjust for the presence of a trend in their investigation where all arms start recruiting at the same time. We explore the following linear regression analyses for the trial that adds an arm at the end of stage one: where β 0 = 0 such that β k represents the difference in mean responses of treatment k and the control treatment; and • M b1 : represent the difference in mean responses of the newly added treatment and the control treatment. Note that models M a1 and M a2 use all data to estimates all β 1 , ..., β K , β K+1 , whereas model M b1 (and M b2 ) uses the data of the control arm (both stages) and the newly added arm to estimate β b1 K+1 (and β b2 K+1 ) only. The parameter γ in M a1 and M b1 represents the effect of continuous recruitment, whereas ν in M a2 and M b2 represents the effect of recruitment that happens after the new treatment has been added (Elm et al [28] interpret ν as a stage effect). It is worth mentioning that the amount of the control data used to estimate β K+1 in M a2 (and respectively β b2 K+1 in M b2 ) is similar to (same as) that of using only the second stage control data, e.g.X K+1,2 −X 0.2 , since the newly added treatment was not present in stage one and the stage one control data would contribute to the estimation of the stage effect and that of β 1 , ..., β K under such a model set-up. In other words, the stage one control data contributes very little to the estimation of the difference in mean responses of the newly added treatment and the control treatment when we adjust for ν · c j in the analysis.
We note that the inference from M a1 without adjustment for the linear term in patient ordering is similar to that from WLS all ; and that from M a2 is similar to WLS s2 . This is because the joint distribution of {β 1 , ...,β K+1 } is equivalent to the corresponding joint distribution ofδ. The subtle difference is the former approach has one more parameter (i.e. the intercept α) than the weighted regression model, which reflects the mean response of the reference group assuming the model is true. On the other hand, testing β b1 K+1 and β b2 K+1 respectively are similar to considering an independent Z-test (or a t-test when assuming unknown variance) for the hypothesis of H 0,K+1 .

Metrics
We explore the above analysis approaches and consider • bias of estimators; • the type one error rate; • the marginal power of rejecting a false null hypothesis; • the root mean squared error, rMSE = bias of an estimate 2 + variance of an estimate; when including or excluding the stage one control data in the analysis of the newly added treatment. As we assume σ 2 is known, the presence of a trend would affect the accuracy of the estimate but not the variance of an estimate. We include the following metric to explore the reduction in the variance of the estimate: where BoS stands for borrowing of strength [30], V 0 and V a correspond to the variance of an estimate that is computed with only the concurrent control data and that with all the control data respectively. Note that BoS increases as V a decreases. This measure is similar to the R-squared that measures how close the data are to the fitted regression model. Here BoS ≈ 0 indicates that the variability of the response data around its mean cannot be further explained by including non-concurrent control data; whereas BoS ≈ 1 reflects that using all the control data results in a perfect estimate that has negligible variability. One may interpret BoS as an indicator of the benefit of including non-concurrent control data in the analysis of the newly added treatments. Small values show that including the non-concurrent control data provides little benefit to the estimation of the parameters in terms of precision gain. High value is desirable especially when the estimates of the variance are unbiased and are not affected by the presence of a trend.
In the next section, we consider K = 1 and a new arm is added at the end of stage one. Let the effect size of the difference in treatment effects be δ 1 = δ 2 = 0.15 under the alternative scenario, and δ 1 = δ 2 = 0 under the null scenario, with σ 2 = 1, n 11 = n 01 = N 1 /2, and n 12 = n 02 = N 2 /2. We choose n 0 = n 1 = 550 to obtain 80% power and 5% type one error rate of rejecting H 01 using the Z-test.
We conduct simulation studies to examine the impact of a trend with the timing of adding a new treatment, n 01 /n 0 = {0.25, 0.5, 0.75}, and the sample size of the new arm, n 22 = {n 02 , n 0 , 2n 0 }. Following (1), we adjust the generated responses post randomization with λ = {0.02, 0.04, 0.06, 0.08} in the linear trend (2) and the stepwise trend (3) respectively. These values indicate a trend of λ × 100% of standard deviation. Each scenario is replicated 100 000 times. We illustrate n 22 = 2n 0 as one of the options to explore the marginal benefit of having more patients (than necessary) in the newly added arm while keeping the size of the control arm the same as the initial plan.

Analytical power and BoS when there is no trend
We now compare BoS and the power of rejecting H 02 when there is no trend and either all, or only concurrent control data are used in the inference.
We first demonstrate the differences between the independent Z-test (computed with only concurrent control data) and the model-based approach, WLS all and WLS s2 , in terms of power. For ease of presentation, we omit the comparisons to the presented linear regression models as they make adjustment for a trend (see the first row of plots in Figs. 2 and 3 for the corresponding marginal power). The left plot in Fig. 1 shows power curves when the individual Z-test (red lines) and WLS s2 , i.e. the approach with the bivariate normal distribution of (δ 1 ,δ 2 ) (blue lines), are used respectively for testing H 02 , both using only the concurrent control responses. The timing of adding the new arm to the on-going trial is reflected on the x-axis; small values indicate the arm is added after a small number of patients have been randomized to the initial treatment arms. Different lines correspond to having different n 22 .
As expected, the power decreases with n 01 /n 0 when excluding stage one control responses in the inference about the newly added treatment. Comparing the red lines to the blue lines in the left plot of Fig. 1, the power of the individual Z-test is lower than the marginal power obtained from WLS s2 given the same n 22 . When n 22 = n 02 , the difference between the power curves computed by the two approaches is less than 6% (shown by comparing the dashed lines). When n 22 ≥ n 0 , the magnitudes of the differences between the power curves are larger especially for large n 01 /n 0 ; e.g. compare the dotted-dashed lines at n 01 /n 0 = 0.8, the marginal power obtained from WLS s2 is 15% more than that computed from the individual Z-test. This finding highlights that considering the joint outcomes of the hypotheses through the joint distribution of the parameters is more efficient than using the individual Z-test when only the concurrent control data is used. This is because the former accounts for using a shared control group in the inference, whereas the Z-test uses the control data independently for each hypothesis test as if they were from several trials. Besides that, compare the (vertical) difference between the dotted-dashed line (n 22 = 2n 0 ) and the dotted line (n 22 = n 0 ), either red or blue, we see that the magnitude of the difference in the power decreases as n 01 /n 0 increases. This indicates that the marginal benefit of having a larger sample size in one arm than necessary can be small, depending on when the new arm is added to the trial.
For WLS all with n 22 = n 02 , the power curve coincides with the red dotted line, i.e. the individual Z-test using only the concurrent control data and with n 22 = n 0 . This is not surprising as the same number of patients are used in the inference, albeit the number of patients randomized to the newly added arm is different for the two comparators (i.e. WLS all considers n 02 responses of the new treatment and n 0 responses of the control treatment; the individual Z-test uses n 0 responses of the new treatment and n 02 responses of the stage two control treatment). For WLS all with n 22 = n 0 and n 22 = 2n 0 , the marginal power of rejecting H 02 is 0.8 and 0.89 respectively across all n 01 /n 0 , which is close to the respective power obtained from Z-test using all the control data.
We now consider the BoS values that are computed with var(δ 2 ) obtained from WLS all and WLS s2 respectively. The right plot in Fig. 1 shows that when all the control data is included in the estimation, BoS increases with n 01 /n 0 . When n 22 = n 02 , BoS is close to 30% when the new treatment is added at n 01 /n 0 = 0.9. When n 22 has the same number as n 0 , we see that BoS becomes even larger. However, the marginal return of having large n 22 than necessary can be less notable: comparing the dotted-dashed line to the dotted line shows that the magnitudes of the differences between the BoS perhaps are less impressive compared to the likely cost of enrolling an extra n 0 patients for the new arm.

Bias of the estimate when there is a trend
We now present the simulation findings of the inference about the newly added treatment when there is a trend.
Consider the estimated difference in mean responses of the newly added treatment and the control treatment. Table 1 shows the maximum absolute bias (and median absolute bias) in the parameter estimations when there is a trend. Each row corresponds to a trend with a value of λ for all the scenarios with different combination of n 01 /n 0 = {0.25, 0.5, 0.75} and n 22 = {n 02 , n 0 , 2n 0 }. The values with an order of magnitude of -4 are set to zero, as we also obtain such a small number for the unbiased esti-  mators when there is no trend in our simulation, which is due to the Monte Carlo simulation error. When there is a positive trend in the simulation, we find that the estimates obtained from WLS s2 , M a2 and M b2 respectively are unbiased. For those obtained from WLS all , the magnitude of the bias is larger when there is a step trend than when there is a linear trend. This is because the presence of a positive trend inflates the estimate of the overall mean responses of the control treatment while the mean responses of the newly added treatment only have a trend that is similar to the mean responses of the second stage control treatment.
For those obtained from M a1 and M b1 , the estimated parameter is almost unbiased when there is a linear trend but is biased when there is a step trend. This shows that adjustment with a linear term may not always yield accurate estimate especially when the structure of a trend is not linear.

Marginal power in the presence of a trend
The first row of plots in Figs. 2 and 3 show the marginal power of rejecting H 02 when there is no trend; other rows of plots correspond to different magnitudes of λ when there is a linear and a step trend respectively. The x-axis of the plots indicates the value of n 22 and n 01 /n 0 . Each column of plots has a different range of y-axis due to having different values of n 22 when the new arm is added to the on-going trial. As expected, the marginal power obtained from WLS s2 is similar to that from M a2 : the blue line (with ×) superimposes the red line (with ).
We see that the marginal power obtained from WLS all is the highest and from M b2 is the lowest among all the comparators. However, the marginal power obtained from WLS all is inflated when there is a positive trend. From our simulation, we find that relative difference in the inflated power to the non-inflated power is larger when n 22 = 550 compared to when n 22 = 1100. The reason could be when n 22 is small (relative to the fixed n 01 ), stage one control responses deviate more on average from the responses in stage two, which leads to a larger bias when using all the control data and hence increases the average number of rejected hypotheses.
On the other hand, the marginal power obtained from M a1 and M b1 respectively when there is a linear trend remains the same as when there is no trend (i.e. compare the black lines and cyan lines respectively across the row of plots in Fig. 2); but it is inflated when there is a step trend. The magnitude of the inflation increases with the values, λ, of the step trend (see Fig. 3). We find that the marginal power obtained from WLS s2 , M a2 and M b2 respectively remains unchanged when there is a trend. This is not surprising as the impact of a positive trend in the mean responses of the newly added arm and the second stage control arm are of similar magni-tude, which is being removed when the difference in mean responses is considered. Figures 4 and 5 show the type one error rate of rejecting H 02 in the presence of a linear trend and of a step trend respectively, when δ 1 = δ 2 = 0. A subtle difference in the type one error rate obtained from WLS s2 and M b2 respectively can be seen in the plots. We observe that the presence of a step trend would lead to a larger inflation in the type one error rate obtained from WLS all than when there is a linear trend with the same λ, i.e. from comparing the green line (with + sign) in Fig. 4 to those in Fig. 5. Besides, the magnitude of inflation is the largest when both the values of n 01 /n 0 and λ are the largest. Having larger n 22 also causes more inflation in the type one error rate given the same n 01 /n 0 . This may be because, given the same magnitude of the bias, large n 22 makes the denominator of the test statistics smaller, and hence increases the number of hypothesis rejections.

Type one error rate in the presence of a trend
For the type one error rate obtained from M a1 and M b1 , we see a similar finding to that of the marginal power. That is, when there is a linear trend, the type one error rate is maintained within the Monte Carlo simulation error, whereas it is inflated when there is a step trend (see the black line and cyan line respectively in Fig. 5). We find that the type one error rate obtained from WLS s2 , M a2 and M b2 respectively is maintained within the Monte Carlo simulation error when there is a trend.

rMSE in the presence of a trend
The plots of rMSE are presented in Additional file 1: Figures 1 and 2 in the supplement. When there is a positive trend, we find that M b2 has the highest rMSE, which remains the same across all the considered values of λ. This is not surprising as the least amount of the data is used. The rMSE values obtained from WLS s2 and M a2 also remain unchanged when there is a trend.
When there is a linear trend, the rMSE obtained from WLS all increases with λ while remaining smaller than that from M a1 and M b1 (which remain unchanged across the linear trends). However, when there is a step trend, the rMSE obtained from these approaches are inflated; the rMSE obtained from WLS all is higher than that from M a1 when λ ≥ 4%. Nevertheless, across all the considered scenarios, the rMSE obtained from M a1 is always smaller than the approaches that use less data, i.e. M a2 , M b1 , M b2 , and WLS s2 .
These findings highlight that even though the presence of a small trend would lead to a small bias in the estimation, the quality of the estimate computed using all the control data can be better than that computed using only the concurrent control data. This does not hold for larger trends.

Inference about the initial treatment
For the inference about the initial treatment, we find that the bias, type one error rate and power of rejecting H 01 are maintained at the same levels as those when there is no trend in the simulation. This finding is observed for all the model-based approaches. This may be due to using the restricted randomization procedure within each stage of the trial in the simulation, which ensures that not all n ks patients are being enrolled to a particular arm k during the early or the late phase of the trial. As a result, the average impact of a trend on the stage-wise mean responses of the initial treatment and of the control treatment are similar, which are then being subtracted when the difference in the mean responses is considered. This observation is  [17] who compare different randomization procedures in the context of the standard ANOVA F-test.

Discussion
When there is no trend, the advantages of including the non-concurrent control data are greater when the new arm is added at a later time point. Our simulation studies show that given the same magnitude of λ > 0, the presence of a step trend affects the validity of the inference more than that of a linear trend when all the control data are used in the analysis of the new arm. We also examine that using a linear regression model adjusting for a linear term may lead to spurious findings when there is a Fig. 4 The type one error rate of rejecting H 02 in the presence of a linear trend with a magnitude of λ > 0 standard deviation (row-wise); x-axis indicates the value of n 22 and the timing of adding the new arm non-linear trend, though the rMSE of the corresponding estimate can be lower than other approaches that use only the concurrent control data. This represents a caveat for including non-concurrent control data in the analysis plan as it is impossible to know the structure of a trend at the planning stage of a trial. After data collection, one can only test the presence of a trend using some quality control techniques such as control charts and CUSUM analysis, see for example a review by Noyez [31]. The analogy between clinical trials and industrial processes is becoming clearer especially when the trial has a long enrolment period. [32][33][34][35][36][37][38] This work has highlighted the impact of a trend when the variance of the observations is known. We emphasise that not all methods assume known variance. We conjecture that the presented results would hold when an estimated variance is unbiased, i.e. the estimate is robust to the presence of a trend. Otherwise, the over-or underestimated variance would cause the test statistics to be biased and hence provide a spurious result [28]. We also note that when there is a negative trend, i.e. λ < 0, the type one error and the marginal power will be deflated, i.e. smaller than the nominal value. This is a direct consequence of a trend that causes responses to be smaller than the true values as the trial progresses.
All things considered, we would recommend that at the design stage of phase III trials, sample size calculations assume only concurrent control information will be used. When the magnitude of a trend is small, we find including the non-concurrent control data in the analysis can improve the efficiency of estimating the parameter of interest. However there could be an inflation (or deflation) in the type one error rate and the marginal power of testing the corresponding hypothesis unless the positive (or negative) trend has a magnitude of less than 0.5% of the standard deviation (based on simulation results not presented here). This finding is similar to the finding of Kopp-Schneider et al [39] for the situation where external information is used in the inference of clinical trials: power gain is not possible when requiring a strict type one error rate control. When the recruitment to all arms do not finish simultaneously, the control data can be sepa-rated into before and after adding an arm and those after the initial treatment arms finish recruitment. In this case, the presence of a trend across the three stages can be tested before utilising all the control data to increase the precision of the inference.
We acknowledge that an inflated power is not an issue if we are confident that the intervention is effective. However, using the trial result for other purposes may lead to negative consequences since an inflated power could mean that the estimated effect size is likely to be higher than it should. For example, doing a cost-effectiveness analysis using the estimated effect size may over-estimate the value of the intervention due to the presence of a positive trend. Future work could review real trials that add arms and explore the presence of trends and their structure in the data. One can also investigate the potential of using non-concurrent control data for the situations where i) there are more than one treatment arm being added to the on-going trial; ii) there are more than twostages; and iii) adaptive randomization procedures are considered.

Conclusion
Platform trials can potentially speed up the drug development processes. The feature of continuous recruitment to the control arm may increase the precision of the inference about the newly added interventions in some situations; or reduce the required sample size for the evaluation of the newly added interventions at the cost of having a less stringent control of the error rates. In light of the presence of a potential trend, it is wise to compare the results of including and excluding nonconcurrent control data in the analysis of the newly added interventions.