Sequential boundaries approach in clinical trials with unequal allocation ratios

Background

In clinical trials, both unequal randomization design and sequential analyses have ethical and economic advantages. In the single-stage-design (SSD), however, if the sample size is not adjusted based on unequal randomization, the power of the trial will decrease, whereas with sequential analysis the power will always remain constant. Our aim was to compare sequential boundaries approach with the SSD when the allocation ratio (R) was not equal.

Methods

We evaluated the influence of R, the ratio of the patients in experimental group to the standard group, on the statistical properties of two-sided tests, including the two-sided single triangular test (TT), double triangular test (DTT) and SSD by multiple simulations. The average sample size numbers (ASNs) and power (1-β) were evaluated for all tests.

Results

Our simulation study showed that choosing R = 2 instead of R = 1 increases the sample size of SSD by 12% and the ASN of the TT and DTT by the same proportion. Moreover, when R = 2, compared to the adjusted SSD, using the TT or DTT allows to retrieve the well known reductions of ASN observed when R = 1, compared to SSD. In addition, when R = 2, compared to SSD, using the TT and DTT allows to obtain smaller reductions of ASN than when R = 1, but maintains the power of the test to its planned value.

Conclusion

This study indicates that when the allocation ratio is not equal among the treatment groups, sequential analysis could indeed serve as a compromise between ethicists, economists and statisticians.

One of the key reasons for using sequential methods, instead of single-stage design (SSD), in planning clinical trials is that the expected number of patients is decreased while maintaining the pre-specified significance level and power. Sequential designs have become common practice in interim monitoring of clinical trials because of their ethical and economic advantages. Nowadays, investigators planning a clinical trial have a wide range of sequential methods available to choose. These methods can be categorized in two different types: boundaries approach and repeated significance tests [1–6]. In this paper we have only considered boundaries approach, including triangular test (TT) and double triangular test (DTT), because of their interesting properties [2, 7–9]. Sebille and Bellissant presented a nice account of the properties of TT and DTT [7, 8, 10]. However, the properties of these methods, in some cases, are still unknown. In particular, situations not dealt with in previous articles are sequential designs with unequal randomization ratios.

In a randomized controlled clinical trial with two treatments, it is a standard practice to have approximately equal-sized treatment groups since it maximizes the statistical power for a given total sample size. However, there are several research papers with topics on unequal randomization that have demonstrated the efficiency of this method in clinical trials [11–16]. They showed that in some trials, which compare a new treatment against a standard, using unequal randomization could be helpful from the ethical and economic viewpoints. Yet there is no consensus between ethicists and economists on the issue. On the other hand, since sequential designs have ethical and economic advantages per se, it seems reasonable to use these methods when investigators decide to randomize patients to the experimental and standard treatment groups in unequal ratios.

Hence, the purpose of this study is to assess the effect of unequal randomization on the statistical properties of TT, DTT and SSD adjusted for unequal allocation ratio (SSD_adj) by multiple simulation, and SSD using the formulas used by Pocock [16]. In all of these methods the power and average sample size numbers (ASNs) were computed when the patients were allocated to the experimental and standard treatments in different ratios.

We follow notations similar to that used by Sebille and Bellissant [8]. Let θ be a measure of the difference between the experimental and standard treatments. The clinical trial can be viewed as a test of the null hypothesis of no treatment difference H₀ (θ = 0) against the alternative that there is a difference H₁ (θ ≠ 0). This parameter is designed such that θ = 0 when treatments are equivalent, θ > 0 ({\text{H}}_{\text{1}}^{\text{+}}) when the experimental treatment is better than the Standard one, and θ < 0 ({\text{H}}_{\text{1}}^{\text{-}}) when the experimental treatment is worse.

The trial considered here only involves the comparison of two normally distributed responses in two-sided tests. We defined the effect size as the difference between treatments in units of standard deviation, θ_R = (μ₂-μ₁)/σ where μ₁ and μ₂ are the means for the standard and experimental groups, respectively, and σ is the common standard deviation (σ₁ = σ₂ = σ).

Single stage design (SSD)

The traditional statistical approach in the analysis of clinical trials is SSD with equal patients in each group. In this method, the sample size is computed at the design phase based on the significance level (α), difference of clinical interest (θ_R), and power (1-β). In a two-group comparative study where the response measure is normally distributed, the total sample size formula would be:

\text{N = 4}{\left(\frac{{\text{Z}}_{\text{1}-\alpha /2}+{\text{Z}}_{\text{1}-\beta }}{{\theta }_{\text{R}}}\right)}^2\left(1\right),

where {\theta }_{\text{R}}=\frac{{\mu }_2-{\mu }_1}{\sigma } and Z_α is the upper 100α% percentile of N(0,1), that is, α = 1 - Φ(Z_α). Z _1-β is defined similarly.

If N_E and N_S denote the numbers of patients assigned to experimental and standard treatments with N_E + N_S = N being fixed and \text{r = }\frac{{\text{N}}_{\text{E}}}{\text{N}} denotes the proportion on the experimental treatment, then the power under H₁ is given by [16]:

\text{power = }\Phi \left\{\text{2}\left[{\Phi }^{-1}(1-\alpha )+{\Phi }^{-1}(1-\beta )\right]\times \sqrt{\text{r(1}-\text{r)}}-{\Phi }^{-1}(1-\alpha )\right\}\left(2\right).

In this formula Φ(·) denotes the cumulative function of the standard normal distribution N(0,1).

However, if the investigator decides to allocate patients in unequal ratio and aims to achieve the pre-specified power, then the total sample size for SSD should be adjusted by a factor dependent on the allocation ratio. Therefore, the total sample size for SSD_adj is equal to [2]:

{\text{N}}_{\text{adj}}=\frac{{(\text{R+1)}}^{\text{2}}}{\text{R}}{\left(\frac{{\text{Z}}_{\text{1}-\alpha /2}+{\text{Z}}_{\text{1}-\beta }}{{\theta }_{\text{R}}}\right)}^2=\frac{{(\text{R+1)}}^{\text{2}}\text{N}}{\text{4R}}\left(3\right),

where R is the ratio of patients in the experimental group to the standard group or the reverse ratio.

Once the data have been collected, the statistical analysis is conducted. Based on the SSD or SSD_adj we cannot stop an ongoing trial before inclusion of a predetermined sample size, even if the early data show a clear difference between treatments.

Boundaries approach: triangular and double triangular tests (TT and DTT)

Sequential boundaries approach, the TT and DTT, permit repeated statistical analyses to be performed throughout the trial recruitment period in order to allow for early termination of a trial while maintaining a pre-specified α and β level. This reduction in sample size has ethical and economic advantages [8]. From the ethical viewpoint, this reduction minimizes the number of patients who will be given an inferior or ineffective treatment. Moreover, from the economic viewpoint, it leads to saving in time and resources. As shown in Figures 1 and 2, the TT and DTT are based on the two-perpendicular axes [2, 9]. These two axes are two sample statistics that play particularly important role in the investigation of θ and are fundamental to sequential trials. The vertical axis is a cumulative measure of the advantage of the experimental treatment, and will be denoted by Z (efficient score for that calculated under the null hypothesis). The horizontal axis, denoted by V, indicates the amount of information about θ contained in Z (Fisher's information) and it will increase as the trial progresses [7]. The straight lines, the boundaries of the tests, delineate a continuation and stopping region. The equations of the straight line boundaries depend on the values of the benefit to detect, and α and β, as well as on the frequency of the analyses, defined in terms of the number of patients included between two analyses [2]. At each analysis, the two statistics V and Z are calculated from all the data collected since the beginning of the study and a point (V, Z) is defined on the sequential plan. The consecutive points define a sample path from the left to the right of the sequential plan. As long as the sample path stays within the two boundaries, the study is continued and new patients are included. When the sample path crosses one of the boundaries, the trial is stopped.

Stopping boundaries based on the TT for α = 0.05, β = 0.05 and R = 1 with θ_R = 0.7.

Full size image

Stopping boundaries based on the DTT for α = 0.05, β = 0.05 and R = 1 with θ_R = 0.7.

Full size image

The triangular tests can be categorized in two classes based on their power function. The power function, denoted by C (θ), is defined as the probability that H₀ is rejected when the parameter θ is true. When the true treatment difference is θ, C⁺ (θ) and C^- (θ) are the probability of reaching the conclusion that the experimental treatment is significantly better and worse than the standard, respectively. Based on this definition two alternative power requirements will be specified: power requirement I and power requirement II. TT is designed to satisfy power requirement I. In this situation C⁺(θ_R) = 1-β but no specification is made for C^- (-θ_R) and also C^- (θ_R) is usually negligible. On the other hand, DTT is designed to satisfy power requirement II. In this situation, C⁺(θ_R) = C^- (-θ_R) = 1-β and both C⁺(-θ_R) and C^- (θ_R) are negligible [2, 17].

Simulation study

We studied the ASN for the TT and DTT by multiple simulations in PEST3 [17]. Our simulation design was very similar to that used by Sebille and Bellissant [8]. For each studied situation, we generated 30,000 independent comparative trials in which patient responses were drawn from a normal distribution with mean μ₁ (mean response in standard group) equal to 10 and the standard deviation equal to 5. The influences of different values of β and θ_R (μ₂) on the statistical properties of all tests were evaluated. The total number of patients at each interim analysis (n) was equal to 12. We also evaluated the influence of the allocation ratio (R) on the statistical properties. R is defined as the ratio of the patients in the experimental group to the standard group. Namely, we chose two different values for β(0.05 and 0.1), seven values for θ_R (0.4, 0.5, 0.6, 0.7, 0.8, 0.9 and 1.0), one value for n (n = 12) and two values for R (1 and 2). The value of α was set to 0.05 for all simulated trials. We also calculated the required sample size for SSD and SSD_adj for the same values of θ_R, R, β and α as for TT and DTT by Formulas (1) and (3), respectively. Moreover, we simulated the required ASN and power for the TT (n = 12), DTT (n = 12), and two-sided SSD_adj for different values of R, when θ_R = 0.7 and β = α = 0.05. For SSD, the required sample size and power were calculated for the same value of R, θ_R, α and β using Formulas (1) and (2).

Table 1 shows the ASN of patients required to reach a conclusion under H₀, H₁ and θ = θ_R/2 for different values of θ_R and β for the SSD when R = 1, SSD when R = 2 (SSD_adj), TT when R = 1 and when R = 2, and DTT when R = 1 and when R = 2. The ASNs under H₀, H₁ and θ = θ_R/2 were smaller for all sequential tests than for SSD (R = 1), whatever values of θ_R, β and R were considered. Indeed, as compared with the SSD (R = 1), there were decreases of approximately 39% and 20% under H₀, 29% and 28% under H₁, and 14% and 12% under θ = θ_R/2 in the ASNs for the TT (R = 2) and DTT (R = 2), respectively. Moreover, Table 1 shows that choosing R = 2 instead of R = 1 increases the sample size of SSD by approximately 12% and the ASN of the TT and DTT by the same proportion under H₀, H₁ and θ = θ_R/2. On the other hand, as compared with SSD when R = 2 (SSD_adj), there were decreases of approximately 46% and 29% under H₀, 37% and 36% under H₁, and 24% and 21% under θ = θ_R/2 in the ASNs for the TT (R = 2) and DTT (R = 2), respectively.

Figure 3 displays the sample size required by the SSD and the ASN for SSD_adj, TT (n = 12) and DTT (n = 12) as a function of R when θ_R = 0.7 and β = 0.05. The ASN curves for the TT and DTT under {\text{H}}_{\text{1}}^{\text{+}} always stayed beneath the sample size required by the SSD_adj and were similar to one another. Indeed, as compared with SSD_adj, there were decreases of approximately 39.5%, 39.5%, 40.4%, 41.5%, 42% and 42.7% in ASNs for TT and DTT when R was equal to 1, 2, 3, 4, 5 and 9, respectively. Also, for R ≤ 4, the ASN curves of the TT and DTT stayed beneath the sample size required by the SSD (N = 106). Indeed, as compared with SSD, for R ≤ 4, they were decreased by approximately 39.5%, 32%, 21% and 8.5% in ASNs for TT and DTT when R was equal to 1, 2, 3 and 4 respectively. On the other hand, for R ≥ 2 the ASN curve of SSD_adj remained above the sample size required by SSD. Indeed, as compared with SSD, they were increased by approximately 12.3%, 33%, 56.6%, 80% and 178% in ASNs for SSD_adj when R was equal to 2,3,4,5 and 9, respectively.

Sample size required by two-sided SSD and SSD_adj and average sample number for the TT and the DTT (under {\text{H}}_{\text{1}}^{\text{+}}), for different values of R, when n = 12, θ_R = 0.7 and β = α = 0.05.

Full size image

Figure 4 shows that in the SSD, with a fixed total sample size, unequal randomization can lead to a reduction in statistical power when the R increases. In contrast, for the TT, DTT and SSD_adj where there is no limit on patient recruitment, the power curves remain constant.

Exact power for the SSD, and estimated power for the TT, the DTT (under {\text{H}}_{\text{1}}^{\text{+}}) and the SSD_adj for different values of R, when n = 12, θ_R = 0.7 and β = α = 0.05.

Full size image

Sequential methods and unequal randomization design are two different techniques in clinical trials, with their ethical and economic advantages. However, no previous study has combined unequal randomization with sequential analyses. In other words, the debates concerning unequal randomization were restricted to the SSD [11–16] and sequential analyses were only discussed in situations where the patients were equally randomized between the treatment groups [7, 8]. Sebille and Bellissant [8] showed that, of the one-sided sequential tests, the one-sided TT (R = 1) offers a substantial decrease in sample size compared with the one-sided SSD (R = 1); namely 40% under H₀ and H₁ and 25% under θ = θ_R/2. In addition, they showed that the two-sided TT (R = 1) offers a two-sided conclusion with much fewer patients than the double TT (R = 1) and two-sided SSD (R = 1), but at the expense of a high decrease in power under {\text{H}}_{\text{1}}^{\text{-}}[7]. On the other hand, according to Avins [13], Edwards [14] and Pocock [16], unbalanced randomization has ethical advantages since more patients are randomized to what is thought to be the superior therapy. Also, Torgerson and Campbell [11, 12] showed that when research costs differ between treatments and there is no constraint on total sample size, it is more cost-effective to randomize more patients to the less expensive treatment. However, Pocock [16] showed that, when there is a ceiling on total sample size, unequal randomization leads to a reduction in statistical power. Hence, we expected that, in the practical situation, in which the sample size of the SSD cannot be adjusted, the TT (R = 2) and DTT (R = 2), compared with unadjusted SSD, decrease the sample size while maintaining the power of the trial to its planned value. As an important result, our simulation study showed that even with the maximum ASN which occurs at θ = θ_R/2, the TT (R = 2) and DTT (R = 2) have smaller ASNs than the SSD (R = 1). Also before the start of the work, we could not estimate how much using R = 2 instead of R = 1 would increase the sample size in the TT, DTT and SSD. However, based on our findings, choosing R = 2 instead of R = 1 equally increases the sample size in the sequential methods and SSD up to 12%. Nevertheless, when the costs of the two treatment groups are very different, allocation of more patients to the cheaper treatment in the TT and DTT will compensate for this increase rate in the sample size. This decreases the total cost of the trial substantially.

However, it is necessary to present some characteristics of our study. Firstly, to present a fair comparison, we have only evaluated the statistical properties of the two-sided TT and DTT under H₀, {\text{H}}_{\text{1}}^{\text{+}} and θ_R/2, because, under these hypotheses, their power functions are identical [7]. Secondly, we did not compare the one-sided TT with other two-sided tests simultaneously because it is quite controversial in the literature [18, 19].

This study shows that if we allocate patients unequally in the SSD among the treatment groups and sample size ceiling cannot be increased to maintain the power of the trial due to economic restrictions, then an amalgamation of the sequential analysis and unequal randomization, compared with SSD, can be a compromise between statistical, ethical and economic requirements.

We are thankful to Keivan Shalileh for his comments and reading the final draft. We are also thankful to the referees for their invaluable comments. The financial support for this study was provided by Shiraz University of Medical Sciences, Shiraz, Iran.

Author notes

Affiliations

1. Department of Biostatistics and Epidemiology, Faculty of Health, Shiraz University of Medical Sciences, Shiraz, Iran

   • Peyman Jafari
   •  & Seyyed Mohammad Taghi Ayatollahi

2. Department of Statistics, Faculty of Science, Shiraz University, Shiraz, Iran

   • Javad Behboodian

Corresponding author

Correspondence to Peyman Jafari.

Competing interests

The author(s) declare that they have no competing interests.

Authors' contributions

PJ was responsible for the design, simulations, analyses and interpretation. SMTA and JB contributed to the analyses and interpretation.

Seyyed Mohammad Taghi Ayatollahi and Javad Behboodian contributed equally to this work.

Below are the links to the authors’ original submitted files for images.

Reprints and Permissions