We performed a simulation study with normally distributed endpoints and one protocol change. This protocol change takes place when half of the patients are recruited (scenario 1) or when a third of the patients is recruited (scenario 2). After the protocol change the variance was inflated, thus the second phase had a larger variability. We assumed a two group setting with one group being the treatment group and the other being the control group.
The factor for the variance change in the second phase (variance inflation factor) was always equal for both groups. To be precise, we used the following variance inflation factors: 1 (i.e. no change), 1.5, 2, 2.5, and 3. The configurations of means for the simulations can be divided into four groups:
1. Investigation of the type I error rate: = (0, 0, 0, 0).
2. Investigation of the power in the case of constant means within groups and with a constant non-zero between-groups mean difference s: = (s, s, 0, 0), for the shift (i.e. the mean difference) s the values 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.40, 0.50, 0.60, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95, and 1.00 were used.
3. Investigation of the power in the case of
-
a.
non-constant means, i.e. within the two groups, means can differ between phases by and simultaneously
-
b.
a constant between-groups mean difference
leading to = (s, s + d, 0, d). For s as well as for d the values 0.1, 0.5, and 1 were used.
4. Investigation of the power in the case of non-constant means (i.e. within groups, means can differ between phases) and a non-constant between-groups mean difference: The following special cases are presented in this paper: = (0.5, 0.2, 0, 0) for scenario 1 and = (0.7, 0.2, 0, 0) for scenario 2.
In addition, we investigated the effect of a decrease in variance. To be precise, we simulated data with a shift of 0.1, 0.5, or 1, respectively, and a reduction of the standard deviation to 0.25, 0.5 and 0.75, respectively.
We used two possible strategies of evaluating the data. The first strategy is simply pooling the data and performing a one-sided t-test with the assumption of homoscedasticity (which is in fact fulfilled). The second analysis is to perform a one-sided t-test for each of the two phases separately and then using Fisher's combination test to obtain an overall result. It should be noted that the combination of p-values across populations is essentially a meta-analytic method. Regarding the combination test we present results for testing the intersection hypothesis as well as for identifying efficacy in at least one population. The latter one will be abbreviated as "com & one" in the figures. For all strategies the (empirical) α-level of the simulation was determined as well as the (empirical) power of the test. All simulations were performed with SAS (version 9.1) and 10 000 simulation runs, except for estimating the type I error rate which is based on 100 000 simulation runs. We set α = 0.05.
We use the following notation:
: number of patients in control group before the amendment,
: number of patients in treatment group before the amendment,
: number of patients in control group after the amendment, and
: number of patients in treatment group after the amendment.
In the first scenario all four sample sizes , , and are 50. In the second scenario, we have = = 25 and = = 50.