Comparison of three tests of homogeneity of odds ratios in multicenter trials with unequal sample sizes within and among centers

Background Mixed effects logistic models have become a popular method for analyzing multicenter clinical trials with binomial data. However, the statistical properties of these models for testing homogeneity of odds ratios under various conditions, such as within-center and among-centers inequality, are still unknown and not yet compared with those of commonly used tests of homogeneity. Methods We evaluated the effect of within-center and among-centers inequality on the empirical power and type I error rate of the three homogeneity tests of odds ratios including likelihood ratio (LR) test of a mixed logistic model, DerSimonian-Laird (DL) statistic and Breslow-Day (BD) test by simulation study. Moreover, the impacts of number of centers (K), number of observations in each center and amount of heterogeneity were investigated by simulation. Results As compared with the equal sample size design, the power of the three tests of homogeneity will decrease if the same total sample size, which can be allocated equally within one center or among centers, is allocated unequally. The average reduction in the power of these tests was up to 11% and 16% for within-center and among-centers inequality, respectively. Moreover, in this situation, the ranking of the power of the homogeneity tests was BD≥DL≥LR and the power of these tests increased with increasing K. Conclusions This study shows that the adverse effect of among-centers inequality on the power of the homogeneity tests was stronger than that of within-center inequality. However, the financial limitations make the use of unequal sample size designs inevitable in multicenter trials. Moreover, although the power of the BD is higher than that of the LR when K≤6, the proposed mixed logistic model is recommended when K≥8 due to its practical advantages.


Background
The results from multicenter clinical trials or meta-analysis studies with binomial data are often summarized in K 2 × 2 contingency tables, where K denotes the total number of centers or studies. Combining data in such tables and proposition a summary measure is the primary objective of such studies. However, before computing the overall odds ratio, we often need to assess whether the specific odds ratios are homogeneous across tables [1][2][3][4].
Nowadays, investigators have a wide range of methods available for this purpose, including model-based and test-based approaches. The excellent simulation studies conducted by pioneer researchers in this field assist us in choosing the most appropriate test for the assessment of homogeneity among K 2 × 2 tables [1][2][3][4][5][6]. Nevertheless, the results of these simulation studies indicate that homogeneity tests show different behaviors under combinations of parameters such as the number of centers, center sizes and amount of heterogeneity [3,5,7].
In recent years, a class of models called mixed logistic models has been used for analysis of multicenter clinical trials with binomial data. Although Agresti has discussed a likelihood ratio (LR) test based on a mixed logistic model for testing homogeneity of odds ratios in K 2 × 2 contingency tables [8], the statistical properties of this test and the other traditional homogeneity tests such as Breslow-Day (BD) [9] and DerSimonian-Laird (DL) [10] are still unknown. A situation which occurs frequently in multicenter trials and has not been evaluated in previous studies is the effect of unequal sample size designs on the statistical properties of these homogeneity tests. For example, in some multicenter clinical trials, when one center is larger, it may seem reasonable to select a larger sample from it, but this leads to among-centers inequality. Moreover, within-center inequality could occur when the costs of two treatment groups are very different. In this situation, due to financial restrictions, it is reasonable to allocate more patients to the cheaper treatment in each center. This simulation study hence compares the empirical power and type I error rate of the three tests of homogeneity of odds ratios, including LR, DL and BD tests when the sample size is unequal within one center or among centers.

Methods
Consider a series of K independent 2 × 2 contingency tables, with the data in the kth table denoted as shown in Table 1. In this paper, K is the number of centers in multicenter clinical trials.
Suppose that each y ik follows a binomial distribution with parameters n ik and π ik (i = 1,2; k = 1,2,...,K). Let n ik denote the total number of observations in the ith treatment arm and kth center, and let π ik denote the success probability at treatment level x ik in the kth center, where x ik is the treatment indicator with x ik = 1 representing treatment 1 and x ik = 0 treatment 2.
In this paper, we focus on the following logistic-normal probability model in order to assess the homogeneity of odds ratios: In this model, the segment a + bx ik is the fixed effect part in which b is the common treatment effect. Here, u k and b ik are independent random components of the model, where u k is the center effect and u k ∼ N(0, σ 2 u ) and the parameter σ 2 u summarizes center heterogeneity. In addition, b ik is the center-by-treatment interaction random effect, b k ∼ N(0, σ 2 b ) and the parameter σ 2 b describes variability in the log-odds ratios [8]. The two advantages of working with this model are: first, a test of homogeneity of odds ratios can be performed by testing the null hypothesis: H 0 :σ 2 b = 0 against H 1 :σ 2 b > 0; and, second, the common treatment effect and also center-specific odds ratios can be obtained by estimating b and b ik [8].
It should be noted that the homogeneity test can be performed using likelihood ratio (LR) test Δ = -2(l 0l 1 ), where l 0 is the log-likelihood under the assumption of homogeneity of odds ratios and when there is just one random effect, u k , in the Model 1 and l 1 is the log-likelihood when both u k and b ik are in the model. Under the null hypothesis, the asymptotic distribution of the LR test is a mixture of a chi-squared distribution with zero and one degrees of freedom, respectively, both with weight of 1/2 [11,12].
Since the properties of the LR test in order to assess homogeneity of odds ratios had not been evaluated in the previous studies, we investigated the behavior of this test under various conditions, particularly under withincenter and among-centers inequality in multicenter trials and also compared it with the other two most common test statistics, including DL and BD tests. Brief calculation details of these statistics are given in the appendix [9][10][11][12].

Simulation study
We studied the statistical properties of the three mentioned-above homogeneity tests, in terms of empirical power and type I error rate, by simulation in SAS statistical package v 9.1. Our simulation design was based on the mixed logistic model 1; we set the fixed effect parameters for all simulation scenarios arbitrarily as: a = -1 and b = 1 . The random components u k and b ik are generated independently from normal distribution with mean zero and variance σ 2 u and σ 2 b . In order to generate the response variable, the binomial distribution with parameters n ik and π ik = exp(α used. The influence of within-center and among-centers inequality, different values of K, σ 2 u , σ 2 b and also number of observations in each centers, n k , on the statistical properties of all the homogeneity tests were evaluated. We generated 1000 random data sets for each simulation scenario. Namely, we chose three values for K (4, 6 and 8), two values for σ 2 u (0.1 and 0.5) and five values for σ 2 b (0, 0.2, 0.4, 0.6 and 0.8). In the equal sample size design, in which equal numbers of patients were allocated between two treatment arms and also among all centers, the number of individuals per center, n k , was set at 40, 100 and 200. In addition, in this study two forms of unequal sample size designs were considered. In the first one, ie within-center inequality, the same Table 1 Summary of data from the kth 2 × 2 contingency table

Success
Failure Total Total t k t k -n k n k sample size, which can be allocated equally between two treatment arms within a center, is allocated unequally in the ratio of 3:1. In this case, the sample sizes were considered equal in all centers. In the latter one, ie amongcenters inequality, the same total sample size, which can be allocated equally among centers, is allocated unequally. In this situation, one center has a much larger sample size in comparison with the other ones. The exact details of different sample size configurations are described in Table 2. In addition, SAS simulation code for performing the three homogeneity tests is presented in the additional file 1.

Equal sample size design
Tables 3 shows empirical type I error rate (when σ 2 b = 0) and power (when σ 2 b = 0) of the three homogeneity tests under the equal sample size design and various combinations of n k , K, σ 2 u , and σ 2 b . In terms of power, the tests are ordered BD≥DL≥LR in all cases and they perform similarly when the total sample size in each center is as large as 200 and σ 2 b ≥ 0.4. In addition, our findings reveal that substantial gain in the power of all homogeneity tests occurred with increasing number of centers, sample size per center and degree of heterogeneity of odds ratios, σ 2 b . However, increasing degree of heterogeneity of center, σ 2 u , from 0.1 to 0.5 resulted in a reduction of approximately 4.93% and 3.76% in the power of DL and BD, respectively, while no regular trend was detected for the change in LR. Table 4 shows type I error rate and the power of the homogeneity tests under within-center inequality for different values of n k , K, σ 2 u , and σ 2 b . Indeed, when σ 2 u = 0.1, as compared with the equal sample size design, there were decreases of approximately 16.35%, 9.93% and Table 3 Type I  The notation of E i is described in Table 2   Table 4 Type I error rate (when σ 2 b = 0) and the statistical power (when σ 2 b = 0) of homogeneity tests under withincenter inequality. The notation of W i is described in Table 2 Bagheri 11.62% in the power of LR test for within-center inequality, when K was equal to 4, 6 and 8, respectively. In addition, the power of the BD statistic decreased approximately by 10.67%, 7.79% and 7.15% and the power of DL was reduced approximately by 14.46%, 11.24% and 9.76% for K = 4, 6 and 8, respectively. It should be noted that, for a given value of K, the reported amount of reduction was the average reduction for all values of σ 2 b . Type I error rate and the power of the homogeneity tests under among-centers inequality are illustrated in Table 5. In fact, when σ 2 u = 0.1 as compared with the equal sample size design, there were decreases of approximately 17.15%, 11.78% and 11.89% in the power of LR test and 16.63%, 13.62% and 12.57% in the power of the BD statistic for among-centers inequality, when K was equal to 4, 6 and 8, respectively. In addition, this amount of reduction in the power of DL test was more critical and approximately equal to 21.56%, 18.4% and 16.61% for the same sequence of K.

Unequal sample size designs
It should be pointed out that the amount of reduction in the power of the three homogeneity tests for σ 2 u = 0.5 under two forms of unequal sample size designs was approximately similar to those of σ 2 u = 0.1.

Type I error rate
Tables 3, 4 and 5, when σ 2 b = 0, represent empirical type I error rate of the three homogeneity tests at the nominal significance level of 0.05. As indicated, LR test showed conservative behavior when K = 4, otherwise, the type I error rate was close to the nominal level. In addition, BD performed adequately in terms of type I error rate in almost all cases. On the other hand, type I error rate of the DL statistic was close or below the nominal level.

Discussion and conclusions
In a simple randomized clinical trial, the use of unequal allocation ratios, particularly the allocation ratio of 3:1, will significantly reduce the power of study for detecting significance difference between two treatments [13][14][15]. To our knowledge, few published studies investigated the impact of within-center and among-centers inequality on the statistical properties of the tests of homogeneity of odds ratios in multicenter clinical trials [1,3,4]. As illustrated in Tables 3, 4 and 5, the type I error rate of the three homogeneity tests is approximately close to the nominal level of 0.05 except for LR when K = 4. Since the results show that these tests have almost the same type I error rate, power comparisons are possible. As compared with the equal sample size design, the power of the LR, BD and DL tests will decrease if the same total sample size, which can be allocated equally within one center or among centers, is allocated unequally. In this case, the power ranking of the tests was BD≥DL≥LR. It is worth mentioning that, as compared with within-center inequality, among-centers inequality has stronger adverse effect on the power of the homogeneity tests. Despite the use of different tests, these findings are inconsistent with those of Paul, who reported the adverse effect of withincenter inequality to be stronger [3]. Table 5 Type I  The notation of A i is described in Table 2 Also, this paper shows how to use a mixed logistic model to test homogeneity of odds ratios in multicenter trials. In Model 1, there are two types of homogeneity: homogeneity of odds ratios among centers and homogeneity of centers. However, removing the center-by-treatment interaction from Model 1 leads to a model which can only be used to test homogeneity of centers. This model, which has been previously discussed by Gao, assumes that the odds ratios are constant over centers [16]. Therefore, it should not be used to generate data for comparing the tests of heterogeneity of odds ratios. Furthermore, the power of the three tests of homogeneity increases more when we increase the number of K and n k compared to when we increase the number of K and σ 2 b . This result is in agreement with previous studies which have evaluated the influence of K and n k on the power of the homogeneity tests [5,[17][18][19]. Nevertheless, our simulation study shows that the degree of amongcenters heterogeneity, σ 2 u , has little or no effect on the power of the three tests of homogeneity, except for DL when σ 2 b ≤ 0.4 and sample size is small. In addition, it is noteworthy that we used the DL statistic calculated from the one-way random effects model, which has approximately a chi-square distribution. However, Biggerstaff and Jackson [20] have calculated the exact distribution and power of the wellknown Q statistic based on the same random effects model, which can be used for testing homogeneity of odds ratios and be compared with the tests used in the present study.
In conclusion, of the three tests of homogeneity, the BD seems to be the most appealing with regard to its statistical properties: its type I error rate is close to the nominal level and its power is greater than that of DL and LR. Moreover, it has the advantage of simplicity of calculation and is recommended by a number of authors [1,[4][5][6]. However, one limitation of BD test is that it has low power when the sample size within each center is small, even if the number of centers is large [1,2]. Nevertheless, despite having low power under small number of centers and its complexity, Model 1 has its own advantages. Firstly, when the centers are a random sample themselves, the LR test from the Model 1 enables inferences to extend to the population of centers. Secondly, a further consideration is that common odds ratio can be estimated from the fixed part of the Model 1, even when the odds ratios are not homogeneous. Thirdly, in each center, Model 1 provides a predicted log-odds ratio that shrinks the sample value toward the mean. This is especially useful when the sample size in a center is small and the ordinary sample odds ratio has a large standard error [8]. In addition, the mixed logistic model described in this study will potentially be applicable to meta-analysis studies.
It is clear that, based on Model 1, the odds ratio in the kth center, as given in the appendix 4, is exp(b + b 1k b 2k ), which can be written as C × exp(b 1kb 2k ) where C = exp(b) .This indicates that the odds ratio in each center is absolutely independent of a and u k . Indeed, the odds ratios are affected by b 1k and b 2k , and b has the same effect on odds ratio in all centers. Hence, to generate heterogeneous odds ratios among centers, the fixed simulation parameters, ie a and b, can be chosen arbitrarily.
It should be noted that, although using unequal sample size designs in multicenter clinical trials reduces both the power of the study and the power of the homogeneity tests, a substantial reduction in the total cost of the trial will compensate for the reduction in the power of the statistical tests [14,15]. Finally, further research is warranted to investigate the influence of the number of centers, unequal sample size design, sparseness and also deviation from normal assumption of the random effects on the robustness and accuracy of the estimates of the fixed and random parameters of the Model 1.