Skip to main content

Archived Comments for: Bias, precision and statistical power of analysis of covariance in the analysis of randomized trials with baseline imbalance: a simulation study

Back to article

  1. Issues regarding simulation study and conclusions

    Jonathan Bartlett, London School of Hygiene and Tropical Medicine

    28 July 2014

    The topic of covariate adjustment in randomised trials is an important one. However, I believe the simulation study and conclusions of Edgewale are flawed, and moreover my concerns mirror those of one of the paper's original reviewer's (Gillian Raab), which appear not to have been dealt with.

    The authors investigate the performance of three methods for analysing randomised trials with a single continuous outcome and a corresponding baseline measure. They focus on the issue of baseline imbalance, and conduct a simulation study where trial data are generated such that there is, on average, an imbalance at baseline between the two treatment groups. From their simulation results, the authors conclude that ANCOVA is unbiased, whereas analysis of change scores and an unadjusted comparison of outcomes (ignoring baseline) are biased.

    The problem is that the above approach for generating data does not correspond to how data arise in randomised trials. As the authors explain in the introduction, randomisation guarantees balance in expectation or on average, but not balance in any given study. But in the simulation study conducted, data are generated so that there is systematically (i.e. on average) imbalance at baseline. It is therefore unsurprising that in this case the ANOVA analysis (a t-test of outcomes ignoring baseline) is biased. All this demonstrates is that if one has a confounder, and one does not adjust for the confounder, estimates are biased. Put another way, the simulations show the following: if one performs repeated randomised trials where patients are more likely to be allocated to one of the treatment groups if they have higher baseline values, then a ANOVA or analysis of change scores is biased. But of course this is not how patients are allocated to groups in a simple randomised trial!

    In a randomised trial, provided that the randomisation procedure is not compromised, all three of the methods considered by the authors are unbiased, at least according to the statistical definition of bias of an estimator as the difference between the expectation of the estimator and the true parameter value. The methods do however differ in terms of precision/efficiency, and previously it has been shown that ANCOVA is superior in this regard to the other two methods. All of these results can be found in the following paper:

    Yang L, Tsiatis A (2001). Efficiency study of estimators for a treatment effect in a pretest-posttest trial. The American Statistician; 55: 314-321.

    Competing interests

    No competing interests.