Fig. 3

Simulation distribution of AUC estimates obtained by different strategies. Boxplots showing distribution of AUC estimates across the 250 simulated data sets in a setting with moderate sample size (n=200), p=10 covariates, moderate missing at random (MAR) missingness (miss=25 % of values missing), balanced outcome class distribution (frac=0.5) and uncorrelated covariates (ρ=0) in the absence (theoretical auc=0.5; a) and presence (theoretical auc=0.66, see text; b) of a moderate true effect of the covariates on the outcome. The horizontal line denotes ‘true’ AUC related to a complete data set of size 200 (which is not necessarily equal to theoretical auc; see text). BS, bootstrap; CVK, K-fold CV; CVKrep, repeated K-fold CV; MI, multiple imputation; MI(-y), multiple imputation without including the outcome; No val., no validation (i.e., apparent performance); OOB, out-of-bag estimate; opt.corr., ordinary optimism-corrected estimate; SS, subsampling; Val, validation