Skip to main content

Table 2 Simulation results for the setting in which there are 10 useful predictors and no noise variables. For bootstrap imputation methods on incomplete data, we show results corresponding to the best threshold values of π based on F1. The sample size n=1000. Two missingness proportions were considered: 40% missingness in Y and 60% overall missingness; 20% missingness in Y and 40% overall missingness. The performance measures were computed across 250 data replications

From: A flexible approach for variable selection in large-scale healthcare database studies with missing covariate and outcome data

  AUC Precision Recall F1 Type I error
Fully observed data
BART 0.74 (0.68, 0.80) 1.00 0.62 0.70 NA
XGBoost 0.75 (0.69, 0.81) 1.00 0.61 0.69 NA
Incomplete data: 40% missingness inY and 60% overall missingness
RR-BART 0.73 (0.67, 0.79) 1.00 0.36 0.48 NA
RR-BART (all selected) 0.97 (0.95, 0.99) 1.00 1.00 1.00 NA
BI-BART π=0.1 0.73 (0.67, 0.79) 1.00 0.38 0.50 NA
BI-XGB π=0.2 0.79 (0.73, 0.85) 1.00 0.54 0.64 NA
BART Complete cases 0.50 (0.43, 0.57) 1.00 0.15 0.35 NA
XGBoost Complete cases 0.55 (0.48, 0.62) 1.00 0.18 0.38 NA
MIA-BART (Impute missing Y) 0.66 (0.60, 0.72) 1.00 0.31 0.42 NA
MIA-BART (Exclude missing Y) 0.63 (0.56, 0.70) 1.00 0.25 0.40 NA
MIA-XGB (Impute missing Y) 0.73 (0.67, 0.69) 1.00 0.50 0.59 NA
MIA-XGB (Exclude missing Y) 0.70 (0.62, 0.77) 1.00 0.46 0.55 NA
Incomplete data: 20% missingness inY and 30% overall missingness
RR-BART 0.77 (0.72, 0.82) 1.00 0.51 0.67 NA
RR-BART (all selected) 0.98 (0.96, 0.99) 1.00 1.00 1.00 NA
BI-BART π=0.1 0.75 (0.70, 0.80) 1.00 0.50 0.69 NA
BI-XGB π=0.2 0.80 (0.75, 0.85) 1.00 0.52 0.70 NA
MIA-BART (Impute missing Y) 0.70 (0.65, 0.75) 1.00 0.46 0.60 NA
MIA-BART (Exclude missing Y) 0.67 (0.61, 0.73) 1.00 0.43 0.57 NA
MIA-XGB (Impute missing Y) 0.70 (0.65, 0.75) 1.00 0.46 0.64 NA
MIA-XGB (Exclude missing Y) 0.67 (0.61, 0.73) 1.00 0.42 0.61 NA
BART Complete cases 0.54 (0.49, 0.59) 1.00 0.16 0.39 NA
XGBoost Complete cases 0.56 (0.51, 0.61) 1.00 0.20 0.40 NA