Skip to main content

Table 3 Performance measures for the model s with both weak and strong predictive ability. Results were summarized over the number of simulations for which convergence is achieved. The maximum failure rate of convergence for RIDGE with weak predictive ability, out of 1000 simulations, is 40% for the lowest EPV

From: Performance of Firth-and logF-type penalized methods in risk prediction for small or sparse binary data

Model with weak predictive ability
   Calibration slope, Max MCE=0.0235 AUC, Max MCE=0.0012
EPV (N)   MLE FIRTH logF(1,1) logF(2,2) RIDGE MLE FIRTH logF(1,1) logF(2,2) RIDGE
2(67) Mean 0.367 0.414 0.383 0.424 1.029 0.606 0.605 0.605 0.607 0.628
  SD 0.277 0.303 0.281 0.302 0.847 0.060 0.058 0.059 0.059 0.042
3(100) Mean 0.472 0.512 0.487 0.517 1.027 0.613 0.613 0.613 0.614 0.626
  SD 0.305 0.326 0.311 0.324 0.757 0.054 0.054 0.054 0.054 0.041
5(167) Mean 0.621 0.658 0.637 0.658 1.055 0.629 0.630 0.630 0.630 0.635
  SD 0.317 0.328 0.317 0.323 0.667 0.046 0.046 0.046 0.046 0.039
10(334) Mean 0.797 0.814 0.801 0.812 1.076 0.645 0.645 0.645 0.646 0.646
  SD 0.286 0.289 0.282 0.286 0.504 0.037 0.037 0.037 0.037 0.035
   root Brier Score, Max MCE=0.0007 APP (True 0.152), Max MCE=0.0015
EPV(N)   MLE FIRTH logF(1,1) logF(2,2) RIDGE MLE FIRTH logF(1,1) logF(2,2) RIDGE
2(67) Mean 0.370 0.369 0.367 0.365 0.360 0.159 0.178 0.154 0.153 0.156
  SD 0.022 0.019 0.019 0.018 0.017 0.045 0.041 0.044 0.044 0.044
3(100) Mean 0.363 0.362 0.361 0.360 0.358 0.156 0.171 0.154 0.154 0.155
  SD 0.018 0.017 0.017 0.017 0.016 0.035 0.033 0.035 0.035 0.035
5 (167) Mean 0.357 0.357 0.357 0.356 0.355 0.153 0.163 0.153 0.153 0.152
  SD 0.017 0.016 0.017 0.016 0.016 0.028 0.027 0.027 0.027 0.027
10 (334) Mean 0.354 0.354 0.354 0.354 0.354 0.151 0.157 0.151 0.151 0.151
  SD 0.016 0.015 0.016 0.016 0.015 0.020 0.019 0.020 0.020 0.019
Model with strong predictive ability
   Calibration slope, Max MCE=0.0344 AUC, Max MCE=0.0024
EPV (N)   MLE FIRTH logF(1,1) logF(2,2) RIDGE MLE FIRTH logF(1,1) logF(2,2) RIDGE
2(67) Mean 0.659 0.825 0.784 0.890 1.252 0.831 0.831 0.832 0.834 0.832
  SD 0.296 0.310 0.268 0.273 0.742 0.039 0.039 0.038 0.037 0.037
3 (100) Mean 0.774 0.888 0.857 0.931 1.125 0.845 0.845 0.846 0.846 0.845
  SD 0.236 0.251 0.231 0.233 0.292 0.028 0.028 0.028 0.028 0.028
5(167) Mean 0.868 0.934 0.917 0.963 1.066 0.854 0.854 0.854 0.855 0.854
  SD 0.218 0.226 0.216 0.217 0.224 0.024 0.023 0.023 0.023 0.023
10(334) Mean 0.933 0.959 0.955 0.979 1.016 0.860 0.860 0.860 0.860 0.860
  SD 0.167 0.169 0.166 0.167 0.159 0.022 0.022 0.022 0.022 0.022
   root Brier Score, Max MCE=0.0009 APP (True 0.162), Max MCE=0.0014
EPV (N)   MLE FIRTH logF(1,1) logF(2,2) RIDGE MLE FIRTH logF(1,1) logF(2,2) RIDGE
2 (67) Mean 0.338 0.331 0.330 0.327 0.328 0.172 0.182 0.164 0.163 0.167
  SD 0.030 0.022 0.021 0.020 0.019 0.045 0.040 0.042 0.042 0.042
3(100) Mean 0.323 0.321 0.321 0.320 0.320 0.165 0.175 0.163 0.163 0.164
  SD 0.018 0.016 0.017 0.016 0.016 0.033 0.032 0.033 0.033 0.032
5(167) Mean 0.316 0.315 0.315 0.315 0.315 0.163 0.170 0.163 0.163 0.163
  SD 0.016 0.015 0.015 0.015 0.015 0.026 0.025 0.025 0.025 0.025
10(334) Mean 0.310 0.310 0.310 0.310 0.310 0.163 0.166 0.163 0.163 0.163
  SD 0.016 0.015 0.015 0.015 0.015 0.019 0.019 0.019 0.019 0.019
  1. APP: Average Predicted Probability