Skip to main content

Table 4 Discrimination and Calibration Performance in the Training and Testing Sets for Multivariate Adaptive Regression Spline Algorithm

From: Machine learning in medicine: a practical introduction to techniques for data pre-processing, hyperparameter tuning, and model comparison

 

Training set (cross-validation)

Testing set

Discrimination performance

  AUROC

0.84 (95% CI 0.82–0.86)

0.88 (95% CI 0.83–0.93)

  Accuracy

77.6% (95% CI 75.6%—79.5%)

81.2% (95% 74.9%—86.4%)

  No-information rate

53.7%

53.9%

  p-value [accuracy > no-information rate]*

< 0.001

< 0.001

  p-Value [Chi-Square goodness of fit]

0.10

0.08

Calibration performance

  Spiegelhatler’s Z-score

0.83

-1.31

  p-value

0.20

0.09

  1. *Refers to a one-sided binomial test determining whether the accuracy proportion is higher than the no-information rate