BMC Medical Research Methodology

Table 2 Overview of discrimination, calibration, and utility performance on external validation data

From: Multiclass risk models for ovarian malignancy: an illustration of prediction uncertainty due to the choice of algorithm

	PDI (95% CI)			ECI				NB at 10% (referrals avoided)
Model	PDI (95% CI)	Benign	BOT	Stage I	Stage II-IV	Sec. meta	Mean	NB at 10% (referrals avoided)
Models with CA125
MLR	0.54 (0.50–0.59)	0.060	0.098	0.023	0.013	0.035	0.046	0.33 (0.23)
Ridge MLR	0.49 (0.46–0.53)	0.113	0.240	0.089	0.038	0.158	0.128	0.33 (0.21)
RF	0.54 (0.50–0.59)	0.014	0.083	0.020	0.0002	0.037	0.031	0.34 (0.28)
XGBoost	0.55 (0.51–0.60)	0.017	0.055	0.009	0.007	0.041	0.026	0.34 (0.27)
NN	0.54 (0.50–0.58)	0.042	0.114	0.058	0.005	0.155	0.075	0.33 (0.23)
SVM	0.41 (0.39–0.43)	0.161	0.179	0.271	0.279	0.069	0.192	0.33 (0.23)
Models without CA125
MLR	0.51 (0.47–0.54)	0.065	0.131	0.043	0.029	0.013	0.056	0.34 (0.24)
Ridge MLR	0.47 (0.44–0.49)	0.038	0.206	0.073	0.004	0.111	0.086	0.34 (0.25)
RF	0.50 (0.46–0.54)	0.013	0.076	0.009	0.001	0.086	0.037	0.34 (0.24)
XGBoost	0.50 (0.46–0.54)	0.016	0.047	0.012	0.005	0.068	0.030	0.34 (0.25)
NN	0.50 (0.46–0.54)	0.048	0.049	0.048	0.008	0.089	0.048	0.33 (0.19)
SVM	0.42 (0.39–0.45)	0.215	0.254	0.414	0.228	0.022	0.227	0.33 (0.16)

PDI, polytomous discrimination index; ECI, estimated calibration index; BOT, Borderline tumor; NB, net benefit; CI, confidence interval; Sec, secondary; MLR, multinomial logistic regression; RF, random forest; NN, neural network; SVM, support vector machine
Net Benefit: the Net Benefit of treat all is 0.31
Referrals avoided: the net proportion of patients where an unnecessary referral (i.e. a false positive) was avoided relative to treat all (referring everyone)

Back to article page

ISSN: 1471-2288

Contact us

General enquiries: journalsubmissions@springernature.com