Facilitating validation of prediction models: a comparison of manual and semi-automated validation using registry-based data of breast cancer patients in the Netherlands

Table 1 Comparison of manual and semi-automated validation using Evidencio’s validation tool

Model	Outcome	Calibration: intercept			Calibration: slope			Discrimination: AUC
Model	Outcome	Manual (M)	Semi-automated (S)	Difference (M-S)	Manual (M)	Semi-automated (S)	Difference (M-S)	Manual (M)	Semi-automated (S)	Difference (M-S)
CancerMath^a (n = 8900)	5-year OS	−0.02	−0.03	0.01	1.12	1.09	0.03	0.76	0.76	0
INFLUENCE^b (n = 12,308)	Conditional 5-year LRR	−0.01	−0.01	0	1.06	1.06	0	0.74	0.74	0
PPAM^c (n = 1416)	Probability of axillary metastasis	0	0	0	0.99	0.98	0.01	0.77	0.77	0
PREDICT 2.0^d (n = 8834)	5-year OS	−0.01	−0.01	0	1.03	1.02	0.01	0.80	0.80	0
PREDICT 2.0^d (n = 8834)	10-year OS	−0.01	−0.01	0	1.03	1.02	0.01	0.77	0.77	0

^a,b,c = logistic regression, ^d = Cox proportional hazard regression. PPAM Predicted Probability of Axillary Metastasis, n number of patients included for validation, OS overall survival, LRR locoregional recurrence, AUC area under the receiver operating characteristic curve

ISSN: 1471-2288