Performance of Firth-and logF-type penalized methods in risk prediction for small or sparse binary data

Rahman, M. Shafiqur; Sultana, Mahbuba

doi:10.1186/s12874-017-0313-9

BMC Medical Research Methodology

Table 3 Performance measures for the model s with both weak and strong predictive ability. Results were summarized over the number of simulations for which convergence is achieved. The maximum failure rate of convergence for RIDGE with weak predictive ability, out of 1000 simulations, is 40% for the lowest EPV

From: Performance of Firth-and logF-type penalized methods in risk prediction for small or sparse binary data

Model with weak predictive ability
		Calibration slope, Max MCE=0.0235					AUC, Max MCE=0.0012
EPV (N)		MLE	FIRTH	logF(1,1)	logF(2,2)	RIDGE	MLE	FIRTH	logF(1,1)	logF(2,2)	RIDGE
2(67)	Mean	0.367	0.414	0.383	0.424	1.029	0.606	0.605	0.605	0.607	0.628
	SD	0.277	0.303	0.281	0.302	0.847	0.060	0.058	0.059	0.059	0.042
3(100)	Mean	0.472	0.512	0.487	0.517	1.027	0.613	0.613	0.613	0.614	0.626
	SD	0.305	0.326	0.311	0.324	0.757	0.054	0.054	0.054	0.054	0.041
5(167)	Mean	0.621	0.658	0.637	0.658	1.055	0.629	0.630	0.630	0.630	0.635
	SD	0.317	0.328	0.317	0.323	0.667	0.046	0.046	0.046	0.046	0.039
10(334)	Mean	0.797	0.814	0.801	0.812	1.076	0.645	0.645	0.645	0.646	0.646
	SD	0.286	0.289	0.282	0.286	0.504	0.037	0.037	0.037	0.037	0.035
		root Brier Score, Max MCE=0.0007					APP (True 0.152), Max MCE=0.0015
EPV(N)		MLE	FIRTH	logF(1,1)	logF(2,2)	RIDGE	MLE	FIRTH	logF(1,1)	logF(2,2)	RIDGE
2(67)	Mean	0.370	0.369	0.367	0.365	0.360	0.159	0.178	0.154	0.153	0.156
	SD	0.022	0.019	0.019	0.018	0.017	0.045	0.041	0.044	0.044	0.044
3(100)	Mean	0.363	0.362	0.361	0.360	0.358	0.156	0.171	0.154	0.154	0.155
	SD	0.018	0.017	0.017	0.017	0.016	0.035	0.033	0.035	0.035	0.035
5 (167)	Mean	0.357	0.357	0.357	0.356	0.355	0.153	0.163	0.153	0.153	0.152
	SD	0.017	0.016	0.017	0.016	0.016	0.028	0.027	0.027	0.027	0.027
10 (334)	Mean	0.354	0.354	0.354	0.354	0.354	0.151	0.157	0.151	0.151	0.151
	SD	0.016	0.015	0.016	0.016	0.015	0.020	0.019	0.020	0.020	0.019
Model with strong predictive ability
		Calibration slope, Max MCE=0.0344					AUC, Max MCE=0.0024
EPV (N)		MLE	FIRTH	logF(1,1)	logF(2,2)	RIDGE	MLE	FIRTH	logF(1,1)	logF(2,2)	RIDGE
2(67)	Mean	0.659	0.825	0.784	0.890	1.252	0.831	0.831	0.832	0.834	0.832
	SD	0.296	0.310	0.268	0.273	0.742	0.039	0.039	0.038	0.037	0.037
3 (100)	Mean	0.774	0.888	0.857	0.931	1.125	0.845	0.845	0.846	0.846	0.845
	SD	0.236	0.251	0.231	0.233	0.292	0.028	0.028	0.028	0.028	0.028
5(167)	Mean	0.868	0.934	0.917	0.963	1.066	0.854	0.854	0.854	0.855	0.854
	SD	0.218	0.226	0.216	0.217	0.224	0.024	0.023	0.023	0.023	0.023
10(334)	Mean	0.933	0.959	0.955	0.979	1.016	0.860	0.860	0.860	0.860	0.860
	SD	0.167	0.169	0.166	0.167	0.159	0.022	0.022	0.022	0.022	0.022
		root Brier Score, Max MCE=0.0009					APP (True 0.162), Max MCE=0.0014
EPV (N)		MLE	FIRTH	logF(1,1)	logF(2,2)	RIDGE	MLE	FIRTH	logF(1,1)	logF(2,2)	RIDGE
2 (67)	Mean	0.338	0.331	0.330	0.327	0.328	0.172	0.182	0.164	0.163	0.167
	SD	0.030	0.022	0.021	0.020	0.019	0.045	0.040	0.042	0.042	0.042
3(100)	Mean	0.323	0.321	0.321	0.320	0.320	0.165	0.175	0.163	0.163	0.164
	SD	0.018	0.016	0.017	0.016	0.016	0.033	0.032	0.033	0.033	0.032
5(167)	Mean	0.316	0.315	0.315	0.315	0.315	0.163	0.170	0.163	0.163	0.163
	SD	0.016	0.015	0.015	0.015	0.015	0.026	0.025	0.025	0.025	0.025
10(334)	Mean	0.310	0.310	0.310	0.310	0.310	0.163	0.166	0.163	0.163	0.163
	SD	0.016	0.015	0.015	0.015	0.015	0.019	0.019	0.019	0.019	0.019

APP: Average Predicted Probability

Back to article page

ISSN: 1471-2288

Contact us

General enquiries: journalsubmissions@springernature.com