Artificial neural networks versus proportional hazards Cox models to predict 45year allcause mortality in the Italian Rural Areas of the Seven Countries Study
 Paolo Emilio Puddu^{1}Email author and
 Alessandro Menotti^{2}
DOI: 10.1186/1471228812100
© Puddu and Menotti; licensee BioMed Central Ltd. 2012
Received: 19 October 2011
Accepted: 23 June 2012
Published: 23 July 2012
Abstract
Background
Projection pursuit regression, multilayer feedforward networks, multivariate adaptive regression splines and trees (including survival trees) have challenged classic multivariable models such as the multiple logistic function, the proportional hazards life table Cox model (Cox), the Poisson’s model, and the Weibull’s life table model to perform multivariable predictions. However, only artificial neural networks (NN) have become popular in medical applications.
Results
We compared several Cox versus NN models in predicting 45year allcause mortality (45ACM) by 18 risk factors selected a priori: age; father life status; mother life status; family history of cardiovascular diseases; jobrelated physical activity; cigarette smoking; body mass index (linear and quadratic terms); arm circumference; mean blood pressure; heart rate; forced expiratory volume; serum cholesterol; corneal arcus; diagnoses of cardiovascular diseases, cancer and diabetes; minor ECG abnormalities at rest. Two Italian rural cohorts of the Seven Countries Study, made up of men aged 40 to 59 years, enrolled and first examined in 1960 in Italy. Cox models were estimated by: a) forcing all factors; b) a forward; and c) a backwardstepwise procedure. Observed cases of deaths and of survivors were computed in decile classes of estimated risk. Forced and stepwise NN were run and compared by Cstatistics (ROC analysis) with the Cox models. Out of 1591 men, 1447 died. Model global accuracies were extremely high by all methods (ROCs > 0.810) but there was no clearcut superiority of any model to predict 45ACM. The highest ROCs (> 0.838) were observed by NN. There were intermodel variations to select predictive covariates: whereas all models concurred to define the role of 10 covariates (mainly cardiovascular risk factors), family history, heart rate and minor ECG abnormalities were not contributors by Cox models but were so by forced NN. Forced expiratory volume and arm circumference (two protectors), were not selected by stepwise NN but were so by the Cox models.
Conclusions
There were similar global accuracies of NN versus Cox models to predict 45ACM. NN detected specific predictive covariates having a common thread with physical fitness as related to job physical activity such as arm circumference and forced expiratory volume. Future attention should be concentrated on why NN versus Cox models detect different predictors.
Keywords
Neural networks Cox models Prediction Allcause mortality 45year followup Epidemiology Seven countries studyBackground
The predictive power assessment of risk factors by multivariable models such as the multiple logistic function, the proportional hazards life table Cox model, the Poisson’s model, and the Weibull’s life table model, is one of the cogent problems of contemporary cardiovascular epidemiology since the selection among these standard methods [1–3] has been challenged by other methods to perform multivariable predictions. New models included projection pursuit regression [4], multilayer feedforward networks [5] and multivariate adaptive regression splines [6]. Other methods, particularly trees (including survival trees) have been employed [7–10], although back prop nets may well perform better. However, among new comers [4–12] only artificial neural networks have become popular in medical applications [11, 12], which could also be possibly due to a lot of attention being focused on these techniques in other fields and so have become known in medicine. This larger acceptance and applicability relates to widespread availability of free, share, and commercialware [see: http://neuralnetworks.aidepot.com/Software.html and the recent increase of personal computer power. On the other hand, the necessity has been felt to cope with the limitations of methods such as logistic regression [13]. In fact, receiver operating characteristic (ROC) curves, indexing global predictive accuracy of logistic models [14–17] also comparatively [18–20], rarely exceeded 0.75 in the majority of epidemiological or clinical cardiovascular investigations [21, 22].
Implementation
When the performance and/or reliability of predictive models is limited, or of low sensitivity and specificity, their capability may be hampered to identify high risk subjects who deserve individualized treatment [21]. The neural network method stems [11, 12, 23, 24] from its potential for improved predictive performance by exploring hidden layers to find nonlinearities, interactions and nonlinear interactions among predictors, particularly when the data are continuous [25]. The attraction of neural networks is quite evident from the impressive growth of results published with these methods in the last 20 years [12]. However, there are relatively few comparative reports on the performance and accuracy of neural networks, which were assessed only versus multiple logistic function, to predict events in clinical [26, 27] or epidemiological [28, 29] cardiovascular studies and none has been performed versus models taking time into account such as the proportional hazards Cox model.
For the purpose of the present investigation we selected a priori[30] a series of covariates among those previously studied [31] to assess 40year allcause mortality predictive power among middleaged men of the Italian Rural Areas (IRA) of the Seven Countries Study (SCS). We used the 45year survival data to compare the global predictive accuracy of Cox and neural network models.
Cohorts and risk factors
The epidemiological material used for this analysis derives from the two Italian rural cohorts of the Seven Countries Study of Cardiovascular Diseases, made up of men aged 40 to 59 years, enrolled and first examined in 1960 [32] using standard methods [33, 34]. They represented 98.8% (n = 1712) of defined samples belonging to the rural communities of Crevalcore in Northern Italy and Montegiorgio in Central Italy. For the purposes of this analysis only baseline measurements of risk factors were considered, together with information on mortality over 45 years, although several reexaminations were conducted after the entry one.
Risk factors used in this analysis were those identified as significant in a previous analysis dealing with 40 years of followup of all cause mortality (except xantelasma, too rare and unstable) [31], plus heart rate, minor ECG abnormalities and family history of cardiovascular diseases which were promising but not reaching significance in the previous analysis. Altogether they were the following: age; father life status; mother life status; family history of cardiovascular diseases; jobrelated physical activity; cigarette smoking; body mass index (linear and quadratic terms); arm circumference; mean blood pressure; heart rate; forced expiratory volume; serum cholesterol; corneal arcus; diagnoses of cardiovascular diseases, cancer and diabetes; minor ECG abnormalities at rest. Unit of measurements and technical details, are reported in Additional file 1: Appendix 1.
Collection of data on vital status and causes of death was complete for 45 years. Causes of death were coded but not used for this analysis. The baseline survey was conducted well before the era of the Helsinki Declaration. On the occasion of subsequent examinations, verbal consent was obtained in view of collecting followup data. The endpoint of this analysis was allcause mortality in 45 years, and the corresponding survival in some analyses. The analysis was conducted on 1591 men who had all the measurements available.
Statistical analysis
Data are expressed as means ± SD or proportions and SE (when appropriate). Followup data, during 45 years, were investigated by modelling the presence (coded 1) or absence (coded 0) of allcause mortality using the proportional hazards model [31]. Cox proportional hazards models were estimated by: a) forcing all factors; b) a forward stepwise procedure; and c) a backward stepwise procedure (with p = < 0.05 as selection criterion for stepwise procedures). Plots of Schönfeld residuals over time were produced to test the proportionality of hazard. The coefficients and constants of the models were applied back to the original risk factor levels of all men, to obtain an estimated risk of death. Observed cases of survivors were computed in decile classes of estimated risk. NCSS software version 2007 (released August 14, 2007 by J Hintze, Kaysville, Utah; see http://www.ncss.com) was used.
Tiberius Data Mining © software (version 5.4.3; see http://www.tiberius.biz) was used to obtain multilayer perceptron (MLP) neural network solutions (see Additional file 1: Appendix 2 for details). Briefly, these were from a 3layer network, including the hidden unit containing 2 neurons (one linear and the second nonlinear), with 18 input nodes (corresponding to the 18 risk factors selected for Cox model) and one output unit, modelling the dichotomous risk outcome (for details and examples see Additional file 1: Appendix 2). MLPs were trained on all patterns but preventing overfitting [12], using procedures substantially similar to the forced or forward stepwise methods used by the Cox model. A method similar to bootstrap was used on 10 consecutive runs to obtain MPLs by both forced and forward methods. Corrado Gini’s coefficient and graph [35] were produced. A Gini coefficient is the area under the diagonal and the curve whereas the area under the curve is the total area under an ROC. Therefore it is easy to obtain: ROC = (Gini*0.5) + 0.5. MedCalc software (version 9.6.3.0; see http://www.medcalcsoftware.com) was used to calculate the area under an ROC with 95% confidence intervals (CI) and make comparisons [14, 15, 19]. ROCs were compared between models and among solutions obtained. A value of p<0.05 was considered statistically significant in all cases.
Results
Proportional hazards models predicting 45year allcause mortality (1447 deaths among 1591 men) by three methods
Forced Cox model  Forward stepwise Cox model  Backward stepwise Cox model  

(N = 1591)  (N = 1591)  (N = 1591)  
β  t  HR(±95%CI)  β  t  HR(±95%CI)  β  t  HR(±95%CI)  
Age (years)  0.1024  16.81  1.68(1.581.79)  0.1017  16.78  1.67(1.581.78)  0.1029  17.01  1.69(1.591.79) 
Father status (codes 0–1)  0.1422  2.19  1.15(1.011.31)  0.1421  2.19  1.15(1.011.31)  0.1436  2.21  1.15(1.021.31) 
Mother status (codes 0–1)  0.2063  3.11  1.23(1.081.40)  0.2121  3.22  1.24(1.091.41)  0.2072  3.16  1.23(1.081.40) 
Family history of CVD (codes 0–1)  0.0326  0.60  1.03(0.931.15)  =  =  =  =  =  = 
Corneal arcus (codes 0–1)  0.2092  2.70  1.23(1.061.43)  0.1988  2.57  1.22(1.051.42)  0.2037  2.63  1.23(1.051.43) 
Physical activity (codes 123)  −0.0830  −1.92  0.92(0.851.00)  =  =  =  −0.1042  −2.51  0.90(0.830.98) 
Cigarettes smoked per day (N)  0.0174  6.26  1.18(1.121.24)  0.0177  6.37  1.18(1.121.25)  0.0177  6.36  1.18(1.121.25) 
Body mass index (Kg/m^{2})  −0.1840  −2.77  0.51(0.310.82)  −0.1711  −2.62  0.53(0.330.85)  −0.2097  −3.30  0.46(0.290.73) 
Body mass index^{2} [(Kg/m^{2})^{2}]  0.0033  2.70  1.90(1.193.04)  0.0031  2.62  1.85(1.172.93)  0.0036  3.03  2.02(1.283.18) 
Arm circumference (cm)  −0.0026  −1.64  0.94(0.881.01)  −0.0034  −2.21  0.92(0.860.99)  =  =  = 
Mean blood pressure (mmHg)  0.0180  7.76  1.28(1.201.36)  0.0186  8.24  1.29(1.211.36)  0.0189  8.43  1.29(1.221.37) 
Heart rate (beats/min)  0.0018  0.78  1.02(0.971.09)  =  =  =  =  =  = 
Serum cholesterol (mmol/l)  0.0722  2.81  1.08(1.021.14)  0.0758  2.97  1.08(1.031.14)  0.0751  2.94  1.08(1.031.14) 
Forced expiratory volume (l/m^{2})  −0.4691  −3.93  0.89(0.840.94)  −0.4815  −4.05  0.89(0.840.94)  −0.4742  −3.99  0.89(0.840.94) 
ECGm abnormalities (codes 0–1)  0.0518  0.47  1.05(0.851.31)  =  =  =  =  =  = 
Prevalence CVD (codes 0–1)  0.3658  2.85  1.44(1.121.85)  0.3520  2.75  1.42(1.111.83)  0.3905  3.05  1.48(1.151.90) 
Prevalence K (codes 0–1)  2.1479  4.72  8.57(3.5120.89)  2.2056  4.87  9.08(3.7322.07)  2.1644  4.76  8.71(3.5821.22) 
Prevalence DIAB (codes 0–1)  0.2545  2.06  1.29(1.011.64)  0.2502  2.02  1.28(1.011.64)  0.2674  2.16  1.31(1.031.66) 
ROC ± standard error (±95%CI)  0.829 ± 0.0137 (0.8100.847)  0.830 ± 0.0136 (0.8110.849)  0.830 ± 0.0137 (0.8100.848) 
Multilayer perceptron models predicting 45year allcause mortality (1447 deaths among 1591 men) by two methods
Rank  Gini  Variable  Keep 

NEURAL NETWORK FORCED  
1  0.15950  Age (AGE0: years)  1 
2  0.60615  Cigarettes smoked per day (CIG0: N)  1 
3  0.64475  Family history of CVD(famcv0: codes 0–1)  1 
4  0.64790  Mean blood pressure (MBP: mmHg)  1 
5  0.65872  Serum cholesterol (CHOL0: mmol/l)  1 
6  0.67052  Corneal arcus (gero0: codes 0–1)  1 
7  0.67484  Mother status (Mother0: codes 0–1)  1 
8  0.67680  Prevalence DIAB (pdiab0: codes 0–1)  1 
9  0.68081  Forced expiratory volume (fev0trans0: l/m^{2})  1 
10  0.68321  Heart rate (hr0: beats/min)  1 
11  0.68362  Father status (Father0: codes 0–1)  1 
12  0.68414  Body mass index (BMI0: Kg/m^{2})  1 
13  0.68516  Minor ECG abnormalities (MinorECG0: codes 0–1)  1 
14  0.68524  Prevalence CVD (Pcvd0: codes 0–1)  1 
15  0.68579  Arm circumference (midclean0: cm)  1 
16  0.68664  Body mass index^{2} (BMIsq0) [(Kg/m^{2})^{2}]  1 
17  0.68813  Prevalence cancer (pcan0: codes 0–1)  0 
18  0.68872  Physical activity (PHYAC0: codes 123)  0 
NEURAL NETWORK STEPWISE  
1  0.17824  Age (years)  1 
2  0.60661  Cigarettes smoked per day (N)  1 
3  0.65370  Mean blood pressure (mmHg)  1 
4  0.67844  Mother status (codes 0–1)  1 
5  0.67918  Corneal arcus (codes 0–1)  1 
6  0.68104  Heart rate (beats/min)  1 
7  0.68347  Father status (codes 0–1)  1 
8  0.68468  Minor ECG abnormalities (codes 0–1)  1 
9  0.69231  Physical activity (codes 123)  1 
10  0.69329  Family history (codes 0–1)  1 
11  0.69480  Prevalence DIAB (codes 0–1)  1 
12  0.69603  Serum cholesterol (mmol/l)  1 
13  0.69854  Prevalence CVD (codes 0–1)  1 
Comparisons among receiver operating characteristics curves obtained by Cox versus neural network models predicting 45year allcause mortality (1447 deaths among 1591 men)
First Model  Compared to the Model  p 

Cox forced  Cox forward stepwise  0.9587 
Cox forced  Cox backward stepwise  0.9588 
Cox forced  Neural network forced  0.3478 
Cox forced  Neural network stepwise  0.3681 
Cox forward stepwise  Cox backward stepwise  1.0000 
Cox forward stepwise  Neural network forced  0.3478 
Cox forward stepwise  Neural network stepwise  0.4236 
Cox backward stepwise  Neural network forced  0.3861 
Cox backward stepwise  Neural network stepwise  0.4269 
Neural network forced  Neural network stepwise  0.7237 
Discussion
This is the first investigation to have ever compared in epidemiological material several methods to run Cox versus neural network models to predict 45year allcause mortality by a set of 18 risk factors (of which half were continuous) selected a priori. The global accuracies, assessed by Cstatistics (ROC analysis) were extremely high by all methods but there was no clearcut superiority of any model to predict 45year allcause mortality. There are intermodel variations to select predictive covariates among baseline variables. In particular, whereas all models concurred to define the role of 10 covariates (age, father and mother status, corneal arcus, cigarettes smoked per day, mean blood pressure, serum cholesterol and the prevalences of cardiovascular disease, cancer and diabetes), family history of CVD, heart rate and minor ECG abnormalities were not contributors by Cox models but were so by forced neural network model. Special attention needs be directed to the protective roles of forced expiratory volume and arm circumference, which may have a common thread with physical fitness as related to job physical activity [31, 32], since these variables were not selected by stepwise neural network (selecting instead physical activity) but were so by Cox models. Finally, the overall picture indicates an inverse J shaped relations for body mass index (except than by stepwise neural network).
Multivariable statistics and neural networks
There are excellent recent books [1, 2, 17] to have covered proportional hazards life table Cox model [36] and its use to assess the relationship between covariates and events including mortality. On the other hand, multilayer feedforward networks were demonstrated by Hornik et al. with appropriate internal parameters (weights) to approximate an arbitrary nonlinear function [5]. Because prediction can be restated as a function approximation problem, it follows that artificial neural networks have the potential to solve major problems in a wide range of applications where their use has been reviewed to show advantages and disadvantages for predicting medical outcomes [12]. What is particularly important with neural networks is that a multifactorial function can be fitted in such a way that creating the functional form and fitting the function are performed at the same time, unlike nonlinear regression in which a fit is forced to a prechosen function. This capability gives neural networks, at least potentially, an advantage over traditional statistical multivariable regression techniques [12].
Dayhoff and De Leo have recently reviewed what is inside the black box of neural network models in describing the most popular squashing function (also known as activation function) by which multilayered perceptron actually operates (see Additional file 1: Appendix 2). They have pointed out that with neural networks it is possible to mediate predictions for individual patients with prevalence and misclassification cost considerations using ROC methodology [12]. When a neural network is trained on a compendium of data, it builds a predictive model based on those data, by reflecting a minimization in error when the network’s prediction (its output) is compared with a known or expected outcome. Performance measurements would be taken to report the neural network’s level of success. The trained neural network then can be used to classify each new individual subject. This represents “a paradigm shift”, compared to previous methods whereby statistics concerning given populations are computed and published and a new individual subject then is referenced to the closest matching patient population for clinical decision support [12].
With all modelling methods an important part is the selection (and the number) of prognostic variables to be included in the model. The selection may be done a priori based on previous knowledge, as it was done in the present investigation, to prevent the data driven method used more often than not, which leads to a different set of variables being selected each time [30]. However, some inspiration was taken by a previous experience on this material exploring 40year predictive capabilities of allcause mortality [31]. The results of our study also showed that it is indeed important to take into consideration also the methods used to run the predictive models. In fact, when several variables are included, one may not obtain directly comparable solutions among different studies (or here different models), if the selected procedure is stepwise. This reinforces the importance to have forced solutions with a full model considering all selected covariates.
Comparing different predictive models
In the cardiovascular area, systematic comparisons of neural networks versus standard multivariable predictive models such as multiple logistic function has not been a common practice in either epidemiological [28, 29] or clinical [21–27] investigations. Cox and neural networks were not previously compared. Voss et al. [28] analysed 10year fatal (104 of 5.159 men, or 2%) and nonfatal (235 of 5.159 men, or 4.6%) CHD events among working men aged 35–65 years from the PROCAM study in Germany [28]. However, neural network and multiple logistic models were run with dissimilar covariates [30]. For example body mass index, height, presence and family history of hypertension were considered for neural network and not for multiple logistic function model. The conclusion was a superior performance of neural network versus logistic regression model in predicting 10year CHD events. ROCs were 0.897 (95% CI 0.8880.906) and 0.840 (95% CI 0.8300.851), respectively [28].
Since the PROCAM experience lacked external validation of the neural network model [28], as commented by May [30], a necessary step to delineate not only its predictive accuracy and potency, but also its generalization, which might be a potential advantage over conventional regression techniques [11, 12, 37, 38] we investigated 12763 men enrolled in the SCS. We compared 25year CHD mortality and the predictive discrimination of the multilayer perceptron neural network versus multiple logistic function based on 4 standard, continuous risk factors, selected a priori. CHD mortality prediction by training neural network or multiple logistic function had similar ROCs (below 0.699). The external validation of neural network models derived from the high (USA) and low (Italy) risk populations yielded comparable ROCs similar to the logistic solutions in Northern and Eastern Europe, but higher ROCs in two areas [0.633 (logistic) vs. 0.665 or 0.666 (neural network: p<0.05) in Southern Europe and 0.676 (logistic) vs. 0.725 or 0.737 (neural network: p<0.01) in Japan]. Thus 25year CHD prediction based on 4 continuous covariates showed lower global accuracies, both by neural network and logistic models [29], than 10year CHD prediction based on 13 covariates [28].
How to take advantage of all this
A lot of models in epidemiology, historically [32, 39], have been linearbased on the principle of parsimony [1–3]. If the principle of linearity predicts the data well, why go to more complex models? The question has received replies from the continuous developments of theoretical mathematicians and the parallel increase of computer computation power, so that curvilinear models [13, 36] are no more a problem also with personal computers [3, 17, 40–42]. This prompted new methods in the domain of survival prediction, among which neural networks [11, 12, 37, 38] are only a few [4–10]. If neural networks are interesting, the interest should be in two aspects. First, the overall predictability of the model impact is at increasing the identification of high risk subjects who deserve individualized treatment [21]. The stakes are high as better models may lead to prevention of more deaths from CHD [30]. The second should be the actual pattern of prediction. Does the neural network give an answer which is intuitively more appealing and explanatory than the logistic regression or other models? This would be the most interesting aspect of the method [12]. With neural network methodology a meaningful prediction that is unique to each individual might be produced (see Additional file 1: Appendix 2). By applying ROC methodology to model outputs, the decision for individual subjects can be tailored further, since cost tradeoff between falsepositive and falsenegative classifications might be examined [12].
There are obviously some limitations with neural networks but these may largely apply to standard multivariable methods as well. For example when applying neural networks for longterm (say more than 25 years) prediction of CHD deaths, the number of nonCHD deaths are too many and the time is not considered by neural networks. In this case, if nonCHD deaths are retained (as non cases) the predictive power of risk factors is diluted; if they are excluded the structure of the population and of its destiny are deformed. However, when prediction is referred to allcause mortality as in the present study, these limitations do not operate. A further limitation of this analysis is bound to the use of a single measurement of personal characteristics employed for prediction, ignoring the time changes occurred along time. The satisfactory, and even more significant results separately obtained in the 40year run of the IRA SCS cohort [31] made us confident in a valuable outcome of this analysis. The present essay may thus provide a baseline material to construct upon in the hope of further advancing our knowledge on risk factors for allcause mortality prediction by neural networks.
Consideration for the statisticallyeducated clinician
It is important to understand than global accuracy comparisons (using ROC curves) are often made on ‘holdout’ data, eg data that was not used to generate models in the first place, so as to test the generality of the models. It is instead best to compare on new data, although this has been done quite rarely [29, 43, 44]. On the other hand, similar to the present results, several medical studies have found conventional regression techniques to perform as well or better than more complex techniques [27–29, 44–46]. As for conventional techniques it is important to consider that full forced models should be used, since stepwise methods may convey unstable (or difficult to compare) results. As for more complex methods, an extremely important feature of the present study is the potential for a ‘paradigm shift’ in prediction, mentioned above in relation to neural network models. Although neural nets (and trees and various other machine learning techniques such as boosting, random forests etc.) allow individual predictions, back propagation neural network are still something of a ‘black box’ to the average clinician. Outputs of neural networks can be difficult to interpret for the ‘uninitiated’, although progress is being made.
Longterm allcause mortality prediction by risk factors
Comparisons with other studies are not easy due to different length of followup, different choice of risk factors and predictive models, and also because most of them dealt with a single risk factor, or few related risk factors. In the 30year followup experience of the Framingham Study serum cholesterol was directly associated with allcause mortality, at least for relatively younger subjects [47]: this was also observed here by all models, but was not the case in larger aggregate experience of the SCS in Europe [42]. Allcause mortality investigations were reviewed [42]. In the context of the General Post Office study in UK [48] among women and men aged 35–70, associations with systolic blood pressure were equally strong for women and men, that of serum cholesterol was higher in women, while associations with 2hour glucose levels was observed only in men. The strongest, most consistent predictor of mortality was smoking in women and poor lung function in men. ECG ischemia, although associated with cardiovascular mortality in both sexes was not associated with allcause mortality. In a Japanese study on elderly people, health behavior and social role were risk factors for allcause mortality along with age, low serum albumin, high blood pressure and ECG abnormalities, among a total of 30 personal characteristics [49]. Other studies (always reviewed in [42]) considered single or very specific risk factors for allcause mortality such as the limited influence of soilcadmium levels, the null influence of radar equipment, the direct role of respiratory symptoms, alcohol abuse, left bundle branch block, postload plasma glucose, high basal metabolic rates, high body mass index, pessimistic side of the Minnesota Multiphasic Personality Inventory OptimismPessimism Scale scores, and high physical work demand. None of previous investigations considered neural networks or undertook a comparative study of global predictive performance vs. Cox models or logistic regression.
Conclusions
Following the external validation of neural networks [29], at least in the context of 25year prediction of CHD mortality, a global conclusion should be that neural network models present some potential advantage [12], although the statistical difference as compared to standard multivariable methods such as logistic regression [28] or other more complex models [43] may have not a tremendous impact to call for their wider application. The evidence presented here about 45year allcause mortality prediction by 18 covariates is in line with these conclusions as ROCs were higher by neural networks (also in absolute terms since > 0.838: Figure 1) but there was no statistical differences vs. Cox models (ROCs > 0.810: Tables 1 and 3). A peculiarity may still reside on the capability of selecting covariates (such as in here arm circumference) that may go underestimated by different methods. Future attention should be concentrated on why neural network versus Cox models detect different predictors.
Abbreviations
 ACM:

Allcause mortality
 CHD:

Coronary heart disease
 IRA:

Italian rural areas
 ROC:

Receiver operating characteristic
 SCS:

Seven countries study.
Declarations
Acknowledgments
The cooperation of Dr Phil Brierley from NeuSolutions is acknowledged not only for having granted an Academic licence for Tiberius software, but also for suggestions and collaboration during the development of the analyses reported here.
Authors’ Affiliations
References
 Afifi AA, Clark V: Computer aided multivariate analysis. 1990, Van Nostrand Reinhold Co, New YorkGoogle Scholar
 Miller Ch C, Reardon MJ, Safi HJ: Risk stratification. 2001, University Press, A practical guide for clinicians. CambridgeView ArticleGoogle Scholar
 Menotti A, Puddu PE, Lanti M: Il rischio in Cardiologia: dalla teoria alla pratica. 2004, Edizioni Internazionali srl, PaviaGoogle Scholar
 Friedman JH, Stuetzle RJ: Projection pursuit regression. J Am Stat Assoc. 1981, 76: 817823. 10.1080/01621459.1981.10477729.View ArticleGoogle Scholar
 Hornik K, Stinchcomb X, White X: Miltilayer feedforward networks are universal approximators. Neural Net. 1989, 2: 359366. 10.1016/08936080(89)900208.View ArticleGoogle Scholar
 Friedman JH: Multivariate adaptive regression splines. Ann Stat. 1991, 19: 1141. 10.1214/aos/1176347963.View ArticleGoogle Scholar
 Ciampi A, Hogg SA, McKinney S, Thiffault J: RECPAM, a computer program for recursive partition and amalgamation for censored survival data and other situations frequently occurring in biostatistics. I. methods and program features. Comp Meth Progr Biomed. 1988, 26: 239256. 10.1016/01692607(88)900041.View ArticleGoogle Scholar
 Zhang H: Recursive partitioning and treebased methods. Handbook of computational statistics. Edited by: Gentle JE, Hardle W, Mori Y. 2004, Springer, BerlinGoogle Scholar
 Lee JW, Um SH, Lee JB, Mun J, Cho H: Scoring and staging systems using Cox linear regression modeling and recursive partitioning. Meth Inform Med. 2006, 45: 3743.PubMedGoogle Scholar
 Delen D, Oztekin A, Kong ZJ: A machine learningbased approach to prognostic analysis of thoracic transplantations. Artif Intel Med. 2010, 49: 3342. 10.1016/j.artmed.2010.01.002.View ArticleGoogle Scholar
 Tu JV: Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol. 1996, 49: 12251231. 10.1016/S08954356(96)000029.View ArticlePubMedGoogle Scholar
 Dayhoff JE, DeLeo JM: Artificial neural networks. Opening the black box. Cancer. 2001, 91: 16151635. 10.1002/10970142(20010415)91:8+<1615::AIDCNCR1175>3.0.CO;2L.View ArticlePubMedGoogle Scholar
 Hosmer DW, Lemeshow S: Applied logistic regression. 2000, John Wiley and Sons, New York, 2View ArticleGoogle Scholar
 Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982, 143: 2936.View ArticlePubMedGoogle Scholar
 Zweig MH, Campbell G: Receiveroperating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem. 1993, 39: 561577.PubMedGoogle Scholar
 Obuchowski NA: Receiver operating characteristic curves and their use in radiology. Radiology. 2003, 229: 38. 10.1148/radiol.2291010898.View ArticlePubMedGoogle Scholar
 Pepe MS: The statistical evaluation of medical tests for classification and prediction. 2003, Oxford University Press, New YorkGoogle Scholar
 Hanley JA, McNeil BJ: A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983, 148: 839843.View ArticlePubMedGoogle Scholar
 DeLong ER, DeLong DM, ClarkePearson DL: Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988, 44: 837845. 10.2307/2531595.View ArticlePubMedGoogle Scholar
 Bandos AI, Rockette HE, Gur D: A conditional nonparametric test for comparing two areas under the ROC curves from paired design. Acad Radiol. 2005, 12: 291297. 10.1016/j.acra.2004.08.013.View ArticlePubMedGoogle Scholar
 Hense HW: Observations, predictions and decisions assessing cardiovascular risk assessment. Int J Epidemiol. 2004, 33: 235239. 10.1093/ije/dyh118.View ArticlePubMedGoogle Scholar
 Shahian DM, Blackstone EH, Edwards FH, Grover FL, Grunkemeier GL, Naftel DC, Nashef SAM, Nugent WC, Peterson ED: Cardiac surgery risk models: a position article. Ann Thorac Surg. 2004, 78: 18681877. 10.1016/j.athoracsur.2004.05.054.View ArticlePubMedGoogle Scholar
 Warner BA: Thoughts and considerations on modelling coronary bypass surgery risk. Ann Thorac Surg. 1997, 63: 15291530.View ArticlePubMedGoogle Scholar
 Orr RK: Use of a probabilistic neural network to estimate the risk of mortality after surgery. Med Decis Making. 1997, 17: 178185. 10.1177/0272989X9701700208.View ArticlePubMedGoogle Scholar
 Altman DG: Categorizing continuous variables. Br J Cancer. 1991, 64: 97510.1038/bjc.1991.441.View ArticlePubMedPubMed CentralGoogle Scholar
 Lippmann RP, Shahian DM: Coronary artery bypass risk prediction using neural networks. Ann Thorac Surg. 1997, 63: 16351643. 10.1016/S00034975(97)002257.View ArticlePubMedGoogle Scholar
 Nilsson J, Ohlsson M, Thulin L, Höglund P, Nashef SAM, Brandt J: Risk factor identification and mortality prediction in cardiac surgery using artificial neural networks. J Thorac Cardiovasc Surg. 2006, 132: 1219. 10.1016/j.jtcvs.2005.12.055.View ArticlePubMedGoogle Scholar
 Voss R, Cullen P, Schulte H, Assmann G: Prediction of risk of coronary events in middleaged men in the prospective cardiovascular Münster study (PROCAM) using neural networks. Int J Epidemiol. 2002, 31: 12531262. 10.1093/ije/31.6.1253.View ArticlePubMedGoogle Scholar
 Puddu PE, Menotti A: Artificial neural network versus multiple logistic function to predict 25year coronary heart disease mortality in the Seven Countries Study. Eur J Cardiovasc Prev Rehabil. 2009, 16: 583591. 10.1097/HJR.0b013e32832d49e1.View ArticlePubMedGoogle Scholar
 May M: Commentary: improved coronary risk prediction using neural networks. Int J Epidemiol. 2002, 31: 12621263. 10.1093/ije/31.6.1262.View ArticleGoogle Scholar
 Menotti A, Lanti M, Maiani G, Kromhout D: Determinants of longevity and allcause mortality among middleaged men. Role of 48 personal characteristics in a 40year followup of Italian Rural Areas in the Seven Countries Study. Aging Clin Exp Res. 2006, 18: 394406.View ArticlePubMedGoogle Scholar
 Keys A, Blackburn H, Menotti A, Buzina R, Mohacek I, Karvonen MJ, Punsar S, Aravanis C, Corcondilas A, Dontas AS, Lekos D, Fidanza F, Puddu V, Taylor HL, Monti M, Kimura N, Van Buchem FSP, Djordjevic BS, Strasser T, Anderson JT, Den Hartog C, Pekkarinen M, Roine P, Sdrin H: Coronary heart disease in seven countries. Circulation. 1970, 41 (suppl 1): 1211.Google Scholar
 Anderson JT, Keys A: Cholesterol in serum and lipoprotein fractions: its measurement and stability. Clin Chem. 1956, 2: 145159.PubMedGoogle Scholar
 Rose G, Blackburn H: Cardiovascular survey methods. 1968, World Health Organization, GenevaGoogle Scholar
 Gini C: Measurement of inequality of incomes. Econ J. 1921, 31: 124126. 10.2307/2223319.View ArticleGoogle Scholar
 Cox DR: Regression models and life tables. J Roy Stat Soc. 1972, B43: 185220.Google Scholar
 White H: Learning in artificial neural networks: a statistical perspective. Neural Comput. 1989, 1: 425464. 10.1162/neco.1989.1.4.425.View ArticleGoogle Scholar
 Liestol K, Andersen PK, Andersen U: Survival analysis and neural nets. Stat Med. 1994, 13: 11891200. 10.1002/sim.4780131202.View ArticlePubMedGoogle Scholar
 Keys A, Aravanis C, Blackburn H, Buzina R, Djordjevic BS, Dontas AS, Fidanza F, Karvonen MJ, Kimura N, Menotti A, Mohacek I, Nedeljkovic S, Puddu V, Punsar S, Taylor HL, Van Buchem F: Seven Countries Study. A multivariate analysis of death and coronary heart disease. Edited by: Keys A. 1980, Harvard Univ Press, Cambridge, MassView ArticleGoogle Scholar
 Puddu PE, Brancaccio G, Leacche M, Monti F, Lanti M, Menotti A, Gaudio C, Papalia U, Marino B, OPRISK Study Group: Prediction of early and delayed postoperative deaths after coronary artery bypass surgery in Italy. Multivariate prediction based on Cox and logistic models and a chart based on the accelerated failure time model. Ital Heart J. 2002, 3: 166181.PubMedGoogle Scholar
 Sciangula A, Puddu PE, Schiariti M, Acconcia MC, Missiroli B, Papalia U, Gaudio C, Martinelli G, Cassese M: Comparative application of multivariate models developed in Italy and Europe to predict early (28 days) and late (1 year) postoperative death after on or offpump coronary artery bypass grafting. Heart Surg Forum. 2007, 10: E258E266. 10.1532/HSF98.20071021.View ArticlePubMedGoogle Scholar
 Puddu PE, Menotti A, Tolonen H, Nedeljkovic S, Kafatos A: Determinants of 40year allcause mortality in the European cohorts of the Seven Countries Study. Eur J Epidemiol. 2011, 26: 595608. 10.1007/s1065401196007. 8View ArticlePubMedGoogle Scholar
 Lim TS, Loh WY, Shih YS: A comparison of prediction accuracy, complexity, and training time of thirtythree old and new classification algorithms. Machine Learn. 2000, 40: 203228. 10.1023/A:1007608224229.View ArticleGoogle Scholar
 Wolfe R, McKenzie DP, Black J, Simpson P, Gabbe BJ, Cameron PA: Models developed by three techniques did not achieve acceptable prediction of binary trauma outcomes. J Clin Epidemiol. 2006, 59: 2635. 10.1016/j.jclinepi.2005.05.007.View ArticlePubMedGoogle Scholar
 Austin PC: A comparison of regression trees, logistic regression, generalized additive models, and multivariate adaptive regression splines for predicting AMI mortality. Stat Med. 2007, 26: 29372957. 10.1002/sim.2770.View ArticlePubMedGoogle Scholar
 Austin PC, Tu JV, Lee DS: Logistic regression had superior performance compared with regression trees for predicting inhospital mortality in patients hospitalized with heart failure. J Clin Epidemiol. 2010, 63: 11451155. 10.1016/j.jclinepi.2009.12.004.View ArticlePubMedGoogle Scholar
 Anderson KM, Castelli WP, Levy D: Cholesterol and mortality. 30 years of followup from the Framingham study. JAMA. 1987, 257: 21762180. 10.1001/jama.1987.03390160062027.View ArticlePubMedGoogle Scholar
 Ferrie JE, SinghManoux A, Kivimäki M, Mindell J, Breeze E, Smith GD, Shipley MJ: Cardiorespiratory risk factors as predictors of 40year mortality in women and men. Heart. 2009, 95: 12501257. 10.1136/hrt.2008.164251.View ArticlePubMedPubMed CentralGoogle Scholar
 Goto A, Yasumura S, Nishise Y, Sakihara S: Association of health behavior and social role with total mortality among Japanese elders in Okinawa, Japan. Aging Clin Exp Res. 2003, 15: 443450.View ArticlePubMedGoogle Scholar
 The prepublication history for this paper can be accessed here:http://www.biomedcentral.com/14712288/12/100/prepub
Prepublication history
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.