Skip to main content
Download PDF

Machine learning is an effective method to predict the 90-day prognosis of patients with transient ischemic attack and minor stroke

BMC Medical Research Methodology volume 22, Article number: 195 (2022) Cite this article

Abstract

Objective

We aimed to investigate factors related to the 90-day poor prognosis (mRS≥3) in patients with transient ischemic attack (TIA) or minor stroke, construct 90-day poor prognosis prediction models for patients with TIA or minor stroke, and compare the predictive performance of machine learning models and Logistic model.

Method

We selected TIA and minor stroke patients from a prospective registry study (CNSR-III). Demographic characteristics,smoking history, drinking history(≥20g/day), physiological data, medical history,secondary prevention treatment, in-hospital evaluation and education,laboratory data, neurological severity, mRS score and TOAST classification of patients were assessed. Univariate and multivariate logistic regression analyses were performed in the training set to identify predictors associated with poor outcome (mRS≥3). The predictors were used to establish machine learning models and the traditional Logistic model, which were randomly divided into the training set and test set according to the ratio of 70:30. The training set was used to construct the prediction model, and the test set was used to evaluate the effect of the model. The evaluation indicators of the model included the area under the curve (AUC) of the discrimination index and the Brier score (or calibration plot) of the calibration index.

Result

A total of 10967 patients with TIA and minor stroke were enrolled in this study, with an average age of 61.77 ± 11.18 years, and women accounted for 30.68%. Factors associated with the poor prognosis in TIA and minor stroke patients included sex, age, stroke history, heart rate, D-dimer, creatinine, TOAST classification, admission mRS, discharge mRS, and discharge NIHSS score. All models, both those constructed by Logistic regression and those by machine learning, performed well in predicting the 90-day poor prognosis (AUC >0.800). The best performing AUC in the test set was the Catboost model (AUC=0.839), followed by the XGBoost, GBDT, random forest and Adaboost model (AUCs equal to 0.838, 0, 835, 0.832, 0.823, respectively). The performance of Catboost and XGBoost in predicting poor prognosis at 90-day was better than the Logistic model, and the difference was statistically significant(P<0.05). All models, both those constructed by Logistic regression and those by machine learning had good calibration.

Conclusion

Machine learning algorithms were not inferior to the Logistic regression model in predicting the poor prognosis of patients with TIA and minor stroke at 90-day. Among them, the Catboost model had the best predictive performance. All models provided good discrimination.

Peer Review reports

Introduction

Stroke was the second leading cause of death worldwide and the leading cause of mortality and disability in China [1, 2]. In previous studies in multiple countries, stroke recurrence rate 90 days after TIA/minor stroke was 18%-20% [3]. About 40% of stroke survivors have a poor prognosis (modified Rankin Scale [mRS] score ≥3) between 1 month and five years after stroke [4]. In the past clinical work, both patients and medical workers tend to pay more attention to the secondary prevention of non-minor stroke patients, ignoring the poor prognosis of minor stroke. According to the data of the Third China National Stroke Registry (CNSR-III), TIA and minor stroke (a National Institutes of Health Stroke Scale (NIHSS) score≤5) account for about 73% of acute ischemic stroke cases. Therefore, it was essential to predict the prognosis of patients with TIA and minor stroke, find risk factors, identify high-risk patients, and accurately carry out early intervention for patients with TIA and minor stroke. With the improvement of computer computing power, the advent of the era of big data, and the update of algorithms, machine learning has made good progress in disease prediction. However, there were few comparative studies on multiple tree model machine learning algorithms. This study used the CNSR-III database to focus on factors related to the 90-day prognosis of TIA and minor stroke patients and compared the predictive performance of machine learning models and the Logistic model to provide references for related research and clinical work.

Method

Data availability statement

All anonymized data in this study could be shared by request from any qualified investigator.

Study design and patients

The CNSR-III database was a nationwide prospective clinical registry of ischemic stroke or TIA in China based on etiology, imaging, and biological markers. The detailed study design of the CNSR-III trial has been described elsewhere [5]. Between August 2015 and March 2018, the CNSR-III recruited consecutive patients with ischemic stroke or TIA from 201 hospitals that covered 22 provinces and four municipalities in China. Written informed consent was obtained from the patients or their legal representatives. Clinical data were collected prospectively using an electronic data capture system by face-to-face interviews. Brain imaging, including brain magnetic resonance imaging (MRI) and computed tomography (CT), were completed at baseline. Blood samples were collected and biomarkers were tested at baseline. The registry recruited consecutive patients who met the following criteria: age >18 years; ischemic stroke or TIA; within 7 days from the onset of symptoms to enrolment; Acute ischemic stroke was diagnosed according to the World Health Organization (WHO) criteria [6] and confirmed by MRI or brain CT. Patients who had silent cerebral infarction with no manifestation of symptoms and signs or who refused to participate in the registry were excluded. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the ethics committee of Beijing Tiantan Hospital (No.: KY2015-001- 01) and all study centers gave ethical approval of the study protocol. Written consents were obtained from all participants or their legal representatives.

In this study, minor stroke was defined as an NIHSS score≤5. There were 15,166 patients in CNSR-III, and 4086 patients with an NIHSS score >5 were excluded.

There were 11,080 patients with TIA and minor stroke (an NIHSS score≤5). 113 patients were excluded (Including patients whose mRS score was missing for 90-day in the follow-up data). A total of 10967 patients were eligible for the study. Supplementary Figure 1 shows a detailed flow chart for the study population selection from CNSR-III.

Baseline variables

For baseline variables, we investigated demographic characteristics (sex, age, BMI, race, family income/monthly, education level, living conditions), smoking history, drinking history (≥20g/day), physiological data (systolic blood pressure, diastolic blood pressure, heart rate), medical history (including stroke, hypertension, diabetes, heart disease and lipid metabolism disorders),secondary prevention treatment, in-hospital evaluation and education (including swallowing function, Limb rehabilitation and stroke-related education), laboratory data, neurological severity (admission and discharge), mRS score (admission and discharge) and TOAST classification. Finally, a total of 44 variables were included as baseline variables for analysis (Supplementary Table 1). Neurological severity was scaled by the National Institutes of Health Stroke Scale score.

Clinical outcomes

The clinical outcome of this study was poor prognosis at 90-day. In this study, a poor prognosis was defined as the modified Rankin Scale (mRS) ≥ 3, and 0-2 was defined as a good prognosis

Classification algorithm

We used various machine learning techniques to predict poor prognosis at 90-day: CatBoost (CB) [7], XGBoost (XGB) [8], Gradient Boosting Decision Tree (GBDT) [9], Random forest (RF) [10], and AdaBoost (Ada) [11].

CB: CatBoost is an innovative ordered gradient boosting algorithm, which uses ordered target-based statistics for categorical features processing and permutation strategies to avoid prediction shift. Its base learner was an oblivious tree and each tree corresponds to a partition of the feature space. The model learns the feature space partition at each training iteration and finally obtains the aggregated data as a classification result [7].

XGB: XGBoost was a Boosting library developed by Chen Tianqi of the University of Washington in 2016. It has both a linear scale solver and a tree learning algorithm. XGBoost was a second-order Taylor expansion of the loss function, and a regular term was added to the objective function to find the optimal solution as a whole, which was used to weigh the decline of the objective function and the complexity of the model, avoid overfitting, and improve the model The efficiency of the solution [8].

GBTD: The decision tree used in GBDT was a regression tree. The goal of each training was to reduce the error of the last training and finally get the minimum error. The model uses the gradient descent method to reduce the error [9, 12].

RF: RF was an integrated supervised learning method, which consists of multiple decision trees corresponding to different sub-data sets. Calculate the results for each tree, and get the average of the predicted results. This approach allows reducing variance in decision trees [10, 13].

Ada: The basic idea of the AdaBoost algorithm was to classify a group of weak learners through weighted majority voting (or sum). It takes into account the mistakes of previous weak learners and repeatedly updates the data [11, 14, 15].

Data preprocessing

Missing value processing: Continuous variables were filled using linear imputation, and categorical variables were filled using mode. Most of the 44 variables had no missing values, and 4 variables had missing values greater than 5%. We examined the distribution of the laboratory data after imputation of the variables used in the modeling and found no significant difference in the distribution of the data before and after imputation (Supplementary Tables 1 and 3).

Feature selection

Logistic regression was used for feature selection in our study. Logistic regression was the most commonly used model for characterizing the relationship between a dependent variable and one or more explanatory variable. LR models had a long and well-known theoretical and computational background, and their regression parameters and language were generally accepted. In the training set, we used univariate logistic analysis to compare baseline characteristics of 90-day prognosis. The risk factors selected by univariate analysis were included in multivariate analysis using stepwise regression method. Variables with P < 0.05 were used as predictors for the establishment of the 90-day poor prognosis prediction model in this study.

Statistical analysis

Continuous variables were expressed as the mean ± standard deviation, and classification variables were expressed as a percentage. After data preprocessing, the good prognosis group (mRS≤2) and the poor prognosis group (mRS≥3) were randomly grouped according to a ratio of 70:30, divided into a training set and test set, repeated five times. The test set was only used for model testing. We use GridSearch CV for 5-fold cross-validation tuning in the training set. After tuning each model to the optimum in the training set, we performed model evaluation on a clean test set to test the predictive performance of the five models established in this study. Supplementary Table 2 showed the parameters involved in each model in this study. Data analysis application SAS software (SAS9.4) and Python software (Python v3.6.8) completed. When comparing the predictive performance of different models in this study, the comparison and evaluation were mainly conducted from the two aspects of discrimination and calibration. In this study, the discriminative index was the area under the curve (AUC). The higher the AUC value, the higher the discriminative degree of the model. The calibration index of this study adopts Brier score [16] (scoring range was 0 ~ 1). The closer the Brier score was to 0, the better the calibration of the model. Two-sided probability values <0.05 were considered statistically significant.

Result

Demographic and clinical characteristics

A total of 15166 patients were registered to the cohort during the study period. After excluding 4086 patients with NIHSS score >5 and 113 patients with missing clinical outcomes(90-day mRS), 10967 patients were finally included (Supplementary Figure 1).

The mean age of the 10967 patients was 61.77±11.18 years and 30.68% were female (Supplementary Figure 1 and Table 1). The training set included 7676 patients and the test set included 3291 patients. We performed feature selection in the training set (n=7676). First, we performed univariate analysis in the training set and found that 28 variables (P<0.05) differed between patients with and without a poor functional outcome at 90-day (based on 44 baseline variables). They were demographics (including sex, age, family income/monthly and education level), smoking history, drinking history (≥20g/day), physiological data (systolic blood pressure and heart rate),medical history (including stroke, hypertension, diabetes and heart disease), secondary prevention treatment, in-hospital evaluation (swallowing function, Limb rehabilitation), laboratory data, neurological severity (admission and discharge), mRS score (admission and discharge) and TOAST classification. Table 1 showed the total population and baseline data for both groups.

Table 1 Baseline data for the total population and for both groups
Full size table

Determining predictors of poor prognostic outcome

In the training set, the risk factors selected by the univariate analysis were included in the multivariate analysis. Multivariate Logistic regression results showed that sex, age, stroke history, heart rate, D-dimer, creatinine, TOAST classification, admission mRS, discharge mRS and discharge NIHSS score were related to the 90-day poor prognosis (P<0.05). The above factors can be used as predictive indicators to build a predictive model. In TOAST classification, LAA was set as a control variable. In laboratory indicators, D-dimer and creatinine were risk factors for a poor prognosis of 90-day (OR=1.051, OR=1.003, respectively). Compared with males, females were risk factors for a poor prognosis of 90-day (OR= 1.226),other factors were showed in Table 2 below.

Table 2 Association between predictors and poor functional outcome in multivariable analysis (In the training set)
Full size table

Predictive performance of different models

In the training set, We used the multivariate Logistic regression model as a feature selection method, and determined 10 variables as predictors to establish different prediction models. Five tree model classifiers, namely CB, XGB, GBDT, RF and Ada, were trained on the training data set repeated 5 times, and 5-fold cross-validation was performed to adjust the optimal parameters. Finally, the five optimal models adjusted training sets will be evaluated in test sets. The results of AUC, accuracy, PPV, NPV, F1-score, and Brier score in the test set were showed in Table 3. Compared with other machine learning classifiers, and the CB model had the highest AUC of 0.839, followed by XGB, GBDT, RF, and Ada models (0.838, 0, 835, 0.832, 0.823, respectively). The AUC of all machine learning classifiers was higher than the Logistic model (AUC=0.822). Supplementary Figure 2 showed the ROC curve and the area under the curve (AUC) of each machine learning classifier in test sets compared with the Logistic model. The prediction performance of the CB and XGB model was better than the Logistic model, and the difference was statistically significant (P<0.05). Figure 1 showed ROC curve and calibration plots of CB and XGB models on test sets. In terms of calibration, CB, XGB, and GBDT had the best calibration (Brier scores were all 0.047), and the Ada model had the worst calibration (Brier score=0.159) (Table 3). In addition, Supplementary Figure 3 showed calibration plots of each model on test sets. SHAP values for the two models (the CB model and the XGB model) were assessed in the test set, and are shown in Fig. 2, respectively.

Table 3 Test sets result of machine learning models and the Logistic model on 90-day stroke outcome prediction
Full size table
Fig. 1
figure 1

Calibration plots for prediction of stroke outcome at 90-day on test sets: A the Catboost model, B the XGBoost model

Full size image
Fig. 2
figure 2

SHapley Additive exPlanations (SHAP) plots, ranking plot of shap values on test sets. The blue to red color represents the feature value (red high, blue low). The x-axis measures the impacts on the model output (right positive, left negative). (A) the Catboost model, (B) the XGBoost model

Full size image

Discussion

Machine learning (ML) methods have gained increasing popularity in medical research. Machine learning-based algorithms may be used for screening, diagnostic, or prognostic purposes. ML methods have been tested in several medical conditions to predict a future health state in cardiovascular medicine. This study aimed two-fold: On the one hand, the training set of this study was used to discover the related factors of poor 90-day prognosis in patients with TIA and minor stroke, which provided reference for related research and clinical work. On the other hand, factors found in training sets were used as indicators for predicting poor patient prognosis at 90-day. 90-day poor prognosis prediction models for TIA and minor stroke patients were constructed, and the prediction performance of the ML algorithm and the Logistic model was compared.

There were two significant findings of this study:

  1. (1).

    We performed univariate and multivariate analyses of patients with TIA and minor stroke in the training set and found that the 90-day prognosis prediction of patients with TIA and minor stroke was determined by many factors: sex, age, stroke history, heart rate, D-dimer, creatinine, TOAST classification, admission mRS, discharge mRS, and discharge NIHSS score.

  2. (2).

    The discrimination and the calibration of each model were good. The discrimination (AUC) between the CB model and the XGB model was better than the Logistic model (P<0.05), and CB model has the best prediction performance.

In previous studies, the prognosis of stroke was affected by factors such as stroke severity (NIHSS score) [18, 19], age [18,19,20,21], sex [20,21,22], and comorbidities [22,23,24]. In this study, the 90-day prognosis of TIA and minor stroke patients were determined by many factors. Such as sex, age, stroke history, heart rate, D-dimer, creatinine, TOAST classification, admission MRS, discharge MRS , and discharge NIHSS score.

In this study, we found that the risk of 90-day poor prognosis was higher for older TIA and minor patients, and the risk of 90-day poor prognosis was higher for females than for males. The human body's natural aging process was irreversible with age Therefore, age and sex were non-intervention factors. In addition, the proportion of stroke history in the poor prognosis group was significantly higher than that in the good prognosis group, suggesting that stroke history was a predictor of the 90-day poor prognosis. Previous studies had shown that high heart rate was an independent risk factor for stroke and cardiovascular and cerebrovascular events. Patients with high a heart rate had a significantly higher incidence of stroke and adverse cardiovascular and cerebrovascular outcomes [25,26,27,28]. This study found that TIA and minor patients with high heart rate levels had a higher risk of poor prognosis at 90-day. This might be related to the high level of heart rate that increased myocardial oxygen consumption and reduces cardiac reserve. It was also possible that the excessive activation of sympathetic nerves damages the cardiovascular system through multiple mechanisms, leading to a poor prognosis of stroke patients at 90-day. This study suggested that the heart rate level was related to the 90-day poor prognosis prediction for patients with TIA and minor stroke.

This study found that a high level of creatinine at admission was associated with a poor prognosis at 90-day, and patients with a high creatinine level were at higher risk of a poor prognosis at 90-day. In recent years, studies found a specific correlation between creatinine and nerve damage, but there were few clinical correlations between creatinine and nerve damage and stroke prognosis [29]. Unlike previous studies [30], this study suggested that early detection of creatinine was of great significance for the prognosis prediction of patients with TIA or minor stroke. Although the D-dimer levels were elevated in patients with ischemic stroke, the relationship with a poor prognosis was unclear. Elevated D-dimer levels may be related to several factors [31]. (1) The level of D-dimer in stroke patients was positively correlated with the infarct volume; (2) The high level of D-dimer was an indicator of systemic hypercoagulability. (3)D-dimer could activate inflammation. (4) The high level of D-dimer could indicate that the thrombus has a higher tolerance for endogenous fibrinolytic system thrombolysis. (5) D-dimer levels were high, often accompanied by venous thrombosis of the lower extremities. However, D-dimer had the advantages of high in vitro activation tolerance and long half-life and rarely increased in the blood-brain barrier of healthy people [32]. This study suggested that high levels of D-dimer were associated with a poor prognosis at 90-day. Patients with high D-dimer levels were at high risk of poor prognosis at 90-day. Previous studies had also showed that higher D-dimer levels could predict the prognosis of acute myocardial infarction [33]. In short, the early detection of changes in patients' creatinine and D-dimer levels has important application value in predicting the outcome and prognosis of TIA and minor stroke patients. At present, studies had showed that TOAST classification was related to the severity and prognosis of stroke patients [34]. This study suggested that TOAST classification was correlated with the prognosis prediction of TIA or minor stroke patients. Therefore, early determination of the TOAST classification of TIA and minor stroke patients was of great significance for secondary prevention and predicting the poor prognosis of patients. NIHSS score was currently the most commonly used scale for acute ischemic stroke in the world. The NIHSS score was closely related to the patient's cerebral infarction volume, location, and other factors. It could be used to efficiently and effectively evaluate the degree of neurological impairment in patients with acute ischemic stroke. Patients discharged from the hospital with a high NIHSS score were generally more severely ill and had a larger brain tissue infarction volume, which had a lot to do with the poor prognosis of patients [35]. In addition, this study showed that the mRS score at admission and discharge was related to the 90-day prognosis of patients with TIA and minor stroke, and the higher the score, the higher the risk of adverse prognosis.

All six models showed good discrimination in this study (AUC>0.80), which was consistent with previous studies [36,37,38,39]. Among them, the CB model had the highest prediction performance, followed by XGB, GBDT, RF, Ada, and Logistic (AUC were 0.839, 0.838, 0.835, 0.832, 0.823, and 0.822, respectively), where CB and XGB were better than the Logistic model, the difference was statistically significant (P<0.05). Conventionally, AUC values>0.70 were considered to represent moderate discrimination, values>0.80 good discrimination, and values>0.90 excellent discrimination. In terms of calibration, the Brier score of each model was better, and the CB, XGB and GBDT models have the best correction effect (Brier score=0.047). In addition, Supplementary Figure 3 showed calibration plots of each model on test sets. It can be seen that the calibration curves of each model were better, especially the CB model and the XGB model, indicating that there was no overfitting on test sets.

In previous studies, when comparing ML models and the traditional Logistic model, ML models were superior to logistic model likely due to the features instead of the algorithm itself [36, 40,41,42], because different models often use different predictors. In addition to the predictors, the characteristics of the algorithm are also important factors to consider when selecting a model suitable for medical application. Unlike most of previous studies in machine learning, this study used the same features selected by the logistic model. Our finding at ML models outperformed logistic model in that circumstance. In recent years, many studies had used SHAP value to explain the "black boxes", and its core idea is to calculate the marginal contribution of features to the model output. Although the degree of contribution and the impact on outcome variables can be seen in the SHAP plot, in clinical studies, there is a lack of quantitative explanations for the specific impact of each clinical variable. Logistic regression, as a traditional statistical model, has the advantage of being able to explain the relationship between variables and outcomes well compared to pure machine learning algorithms for feature selection. In the process of feature selection, traditional logistic regression can use its confidence interval and OR indicator to specifically represent the relationship between predictors and outcomes, making up for the problem of machine learning “black boxes”.

This study showed that compared with the Logistic model, the CB model showed the highest AUC. This result showed that the CatBoost algorithm can well predict patients with poor 90-day prognosis from TIA and minor stroke patients. CatBoost was a new integrated algorithm based on decision tree gradient lifting developed by researchers and engineers of Yandex in Russia. Previously, the two mainstream algorithms in the Boosting family were XGBoost and LightGBM, and according to official evaluation, the new CatBoost member model in the Boosting family had better performance than the above two algorithms. It was also consistent with the results of this study. CatBoost’s success might have been explained by its ability to process categorical features and model feature combinations. Additionally, CatBoost’s new capacity in undertaking feature combinations increased its nonlinear modeling abilities. In addition, although the LR algorithm performed well, it may also be because we used the stepwise regression method to screen the characteristic variables. Perhaps the CatBoost algorithm will be more convenient when there are more feature variables, a larger amount of data, or even multivariate heterogeneous data, and the performance difference between the two algorithms will be greater. As an emerging algorithm, CatBoost has unique advantages. It competes with any leading machine learning algorithm and handles categorical features automatically, requiring less hyperparameter tuning, enhancing model stability and reducing the possibility of overfitting.

As our analysis showed, machine learning models, especially CatBoost algorithm, showed promising results in predicting poor outcome in patients with 90-day TIA and minor stroke. Due to the particularity of clinical research, it is extremely important to select variables based on prior knowledge, and to use stepwise regression with more interpretability for features selection. Although our study used only 10 predictors for modeling, the machine learning model, especially the CB model, showed better predictive performance without sacrificing accuracy. And due to the low cost of the number of features, it is more convenient to perform external verification and input into Clinical Decision Support System (CDSS) for clinical practical application prediction in the future. The knowledge-driven model of combining new algorithms in machine learning may complement the purely data-driven approach of previous research in the field of machine learning for disease. I believe that our research can be applied to electronic health records to provide services for doctors, health care workers.

This study has several limitations. First, although the ML algorithm could have high accuracy and AUC performance, especially CB models, we still need more external validation to check the robustness of ML model. Second, it could be combined with a better selection algorithm to improve accurate prediction In the future. Third, imaging and omics data were not included in this study, which may limit the predictive performance to some extent. Fourth, this study performed a single imputation of missing values, which will inevitably lead to certain bias, but if the missing values are deleted, certain selection bias cannot be avoided; Since mehtods of directly testing missing at random is not available yet, we are not confident to state that variables with >5% missing values are missing at random [43]. Nevertheless, we believe that the selection bias would be minimized in this study as the baseline characteristics between included and excluded patients were largely comparable.

We will further explore the adaptation conditions of different models for ischemic stroke and conduct comprehensive studies on the development of predictive models and predictive performance to provide a more comprehensive reference for establishing a perfect prognosis prediction of ischemic stroke.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Collaborators GBDCoD. Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392(10159):1736–88. https://doi.org/10.1016/S0140-6736(18)32203-7 Epub 2018/11/30.

    Article  Google Scholar 

  2. Zhou M, Wang H, Zhu J, Chen W, Wang L, Liu S, et al. Cause-specific mortality for 240 causes in China during 1990-2013: a systematic subnational analysis for the Global Burden of Disease Study 2013. Lancet. 2016;387(10015):251–72. https://doi.org/10.1016/S0140-6736(15)00551-6 Epub 2015/10/30.

    Article  PubMed  Google Scholar 

  3. Rothwell PM, Giles MF, Chandratheva A, et al. Effect of urgent treatment of transient ischaemic attack and minor stroke on early recurrent stroke (EXPRESS study): a prospective population-based sequential comparison. Lancet. 2007;370(9596):1432–42.

    Article  Google Scholar 

  4. Luengo-Fernandez R, Paul NL, Gray AM, Pendlebury ST, Bull LM, Welch SJ, et al. Population- based study of disability and institutionalization after transient ischemic attack and stroke: 10-year results of the Oxford Vascular Study. Stroke. 2013;44(10):2854–61. https://doi.org/10.1161/STROKEAHA.113.001584 Epub 2013/08/08.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Wang Y, Jing J, Meng X, Pan Y, Wang Y, Zhao X, et al. The Third China National Stroke Registry (CNSR-III) for patients with acute ischaemic stroke or transient ischaemic attack: design, rationale and baseline patient characteristics. Stroke Vasc Neurol. 2019;4(3):158–64. https://doi.org/10.1136/svn-2019-000242 Epub 2019/11/12.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Stroke--1989. Recommendations on stroke prevention, diagnosis, and therapy. Report of the WHO Task Force on Stroke and other Cerebrovascular Disorders. Stroke (1989) 20(10):1407-31. Epub 1989/10/01. doi: 10.1161/01.str.20.10.1407.

  7. A.V. Dorogush, V. Ershov, A.J.a.p.a. Gulin, CatBoost: gradient boosting with categorical features support, (2018).

  8. Chen T Q, Guestrin C. Xgboost:a scalable tree boosting system [C]//Association for Computing Machinery Special Interest Group on management of data, ACM Special Interest Group on Knowledge Discovery in Data. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: NY ACM,2016:785-794.

  9. Peng T, et al. The Prediction of Hepatitis E through Ensemble Learning. Int J Environ Res Public Health. 2020;18(1). https://doi.org/10.3390/IJERPH18010159.

  10. L.J.M.l. Breiman, Random forests, 45 (2001) 5-32.

  11. T. Hastie, S. Rosset, J. Zhu, H.J.S. Zou, i. Interface,Multi-class adaboost,2 (2009) 349- 377 360.

  12. Friedman JH. Greedy function approximation: A gradient boosting machine. Ann Statist. 2001;29(5):1189–232.

    Article  Google Scholar 

  13. Wang C, Chen X, Du L, Zhan Q, Yang T, Fang Z. Comparison of machine learning algorithms for the identification of acute exacerbations in chronic obstructive pulmonary disease. Comput Methods Programs Biomed. 2020;188:105267. https://doi.org/10.1016/j.cmpb.2019.105267.

    Article  PubMed  Google Scholar 

  14. Tran BX, Ha GH, Nguyen LH, Vu GT, Hoang MT, Le HT, et al. 401 Ho, R.C.M. Ho, Studies of Novel Coronavirus Disease 19 (COVID-19) Pandemic: A Global 402 Analysis of Literature. Int J Environ Res Public Health. 2020;17:403. https://doi.org/10.3390/ijerph17114095.

    Article  CAS  Google Scholar 

  15. Liu L, Zhang C, Zhang G, Gao Y, Luo J, Zhang W, et al. A study of aortic 405 dissection screening method based on multiple machine learning models. J Thorac Dis. 2020;12:605–14. https://doi.org/10.21037/jtd.2019.12.119.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Brier GW. Verification of forecasts expressed in terms of probability. Mon Weather Rev. 1950;78:1–3. https://doi.org/10.1175/1520-0493 (1950)078<0001:VOFEIT>2.0.CO;2. [2021-02-05].

    Article  Google Scholar 

  17. Adams HP, Bendixen BH, Kappelle LJ, Biller J, Love BB, Gordon DL, et al. Classification of subtype of acute ischemic stroke. Definitions for use in a multicenter clinical trial. TOAST. Trial of Org 10172 in Acute Stroke Treatment. Stroke. 1993;24(1):35–41.

    Article  Google Scholar 

  18. Meyer MJ, Pereira S, McClure A, Teasell R, Thind A, Koval J, et al. A systematic review of studies reporting multivariable models to predict functional outcomes after post-stroke inpatient rehabilitation. Disabil Rehabil. 2015;37:1316–23.

    Article  Google Scholar 

  19. Veerbeek JM, Kwakkel G, van Wegen EE, Ket JC, Heymans MW. Early prediction of outcome of activities of daily living after stroke: a systematic review. Stroke. 2011;42:1482–8.

    Article  Google Scholar 

  20. Kelly-Hayes M, Beiser A, Kase CS, Scaramucci A, D’Agostino RB, Wolf PA. The influence of gender and age on disability following ischemic stroke: the Framingham study. J Stroke Cerebrovasc Dis. 2003;12:119–26.

    Article  Google Scholar 

  21. Nichols-Larsen DS, Clark P, Zeringue A, Greenspan A, Blanton S. Factors influencing stroke survivors’ quality of life during subacute recovery. Stroke. 2005;36:1480–4.

    Article  Google Scholar 

  22. Reeves MJ, Bushnell CD, Howard G, Gargano JW, Duncan PW, Lynch G, et al. Sex differences in stroke: epidemiology, clinical presentation, medical care, and outcomes. Lancet Neurol. 2008;7:915–26.

    Article  Google Scholar 

  23. Spence D. China Stroke Statistics 2019: a wealth of opportunities for stroke prevention. Stroke Vasc Neurol. 2020;5(3):doi:10.1136/svn-2020-000529.

    Article  Google Scholar 

  24. Wang Y-J, Li Z-X, Gu H-Q, Zhai Y, Jiang Y, Zhao X-Q, et al. China Stroke Statistics 2019: A Report From the National Center for Healthcare Quality Management in Neurological Diseases, China National Clinical Research Center for Neurological Diseases, the Chinese Stroke Association, National Center for Chronic and Non-communicable Disease Control and Prevention, Chinese Center for Disease Control and Prevention and Institute for Global Neuroscience and Stroke Collaborations. Stroke Vasc Neurol. 2020;5(3). https://doi.org/10.1136/svn-2020-000457.

  25. Lonn EM, Rambihar S, Gao P, Custodis FF, Sliwa K, Teo KK, et al. Heart rate is associated with increased risk of major cardiovascular events, cardiovascular and all-cause death in patients with stable chronic cardiovascular disease: an analysis of ONTARGET/TRANSCEND. Clin Res Cardiol. 2014;103(2). https://doi.org/10.1007/s00392-013-0644-4.

  26. Fox K, Bousser M-G, Amarenco P, Chamorro A, Fisher M, Ford I, et al. Heart rate is a prognostic risk factor for myocardial infarction: A post hoc analysis in the PERFORM (Prevention of cerebrovascular and cardiovascular Events of ischemic origin with teRutroban in patients with a history of ischemic stroke or tRansient ischeMic attack) study population. Int J Cardiol. 2013;168(4). https://doi.org/10.1016/j.ijcard.2013.04.206.

  27. Mao Q, Jian-Feng H, Lu X, Wu X, Jichun C, Jie C, et al. Heart rate influence on incidence of cardiovascular disease among adults in China. Int J Epidemiol. 2010;39(6). https://doi.org/10.1093/ije/dyq119.

  28. Therese CM, Erkki V, Tiina L, Tinna L, Anne J, Alexandra D, et al. Elevated resting heart rate is an independent risk factor for cardiovascular disease in healthy men and women. Am Heart J. 2010;159(4). https://doi.org/10.1016/j.ahj.2009.12.029.

  29. Adrien L, MSc GI, Aurélien D, Guillaume G, Bruno F, Virginie Z, et al. Characteristics of and risk factors for severe neurological deficit in patients with pyogenic vertebral osteomyelitis: A case–control study. Medicine. 2017;96(21). https://doi.org/10.1097/MD.0000000000006387.

  30. Becker BA, Thomas Y, Jaffe JT, Samuel S, Yidong C, Teri R, et al. Impact of creatinine screening on contrast-induced nephropathy following computerized tomography for stroke. Am J Emerg Med. 2021:45. https://doi.org/10.1016/J.AJEM.2020.09.044.

  31. Takeuchi D, Inai K, Shinohara T, et al. Blood coagulation abnormalities and the usefulness of D-dimer level for detecting intracardiac thrombosis in adult Fontan patients. Int J Card. 2016;224(17):139–44.

    Article  Google Scholar 

  32. Soomro AY, Guerchicoff A, Nichols DJ, et al. The current role and future prospects of D-Dimer biomarker. Eur Heart J Cardiovasc Pharmacother. 2015;39(5):157–62.

    Google Scholar 

  33. Reihani H, Shamloo AS, Keshmiri A. Diagnostic value of D-Dimer in acute myocardial infarction among patients with suspected acute coronary syndrome. Cardiol Res. 2018;9(1):17–21 LEIRA E C,ADAMS H P,ROSENTHAL G E,et al. Baseline NIH stroke scale responses estimate the probability of each particular stroke subtype [J]. Cerebrovasc Dis,2008,26(6):573-577.DOI:10.1159/000165109.

    Article  Google Scholar 

  34. Leira EC, Adams HP, Rosenthal GE, et al. Baseline NIH stroke scale responses estimate the probability of each particular stroke subtype. Cerebrovasc Dis. 2008;26(6):573–7. https://doi.org/10.1159/000165109.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Verro P, Tanenbaum LN, Borden N, et al. Clinical application of CT angiography in acute ischemic stroke. Clin Neurol Neurosurg. 2007;109(2):138–45.

    Article  Google Scholar 

  36. Yu G, Zheming L, Shuxian L, Jingling L, Moyuan S, Xiaoqing L, et al. The role of artificial intelligence in identifying asthma in pediatric inpatient setting. Ann Transl Med. 2020;8(21). https://doi.org/10.21037/ATM-20-2501A.

  37. Anaraki JR, Usefi H. A Feature Selection based on perturbation theory [J]. Expert Systems With Applications. 2019:127. https://doi.org/10.1016/j.eswa.2019.02.028.

  38. Vida A, Venkatesh A, Durgesh C, Shima S, Ayesha K, Griessenauer Christoph J, et al. Prediction of Long-Term Stroke Recurrence Using Machine Learning Models. J Clin Med. 2021;10(6). https://doi.org/10.3390/JCM10061286.

  39. Alban G, Arnaud H, Otto S, JensUwe V, Virginie LR, Christophe L, et al. Prediction of response to cardiac resynchronization therapy using a multi-feature learning method. Int J Cardiovasc Imaging. 2020. https://doi.org/10.1007/S10554-020-02083-1.

  40. Kim BJ, et al. Diagnosis of Acute Central Dizziness With Simple Clinical Information Using Machine Learning. Front Neurol. 2021:12. https://doi.org/10.3389/fneur.2021.691057.

  41. Tsur A, Batsry L, Toussia-Cohen S, et al. Development and validation of a machine-learning model for prediction of shoulder dystocia. Ultrasound Obstet Gynecol. 2020;56.

  42. Domenico S, Carlo R, Leandro D, Ernesto L, Petronilla B, Pietro G, et al. Machine learning to predict mortality after rehabilitation among patients with severe stroke. Sci Rep. 2020;10(1). https://doi.org/10.1038/s41598-020-77243-3.

  43. Potthoff RF, Tudor GE, Pieper KS, Hasselblad V. Can one assess whether missing data are missing at random in medical studies? Stat Methods Med Res. 2006;15(3):213–34.

    Article  Google Scholar 

Download references

Acknowledgements

We thank all participating hospitals, physicians and nurses, and the Third China National Stroke Registry Steering Committee members. Special thanks to Dr. Yongjun Wang and Dr. Yong Jiang for their support and help. Many thanks to Dr. Youjia for his methodological guidance. In addition, we would like to thank the editors and reviewers for their thorough review and constructive comments.

Funding

This work was supported by grants from CAMS Innovation Fund for Medical Sciences (2019-I2M-5-029), grants from the National Natural Science Foundation of China (81870905, U20A20358), and grants from Capital's Funds for Health Improvement and Research (2020-1-2041).

Author information

Authors and Affiliations

  1. Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, No.119 South 4th Ring West Road, Fengtai District, Beijing, 100070, China

    Si-Ding Chen, Xiao-Meng Yang, Yong Jiang & Yong-jun Wang

  2. China National Clinical Research Center for Neurological Diseases, No.119 South 4th Ring West Road, Fengtai District, Beijing, 100070, China

    Si-Ding Chen, Hong-Qiu Gu, Xin-Ying Huang, Huan Liu, Yong Jiang & Yong-jun Wang

  3. Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, 200433, China

    Jia You & Jian-Feng Feng

  4. Beijing Advanced Innovation Center for Big Data-Based Precision Medicine (Beihang University & Capital Medical University), Beijing, 100091, China

    Yong Jiang

  5. Advanced Innovation Center for Human Brain Protection, Capital Medical University, Beijing, China

    Yong-jun Wang

  6. Clinical Center for Precision Medicine in Stroke, Capital Medical University, Beijing, China

    Yong-jun Wang

  7. Research Unit of Artificial Intelligence in Cerebrovascular Disease, Chinese Academy of Medical Sciences, 2019RU018, Beijing, China

    Yong-jun Wang

  8. Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China

    Yong-jun Wang

  9. Chinese Institute for Brain Research, Beijing, China

    Yong-jun Wang

Authors
  1. Si-Ding Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia You
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao-Meng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hong-Qiu Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xin-Ying Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Huan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jian-Feng Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yong Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yong-jun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Si-Ding Chen wrote the main manuscript text , data analysis, and prepared supplementary contents. Yong Jiang and Yong-jun Wang were responsible for supervision and data provision. Xiao-Meng Yang , Jia You , Hong-Qiu Gu, Xin-Ying Huang, Huan Liu, Jian-Feng Feng participated in the discussion and data provision. All authors read and approved the manuscript for submission.

Corresponding authors

Correspondence to Yong Jiang or Yong-jun Wang.

Ethics declarations

Ethics approval and consent to participate

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the ethics committee of Beijing Tiantan Hospital (No.: KY2015-001- 01) and all study centers gave ethical approval of the study protocol.

Written informed consents were obtained from all participants or their legal representatives.

Consent for publication

Not applicable.

Competing interests

No conflict of interest exits in the submission of this manuscript, and manuscript is approved by all authors for publication.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, SD., You, J., Yang, XM. et al. Machine learning is an effective method to predict the 90-day prognosis of patients with transient ischemic attack and minor stroke. BMC Med Res Methodol 22, 195 (2022). https://doi.org/10.1186/s12874-022-01672-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12874-022-01672-z

Keywords

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us