Skip to main content
Download PDF

Absolute mortality risk assessment of COVID-19 patients: the Khorshid COVID Cohort (KCC) study

BMC Medical Research Methodology volumeĀ 21, ArticleĀ number:Ā 146 (2021) Cite this article

Abstract

Background

Already at hospital admission, clinicians require simple tools to identify hospitalized COVID-19 patients at high risk of mortality. Such tools can significantly improve resource allocation and patient management within hospitals. From the statistical point of view, extended time-to-event models are required to account for competing risks (discharge from hospital) and censoring so that active cases can also contribute to the analysis.

Methods

We used the hospital-based open Khorshid COVID Cohort (KCC) study with 630 COVID-19 patients from Isfahan, Iran. Competing risk methods are used to develop a death risk chart based on the following variables, which can simply be measured at hospital admission: sex, age, hypertension, oxygen saturation, and Charlson Comorbidity Index. The area under the receiver operator curve was used to assess accuracy concerning discrimination between patients discharged alive and dead.

Results

Cause-specific hazard regression models show that these baseline variables are associated with both death, and discharge hazards. The risk chart reflects the combined results of the two cause-specific hazard regression models. The proposed risk assessment method had a very good accuracy (AUCā€‰=ā€‰0.872 [CI 95%: 0.835ā€“0.910]).

Conclusions

This study aims to improve and validate a personalized mortality risk calculator based on hospitalized COVID-19 patients. The risk assessment of patient mortality provides physicians with additional guidance for making tough decisions.

Peer Review reports

Background

The ongoing coronavirus disease 2019 (COVID-2019) pandemic changed priorities all over the world. More than 119 million confirmed infections and over 2.6 million deaths worldwide have been reported [1, 2]. Approximately 20% of such confirmed cases were severe, among which most were admitted in the intensive care unit (ICU) and required early intubation and mechanical ventilation. The health systems thus face financial and facility challenges in light of managing this condition. As the pandemic continuous, hospitals seek effective methods for managing such severe patients. Present guidelines on COVID-19 treatment identify ICU admission, ventilation, and mortality risk as typical outcomes in high-risk patients and consider these patients as potential candidates for medical treatment [3, 4]. Many studies reported the clinical characteristics and outcomes of COVID-19 patients, but few research studies focused on the risk assessment of outcomes [5, 6]. Personal risk profiles can help physicians make the correct decision for optimal patient treatment and hospital capacity management.

The risk assessment procedure and the resulting risk charts have been well documented in community-based cardiovascular Cohorts [7,8,9,10,11]. In such studies, the risk of having an event by the end of the cohort follow-up is estimated using the time-to-event analysis based on Cox proportional hazard regression [12]. By contrast, the analysis of in-hospital data requires special attention. Standard Cox regression models lead to inaccurate results when competing events exist [13]. For example, discharge alive is a competing event when hospital death is the event of interest. Such information should not be ignored since it results in an incomplete reflection of treatment effects [14] and creates competing risk bias. Moreover, standard logistic regression could not be used when some active cases are still hospitalized at the last follow-up date [13, 15]. In fact, excluding such patients creates selection bias.

Recently, machine learning methods were used for survival analysis. Methods, such as Random Survival Forests (RSF), were compared with standard Cox regression models [16]. Although RSF could select significant nonlinear interactions to improve the discrimination ability, they do not provide direct clinical interpretation information and should be further studied before the clinical application [17, 18]. Also, time-to-event data analysis could not be performed appropriately using classification systems, in which the event-of-interest is only used, and time-to-event data is discarded with the same reason mentioned above for the standard logistic regression.

This paper introduces an absolute cause-specific risk regression approach to perform risk assessment [19,20,21] for patients hospitalized with COVID-19. The event-of-interest is death in the hospital, and discharge alive is considered as the competing event. To demonstrate the methodology, the Khorshid COVID-19 Cohort (KCC) dataset [22] with 630 patients is used. To the best of our knowledge, this is the first study in which absolute risk assessment is performed on in-hospital COVID-19 patients considering the competing events. We also provide a death risk chart as a simple tool for clinical applications.

Methods

Absolute risk estimation for hospital mortality

First, we introduce the absolute risk estimation approach, which is used for the risk assessment.

We denote D the event outcome, where Dā€‰=ā€‰1 if the event of interest (e.g., hospital death) occurred, and Dā€‰=ā€‰2 if the competing event (e.g., discharge alive) happened. Then, the probability (i.e., absolute risk) that an event of type 1 (i.e., Dā€‰=ā€‰1) occurred by time t is given by the Eq.Ā (1) [20, 21]:

$${F}_{1}\left(t|x,z\right)={\int }_{0}^{t}S\left(s-|x,z\right){\lambda }_{1,z}\left(s|x\right)ds$$
(1)

where \(s-\) denotes the right-sided limit, \(S\left(s-|x,z\right)\) is the conditional event-free survival function, \({\lambda }_{1,z}\left(s|x\right)\) is the hazard of the event of interest dependent on the baseline covariate vector x, and z is a set of strata variables. The event-free survival function,\(S(s|x,z)\), is the probability to be still hospitalized at time s and thus depends on the patientsā€™ length of hospital stay which is directly linked to both the death hazard, \({\lambda }_{1,z}\left(s|x\right),\) and the discharge hazard, \({\lambda }_{2,z}\left(s|x\right).\) This is in contrast to the classical time-to event settings with only one event of interest (here: death) and without competing events (here: discharge). Thus, the absolute risk of hospital mortality is preliminary determined by the death hazard but also by the discharge hazard. As many clinical factors and biomarkers are associated with the discharge hazard, it is therefore necessary to use competing risk methodology to identify potential predictors for hospital mortality.

To estimate \({F}_{1}\left(t|x,z\right)\), we first need to estimate the event-free survival function. It is done by estimating both cause-specific hazards and then using the product integral estimator [21]. The stratified Cox regression model for cause j, jā€‰=ā€‰1 or 2 [21, 23], is given by the Eq.Ā (2):

$${\lambda }_{0j,z}\left(t|x\right)={\lambda }_{0j,z}(t){e}^{x{\beta }_{j}}$$
(2)

where Ī²j is the vector of log-hazard ratios of the covariate vector x and Ī»0j,z(t) is the baseline hazard function.

Thus, the absolute risk estimation includes estimating the cause-specific hazards for both event types and then calculating the integral Eq.Ā (1).

Obtaining the risk chart

We use the absolute risk estimation procedure to obtain a detailed risk chart for patients hospitalized for COVID-19 (see Fig.Ā 1). First, two cause-specific Cox proportional hazards models for the outcome events death in the hospital and discharged alive are fitted. It is done by using the ā€œcscā€ function of the ā€œriskRegressionā€ package in R [21, 24]. The cause-specific hazard ratios show how the risk factors affect each hazard rate. Both cause-specific hazard ratios have to be considered to conclude the effects of the risk factors on the absolute risk. Thus, even if a variable does not affect the death hazard, it may indirectly affect hospital mortality by in- or decreasing the discharge hazard. By contrast, the subdistribution hazard ratio combines the direct and the indirect effects in a single coefficient. To show how variables may indirectly influence hospital mortality, we additionally estimate the subdistribution hazard for the outcome of death. It is done using the ā€œcrrā€ function of the ā€œcmprskā€ package in R [25,26,27].

Fig. 1
figure 1

The absolute mortality risk chart for COVID-19 hospitalized patients based on sex, age, oxygen saturation, hypertension, and Charlson Comorbidity Index

Full size image

Nonetheless, the absolute risk approach is based on the two cause-specific hazards, as was explained above. Thus, we apply the ā€œpredictā€ function on the csc-object to obtain the estimates of \({F}_{1}\left(t|x,z\right)\) for all patient groups. For each patient, we obtain an individual risk based on the baseline covariates of the patients. Finally, the risk chart as shown in Fig.Ā 1 can be constructed. By color-coding, high-risk and low-risk patients are directly identifiable from the chart.

The area under the curve (AUC) of the receiver operator curve (ROC), defined in Eq.Ā (3) [28,29,30], could be used to assess accuracy for discrimination between patients discharged alive and dead:

$$AUC=\frac{1}{m\times n}\sum_{i=1}^{m}\sum_{j=1}^{n}\varphi \left({P}_{i},{P}_{j}\right)$$
(3)

where m is the number of patients who died, n is the number of patients discharged alive, Pi and Pj are the estimated mortality risks of the data points i and j. The kernel \(\varphi\) is defined in Eq.Ā (4).

$$\varphi \left({P}_{i},{P}_{j}\right)=\left\{\begin{array}{cc}1& {P}_{i}>{P}_{j}\\ 0.5& {P}_{i}={P}_{j}\\ 0& {P}_{i}<{P}_{j}\end{array}\right.$$
(4)

It is done using the ā€œcolAUCā€ function of the ā€œcaToolsā€ package in R [31, 32].

Results were reported as meanā€‰Ā±ā€‰standard deviation (for interval variables) and frequencies (for categorical variables). All data processing was performed using R version 4.0.3 [33].

Dataset and variable selection

In our study, the Khorshid COVID-19 Cohort (KCC) dataset [22, 34] was used. It is a hospital-based open cohort from Isfahan, which was a hot outbreak zone in central Iran. COVID-19 patients were admitted to the Khorshid referral hospital in Isfahan from February 2020 until September 2020. The patient recruitment phase finished at the end of August 2020, and the follow-up continues until the end of August 2021. The patients are followed up for the first, fourth, 12th weeks, and the first year after discharge. In total, 630 COVID-19 patients were enrolled in our study.

Among the recorded data for each COVID-19 patient, the baseline parameters sex, age, hypertension, oxygen saturation (SaO2) at hospital admission, and Charlson Comorbidity Index (CCI) [35, 36] were used as the non-lab input measurements to the model. Such parameters were selected based on clinical knowledge and previous papers published in the literature on the COVID-19 clinical outcome prediction [37, 38]. Age was categorized as less than 45 (years) (B0), 45ā€“54 (B1), 55ā€“64 (B2), 65ā€“74 (B3), and more than 75 (B4) as in Petrilli et al. [39]. Oxygen saturation was categorized as more than or equal to 90% (A0), 80ā€“89 (A1), and less than 80 (A2) by merging some SaO2 sub-categories as mentioned by MejĆ­a et al. [38]. CCI was dichotomized as low (CCIā€‰<ā€‰3) or high (CCIā€‰ā‰„ā€‰3). Such an optimal cut-off was estimated to minimize the (Error-Rate) ER-criteria of the Receiver operating characteristic curve (ROC) [40] for optimal mortality discrimination (AUCā€‰=ā€‰0.767 [CI 95%: 0.697ā€“0.837]). Such a threshold previously was used in the literature for risk stratifications of hospitalized COVID-19 patients [41]. A reference group is considered with the following categories: sex (Female), age (B0), oxygen saturation (A0), and low CCI. Time-to-event was considered until hospital discharge, and the status at discharge (dead or alive) is taken into account. The risk assessment procedure was performed for the time from admission until 58Ā days of hospitalization.

Results

Among 630 patients, 38.7% were female. The average age of the admitted patients was 57.1ā€‰Ā±ā€‰15.4Ā years. The average of the CCI was 2.3ā€‰Ā±ā€‰2.1. 34.9% of the participants had hypertension. Forty-five patients died in the hospital. The descriptive statistics of the admitted patients categorized by the vital state at discharge are listed in Table 1. The cause-specific hazard ratios and their 95%-CIs are shown in Table 2. The subdistribution hazard ratios and their CI 95% for mortality based on the Fine and Gray model [26], with the competing event of discharge alive, are listed in Table 3. For example, we find that low oxygen saturation significantly increases the death hazard rate and decreases the discharge hazard. If it drops below 80%, the death hazard is more than five times higher than for patients in the reference group. At the same time, the discharge hazard is 40% lower. The subdistribution hazard ratio quantifies the combined effect: patients with 80% oxygen saturation or less have a more than ten times higher risk of death in the hospital. This effect is not only explained by the increased death hazard but also by the decreased discharge hazard. Due to the decreased discharge hazards, the patients stay longer in the hospital and therefore longer at risk and thus also at higher risk of dying in the hospital within 58Ā days.

Table 1 The baseline characteristics of 630 patients, discharged alive or dead
Full size table
Table 2 The multivariable Cox proportional hazard regression model for mortality and hospital discharge
Full size table
Table 3 The multivariable subdistribution hazards model for hospital mortality
Full size table

The absolute mortality risk was then calculated within 58Ā days of hospitalization of COVID-19 patients. Its color-coded risk chart is provided in Fig.Ā 1. The proposed risk assessment method had an excellent accuracy (AUCā€‰=ā€‰0.872 [CI 95%: 0.835ā€“0.910]). The ROC of the proposed system is shown in Fig.Ā 2.

Fig. 2
figure 2

The Receiver operating characteristic (ROC) curve of the proposed risk assessment method for discrimination between the dead and discharged alive patients. The parameters Sensitivity (Specificity) are the proportion of patients who died (discharged alive) that were correctly identified by the model

Full size image

The risk chart reflects the combined results of the two cause-specific hazard ratios. We find that the risk of hospital death within each group is about ten times higher for patients with an oxygen saturation lower of 80%.

Discussion

Suitable identification of hospitalized COVID-19 patients at high risk of mortality can significantly improve resource allocation and patient management within hospitals. This study aims to improve and validate a personalized mortality risk calculator based on hospitalized COVID-19 patients. The risk assessment of patient mortality provides physicians with additional guidance for making tough decisions. The presented tool showed a very good accuracy. In this model, baseline clinical parameters collected at hospital admission were used. It requires commonly available demographic and comorbidity data. Also, the oxygen saturation level could be measured using a traditional pulse oximeter. Thus, it does not require advanced blood testing and could be considered as a non-laboratory-based risk chart. The proposed risk assessment procedure is not limited to the input factors used in this paper. In fact, it could be applied to any input variables such as repeated COVID-19 infection, vaccination category, and body mass index.

Predicted mortality increased for patients with low oxygen saturation, corroborating findings that link hypoxemia to mortality [42], as well as the observed prevalence of shortness of breath in severe patients [43]. This measurement additionally serves as a signal of respiratory distress, and respiratory failure has been found clinically as one of the significant causes of COVID-19 mortality [44]. It can also appear in silent hypoxia cases where shortness of breath is not observed [45]. Age was another critical determinant of mortality in the model: older patients have higher mortality risk, as observed in the retrospective patient analysis [46] and subsequently reflected in public health guidance [47]. Similar to our study, it was shown in the literature that a higher CCI score is significantly associated with mortality and disease severity in COVID-19 patients [41, 48].

Moreover, the increase in mortality risk for patients with elevated blood pressure levels is consistent with the reports in other studies [49, 50]. Such findings could have been expected a priori as logical, but the proposed assessment permits quantifying them as a function of time after patient admission. In our analysis, a dichotomized version of CCI was used (CCIā€‰<ā€‰3 vs. CCIā€‰ā‰„ā€‰3). However, more categories of CCI were used in the literature, such as: CCIā€‰=ā€‰0, CCI score of 1ā€“2, and CCI score ofā€‰ā‰„ā€‰3 [41]. Due to the collinearity between age and CCI, our model consisted of a dichotomized version of CCI.

In our dataset, no active patients were in the hospital, and follow-up was complete for hospital mortality. Thus, in principle, the logistic regressions could be alternatively used instead of the absolute risk regression model in our data situation. However, it does not account for time-to-event and censoring. It could only display effects on a cumulative incidence function on a plateau [15]. Moreover, since we aim to use the proposed risk assessment procedure in dynamic situations and integrate it into the hospital information system (HIS), only predictions based on cause-specific regression could be used as there are always active cases in the hospital proceeding time frames.

The example of oxygen saturation categories demonstrated the possible incomplete picture of risk assessment if the competing risk of discharge alive is ignored. In the Cox regression, where discharge alive is not considered (cause-specific hazard ratio of death Tables 2 and 3), the effect of oxygen saturation would have been estimated to be 5.48. However, considering the competing risk, discharge alive showed an indirect effect on hospital mortality via an increased length of stay. Thus, in contrast to a classical survival situation without competing risks, the cause-specific hazard ratio has no one-to-one relationship with the absolute risk. Estimating the subdistribution hazard ratio combined direct and indirect effects and showed that the mortality risk is increased tenfold rather than only fivefold. Such differences affect the estimated mortality risks.

The available data limit our prediction model. Considering more comprehensive variables such as IL-6 levels, D-Dimer, and radiographic diagnosis, may yield more accurate results in the laboratory-based risk charts. While the proposed model focused on the COVID-19 dataset at the national level, some studies considered multi-center region-specific data to measure how major risk factors identify the mortality rate [51]. Furthermore, the variability in treatment protocols across countries and individual organizations might lead to different results [22, 52]. External validation of the proposed risk assessment method is the focus of our future work.

Conclusions

In conclusion, we provided a personalized mortality risk chart based on hospitalized COVID-19 patients to provide physicians with additional guidance for making tough decisions.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

AUC:

Area under the curve

CCI:

Charlson Comorbidity Index

CI:

Confidence Interval

COVID-2019:

Coronavirus disease 2019

HR:

Hazard Ratio

HIS:

Hospital information system

ICU:

Intensive care unit

KCC:

Khorshid COVID Cohort

SaO2:

Oxygen saturation

RSF:

Random Survival Forests

ROC:

Receiver operator curve

References

  1. COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE), JHC [https://www.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6]. Accessed 1 June 2021.

  2. Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis. 2020;20(5):533ā€“4.

    ArticleĀ  CASĀ  Google ScholarĀ 

  3. Shang Y, Pan C, Yang X, Zhong M, Shang X, Wu Z, Yu Z, Zhang W, Zhong Q, Zheng X, et al. Management of critically ill patients with COVID-19 in ICU: statement from front-line intensive care experts in Wuhan, China. Ann Intensive Care. 2020;10(1):73.

    ArticleĀ  Google ScholarĀ 

  4. COVID-19 Treatment Guidelines Panel. Coronavirus Disease 2019 (COVID-19) Treatment Guidelines. National Institutes of Health, [https://www.covid19treatmentguidelines.nih.gov/]. Accessed 1 June 2021.

  5. Barda N, Riesel D, Akriv A, Levy J, Finkel U, Yona G, Greenfeld D, Sheiba S, Somer J, Bachmat E, et al. Developing a COVID-19 mortality risk prediction model when individual-level data are not available. Nat Commun. 2020;11(1):4439.

    ArticleĀ  CASĀ  Google ScholarĀ 

  6. Das AK, Mishra S, Saraswathy Gopalan S. Predicting CoVID-19 community mortality risk using machine learning and development of an online prognostic tool. PeerJ. 2020;8:e10083ā€“e10083.

    ArticleĀ  Google ScholarĀ 

  7. Dā€™Agostino RB, Lee M-L, Belanger AJ, Cupples LA, Anderson K, Kannel WB. Relation of pooled logistic regression to time dependent cox regression analysis: The framingham heart study. Stat Med. 1990;9(12):1501ā€“15.

  8. Schnabel RB, Sullivan LM, Levy D, Pencina MJ, Massaro JM, Dā€™Agostino RB Sr, Newton-Cheh C, Yamamoto JF, Magnani JW, Tadros TM, et al. Development of a risk score for atrial fibrillation (Framingham Heart Study): a community-based cohort study. The Lancet. 2009;373(9665):739ā€“45.

  9. Conroy RM, PyƶrƤlƤ K, Fitzgerald AP, Sans S, Menotti A, De Backer G, De Bacquer D, DucimetiĆØre P, Jousilahti P, Keil U, et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J. 2003;24(11):987ā€“1003.

    ArticleĀ  CASĀ  Google ScholarĀ 

  10. Sarrafzadegan N, Hassannejad R, Marateb HR, Talaei M, Sadeghi M, Roohafza HR, Masoudkabir F, OveisGharan S, Mansourian M, Mohebian MR et al: PARS risk charts: A 10-year study of risk assessment for cardiovascular diseases in Eastern Mediterranean Region. PLOS ONE 2017, 12(12):e0189389.

  11. Kaptoge S, Pennells L, De Bacquer D, Cooney MT, Kavousi M, Stevens G, Riley LM, Savin S, Khan T, Altay S, et al. World Health Organization cardiovascular disease risk charts: revised models to estimate risk in 21 global regions. Lancet Glob Health. 2019;7(10):e1332ā€“45.

    ArticleĀ  Google ScholarĀ 

  12. Dā€™Agostino RB, Pencina MJ, Massaro JM, Coady S. Cardiovascular Disease Risk Assessment: Insights from Framingham. Glob Heart. 2013;8(1):11ā€“23.

  13. Wolkewitz M, Lambert J, von Cube M, Bugiera L, Grodd M, Hazard D, White N, Barnett A, Kaier K. Statistical Analysis of Clinical COVID-19 Data: A Concise Overview of Lessons Learned, Common Errors and How to Avoid Them. Clin Epidemiol. 2020;12:925ā€“8.

    ArticleĀ  Google ScholarĀ 

  14. Latouche A, Allignol A, Beyersmann J, Labopin M, Fine JP. A competing risks analysis should report results on all cause-specific hazards and cumulative incidence functions. J Clin Epidemiol. 2013;66(6):648ā€“53.

    ArticleĀ  Google ScholarĀ 

  15. Poguntke I, Schumacher M, Beyersmann J, Wolkewitz M. Simulation shows undesirable results for competing risks analysis with time-dependent covariates for clinical outcomes. BMC Med Res Methodol. 2018;18(1):79.

    ArticleĀ  Google ScholarĀ 

  16. Wang H, Li G. A Selective Review on Random Survival Forests for High Dimensional Data. Quantitative bio-science. 2017;36(2):85ā€“96.

    ArticleĀ  CASĀ  Google ScholarĀ 

  17. Kurt Omurlu I, Ture M, Tokatli F. The comparisons of random survival forests and Cox regression analysis with simulation and an application related to breast cancer. Expert Syst Appl. 2009;36(4):8582ā€“8.

    ArticleĀ  Google ScholarĀ 

  18. Miao F, Cai Y-P, Zhang Y-T, Li C-Y: Is Random Survival Forest an Alternative to Cox Proportional Model on Predicting Cardiovascular Disease? In: 6th European Conference of the International Federation for Medical and Biological Engineering: 2015// 2015; Cham: Springer International Publishing; 2015: 740ā€“743.

  19. Gerds TA, Scheike TH, Andersen PK. Absolute risk regression for competing risks: interpretation, link functions, and prediction. Stat Med. 2012;31(29):3921ā€“30.

    ArticleĀ  Google ScholarĀ 

  20. Benichou J, Gail MH. Estimates of absolute cause-specific risk in cohort studies. Biometrics. 1990;46(3):813ā€“26.

    ArticleĀ  CASĀ  Google ScholarĀ 

  21. Ozenne B, SĆørensen AL, Scheike T, Torp-Pedersen C, Gerds TA. riskRegression: Predicting the Risk of an Event using Cox Regression Models. R J. 2017;9(2):440.

    ArticleĀ  Google ScholarĀ 

  22. Sami R, Soltaninejad F, Amra B, Naderi Z, Haghjooy Javanmard S, Iraj B, Haji Ahmadi S, Shayganfar A, Dehghan M, Khademi N et al: A one-year hospital-based prospective COVID-19 open-cohort in the Eastern Mediterranean region: The Khorshid COVID Cohort (KCC) study. PLoS One 2020, 15(11):e0241537.

  23. Cox DR: Regression Models and Life-Tables. In: Breakthroughs in Statistics: Methodology and Distribution. edn. Edited by Kotz S, Johnson NL. New York, NY: Springer New York; 1992: 527ā€“541.

  24. Gerds TA, Ozenne B: riskRegression: Risk Regression Models and Prediction Scores for Survival Analysis with Competing Risks. R package version 2020.02.05. 2020.https://CRAN.R-project.org/package=riskRegression.

  25. Gray B: cmprsk: Subdistribution Analysis of Competing Risks. R package version 2.2ā€“10. 2020.https://CRAN.R-project.org/package=cmprsk.

  26. Fine JP, Gray RJ. A Proportional Hazards Model for the Subdistribution of a Competing Risk. J Am Stat Assoc. 1999;94(446):496ā€“509.

    ArticleĀ  Google ScholarĀ 

  27. Gray RJ. A Class of K-Sample Tests for Comparing the Cumulative Incidence of a Competing Risk. Ann Stat. 1988;16(3):1141ā€“54.

    ArticleĀ  Google ScholarĀ 

  28. Fawcett T. ROC graphs: Notes and practical considerations for researchers. Mach Learn. 2004;31(1):1ā€“38.

    Google ScholarĀ 

  29. Mason SJ, Graham NE. Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Q J R Meteorol Soc. 2002;128(584):2145ā€“66.

    ArticleĀ  Google ScholarĀ 

  30. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics. 1988;44(3):837ā€“45.

    ArticleĀ  CASĀ  Google ScholarĀ 

  31. Muschelli J. ROC and AUC with a Binary Predictor: a Potentially Misleading Metric. J Classif. 2020;37(3):696ā€“708.

    ArticleĀ  Google ScholarĀ 

  32. Tuszynski J: caTools: moving window statistics, GIF, Base64, ROC, AUC, etc. R package version 1.18. 1. In.; 2021. https://CRAN.R-project.org/package=caTools.

  33. R_Core_Team: R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2013.http://www.R-project.org/.

  34. Vedaei SS, Fotovvat A, Mohebbian MR, Rahman GME, Wahid KA, Babyn P, Marateb HR, Mansourian M, Sami R. COVID-SAFE: An IoT-Based System for Automated Health Monitoring and Surveillance in Post-Pandemic Life. IEEE Access. 2020;8:188538ā€“51.

    ArticleĀ  Google ScholarĀ 

  35. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373ā€“83.

    ArticleĀ  CASĀ  Google ScholarĀ 

  36. Radovanovic D, Seifert B, Urban P, Eberli FR, Rickli H, Bertel O, Puhan MA, Erne P: Validity of Charlson Comorbidity Index in patients hospitalised with acute coronary syndrome. Insights from the nationwide AMIS Plus registry 2002ā€“2012. Heart (British Cardiac Society) 2014, 100(4):288ā€“294.

  37. Cho KH, Kim SW, Park JW, Do JY, Kang SH: Effect of Sex on Clinical Outcomes in Patients with Coronavirus Disease: A Population-Based Study. Journal of clinical medicine 2020, 10(1).

  38. Mejƭa F, Medina C, Cornejo E, Morello E, VƔsquez S, Alave J, Schwalb A, MƔlaga G: Oxygen saturation as a predictor of mortality in hospitalized adult patients with COVID-19 in a public hospital in Lima, Peru. PLOS ONE 2021, 15(12):e0244171.

  39. Petrilli CM, Jones SA, Yang J, Rajagopalan H, Oā€™Donnell L, Chernyak Y, Tobin KA, Cerfolio RJ, Francois F, Horwitz LI: Factors associated with hospital admission and critical illness among 5279 people with coronavirus disease 2019 in New York City: prospective cohort study. BMJ 2020, 369:m1966.

  40. Unal I. Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach. Comput Math Methods Med. 2017;2017:3762651.

    ArticleĀ  Google ScholarĀ 

  41. Tuty Kuswardhani RA, Henrina J, Pranata R, Anthonius Lim M, Lawrensia S, Suastika K. Charlson comorbidity index and a composite of poor outcomes in COVID-19 patients: A systematic review and meta-analysis. Diabetes Metab Syndr. 2020;14(6):2103ā€“9.

    ArticleĀ  CASĀ  Google ScholarĀ 

  42. Bertsimas D, Lukin G, Mingardi L, Nohadani O, Orfanoudaki A, Stellato B, Wiberg H, Gonzalez-Garcia S, Parra-CalderĆ³n CL, Robinson K, et al. COVID-19 mortality risk assessment: An international multi-center study. PLoS ONE. 2020;15(12):e0243262ā€“e0243262.

    ArticleĀ  CASĀ  Google ScholarĀ 

  43. Ora J, Liguori C, Puxeddu E, Coppola A, Matino M, Pierantozzi M, Mercuri NB, Rogliani P. Dyspnea perception and neurological symptoms in non-severe COVID-19 patients. Neurol Sci. 2020;41(10):2671ā€“4.

    ArticleĀ  Google ScholarĀ 

  44. Hasan SS, Capstick T, Ahmed R, Kow CS, Mazhar F, Merchant HA, Zaidi STR. Mortality in COVID-19 patients with acute respiratory distress syndrome and corticosteroids use: a systematic review and meta-analysis. Expert Rev Respir Med. 2020;14(11):1149ā€“63.

    ArticleĀ  CASĀ  Google ScholarĀ 

  45. Chandra A, Chakraborty U, Pal J, Karmakar P: Silent hypoxia: a frequently overlooked clinical entity in patients with COVID-19. BMJ Case Reports 2020, 13(9):e237207.

  46. Heras E, Garibaldi P, Boix M, Valero O, Castillo J, Curbelo Y, Gonzalez E, Mendoza O, Anglada M, Miralles JC et al: COVID-19 mortality risk factors in older people in a long-term care center. European Geriatric Medicine 2020.

  47. WHO: Guidance on COVID-19 for the care of older people and people living in long-term care facilities, other non-acute care facilities and home care. 2020.

  48. Varol Y, Hakoglu B, Kadri Cirak A, Polat G, Komurcuoglu B, Akkol B, Atasoy C, Bayramic E, Balci G, Ataman S et al: The impact of charlson comorbidity index on mortality from SARS-CoV-2 virus infection and A novel COVID-19 mortality index: CoLACD. International journal of clinical practice 2020:e13858.

  49. Du Y, Zhou N, Zha W, Lv Y: Hypertension is a clinically important risk factor for critical illness and mortality in COVID-19: A meta-analysis. Nutrition, Metabolism and Cardiovascular Diseases 2020.

  50. Liang X, Shi L, Wang Y, Xiao W, Duan G, Yang H, Wang Y. The association of hypertension with the severity and mortality of COVID-19 patients: Evidence based on adjusted effect estimates. J Infect. 2020;81(3):e44ā€“7.

    ArticleĀ  CASĀ  Google ScholarĀ 

  51. Garg P, Joshi D: A region-specific clustering approach to investigate risk-factors in mortality rate during COVID-19: comprehensive statistical analysis from 208 countries. Journal of medical engineering & technology 2021:1ā€“6.

  52. Rieg S, von Cube M, Kalbhenn J, Utzolino S, Pernice K, Bechet L, Baur J, Lang CN, Wagner D, Wolkewitz M et al: COVID-19 in-hospital mortality and mode of death in a dynamic and non-restricted tertiary care model in Germany. PLoS One 2020, 15(11):e0242127.

Download references

Acknowledgements

We would like to acknowledge the Khorshid hospital nurses and interns for this study, who recruited patients and collected follow-up data. Most importantly, we would like to acknowledge all the patients who consented to participate in this study.

Funding

The research leading to these results has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 712949 (TECNIOspring PLUS), and from the Agency for Business Competitiveness of the Government of Catalonia (TECSPR18-1ā€“0017). The TECSPR18-1ā€“0017 project provided the APC. These funders had no role in study design, data collection, analysis, decision to publish, or manuscript preparation.

Author information

Authors and Affiliations

  1. Biomedical Engineering Department, Engineering Faculty, University of Isfahan, Isfahan, Iran

    Hamid Reza Marateb

  2. Biomedical Engineering Research Centre (CREB), Automatic Control Department (ESAII), Universitat PolitĆØcnica de Catalunya-Barcelona Tech (UPC)Building H, Floor 4, Av. Diagonal 647, 08028, Barcelona, Spain

    Hamid Reza Marateb,Ā Marjan MansourianĀ &Ā Miquel Angel MaƱanas

  3. Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany

    Maja von Cube,Ā Harald BinderĀ &Ā Martin Wolkewitz

  4. Department of Internal Medicine, School of Medicine, Isfahan University of Medical Sciences, Isfahan, Iran

    Ramin Sami

  5. Applied Physiology Research Center, Cardiovascular Research Institute, Isfahan University of Medical Sciences, Isfahan, Iran

    Shaghayegh Haghjooy Javanmard

  6. Department of Epidemiology and Biostatistics, School of Health, Isfahan University of Medical Sciences, Isfahan, Iran

    Marjan Mansourian

  7. Bamdad Respiratory Research Center, Isfahan University of Medical Sciences, Isfahan, Iran

    Babak Amra

  8. The Respiratory Research Center, Pulmonary Division, Department of Internal Medicine, School of Medicine, Isfahan University of Medical Sciences, Isfahan, Iran

    Forogh Soltaninejad

  9. Isfahan Kidney Diseases Research Center, Isfahan University of Medical Sciences, Isfahan, Iran

    Mojgan Mortazavi

  10. Isfahan Gastroenterology and Hepatology Research Center (lGHRC), Isfahan University of Medical Sciences, Isfahan, Iran

    Peyman Adibi

  11. School of Medicine, Isfahan University of Medical Sciences, Isfahan, Iran

    Nilufar Khademi,Ā Nastaran Sadat HosseiniĀ &Ā Arash Toghyani

  12. Isfahan Cardiovascular Research Center, Cardiovascular Research Institute, Isfahan University of Medical Sciences, Isfahan, Iran

    Razieh Hassannejad

  13. Biomedical Research Networking Center in Bioengineering, Biomaterials, and Nanomedicine (CIBER-BBN), Madrid, Spain

    Miquel Angel MaƱanas

Authors
  1. Hamid Reza Marateb
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  2. Maja von Cube
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  3. Ramin Sami
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  4. Shaghayegh Haghjooy Javanmard
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  5. Marjan Mansourian
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  6. Babak Amra
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  7. Forogh Soltaninejad
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  8. Mojgan Mortazavi
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  9. Peyman Adibi
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  10. Nilufar Khademi
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  11. Nastaran Sadat Hosseini
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  12. Arash Toghyani
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  13. Razieh Hassannejad
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  14. Miquel Angel MaƱanas
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  15. Harald Binder
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

  16. Martin Wolkewitz
    View author publications

    You can also search for this author in PubMedĀ Google Scholar

Contributions

HRM, MVC, MM, RH, HB, and MW participated in the Conceptualization, Investigation, and Methodology. HRM, MM, MVC, and MW participated in the Visualization, Software, and Validation. RS, SHJ, MM, PA, and MW participated in the Project administration. HRM, MVC, SHJ, MM, BA, FS, MoM, PA, MAM, HB, and MW participated in the formal analysis. RS, SHJ, MM, BA, FS, MoM, PA, NK, NSH, and AT participated in the data acquisition. HRM, RS, MM, NK, NSH, and AT participated in the data curation. RS, SHJ, MM, MAM, and MW participated in funding acquisition. MVC, MAM, and MW participated in supervision. MVC, RS, RH, MAM, and MW participated in the interpretation of the results. HRM, MVC, MM, and MW participated in Writing ā€“ original draft, while RS, SHJ, BA, FS, MoM, PA, NK, NSH, AT, RH, MAM, and HB participated in Writing ā€“ review & editing. All authors read and approved the final manuscript and agreed to be accountable for all aspects of the work.

Corresponding authors

Correspondence to Hamid Reza Marateb or Marjan Mansourian.

Ethics declarations

Ethics approval and consent to participate

The experimental protocol was approved by the ethics committee of the Isfahan University of Medical Sciences (IUMS) and the other national authorities (Longitudinal epidemiologic investigation of patients' characteristics with coronavirus infection referring to Isfahan Khorshid Hospital: IR.MUI.MED.REC.1399.029, Modeling of incidence and outcomes of COVID-19: IR.MUI.RESEARCH.REC.1399.479), conforming to the Declaration of Helsinki. The ethics committee of the IUMS granted permission to use the raw data of the KCC study. Also, the entire subjects gave written informed consent to the experimental procedure. It was given by the first relative family of patients with severe conditions. No minors participated in our study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marateb, H.R., von Cube, M., Sami, R. et al. Absolute mortality risk assessment of COVID-19 patients: the Khorshid COVID Cohort (KCC) study. BMC Med Res Methodol 21, 146 (2021). https://doi.org/10.1186/s12874-021-01340-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12874-021-01340-8

Keywords

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us