- Research article
- Open Access
- Open Peer Review
Positive predictive value of International Classification of Diseases, 10th revision, diagnosis codes for cardiogenic, hypovolemic, and septic shock in the Danish National Patient Registry
BMC Medical Research Methodology volume 15, Article number: 23 (2015)
Large registries are important data sources in epidemiological studies of shock, if these registries are valid. Therefore, we examined the positive predictive value (PPV) of diagnosis codes for shock, the procedure codes for inotropic/vasopressor therapy among patients with a diagnosis of shock, and the combination of a shock diagnosis and a code for inotropic/vasopressor therapy in the Danish National Patient Registry (DNPR).
We randomly selected 190 inpatients with an International Classification of Diseases, 10th revision (ICD-10) diagnosis of shock at Aarhus University Hospital from 2005–2012 using the DNPR; 50 patients were diagnosed with cardiogenic shock, 40 patients with hypovolemic shock, and 100 patients with septic shock. We used medical charts as the reference standard and calculated the PPV with 95% confidence intervals (CI) for overall shock and for each type of shock separately. We also examined the PPV for inotropic/vasopressor therapy and the PPV for shock when a concurrent code for inotropic/vasopressor therapy was also registered.
The PPV was 86.1% (95% CI: 79.7–91.1) for shock overall, 93.5% (95% CI: 82.1–98.6) for cardiogenic shock, 70.6% (95% CI: 52.5–84.9) for hypovolemic shock, and 69.2% (95% CI: 57.7–79.2) for septic shock. The PPV of use of inotropes/vasopressors among shock patients was 88.9% (95% CI: 79.3–95.1). When both a shock code and a procedure code for inotropic/vasopressor therapy were used, the PPV for shock overall was 93.1% (95% CI: 84.5–97.7). ICD-10 codes for subtypes of shock and simultaneously registered use of inotropes/vasopressors provided PPVs of 96.0% (95% CI: 79.6–99.9) for cardiogenic shock, 69.2% (95% CI: 38.6–90.9) for hypovolemic shock, and 82.4% (95% CI: 65.5–93.2) for septic shock.
Overall, we found a moderately high PPV for shock in the DNPR. The PPV was highest for cardiogenic shock but lower for hypovolemic and septic shock. Combination diagnoses of shock with codes for inotropic/vasopressor therapy further increased the PPV of shock overall, and for cardiogenic and septic shock diagnoses.
Shock is a medical emergency in which organs and tissues of the body are not receiving adequate flow of blood, resulting in decreased oxygen delivery for cell metabolism . Shock is defined as sustained systolic blood pressure below 90 mmHg, or a drop of systolic blood pressure of more than 40 mmHg in patients with hypertension . Moreover, cutaneous, renal, and neurological symptoms are often observed as clinical manifestations of shock, and increased serum lactate is often a biochemical sign of abnormal oxygen metabolism in patients with shock . If left untreated, shock can lead to severe organ damage and death [2,3]. Shock is typically categorized as cardiogenic, hypovolemic, obstructive, or distributive shock. The most common cause of distributive shock is septic shock .
Large registries enable efficient and low cost research . However, registry data must be of high validity for use in epidemiologic research.
The positive predictive value (PPV) for shock among patients with myocardial infarction in two Canadian hospital discharge registries has been found to be 75.0% (95% confidence interval [CI]: 19.4–99.4)  and 78.8% (95% CI: 67.0–87.9) [6,7]. In addition, the PPV for severe sepsis, including septic shock, is reported to be 70.7% (95% CI: 51.1–90.4) . No previous study has examined the validity of cardiogenic, hypovolemic, or septic shock diagnoses in the Danish National Patient Registry (DNPR). Therefore, we examined the PPV of (1) diagnosis codes for cardiogenic shock, hypovolemic shock, and septic shock in the DNPR overall and for each code separately, (2) treatment codes for inotropic/vasopressor therapy in the DNPR among patients diagnosed with shock, and (3) shock diagnoses when an in-hospital treatment with inotropes/vasopressors is also registered.
Design and setting
We conducted this cross-sectional validation study using data from medical registries in Denmark. The Danish National Health Service provides tax-supported health care to all Danish residents, including universal and free access to general practitioners and hospitals in Denmark . We used the unique 10-digit Civil Person Registration number, assigned to all inhabitants at birth or upon immigration, to identify medical charts .
Identification of patients with shock in the DNPR
We used the DNPR to identify a random sample of 190 patients admitted with cardiogenic, hypovolemic, or septic shock to Aarhus University Hospital from 1 January 2005 through 31 December 2012. We used both primary and secondary inpatient diagnosis codes to identify the sample of shock patients. Data on all somatic hospital admissions have been routinely registered in the DNPR since 1977, and all outpatient and emergency contacts have been included since 1995 . Each admission is registered by one primary diagnosis code and up to 19 secondary diagnosis codes classified according to the International Classification of Diseases, 8th revision (ICD-8) until the end of 1993, and according to the 10th revision (ICD-10) thereafter . Procedures of inotropic/vasopressor therapy have been registered routinely since 2005 .
We chose to include a minimum of 40 cases for each shock type to allow sufficient precision of the calculated PPV, and because of the maximum available number of coded cases in the study period for all shock types. With this number of cases, a PPV of 90% for shock subtypes would result in a 95% confidence interval (CI) from 76% to 97%, which we considered acceptable. The total number of patients (190) included in the study and the distribution of patients in the three subgroups of shock were based on the distribution of the three shock subtypes registered in the DNPR in the study period. We randomly included patients of the three shock subtypes registered in the DNPR in the study period until at least 40 patients in each subgroup of shock were reached. Data on treatment with inotropes/vasopressors were obtained by in-hospital treatment codes during hospitalization from the DNPR (codes provided in Additional file 1: Table e1).
We obtained data on age and sex from the Danish Civil Registration System .
To assess the level of comorbidities, we used the Charlson Comorbidity Index (CCI) . The CCI is a validated scoring system that has been adapted for use with hospital discharge data . The scoring system assigns a point score from 1 to 6 to a number of diseases depending on their relation to mortality, defined during a one-year period in which the scoring was made . The scoring system generated three levels of comorbidity: 0 (low), 1–2 (moderate), and ≥ 3 (high). For each patient, we used a fixed period of 5 years prior to hospital admission with shock, and included any inpatient or outpatient hospital visits with a diagnosis of the diseases, including both prevalent and incident diseases. ICD-10 codes are provided in Additional file 1: Table e2.
Medical chart review
One author (MDL) obtained and reviewed all available medical charts. Each chart was reviewed, including all notes from date of hospital admission with shock to the date of hospital discharge. This approach was chosen to confirm shock diagnosis and type of shock by predefined diagnostic criteria (Table 1), and to confirm treatment with inotropes/vasopressors.
We tabulated patient characteristics by shock type, including 1) median age and interquartile range, 2) gender and level of comorbidity as counts and percentages.
We used the STATA command diagt (diagnostic test) to estimate the PPV as the proportion of registered diagnoses confirmed by the predefined criteria during medical chart review. Ninety-five percent CIs are given as the exact binomial CI. The calculations were done under the assumption that all the missing medical charts did not differ from the charts found in the archives with respect to the proportion of confirmed shock cases. To ensure that no substantial differences existed between patients with available medical charts and patients with missing medical charts, we (1) compared the two groups of patients with regard to gender, age, and level of comorbidity, (2) examined whether age group, gender, and level of comorbidity were predictors of misclassification using a logistic regression model with verified shock as the outcome, and (3) conservatively estimated the PPV for shock and subtypes of shock by including missing medical charts as non-confirmed shock.
We calculated the PPV for use of inotropes/vasopressors among patients having a diagnosis code of shock. Patients that have a confirmed treatment with inotropes/vasopressors in the medical chart were used as the numerator, and patients with a procedure code of inotropic/vasopressor therapy defined the denominator.
To examine whether the PPVs for shock overall and shock subtypes could be improved using a concurrent procedure code of inotropic/vasopressor therapy, we estimated the PPV as the proportion of confirmed shock cases among patients registered with both a shock diagnosis and an inotropic/vasopressor code. For all statistical analyses, we used STATA statistical software version 13.1 (StataCorp LP, Texas).
Research ethics and informed consent
This validation study was non-experimental and solely used existing data; thus, ethical approval was not needed. The study was approved by the Danish Data Protection Agency (record no. 2006-53-1346).
Among the medical charts reviewed, 46 of 50 (92%) were available for cardiogenic shock validation, 78 of 100 (78%) were available for septic shock validation, and 34 of 40 (85%) were available for hypovolemic shock validation. A flowchart showing the selection process is provided in Figure 1.
Patients with missing charts were slightly younger, and the majority were men (Additional file 1: Table e3). The level of comorbidity did not differ substantially between patients with and without an available medical chart (Additional file 1: Table e3).
The median age at hospital admission among patients with a code of shock was 71 years (interquartile range: 61–80), and the distribution between males and females was equal (Table 2). Patients with cardiogenic shock had an overall lower level of comorbidity compared with patients with hypovolemic and septic shock (Table 2).
Positive predictive value for shock and subtypes of shock
The PPV for shock overall was 86.1% (95% CI: 79.7–91.1) (Table 3). The PPVs were 93.5% (95% CI: 82.1–98.6) for cardiogenic shock, 70.6% (95% CI: 52.5–84.9) for hypovolemic shock, and 69.2 % (95% CI: 73.3–94.2) for septic shock (Table 3). Patients who did not fulfill the diagnostic criteria were equally distributed in groups of patients with unconfirmed shock and patients not fulfilling the specific shock subtype criteria (Figure 1).
When we included missing medical charts as unconfirmed shock in the calculations, the PPV was 71.6% (95% CI: 65.2–78.0) for shock overall, 86.0% (95% CI: 73.3–94.2) for cardiogenic shock, 60.0% (95% CI: 43.3–75.1) for hypovolemic shock, and 54.0% (95% CI: 43.7–64.0) for septic shock (Additional file 1: Table e4).
Male gender, age less than 60 years, and a high level of comorbidities may be predictors of misclassification, although estimates were imprecise (Additional file 1: Table e5).
Predictive values for treatment with inotropes/vasopressors
We found that 72 of the 158 patients diagnosed with shock and with a medical chart available for review had a code for inotropes/vasopressors. Treatment with inotropes/vasopressors was confirmed in the medical chart in 64 of the 72 cases, providing a PPV of 88.9% (95% CI: 79.3–95.1).
Predictive values for combined shock and inotrope/vasopressor codes
The PPV for the shock diagnosis code in combination with inotropes/vasopressors was 93.1% (95% CI: 84.5–97.7) (Table 3). The PPV for shock subtypes combined with inotropic/vasopressor therapy was 96.0% (95% CI: 89.6–99.9) for cardiogenic shock, 69.2% (95% CI: 38.6–90.9) for hypovolemic shock, and 82.4% (95% CI: 65.5–93.2) for septic shock (Table 3).
We found a moderately high PPV for shock diagnoses in the DNPR. For subtypes of shock, cardiogenic shock diagnosis had the highest PPV, whereas more patients diagnosed with hypovolemic and septic shock did not fulfill the diagnostic criteria for these two types of shock.
Procedure codes for treatment with inotropes/vasopressors had also a high PPV. The PPVs for shock were improved when the diagnosis code for shock overall and cardiogenic or septic shock were combined with the procedure code for inotropes/vasopressors. However, the PPV was unchanged for hypovolemic shock when combined with procedure codes for inotropes/vasopressors.
Our finding for the PPV for cardiogenic shock is somewhat higher than results from hospital discharge registries from Canada [6,7]. Lawrence et al. found a PPV of 75.0% (95% CI: 19.4–99.4) for cardiogenic shock among patients with acute myocardial infarction in a hospital discharge registry in Alberta, Canada . However, the study was of limited size, with only four patients registered with ICD-10 data for shock . Another Canadian study of 66 patients diagnosed with an International Classification of Diseases, 9th revision (ICD-9) code for cardiogenic shock in Quebec’s hospital discharge registry reported a PPV of 78.8% (95% CI: 67.0–87.9) . Similar to our finding for septic shock, a study from the University of Michigan Health Systems examined the PPV for ICD-9 codes for severe sepsis, including septic shock, among 111 patients and found a PPV of 70.7% (95% CI: 51.1–90.4) .
Strengths and limitations
Because we reviewed medical chart data from patients with a registered ICD-10 diagnosis code of shock, we could only estimate the PPV and not other measures of validity including negative predictive value, sensitivity, and specificity. Other studies found the sensitivity for myocardial infarction-related shock code to be 60% [6,7] and specificity to be 99% [6,7]. Because we aimed to validate the PPV and not sensitivity of the shock diagnoses, it was not necessary to include a control group.
The medical chart review was not blinded, which could have influenced the data collection. However, strict clinical criteria in congruity with internationally-approved diagnostic criteria [1,15,16] for each shock subtype were defined before the beginning of the data collection; thus, the potential of subjective influence from the data collector was limited. Still, it is a potential limitation that only one author reviewed the data from the medical charts.
Because clinical cases do not always fulfill all diagnostic criteria, some cases are coded with the most suitable code for the course of the disease. Register-based research relies on correct coding procedures, and the PPVs for septic shock and hypovolemic shock reveal some degree of coding inaccuracy. It should also be noted that the criteria for defining shock subtypes may not be mutually exclusive, which may have influenced the PPVs for shock.
Medical charts were not available for all patients in our sample. No substantial differences in the available comorbidity level were found between the two groups of patients, which is why the assumption of no difference between patients with missing charts and those with charts was not disproved.
This study was conducted at Aarhus University Hospital, and all medical charts were randomly selected from the DNPR, including 15 different departments in the hospital. Coding practices between hospital settings might vary. However, the diagnostic criteria for shock subtypes follow national guidelines, which likely make our results representative to other hospitals in Denmark.
No previous study has examined the PPV of shock codes registered in the DNPR, despite the fact that the DNPR may be an important tool for research and quality monitoring of shock. We found fairly high PPVs for shock overall, and for subtypes of shock, which makes these ICD-10 codes usable in studies of risk and prognosis based on data from the DNPR. However, the potential impact of misclassification of shock should be considered when conducting such studies.
We found a moderately high PPV for shock overall in the DNPR using the medical chart as the reference standard. PPV was highest for cardiogenic shock and lower for hypovolemic and septic shock. A feasible approach to increase the PPV of shock overall and of septic shock is to combine the diagnosis with the procedure codes for treatment with inotropes/vasopressors. In summary, the DNPR is a valuable tool for epidemiological research of shock and subtypes of shock, especially cardiogenic shock.
Charlson comorbidity index
The Danish National Patient Registry
International Classification of Diseases, 10th revision
International Classification of Diseases, 8th revision
International Classification of Diseases, 9th revision
Positive predictive value
Vincent JL, De Backer D. Circulatory shock. N Engl J Med. 2013;369(18):1726–34.
Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303–10.
Goldberg RJ, Spencer FA, Gore JM, Lessard D, Yarzebski J. Thirty-year trends (1975 to 2005) in the magnitude of, management of, and hospital death rates associated with cardiogenic shock in patients with acute myocardial infarction: a population-based perspective. Circulation. 2009;119(9):1211–9.
Maier RV: Approach to the Patient with Shock. In Harrison's Principles of Medicine. 18th edition. Edited by Longo DL, Fauci AS, Kasper DL, Hauser SL, Jameson JL, Loscalzo J. New York: McGraw-Hill 2012 http://accessmedicine.mhmedical.com/content.aspx?bookid=331§ionid=40727056. Accessed 06.14.2013.
Sorensen HT, Sabroe S, Olsen J. A framework for evaluation of secondary data sources for epidemiological research. Int J Epidemiol. 1996;25(2):435–42.
So L, Evans D, Quan H. ICD-10 coding algorithms for defining comorbidities of acute myocardial infarction. BMC Health Serv Res. 2006;6:161.
Lambert L, Blais C, Hamel D, Brown K, Rinfret S, Cartier R, et al. Evaluation of care and surveillance of cardiovascular disease: can we trust medico-administrative hospital data? Can J Cardiol. 2012;28(2):162–8.
Iwashyna TJ, Odden A, Rohde J, Bonham C, Kuhn L, Malani P, et al. Identifying patients with severe sepsis using administrative claims: patient-level validation of the angus implementation of the international consensus conference definition of severe sepsis. Med Care. 2014;52(6):e39–43.
National Board of Health. Health Care in Denmark: http://www.sum.dk/Aktuelt/Publikationer/~/media/Filer%20-%20Publikationer_i_pdf/2008/UK_Healthcare_in_dk/pdf.ashx. Accessed 03.28.2014.
Schmidt M, Pedersen L, Sorensen HT. The Danish Civil Registration System as a tool in epidemiology. Eur J Epidemiol. 2014;29(8):541–9.
Lynge E, Sandegaard JL, Rebolj M. The Danish National Patient Register. Scand J Public Health. 2011;39(7 Suppl):30–3.
Blichert-Hansen L, Nielsson MS, Nielsen RB, Christiansen CF, Norgaard M. Validity of the coding for intensive care admission, mechanical ventilation, and acute dialysis in the Danish National Patient Registry: a short report. Clin Epidemiol. 2013;5:9–12.
Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83.
O'Connell RL, Lim LL. Utility of the Charlson comorbidity index computed from routinely collected hospital discharge diagnosis codes. Methods Inf Med. 2000;39(1):7–11.
Dellinger RP, Levy MM, Rhodes A, Annane D, Gerlach H, Opal SM, et al. Surviving Sepsis Campaign Guidelines Committee including The Pediatric Subgroup: Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock, 2012. Intensive Care Med. 2013;39(2):165–228.
Westaby S, Kharbanda R, Banning AP. Cardiogenic shock in ACS. Part 1: prediction, presentation and medical therapy. Nat Rev Cardiol. 2011;9(3):158–71.
Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644–55.
This study was financed by a scholarship from The Danish Council for Independent Research, Medical Sciences (FSS). The funding source had no role in the design, conduct, analysis, or reporting of the study. All authors thank Helle Vester for providing text revision.
The authors declare that they have no competing interests.
HG, MS, and CFC conceived of the study idea and designed the study. MDL and HN collected the data. MDL, HG, MS, and CFC reviewed the literature. MDL performed the statistical analyses, organized the writing, and wrote the initial draft. All authors participated in the discussion and interpretation of the results, critically revised the manuscript for intellectual content, and approved the final version before submission. CFC is the guarantor.
About this article
- Cardiogenic shock
- Hypovolemic shock
- Inotropic therapy
- Positive predictive value
- Septic shock
- Vasopressor therapy