Major adverse cardiovascular event definitions used in observational analysis of administrative databases: a systematic review

Background Major adverse cardiovascular events (MACE) are increasingly used as composite outcomes in randomized controlled trials (RCTs) and observational studies. However, it is unclear how observational studies most commonly define MACE in the literature when using administrative data. Methods We identified peer-reviewed articles published in MEDLINE and EMBASE between January 1, 2010 to October 9, 2020. Studies utilizing administrative data to assess the MACE composite outcome using International Classification of Diseases 9th or 10th Revision diagnosis codes were included. Reviews, abstracts, and studies not providing outcome code definitions were excluded. Data extracted included data source, timeframe, MACE components, code definitions, code positions, and outcome validation. Results A total of 920 articles were screened, 412 were retained for full-text review, and 58 were included. Only 8.6% (n = 5/58) matched the traditional three-point MACE RCT definition of acute myocardial infarction (AMI), stroke, or cardiovascular death. None matched four-point (+unstable angina) or five-point MACE (+unstable angina and heart failure). The most common MACE components were: AMI and stroke, 15.5% (n = 9/58); AMI, stroke, and all-cause death, 13.8% (n = 8/58); and AMI, stroke and cardiovascular death 8.6% (n = 5/58). Further, 67% (n = 39/58) did not validate outcomes or cite validation studies. Additionally, 70.7% (n = 41/58) did not report code positions of endpoints, 20.7% (n = 12/58) used the primary position, and 8.6% (n = 5/58) used any position. Conclusions Components of MACE endpoints and diagnostic codes used varied widely across observational studies. Variability in the MACE definitions used and information reported across observational studies prohibit the comparison, replication, and aggregation of findings. Studies should transparently report the administrative codes used and code positions, as well as utilize validated outcome definitions when possible. Supplementary Information The online version contains supplementary material available at 10.1186/s12874-021-01440-5.

(EMA) in 2012, provided guidance on utilizing a threepoint MACE outcome, which includes acute myocardial infarction (AMI), stroke, and cardiovascular mortality in all trials evaluating the cardiovascular safety of diabetic agents [3]. Some trials have utilized a four-point MACE as well, by including hospitalization for unstable angina or revascularization procedures [3,4]. Five-point MACE further expands on this with the inclusion of heart failure (HF). While MACE is now a better-defined and more ubiquitous outcome among RCTs, its use in observational studies to assess the safety and real-world effectiveness of therapies remains less clear.
Observational studies using large administrative databases can evaluate population-level use and outcomes of therapies in an efficient and cost-effective way [5]. However, several common issues with observational studies limit their reproducibility and comparability to RCTs, thus limiting the utility of MACE as a composite outcome. First, in comparison to RCTs, observational studies often do not consistently report their protocols, including outcome definitions [6]. To allow for useful comparisons between observational studies and RCTs, improved standardization and transparency is needed [7,8]. Even when MACE components are well-defined, such as AMI or stroke, another challenge encountered in observational studies is how to define outcomes using diagnosis codes available from administrative data. For example, the International Classification of Diseases (ICD) is a commonly used coding system for medical reimbursement and is one of the most frequent sources of information available in administrative databases [9]. Unfortunately, ICD codes can be prone to errors and diagnosis misclassifications, as they are primarily collected for reimbursement purposes and rely on clinical documentation that can vary across settings and providers [5]. Though several studies have attempted to validate the diagnosis codes for commonly used MACE components (e.g., AMI, stroke, and HF), with positive predictive values of upwards of 80-90%, it is unclear how these codes have been taken up in MACE composite outcome definitions [10][11][12]. For these reasons, the current use of MACE in observational studies warrants further investigation.
Thus, the purpose of this review was to systematically determine the most common definitions of MACE employed in observational studies using administrative data. With that, our objectives were: i) assess each study's definition of MACE components (e.g., AMI, stroke), ii) assess the diagnostic criteria used for outcome ascertainment such as codes used and position of codes, and iii) assess whether outcomes had been validated. We hypothesized that, across observational studies, there is great variability in the definitions of MACE used and minimal alignment with the classic three, four, or five-point MACE outcomes. Our hope is that this work will promote a standard approach to the definition of MACE in future studies, allowing for the improved transparency and reproducibility of observational studies.

Search strategy and selection criteria
The protocol for this systematic review is based on the 2015 Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) statement [13]. The protocol was not registered in the International Prospective Register of Systematic Reviews (PROS-PERO), but the original protocol and modifications are included in Supplementary Text 1 (see Additional file 1). The reporting of this review is based on the 2020 PRISMA statement and the checklist is provided in Supplementary Table 1 (see Additional file 2) [14]. We searched MED-LINE and EMBASE for literature published from January 1, 2010 to October 9, 2020 that defined composite MACE as a primary or secondary outcome in studies utilizing administrative databases. A 10-year lookback was used in order to restrict to the most recently published studies. The search strategy was developed by the authors using the search terms presented in Supplementary Text 2 (see Additional file 3) and was performed on October 9, 2020. A standardized protocol for study abstract screening and full-text review was developed and piloted using the same 20 studies among the reviewers (EB, ES, LH), and is further described in Supplementary Text 1 (see Additional file 1). The abstracts from each record were initially screened for relevance by three independent reviewers (EB, ES, LH) using Abstrackr (http:// abstr ackr. cebm. brown. edu/), a free semi-automated abstract screening tool [15]. Afterwards, we performed a full-length text review of the abstracts that were retained after screening (EB, ES, LH).
We included studies that identified MACE composite endpoints as the primary or secondary study outcome, used administrative data sources in the study, and defined MACE using International Classification of Diseases, Clinical Modification, Ninth Revision (ICD-9-CM) or Tenth Revision (ICD-10-CM) diagnosis codes. The included studies did not need to overtly name "major adverse cardiovascular events" as the primary or secondary outcome to be included into this study, as many definitions and terms are used across studies. For example, MACE may be referred to as "adverse" or "acute cardiovascular events", "major adverse cardiovascular and cerebrovascular events" or simply "cardiovascular events", among other terms. Thus, we included studies if the composite outcome that was studied utilized a combination of multiple MACE endpoints. We excluded reviews, meta-analyses, conference abstracts, editorials, papers whose primary or secondary outcome was not a composite of multiple MACE endpoints, and those that did not use or report ICD-9-CM or ICD-10-CM codes to define MACE endpoints. Any uncertainty with regards to inclusion or exclusion of an individual paper was resolved by consensus of the three reviewers.

Data extraction and synthesis
After each of the independent reviewers determined that a full-text met inclusion criteria for the study, definitions for composite MACE outcomes were extracted into a shared document using a standardized protocol by two reviewers (LH, EB). ICD-9-CM and ICD-10-CM codes were also extracted for each MACE endpoint, and were acquired either through full-length text review or through review of published supplements. The position of the diagnosis code required in the study outcome criteria was recorded as primary position, any position, or not reported. Further, information on whether the study outcomes had been previously validated was assessed in each individual study or review of supplements, and citations were extracted. Each study's administrative data source and study years were also recorded.
Once data were extracted from included studies, we categorized the specific definitions of MACE based on the individual components referred to by the authors as opposed to the specific diagnosis codes used. We made this decision due to the lack of consistency in diagnosis codes used across studies and chose to categorize based on the MACE components that the authors reported. Component definitions of MACE included: AMI, acute coronary syndrome or ischemic heart disease (ACS/ IHD), stroke (either ischemic or hemorrhagic stroke), revascularization procedures, cardiovascular (CV) death, and all-cause death. Of note, due to the variability of outcomes used across studies, the ACS/IHD component reflects the following definitions used by study authors: ACS, IHD, coronary artery disease (CAD), and unstable angina (UA). Overall, we performed a qualitative assessment of the evidence only, and did not perform a metaanalysis or strength of evidence assessment due to the nature of the research questions.

Included studies
Our search of MEDLINE and EMBASE yielded 920 unique articles, 412 of which were retained for full-text review after abstract screening (Fig. 1). After excluding 354 studies during full-text review, 58 studies were included in the final analysis.

Overall studies
The included observational studies utilized a range of outcome components to define MACE ( Table 1). The majority of the included studies utilized ICD-9-CM to define outcomes. There was poor concordance of MACE definitions used in the included studies when compared to the three-point, four-point, and five-point definitions of MACE commonly used in randomized controlled trials (RCTs). For instance, 8.6% (5/58) of observational studies used a MACE definition that matched the threepoint definition of MACE (AMI, stroke, CV death) while no studies matched the four-point or five-point definitions ( Table 2). Across all included studies, the most common MACE component definitions were: AMI, stroke, 15.5% (9/58); AMI, stroke, all-cause death, 13.8% (8/58); and AMI, stroke, CV death 8.6% (5/58) (Table 3). Overall, 67% of studies (39/58) did not perform a validation or provided no citations validating the outcome definitions used. Additionally, 70.7% (41/58) of studies did not report the position of diagnosis codes used in endpoints, while 20.7% (12/58) used the primary diagnosis position and 8.6% (5/58) used any position.

Acute coronary syndrome / ischemic heart disease
There were 18 studies that defined ACS/IHD as a component of MACE. ICD-9-CM diagnosis codes only were used in 66.7% (12/18)

All-cause death and cardiovascular death
There were nine studies that included cardiovascular death as a component of MACE. Of these, 11.1% (1/9) of studies utilized ICD-9-CM codes only for MACE components and 66.7% (6/9) used ICD-10-CM. The diagnosis position for cardiovascular death was primary 11.1% (1/9), any position 33.3% (3/9), and not reported 55.6% (5/9) of the time. The most common diagnosis codes used were: No codes listed, 100% (1/1) for studies using ICD-9-CM only; and I00.xx-I99.xx, 33.3% (2/6) for those using ICD-10-CM only. There were 15 studies including all-cause death as a component of MACE. Due to the nature of all-cause death, diagnosis codes positions are not necessary and were not reported.

Discussion
In our systematic review, we found substantial heterogeneity for MACE composite endpoints used in the literature. The two most common composite MACE   [99]. These benefits not only potentially reduce research costs but can more expediently answer clinical questions, leading to improved patient care. Therefore, given that MACE endpoints will likely see increasing use as time goes on, there should be a continued effort to standardize and transparently report MACE definitions used in observational studies.
In 2007, findings similar to our study were observed when comparing prospective trials conducted for percutaneous coronary interventions [100]. However, although the FDA in 2008 and the EMA in 2012 attempted to standardize the MACE endpoint definition for RCTs, we found a continued discordance between how observational studies define MACE. Furthermore, we found that the majority of studies did not include mention of the diagnosis position used to define outcomes. Lack of this information prevents the ability to distinguish between incident and prevalent outcomes. While the primary position has been historically used in observational studies to define incident outcomes, the use of only primary diagnosis positions, compared to using secondary positions, may underestimate the rates of MACE for prevalent conditions [101]. The lack of reporting of diagnosis position prevents the ability to make this distinction and prohibits the full interpretation of findings. Acute coronary syndrome / Ischemic Heart disease (n = 18) 12 5 1 Stroke (n = 50) 30 17 3 Heart failure (n = 15) 12 2 1 All-cause death (n = 15) 10 5 0 CV death (n = 9)  Heterogeneity not only existed in defining MACE components but also in which ICD-9-CM and ICD-10-CM codes were used to define each individual MACE component. In our study, none of the top five components of MACE had a consensus ICD-9-CM or ICD-10-CM code that was used among all of the studies. These discrepancies contribute additional variability to our findings, as outcome classification, assuming a standard definition, inherently relies on the providers and medical billers accurately coding diseases. For example, in a study conducted in 11 Canadian emergency departments, the authors reported about 82-86% of agreement with regards to coding hospital conditions [102]. Additionally, a study looking at stroke data collected through the United States Paul Coverdell National Acute Stroke Program found that discordance existed with how strokes were coded, especially at smaller hospitals (< 200 beds) and when differentiating between ischemic strokes, transient ischemic attacks, and subarachnoid and intracerebral hemorrhages [103].
There have been many attempts to validate diagnosis or procedure codes for various MACE component outcomes [32,104]. Research has predominantly focused on the older ICD-9-CM codes, particularly studies using U.S. administrative data, but studies utilizing ICD-10-CM have more recently been published. Using data from the Centers for Medicare and Medicaid Services, a 2018 study found that the most accurate ICD-9-CM code for coding AMI was 410.xx, with a positive predictive value of 67% [104]. Additionally, the U.S. FDA commissioned the Mini-Sentinel pilot program to systematically assess the validation of common healthcare outcomes in administrative databases [32]. These studies were important steps to identify standard outcome definitions for use across disparate data sources and populations. As part of the program, systematic literature reviews from the U.S. and Canada found that the most accurate ICD-9-CM code for heart failure was 428.xx, with a positive predictive value of 84-100%, while the most accurate ICD-9-CM codes for stroke were 430.x, 431.x and 434.x, all with separate positive predictive values > 80%, and 436.x, which had a positive predictive value > 70% [32,33]. The Mini-Sentinel program also assessed the ICD-9-CM codes, 410.× 0 or 410.× 1, for AMI and found a positive predictive value of 76.3-94.3% [105]. Fewer validations exist for ICD-10-CM codes. One study conducted in Japan attempted to validate the ICD-10-CM code I21.x for AMI and found a positive predictive value of 82.5% [106]. Another study in Canada found the positive predictive value for correctly coding stroke was 92% for the ICD-10-CM codes, I60.x-I61.x, I63.x-I64.x, H34.1 and G45.x [69]. Based on our findings, the most common ICD-9-CM and 10-CM codes used for AMI were also the most validated, 410.xx and I21.xx, respectively. The most validated and most common ICD-9-CM code used for heart failure was 428.xx. On the other hand, stroke had some discrepancies when comparing the most validated versus the most commonly used ICD codes. This discordance likely existed because of the differences with which stroke can be defined, Outcome ICD-9 code ICD-10 Code Both ICD-9/10 Codes particularly whether to include acute ischemic strokes with transient ischemic attacks, intracerebral hemorrhages or subarachnoid hemorrhages [69]. There are several limitations to our study. We excluded papers published before 2010, prior to the first published reviews of the Mini-Sentinel program, in order to present a contemporary review of the subject [32]. However, this cutoff likely skewed our results towards more modern MACE definitions and excluded different definitions that were potentially used for older publications. Older studies likely exhibited greater variety in the outcome definitions used. Additionally, because the words "major adverse cardiovascular events" were rarely explicitly written, we had to include any composite definitions that were thought to represent MACE but were listed using another composite name, such "acute cardiovascular events, " based on the discretion of the reviewer. This method potentially excluded studies that used variations of MACE endpoints, but did not conform to similar naming conventions.

Conclusion
Significant heterogeneity exists in how MACE is defined and which ICD-9-CM and 10-CM codes are used to represent each respective MACE outcome in observational studies using administrative databases. The utility of future studies will improve with the use of validated definitions of AMI, stroke, cardiovascular mortality, unstable angina, and HF for the evaluation of MACE outcomes. Further, investigators should ensure that both the ICD diagnosis codes and the code positions are reported in a transparent way. Given the significant heterogeneity that already exists across administrative databases, we recommend the use of more standardized MACE definitions and corresponding ICD-9-CM and ICD-10-CM codes. This practice will allow researchers to more accurately compare findings across studies improve the reproducibility of observational studies, and decrease the potential for misleading conclusions.