Patient population and data
The Australasian Society of Cardiac and Thoracic Surgeons (ASCTS) is responsible for a registry of cardiac and thoracic surgery in Australia covering nearly half of the country's private and public cardiothoracic surgical units.
Data was collected on all patients undergoing cardiac surgery between July 2001 and June 2009 in 18 hospitals in Australia. In each hospital, a data manager was responsible for the completeness of the data collection. All data was verified on receipt by the co-ordinating centre which followed up on queries about missing data, outliers or inconsistent reports. The data was validated locally and also by an external data quality audit program. This program was performed on-site to evaluate the completeness (2.4% missing value) and accuracy of the data collected within the combined database [33].
The ASCTS registry collected information on patient preoperative risk factors (including preoperative cardiac status and previous interventions), intra-operative details (including the procedure performed, myocardial protection and procedural duration), complications and post operative outcomes. In this study, the outcome variable was mortality within 30-days of cardiac surgery. This information was collected by the data managers by contacting medical practitioners, patients or family members by telephone as part of clinical care.
This research project was undertaken following approval from the ASCTS Research Committee which governs access to data from the registry. Ethical approval for the use of de-identified registry data for secondary research purposes such as this project had previously been provided by each participating institution's ethics review committee.
Patient and hospital level characteristics
Patient-level characteristics in the registry included: age, gender, the New York heart association (NYHA) class, urgency of procedure, ejection fraction estimate, lipid-lowering treatment (hypercholesterolaemia), preoperative dialysis, previous cardiac surgery, procedure type, inotropic medication (inotropes), peripheral vascular disease, and body mass index (BMI) [34]. Two hospital-level characteristics were assessed, namely academic affiliation (teaching or non-teaching status) and the median across 2001-2009 of the annual number of cardiac surgeries. Academic affiliation of the 15 teaching hospitals and 3 non-teaching hospitals did not change throughout the study period. All patient and hospital characteristics were included in analysis as categorical risk factors except the median annual number of cardiac surgeries which was continuous.
Statistical techniques
Ordinary, marginal and multilevel logistic regression
Logistic regression [35] was used to assess the influence of risk factors on 30-day mortality. Let pij be the probability and the odds of death for patient j in hospital i. The equation of the ordinary logistic model was
(1)
where the Xijk's represent a patient's values of R risk factors, and β1...βR are regression coefficients corresponding to each risk factor. For a given risk factor, its coefficient βk is the log odds ratio corresponding to a 1-unit difference in continuous Xk or, if a risk factor is an indicator, for example of peripheral vascular disease (1 if yes, 0 if no), then βk is the log odds ratio comparing the effect on mortality of the risk factor's presence with its absence. Exponentiating βk () gives the corresponding odds ratio, OR. In ordinary logistic regression hospital characteristics are treated the same as patient-level risk factors, and patient outcomes Yij are assumed to be independent binomial variables with mortality probability pij.
A model based on generalized estimating equations, GEE [36], may also be used for analysis of patient mortality outcomes. For this, equation (1) is combined with the following assumptions: firstly as before the probability of Yij = 1, a death, is pij and Yij has binomial variance pij(1-pij). Secondly, it is assumed that patients within a hospital have correlated outcomes but patient outcomes in different hospitals are independent, i.e. have zero correlation. An exchangeable working correlation structure in the GEE estimation process assumes that pair-wise correlations between patient outcomes within the same hospital are equal and can be represented by the parameter ρ.
The GEE method includes the calculation of robust estimates for the standard errors of the regression coefficients that ensure consistent inference even if the chosen working correlation structure is incorrect or if the strength of the correlation between patient outcomes within the same hospital varies somewhat from patient to patient.
Multilevel logistic regression [15] assumes that each hospital has its own underlying mortality probability and this varies from hospital to hospital. Specifically, a logistic regression for patients includes an additional term ui, a hospital-level random effect, as a predictor variable:
(2)
Note that in this model is the conditional probability that patient j in hospital i died, and here the probability depends on the value of the random effect, ui, for that hospital. ui is the totality of measured and unmeasured hospital-level variables that predict mortality and are uncorrelated with the individual and hospital-level predictor variables in the model. In other words ui represents the combination of omitted hospital-level variables.
Variation in the mortality propensity between hospitals is accommodated by assuming a normal distribution for ui with mean zero and variance τ2. A hospital with ui = 0 can be thought of as having "average" (compared to other hospitals in the population) mortality probability (on the log odds scale). Higher values of τ2 indicate greater heterogeneity in mortality among hospitals. By including ui in the model as a random effect, the interdependencies among patients within hospitals are explicitly taken into account.
Odds ratio interpretation in ordinary, marginal and multilevel logistic regression
The interpretation of the odds ratio for patient-level risk factors (eβ) in ordinary logistic and marginal models is the same, but differs from the interpretation in the multilevel logistic model [29]. The marginal models estimate population-averaged (or population-marginalized) parameters [30]. In a marginal model, odds ratios characterize the effect of predictors on the population as a whole, averaged over ui, rather than on a typical hospital [30]. Odds ratios in a marginal model represent, across all hospitals, differences in mortality between all patients with one value of a risk factor to all patients with the other value.
In multilevel models, for patient-level variables, the usual odds ratio interpretations apply for comparisons of patients within the same hospital; for example, a body mass index (BMI) effect may be interpreted as an odds ratio between a patient with BMI < 25 and a patient with BMI > 25 belonging to the same hospital and with the same covariates, except for BMI.
Odds ratios for hospital risk factors in marginal models are interpreted as the odds of mortality for hospitals with one value of the factor compared to hospitals with another value of the factor. For example, the odds of 30-day mortality for non-teaching hospitals compared to teaching hospitals.
However, the odds ratio for a hospital-level risk factor in multilevel logistic regression has a different interpretation, namely the odds of mortality for hospitals with one value of the factor compared to hospitals with another value of the factor but with the same value of random effect. For example, the odds of 30-day mortality for a patient treated at a non-teaching hospital compared to a patient treated at a teaching hospital with the same value of ui. Because of this difficult interpretation an additional parameter, the 80% interval odds ratio (IOR-80%), has been developed and is described later in this paper.
The OR for a risk factor in a marginal model is adjusted for the other risk factors included in Xijk. In a multilevel model the ORs are additionally adjusted for unobserved hospital-level characteristics via the random effect. Due to a mathematical property called non-collapsibility of the odds ratio, the odds ratio for a risk factor from a multilevel model is likely to be further from the null value of one than the odds ratio for that risk factor from a marginal model [37].
Intra-class correlation coefficient (ICC)
The fundamental reason for applying special statistical techniques in multilevel analysis is the likely existence of intra-class (intra-hospital) correlation arising from similarity of mortality risk of patients of the same hospital compared to those of different hospitals. Patients operated on at the same hospital may be more similar to each other than patients operated on in other hospitals, as they share a number of economic, social, and other characteristics that may condition similar health status beyond what can be adjusted for by patient-level covariates.
The total variance in the outcome variable is the sum of patient-level and hospital-level variances. In multilevel logistic regression however, the patient-level variance is on the probability scale whereas the hospital-level variance is on the logistic scale. To solve this technical difficulty the linear threshold approximation has been proposed [16] and its solution is to convert the patient-level variance from the probability scale to the logistic scale. The method assumes that the propensity to die is a continuous latent variable and only those patients whose propensity crosses a certain threshold will die as defined by the binary outcome. The unobserved patient variable follows a logistic distribution with patient-level variance equal to 3.29. On this basis, the ICC is calculated as:
(3)
where τ2 is the estimated variance of the random effect of hospital.
In the marginal model, the working correlation structure is the mechanism that accounts for ICC, and the parameter ρ can be interpreted as a measure of intra-hospital correlation in mortality outcomes. There is a subtle distinction between ρ and the ICC defined in (3) for the conceptual propensity-to-die variable. It has been shown that ρ in a marginal model is smaller than ICC in the corresponding multilevel model when there are a small number of clusters [38].
For a binary outcome like mortality, the term ICC can be difficult. Conceptual problems with the ICC (it is a concept from linear regression that has no exact equivalent for logistic regression), interpretational issues as outlined above, and generalisability problems (ICC depends on outcome prevalence) are some of its limitations. Similarly, τ2, the inter-hospital variation in mortality, is difficult to interpret because it is on a log-odds scale [39, 40].
The median odds ratio (MOR)
The MOR is potentially easier to interpret than the ICC because it expresses inter-hospital variance on the OR scale, on which the effects of risk factors are also interpreted.
MOR is defined for a multilevel model as the median of the set of odds ratios that could be obtained by comparing two patients with identical patient-level characteristics from two randomly chosen, different hospitals, i.e. different in hospital random effect value [31, 32]. The MOR is the median odds ratio between the patient in the hospital with higher mortality propensity and the patient in the hospital with lower mortality propensity.
The MOR is a measure of variation between the mortality rates of different hospitals that is not explained by the modelled risk factors. The MOR can be shown [41] to be simply related to τ2 as
(4)
where τ2 is the hospital-level variance. If the MOR is 1, there is no variation between hospitals. If there is considerable between-hospital variation, the MOR will be large.
Because the two measures of inter-hospital variation, τ2 and ICC, are difficult to interpret [31] the MOR is considered as an alternative measure. While ICC and MOR have a direct relationship due to their shared basis on τ2, the different functions of that inter-hospital variation give rise to usefully distinct interpretations.
The 80% interval odds ratio (IOR-80%)
The interpretation of hospital-level effects in multilevel model has been highlighted as problematic. In multilevel models, contrary to patient-level risk factors, hospital-level risk factors only take one value in each hospital and, consequently, it is necessary to compare patients from different hospitals to quantify hospital-level associations with outcome [41, 42]. A multilevel model odds ratio for a hospital-level risk factor needs to be interpreted as the effect of the risk factor given a comparison between two hospitals of identical random effect value whose mortality probabilities differ only with regard to the risk factor under consideration.
To interpret effects of hospital-level risk factors more generally, the unexplained between-hospital variability needs to be taken into account. The IOR-80% achieves this by incorporating both the fixed hospital-level risk factor effect and the unexplained between-hospital heterogeneity in an interval. IOR-80% shows the impact of hospital-level variables on mortality when comparing hospitals with different ui values.
To understand the calculation of the IOR-80%, consider all possible pairs of patients with identical patient-level risk factors from different hospitals but who differ by one unit in a hospital-level risk factor (e.g. one patient in a teaching hospital, the other in a non-teaching hospital). For each pair, the OR between the two patients is calculated, thereby obtaining a distribution of ORs. The IOR-80% is defined as the interval around the median of the distribution that comprises 80% of the OR values. In practice, the lower and upper bounds of the IOR-80% can be computed using the approximation
(5)
where β is the regression coefficient for the hospital-level variable, τ2 is the hospital-level variance, and the values -1.2816 and +1.2816 are respectively the 10th and 90th centiles of the standard normal distribution.
From equation (5), a small amount of between hospital variation, τ2, will lead to a narrow IOR, whereas large τ2 leads to wider intervals. The combination of τ2 with the effect of the hospital-level risk factors in (5), indicates that the IOR-80% will contain 1 if the value of τ2 is large compared to the effect of the hospital-level risk factors.
Model estimation
To fit the ordinary logistic regression model, maximum-likelihood was used with the Stata [43] (version 11) logistic command. The marginal logistic regression model was fitted with Stata's xtgee command and multilevel models were fitted using adaptive quadrature with 12 integration points to evaluate and maximize the marginal log likelihood by Stata's xtlogit command.