Identification of confounder in epidemiologic data contaminated by measurement error in covariates

Empirical confounder identification strategies

Simulation study: overall framework and method of analysis

In specific simulations that we undertook, we assumed that (a) the exposure is associated with the outcome and (b) the putative confounder is indeed a confounder by virtue of its association with both exposure and the outcome (but not the descendant of them). As in many real-life situations, the exposure and confounder are measured with error: for simplicity, we focus on additive classical measurement error models with uncorrelated errors (but our simulation framework can readily be amended to accommodate more complex situations).

The disease model that was considered in our investigation was of the form g(Y) = α + βX + γZ, with g() representing the link function of the generalized linear model, the fixed effects α (background rate or intercept), β (influence of exposure X on outcome Y), and γ (influence of covariate Z on outcome Y). The regression coefficient β is only identical to true value of the effects of interest in linear regression but for logistic and Cox proportional hazard regression, the effects of interest is calculate as relative risk (RR) and hazard ratio (HR), respectively. We denote these true effects of interest as θ for generality.

We assumed that we can only observe realizations of true exposure and confounder with classical additive measurement error models X ^*  = X + ε _x and Z ^*  = Z + ε _z , the error terms are unbiased and independent of each other. The estimates of regression coefficients β from (Y, X ^* , Z ^* ) data with and without adjustment for Z ^* are denoted by β _Z ^* and β ₀ ^* , respectively. These regression coefficients can be used to calculate estimates of the effect of interest θ as θ _Z ^* and θ ₀ ^* , with and without adjustment, respectively; the superscript “^*” denotes variables and estimates contaminated by measurement error.

The screening test for Pearson correlation of X ^* and Z ^* being different from zero used p≤0.05 cutoff. The datasets where covariates Z were not rejected are evaluated using the simulated CIE cutoff calculated as follows.

Simulation study: the specific scenarios

Our example synthetic data scenario features an outcome Y and three different types of outcome were generated, namely binary (with the disease prevalence at follow-up of 10 %), continuous (with a variance of 1), and survival (with the death rate at follow-up of 10 %). The exposure X and true confounder Z both simulated to follow standard normal distributions, Z is associated with both X (via Pearson correlation ρ ≠ 0) and Y (γ ≠ 0). The binary, continuous, and survival data were generated and fitted using a logistic model (ln(P(Y = 1)/P(Y = 0) = ln(1/9) + βX + γZ)), a linear model (Y = βX + γZ + ε _y , ε _y  ~ N(0, 1)), and a Cox model (survival time ~ exp(βX + γZ-min(βX + γZ)), censored at survival time > 0.1), respectively. The survival times were generated as follows: (1) mean survival time for all subjects equaled βX + γZ, (2) the aforementioned means survival times were linear transformed to make then all positive by subtracting the minimum value of (βX + γZ), (3) the survival time for each subject was generated to follow an exponential distribution with rate parameter equal to mean survival time from step (2), (4) survival times were censored at a value of 0.1 so that the outcome was observed in only 10 % of subjects.

In illustrating the kind of information that this tool can yield, we obtained N = 10,000 simulation realizations of a cohort study (yielding a standard error of 0.5 %) of either n = 500 or 2,000 subjects, with ρ∈{0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9} and the true causal associations of X-Y and Z-Y with β = γ =0.1, as well as a situation in which exposure of interest is measured with smaller error than or equal to the putative confounder, i.e. ε _x  ~ N(0, σ_x²∈{0.25, 1}) and ε _z  ~ N(0, σ_z²∈{0.25, 1}). We used K = 10,000 to determine simulation-based CIE for each combination of parameters defined a simulation framework above.

Effect estimate and p-values resulting from these models of the n ^th simulation realization are compared and, depending on the confounder selection rule that was triggered, the final estimate of the effect of X on Y in that particular simulated dataset was computed by either model (a) or (b).

All simulations were carried out using R 3.2.0. The R code we provide allows one to test various CIE cutoffs in order to determine the percentage of the simulated datasets correctly identifying Z ^* as a confounder or effect of X ^* as significant, as well as RMSE resulting from model selected after application of each confounder-identification strategy (Additional file 1).

Application to study design to clarify role of a confounder in NHANES

We illustrate the application of our approach in an example arising from an earlier analysis of exposure to total blood mercury (E) and depression (D) in 6,911 adults aged ≥40 years in the National Health and Nutrition Examination Survey (NHANES) 2009–2010 [18] approved by The National Center for Health Statistics Research Ethics Review Board; informed consent was obtained from all participants at the time of data collection and further consent for specific analyses of this publically available data is not needed. The dataset can be downloaded at http://wwwn.cdc.gov/Nchs/Nhanes/Search/nhanes09_10.aspx.

Contrary to an expected null association between exposure and outcome, a nonsensical inverse (protective) association was reported and the original investigators argued that this was probably due to measurement error leading to residual confounding by fish consumption (F) [18]. That study assessed the binary recall of fish consumption in the 30 days prior to data collection (F _obs ). This variable does not demonstrate statistical properties that support its inclusion as a confounder in the final model because (a) p-value for F _obs in the logistic regression model = 0.82, and (b) inclusion of F _obs the final models does not affect the observed RR_{ED | Fobs} = 0.83 (OR_{ED | Fobs} = 0.79) of depression due to mercury to the third decimal place. Nonetheless, it is important to note that our preliminary test for potential confounding would not have rejected F _obs from the final model because there is evidence that it is correlated with exposure to mercury, albeit “weakly”: Pearson correlations of 0.39 with mercury exposure (95 % CI 0.37–0.41, p< 0.05). Furthermore, given the established effects of habitual fish consumption F on blood mercury E [19] and depression D [20], Ng et al. [18] suspected that F is a confounder of association of total blood mercury with depression (see Fig. 1 for the DAG of the causal association), and that the pattern of results arose because F _obs is a poor measure of unobserved F.

Let us suppose that in the course of future research we have a particular method of measuring fish consumption (W) with a known degree of measurement error that may prove to be superior to F _obs . It is important to note that we do not wish to use F _obs in the new research project with the same sample: it yielded what we believe to be confounded estimates and the motivation of new research would be to improve on quality of data to understand the problem better. We need to simulate W because it is not given in the data but is a reflection of what we can hope to obtain in the study that is being planned under the assumption of given causal structure: we can never hope to measure F itself but need to be generate W and thereby evaluate performance of W. We want to know two things: (a) whether W is a confounder of the mercury-depression relationship, and (b) whether models adjusted for this measure of fish consumption W will result in the hypothesized null mercury-depression association (i.e., RR_{ED | W} = 1, as opposed to the observed estimate RR_{ED | Fobs} = 0.83) Here, W is related to true habitual fish consumption F by classical measurement error: W = F + ε, ε ~ N(0, σ ² ); F ~ N(0,1); ε is independent of both E and F. To more specifically motivate these assumptions, reflecting on common experience in exposure assessment, we consider that F is a measure of fatty acid intake that is measured by a biomarker and then normalized to Gaussian distribution via log-transformation, hence additive measurement error model for W and distributional assumptions can be appropriate. In practice, such assumptions would be verified in a validation or reliability study.

We assumed that total blood mercury E is near-perfectly measured because a precise analytical technique exists. To simplify, we ignored the matter of etiologically relevant windows of exposure, although this may not be trivial because the biologic half-life of mercury in blood is short.

Based on prior research [18], we also assumed: (a) the association between F and D is based on the correlation of underlying continuous measures of F and D, and set it to ρ _FD  = −0.35, and (b) that the correlation of F and E is ρ _FE  = 0.39, same as the observed correlation F _obs and E. With these inputs, we simulated true (F) and mis-measured (W) values of fish consumption subjected to different magnitudes of measurement error. Under various conditions of measurement error, we simulate W 10,000 times. Different degrees of error in measured confounder, σ ² , were examined. We acknowledge that a different model for confounding could have been postulated and included in our simulation framework but we followed the path deemed as sensible by the original investigators in [18].

To empirically determine whether measured fish consumption (W) would correctly remove confounding from effect of mercury on depression, the proportions of simulated datasets in which the adjusted association of mercury on depression, RR_{ED | W}, has p >0.05 were recorded. This is akin to asking how well should we measure confounder in order to have the power to observe true exposure-response association upon adjustment. We also reported the simulation-determined confounder identification criterion described above (i.e. aimed to control type I error at 5 %) to compare it to the conventional CIE of 10 %. Finally, we also determined the average and the 95 % empirical confidence intervals of the estimates of the mercury-depression association with adjustment for simulated values of W based on the 10,000 simulations, in order to determine how well the adjustment for W is able to estimate a specified true causal effect of E-D adjusted for F. (The number of simulations we informed by the observations that it was sufficient to obtain stable behavior of simulation realizations; in every specific situation, a difference size of simulation may be needed.) This reflects a theoretical perspective for confounder identification where based on some pre-determined DAG, W is theoretically a confounder of the exposure-outcome association and should therefore be included in models regardless of measurement properties. To visualize the empirical distribution of RR_{ED | W}, we plotted its histogram from the 10,000 simulated estimates with σ ²  = 1. The R code for the simulations can be found in the Online Supplementary Materials (Additional file 2).

Comparison of confounder identification strategies in a synthetic data: identification of structural confounder

Compared with the empirical criteria tested that were fixed a priori (significance criteria with cutoff levels of p-values of 0.05 or 0.20, and CIE criterion with a cutoff of 10 %), the two simulation-determined CIE criterion exhibited superior performance in selecting the correct model within at least 80 % of simulated datasets for exposure-confounder correlation of 0.2 or higher. In contrast, the three traditional methods perform poorly in all three outcome models.

The traditional methods identified a true confounder generally in less than 60 % of the simulated datasets with binary outcomes (logistic regression), even when the cohort size increased from 500 to 2,000 (Figs. 2 and 3, left-hand panels). The gap in performance in cofounder identification was more apparent for smaller cohort (Fig. 2, left-hand panels): 20–40 % drops in power for the stronger simulated confounding with ρ>0.5. Similar gap in performance remained when cohort was increased to 2,000 while measurement error in exposure was fixed at the lower value, and when measurement error in cofounder was greater than that in exposure (Fig. 3, top 2 left-hand panels). However, as both cohort size and measurement error in exposure increased and confounding became stronger (ρ>0.3), a more regular pattern of power was observed: CIE criteria simulated to control type I error had power 70–100 %, CIE criteria simulated to control type II error had power 70–80 %, significance test p<0.20 had power 40–60 %, significance test p<0.05 had power 20–30 %, and 10 % CIE criteria failed to identify structural confounder in almost all instances (Fig. 3, bottom 2 left-hand panels).

In linear regression, the traditional methods were more comparable to simulated-based CIE approaches but their performance depended on strength of confounding and degree of measurement error in a complex fashion. When higher value of error variance in exposure X were examined, all approaches had similar performance in confounder identification for the large cohort of 2000 (Fig. 6, bottom 2 left-hand panels), except that CIE method designed to control type II error to 20 % behaved erratically as strength of confounding increased beyond ρ = 0.3. In smaller cohort size with the same “large” error in exposure (Fig. 5, bottom 2 left-hand panels), however, there was a clear advantage to simulation-based CIE method catered to control type I error to 5 %, especially when cofounding grew stronger: it maintained power of at least 80 % beyond ρ = 0.3 whereas significance testing and CIE cutoff for control of type II error to 20 % were less successful, with power dropping below 80 % as both the strength of confounding (ρ >0.3) and measurement error in confounder increased (Fig. 5, bottom left-hand panel). The divergence in performance of different criteria was the greatest when error in confounder exceeded error in exposure and the cohort size was smaller (e.g. compare Fig. 5, 2^nd from top left-hand panel vs. Fig. 5, 2^nd from top left-hand panel).

Survival analysis mimicked linear model but deficiency of performance of traditional approaches tended to be greater and, paradoxically, worse with smaller measurement errors for exposure. For example, in survival analysis with cohort size of 2,000 and error variances 0.25 (the smallest tested) and strongest confounding (ρ = 0.9), when simulated CIE criteria correctly included Z as confounder in >80 % of cases, the “significance-testing” approaches had power of 30–60 % only (Fig. 7, top left hand panel). The gap in performance reduced when error variances increased to the largest value tested: 90 % vs. 60–80 % (Fig. 7, bottom left hand panel); it must be noted that the reverse pattern held for the 10 % CIE approach as its power dropped to zero as measurement error increased. It can be observed that when error in confounder increased for this setting, but error in exposure was help constant, the significance criteria suffered greatest loss of performance (from power of 90 % to <40 %) and 10%CIE criterion dropped power from about 60 % to <10 % (Fig. 7, two middle left hand panels). The patterns in smaller cohort of 500 were similar (Fig. 6).

Comparison of confounder identification strategies in a synthetic data: precision

The simulation-determined CIE criterion achieved the smallest RMSE in all survival analyses (Figs. 6 and 7, right hand panels). In linear models, the pattern was complex. For the smaller cohort size, simulation-based CIE approaches led to smaller RMSE only when error in exposure measurement was at the lower tested value (Fig. 4, two upper right hand panels), otherwise, the significance testing approached yielded smaller RMSE (Fig. 4, two bottom right hand panels). For a larger cohort, linear model built using simulation based CIE tended to be associated with lower RMSE (Fig. 5, right hand panels). In logistic regression, simulation-based CIE approaches also tended to produce larger RMSE for the smaller of the tested cohort (Fig. 2, right hand panels), with the 10 % CIE criteria leading to the lowest RMSE across varying degrees of confounding (Fig. 2, bottom right hand panel). When a large cohort was considered in logistic regression analysis, the simulation-based approaches had lower RMSE when measurement error in exposure was fixed at a smaller value only (Fig. 3, right hand panels), just like in the linear model.

Larger degree of measurement error tended to produce lower RMSE values (e.g. survival analysis, Figs. 6 and 7, right hand panels), possibly indicating clustering of estimates around attenuated effect estimate and conveying false certainty in the effect estimate under measurement error. There was also a tendency for RMSE to increase with the degree of cofounding in most studied settings (Figs. 2, 3, 6 and 7, right hand panels) However, the RMSE deceased with the exposure-confounder correlation in linear models, when the measurement errors of exposure and confounder were both “large” (i.e. set at 1) (Figs. 4 and 5, bottom right hand panels). On the other hand, when measurement errors are smaller, in linear model there is an increase in RMSE with the strength of confounding (Figs. 4 and 5, top 3 right hand panels) as in other models.

Influence on power of excluding a true confounder by the screening procedure

We can expect the screening of correlation of potential confounder and exposure by means of testing correlation between them to erode power: observed correlation in a sample can be very weak and imprecisely estimated even when there is true correlation in the population. This can be expected to most serious in small sample sizes and for weak true correlations. We examined this issue by examining loss of power due to the screening procedure in the case of n = 500 (small sample size in our simulation). We noted that our screening procedure excluded variables with little correlation (ρ≤ 0.1 for σ_x² = σ_z² = 0.25, and ρ≤ 0.2 for σ_x² = σ_z² = 1) with the exposure, and for ρ >0.3 these variables were nearly never being excluded (Fig. 8). When confounding was weak, the loss of power was more apparent. Thus, there appeared to be observable loss of power only for the weakest strength of confounding and when error variances are large.

Application to the mercury, fish consumption, and depression example

If we do not have sufficient knowledge to guide a theory-based confounder selection strategy, application of model selection cutoffs may be useful. In this specific setting, a simulation-derived CIE cutoff was small (0.06 %). If such a strategy is adopted, for the observed RR = 0.83 a change of 0.1 % after adjustment for W identifies it as a confounder, even though one can question whether such a small change is discernible from background noise in realistic applications. The degree to which it is important to remove such a degree of confounding depends on the specific application and can range from immaterial to important depending on the association of interest and the role it plays in whatever action is taken on the basis of effect estimation. However, it is clear that the CIE of 10 % is too coarse to detect confounding in this setting with the desired certainty.

Figure 9 shows the empirical distribution of the adjusted RR for the noise-to-signal ratio of 1 (median 0.909, inter-quartile range 0.905–0.914). We can see that we expect all estimates to be closer to null than the naïve and can therefore take the observed effects of that size to support the hypothesis that an apparent mercury-depression association is due to confounding by fish intake. Thus, even if residual confounding is not eliminated after adjusting for a mis-measured confounder, we can still determine whether evidence supports its role as a confounder. This clearly argues for a much more liberal rule for evaluating evidence for confounding, based on statistical grounds alone in the given motivating example, in the presence of measurement error in confounder, than is permitted by the 10 % CIE criteria. It also illustrates the peril of reliance on hypothesis testing: we do not expect to find a statistically significant effect of fish intake in the example illustrated in Fig. 9 and yet all “imperfectly” adjusted point estimates of RR are expected to be less biased than the crude value. This further argues for forcing a variable into a model if there is a theoretical reason to do so, regardless of whether a frequentist hypothesis test indicated an association.

Overview of findings

Our study provides a framework that evaluates the performance of various empirical criteria with considerations of strength of causal and confounding effects, sample size, measurement errors in exposure and confounder, and types of outcomes. This framework is useful for study design and planning. During the stage of study design, researchers often need to choose, among many options, the tools for measuring the exposures and confounders. For example, they can choose among self-report or objectively-measured BMI, physical activity level, and dietary intake. Using the existing results from validation studies [10–13], researchers can make use of our framework to choose the most appropriate measurement tools that maintain a balance between the error of causal effect estimation and the total cost induced.

We were also successful in proposing a solution to the problem of false positive identification of risk factor as confounder in a finite sample. A simple evaluation of correlation between exposure and potential covariate achieved nearly perfect power. We emphasize that this is finite-sample problem that arises due to chance correlation, induced by the fact that X and Z are both related to Y, inducing a chance X-Z association via Y that had a tangible impact (i.e. on the order of simulated CIE cutoff) on estimate of effect of X on Y upon inclusion of Z in a regression model. This phenomenon disappeared when sample size was boosted and worsened in finite samples with large values of β _Z (details available on request). Even if Z is not identified as structural confounder, there is a good reason to include it in the final estimate to remove as much as possible confounding from the estimate of effect of X, however, in doing so, the understanding of problem under study is increased by gaining evidence for chance versus structural confounding by Z.

In practice, epidemiologists that use DAG to identify confounders try to distinguish between several plausible DAGs: this lies at the heart of confounder identification problem. However, all assumptions about plausible DAGs made by investigators can be wrong such that any choice among alternative causal structures does not reflect the true state of nature. Our work does not address such situation but it would be important to consider such a possibility in any truly improved approach to selection of confounder identification method; we believe that this is possible via simulations in a specific setting and allude to this in presentation of empirical example elaborated in this paper.

Given the complexity of factors that influence selection of confounder identification approach and tools even in our relatively simple settings, a practical approach is to customize simulations to reflect uncertainties about causal structures and imperfections of data when making such choices instead of reply on any a priori advice. Previous simulation studies showed that a priori advices such as significance criteria and 10 % CIE may lead to wrong decision of confounder adjustment when the exposure variable is error prone [21]. While a simulation-derived CIE criterion would change for every data situation, our study indicates that using simulations to inform model selection is both feasible and desirable during the study-planning stage, using information that most investigators possess: the knowledge of quality of instruments measuring exposure and confounders, as well as plausible strengths of the associations. It must be emphasized that we propose a solution to a problem that is sensitive to each specific application and, as such, our method is guaranteed to out-perform any general advice such as 10 % CIE, unless, by chance, simulation-based CIE are identical to 10 %, in which case our method will have identical performance relative to the general advice.

Interpretations of findings from analysis of simulated studies

Despite complexities of patters of our results, they seem to exhibit several general tendencies. As the strength of confounding increases, the chance of identifying confounder, when present, also grows across constellations of measurement error, type of outcome and sample size, implying that all confounder identification strategies tend to perform better in picking up stronger effects. Most of our findings were consistent across different outcome types.

With respect to precision of the exposure-outcome association as measured by RMSE, there appear to two competing phenomena. As measurement error of the exposure increases, under the postulated classical measurement error model, for all regression coefficients to be attenuated towards null [7], such that whether an estimate is corrected for confounder or not would make little difference to its attenuated value that tends towards the null and a poorly-measured confounder would do little to remove true confounding in regression model. This would have the net effect of the RMSE to become independent of the strength of confounding or confounder-identification strategy. This is also indeed the case of real data example with confounder (fish consumption) very poorly measured. On the other hand, when measurement error is moderate (i.e. not so strong as to push estimate of the effect of exposure nearly to the null and to make any adjustment inconsequential to its magnitude), RMSE tends to decrease with superior confounder-identification strategy, i.e. correctly specifying the outcome model has tangible benefits. Unsurprisingly, RMSE is smaller for weaker confounding where there are fewer penalties for failing to adjust for confounder. These influence of RMSE create a rotation in RMSE versus the strength of confounding curves such that it may appear that RMSE declines with the strength of confounding with the rise of measurement errors, whereas in fact all we witness is the tendency for RMSE to become independent of measurement begin to dominate the tendency for RMSE to be larger when confounding is stronger.

The RMSE has to be interpreted with cautions, as it is a combination of squared bias and standard error of the causal effect estimate. In logistic regression (n = 2,000) with the largest examined measurement errors, 10 % CIE criteria leads to the smallest RMSE for strong confounding but it is essential to note that this is based almost exclusively on unadjusted estimates because confounder was almost never included in the final model (Fig. 3). We hypothesized that this phenomenon is due to the fact that with a poorly-measured exposure we expect the estimated causal effect would be zero (i.e., an RR of 1). The low RMSE in this case is the result of tight clustering of unadjusted attenuated estimates around wrong value of the effect estimate, a phenomenon that was previously described in theoretical work [14] that leads to over-confidence in wrong estimates in presence of measurement error. Another tendency that is acting on the observed RMSE is due to the fact that when exposure-confounder correlation increases, the standard errors of the maximum likelihood estimate of adjusted causal effect also increases, leading to the increase of RMSE.

We present two types of simulated CIE criteria: designed to control either type I or II errors. When the cutoff values of the two criteria are similar, we are in the fortunate situation where both types of error are controlled to the desired degree. In the synthetic examples that we evaluated, there appears to be little difference in RMSE for the two simulation-based CIE approaches. The two approaches diverged in success in confounder identification per se, but both outperformed significance testing and fixed cutoff of 10 % CIE.

Depression-mercury-fish consumption example

The real data analysis of the NHANES data illustrated how researchers can make use of our framework to choose the most appropriate measurement tools that maintain a balance between the error of causal effect estimation and the total cost induced. By applying our framework, information about the degree of accuracy, validity, or measurement error one needs to achieve in order to obtain a less biased estimate of the causal relationship. For instance, our simulation result informs us that a noise-to-signal ratio of 0.5 or smaller for the variable fish consumption is desirable when we wish to estimate the causal relationship between total blood mercury and depression. In planning such an analysis, in the planning stage researchers should avoid using measurement tools that are only weakly correlated with the true nutrient intake, for example food-frequency questionnaire [10].

The real data analysis is founded on the assumption that here is no reason for exposure to mercury to be protective of risk of depression and therefore cofounding by fish consumption, known to be protective, is suspected. Of course one can assume that there is a small positive effect of mercury that is reversed by stronger negative confounding by fish consumption. One can repeat our simulations in such a way that mercury would have a causal risk factor, e.g. by weakening correlation of mercury with fish consumption or assuming weaker beneficial effect of fish consumption on the outcome. We did not explore this possibility in order to limit future considerations of plausible continuation of work in this setting as discussed in [18].

Limitations

A major limitation of our study is that we considered a limited number of scenarios. If the assumptions of our simulations were violated, for example if the model is mis-specified, or if the errors do not follow normal distribution, the conclusions will be altered. Ideally, readers interested in implementing our approach should consider the validity of these assumptions and implement necessary modification to our R code. Another limitation is that the associations between exposure, confounder, and outcome maybe unknown. We believe it is possible (and desirable) to conduct analyses while acknowledging partial knowledge about causal structures, measurement error and exposure misclassification (e.g. using Bayesian framework [14, 22]): this may prove to be a promising extension to our work.

Another limitation is that we did not evaluate our framework under multiple confounders but this can be done in practice by slight modification of our R code. On the other hand, conceptually, one confounder may stand for multiple confounders that do not cancel each other out (e.g. see [8]), so our approach with one confounder should retain some generalizability.

One can argue that when measurement error is present, suitable analytical approach, for example regression calibration [23, 24] or simulation and extrapolation (SIMEX [25]), should be applied to remove its influence from inference before engaging in the discussion of suitable confounder identification strategy. This is certainly a sensible approach and the one that, to the extent possible, should be advocated. However, we wish to point out that measurement error correction is not routinely practiced by epidemiologists [26] and until such time that this changes, it is still relevant to consider how the historically and currently acceptable analytical strategies for model-building perform in practice.

Ethics approval and consent to participate

NHANES 2009–2010 was approved by The National Center for Health Statistics Research Ethics Review Board; informed consent was obtained from all participants at the time of data collection and further consent for specific analyses of this publically available data is not needed.

Availability of data and materials

The NHANES 2009–2010 dataset can be downloaded at http://wwwn.cdc.gov/Nchs/Nhanes/Search/nhanes09_10.aspx.