Can longitudinal generalized estimating equation models distinguish network influence and homophily? An agent-based modeling approach to measurement characteristics

Past literature on GEE-based approaches to network influence

The empirical work on the network spread of obesity achieved high visibility in both the scientific and lay press. The work has been subjected to a variety of critiques, none convincing to the entire community. For example, Cohen-Cole and Fletcher analyzed data from Add Health to examine the extent of network influence in rates of self-reported acne and headaches [10]. They found that adding school-level fixed effects attenuated the apparent network influence effects towards zero, and removed their statistical significance at conventional levels. In a further analysis, Cohen-Cole and Fletcher demonstrated that failing to account for the auto-correlation in one‘s own height over time can also lead to spurious inference of network influence. However, as these applications hinge on re-analysis of a defined empirically collected data set—in which the true data generating process is, of necessity, unknown—it remains a question as to how relevant to empirical analyses these critiques are. It might also be argued that auto-correlation, collinearity, and shared environment are not unique to networks, but are instead ubiquitous features of longitudinal modeling in social science. In a special section of the Journal of Health Economics, several authors extended these concerns in explorations of Add Health, with different combinations of waves, fixed effects, and parameterizations of the dependent variable—leading to the suggestion that the replicability of the Framingham data in Add Health data might be specification-dependent [2, 11, 12]. More recently, similar results have been obtained from stochastic actor-oriented models, implemented in SIENA [13].

Two additional papers have issued wide-ranging critiques of the GEE model, both making very strong claims on conceptual grounds. Shalizi and Thomas argue that “latent homophily and contagion are generically confounded with each other … and any direct contagion effects cannot be nonparametrically identified from observational data,” and that strong parametric assumptions are needed unless quite strong assumptions can be made about the structure of the data generating process on substantive grounds [14]. Lyons has severely critiqued nearly every aspect of the Christakis & Fowler paper, arguing that the basic modeling assumptions of the GEE suggest that it cannot be used in the network context [15]. Yet VanderWeele et al. have demonstrated that in fact, Lyons’ concerns of model inconsistency and statistical dependence can be solved when testing for contagion [16].

The goal of the present manuscript is not to directly intervene in these theoretical arguments, which have spilled over to the popular press [17]. Rather, we note that there also exists a large body of research suggesting that even when strict regression assumptions are violated, the analysis may still be useful [18, 19]—a crucial question from this perspective is the extent to which the GEE model’s interpretation is sensitive to these violations.

Moving in that direction, Noel and Nyhan examined the robustness of the GEE analyses in a different approach, using simulation-based models [20]. They examined the extent to which the breaking of network ties could lead to bias in apparent network influence. Their examination showed that if more homophilous ties are likely to be more enduring, then this could lead to spurious inference of network influence. This important work highlighted the potential value of a simulation-based approach for establishing the measurement characteristics of the GEE-based analysis of networks, but did not address the fundamental extent to which the GEE-based models could distinguish homophily from network influence. Therefore our primary objective is to determine to what extent, and under what conditions, does the GEE-approach recreate the actual dynamics in an agent-based model and to what extent can longitudinal GEE models distinguish network influence and homophily.

Analytic approach

Our approach to quantifying the GEE model performance is to consider the statistical regression as a test from the perspective of measurement theory. Any test has a sensitivity (the probability of a positive test result being obtained when, in fact, the positive condition holds) and a specificity (the probability of a negative test result being obtained when, in fact, the negative condition holds). Further, this data can be formulated as a likelihood ratio to allow Bayesian updating in light of a test result: given one’s prior odds about whether or not network influence (for example) was present, to what extent should a regression indicating the presence of network influence increase those odds? The positive likelihood ratio is calculated as (Sensitivity/(1—Specificity) ; the negative likelihood ratio is (1—Sensitivity)/Specificity. Posterior odds after having seen the regression result are the prior odds multiplied by the appropriate likelihood ratio. These are related to the familiar concepts of Type I and Type II error, but more explicitly oriented towards understanding the extent to which new empirical results should change one’s prior beliefs about the way the world worked. We further ask the secondary question: when network influence is present in the underlying model, are the GEE parameters sensitive to changes in the strength of that network influence?

It is important to note that the calculation of sensitivity and specificity depend on the dichotomization of the test statistic into a positive or negative. For the purposes of this analysis, we follow Christakis and Fowler and define affirmative evidence as a regression coefficient with a p-value of 0.05 or below. This implies that we might expect specificity of 0.95 under a well-performing regression—that 5% of runs where there is no effect, a regression will nonetheless be “statistically significant” at the p < 0.05 level.

We considered the performance of the GEE model in several cases. Our substantive interest was in network influence and homophily in adolescent obesity. As such, we generated populations of simulated cohorts where either, both, or neither network influence and homophily may occur. All simulated populations were gradually gaining weight, consistent with observed secular trends. We then determined the sensitivity, specificity and likelihood ratios for GEE regression results in these populations. We considered models that do and do not control for these secular trends. In addition to the results here, we have posted the ABM-generating code at an archive site, so that others wishing to ascertain the measurement characteristics of alternative empirical approaches can do so with ease—replicating or extending these results.

Generating populations for simulated cohort studies

Our goal with the ABM was to develop a flexible simulation code for simulating populations as if they were in a cohort study. Each cohort member is simulated using a separate agent. At the beginning of the simulation, each agent has a baseline weight (drawn from a uniform distribution tunable with respect to a set minimum (80lbs) and maximum (300lbs) weight) and an intrinsic rate of weight gain (drawn from a uniform distribution tunable with respect to minimum (0.0lbs) and maximum (2.0lbs) intrinsic gain per month). By “tunable”, we mean a value can be set by the analyst for each simulation run; values are fixed at the beginning of the simulation and for the entire cohort simulation process. In an extension of the model, we had patients draw from a normal distribution of weights with a mean of 190 and a standard deviation of 70, to test if our results were sensitive to the shapes of these parameters.

Each agent also has the capacity to designate a tunable number of friends, which can be fixed (we fix ours at 1). Consistent with observed behavior of adolescents, friend nominations need not be reciprocated. In simulations in which homophily is not present, the selection of friends is done without respect to characteristics of the friend, by choosing at random from other members of the population of agents. In simulations in which homophily is to be present, first the absolute difference between the agent (ego) and all other agents (alters) is calculated. In the base case, each alters’ probability of being chosen as a friend is proportional to the reciprocal of the weight difference; the extent to which weight difference is important to the choice of friends is tunable for each run of the model. In extensions of the model, we include preferential attachment in the choice of friends, using the algorithm described by Newman [21]. Preferential attachment is tested with and without homophily on the basis of weight. In these cases, the probability of being selected as a friend is greater for nodes that already have more friends, in addition to any homophily effects.

We consider static simulations, in which the friendship networks are formed in the first period and do not change thereafter. (Such a situation applies when the rate of change of network ties is slow relative to the time-span under observation in the study, not merely to truly “static” networks for the entire life of the network.) We also consider dynamic networks, in which friendship networks change at tunable intervals (we set ours to every 30 time-steps).

In simulations in which network influence is present, each agent’s actual weight gain is a tunably weighted average of their own intrinsic weight gain rate and the difference between their current weight and the weight of their friends. Weight gain can be negative, if a given agent is friends with much lighter agents.

Having established the basis for each agent, a population of agents is then created and given initial values. At each simulation time-step, the agents’ weights change, including any network influences as specified by the parameters. In dynamic network models, friend nominations are made only after all agents’ weights have been updated. Agents are activated in a random order each time-step, but all are activated once and only once per round. Agents do not enter or exit the model during a run. After a user-defined number of time-steps (120 for us), a network data set is output, enumerating each agent, their current weight, and their current outgoing friend nominations at each time-step.

Initial values for parameters in the present case were set to model a potential study of weight gain in the U.S. Thus for the base case, we set each cohort size to 30; set a minimum intrinsic weight gain of 0.0 lbs and a maximum of 2.0 lbs pounds per time-step (“month”); each agent had one friend; and simulated cohort data collection were output for statistical analysis for each time-step (simulated month). In extensions, we replicated with cohort sizes of 1000 to test the extent to which the GEE’s performance varied across a range of feasible study sizes.

While we have explained this model in terms of weight and friendship, these models are not constructed so as to closely mimic physiology or some other application-specific characteristic. The agents, in fact, simply have one continuous-valued attribute with an intrinsic rate of growth of that attribute (which may be mean zero), and a propensity to develop relationships with other agents that may be based on that attribute.

This initial ABM model intentionally did not feature several additional complications that might be present in the real world—it was designed to examine the baseline performance of the GEE approach. Real world data would include missing data, random variation in weight gain and measurement of weight, heterogenous and variable numbers of friends, and other such complications in the data generating process. Our framework is readily extensible to such conditions, but for clarity they were not included in this first examination.

Statistical analysis using GEE

We simulated the collection of cohort data by examining the characteristics of agents and their network structure as a subset of time-steps—in our case, analyzing data from time-steps 24, 48, 72, 96, and 120 (as if biennial data collection). Since each analysis required lagged values, data from time-step 24 was used only to produce those lagged values.

These terms were interpreted as follows, following Christakis and Fowler [1, 2]. The coefficient on Alter Weight _T was interpreted as the evidence of network influence. Alter Weight _T-1 was interpreted as the homophily parameter. Ego Weight _T-1 was interpreted as controlling for genetic endowments and the past history of the ego.

An exchangeable error structure was used in the GEE model to adjust the standard errors for the non-independence of Ego observations. Christakis and Fowler reported that, in general, their results were not sensitive to the particular form of the error structure that was specified [1]. In the original article on the network spread of obesity, weight was incorporated as a dichotomous variable for obese or not. In our analyses we used continuous variables.

We automated the process of analysis using Stata 10 [22]. Each data set was opened, GEE regressions were run, and coefficients were stored in a summary dataset for further analysis. As is conventional, we considered p < 0.05 statistically significant, and focused on the presence or absence of statistically significant findings rather than on the magnitude of the coefficients. Unless otherwise specified, all regressions controlled for the “survey wave” using a vector of indicator variables.

Availability for replication and extension

The following items are available, permanently archived at the Dryad Digital Repository (http://dx.doi.org/10.5061/dryad.v3s0k): the ABM generating code; each of the populations of simulated cohorts analyzed in this manuscript; and the Stata code used to implement the GEE model.