Analytic approach
Our approach to quantifying the GEE model performance is to consider the statistical regression as a test from the perspective of measurement theory. Any test has a sensitivity (the probability of a positive test result being obtained when, in fact, the positive condition holds) and a specificity (the probability of a negative test result being obtained when, in fact, the negative condition holds). Further, this data can be formulated as a likelihood ratio to allow Bayesian updating in light of a test result: given one’s prior odds about whether or not network influence (for example) was present, to what extent should a regression indicating the presence of network influence increase those odds? The positive likelihood ratio is calculated as (Sensitivity/(1—Specificity) ; the negative likelihood ratio is (1—Sensitivity)/Specificity. Posterior odds after having seen the regression result are the prior odds multiplied by the appropriate likelihood ratio. These are related to the familiar concepts of Type I and Type II error, but more explicitly oriented towards understanding the extent to which new empirical results should change one’s prior beliefs about the way the world worked. We further ask the secondary question: when network influence is present in the underlying model, are the GEE parameters sensitive to changes in the strength of that network influence?
It is important to note that the calculation of sensitivity and specificity depend on the dichotomization of the test statistic into a positive or negative. For the purposes of this analysis, we follow Christakis and Fowler and define affirmative evidence as a regression coefficient with a p-value of 0.05 or below. This implies that we might expect specificity of 0.95 under a well-performing regression—that 5% of runs where there is no effect, a regression will nonetheless be “statistically significant” at the p < 0.05 level.
We considered the performance of the GEE model in several cases. Our substantive interest was in network influence and homophily in adolescent obesity. As such, we generated populations of simulated cohorts where either, both, or neither network influence and homophily may occur. All simulated populations were gradually gaining weight, consistent with observed secular trends. We then determined the sensitivity, specificity and likelihood ratios for GEE regression results in these populations. We considered models that do and do not control for these secular trends. In addition to the results here, we have posted the ABM-generating code at an archive site, so that others wishing to ascertain the measurement characteristics of alternative empirical approaches can do so with ease—replicating or extending these results.
Generating populations for simulated cohort studies
Our goal with the ABM was to develop a flexible simulation code for simulating populations as if they were in a cohort study. Each cohort member is simulated using a separate agent. At the beginning of the simulation, each agent has a baseline weight (drawn from a uniform distribution tunable with respect to a set minimum (80lbs) and maximum (300lbs) weight) and an intrinsic rate of weight gain (drawn from a uniform distribution tunable with respect to minimum (0.0lbs) and maximum (2.0lbs) intrinsic gain per month). By “tunable”, we mean a value can be set by the analyst for each simulation run; values are fixed at the beginning of the simulation and for the entire cohort simulation process. In an extension of the model, we had patients draw from a normal distribution of weights with a mean of 190 and a standard deviation of 70, to test if our results were sensitive to the shapes of these parameters.
Each agent also has the capacity to designate a tunable number of friends, which can be fixed (we fix ours at 1). Consistent with observed behavior of adolescents, friend nominations need not be reciprocated. In simulations in which homophily is not present, the selection of friends is done without respect to characteristics of the friend, by choosing at random from other members of the population of agents. In simulations in which homophily is to be present, first the absolute difference between the agent (ego) and all other agents (alters) is calculated. In the base case, each alters’ probability of being chosen as a friend is proportional to the reciprocal of the weight difference; the extent to which weight difference is important to the choice of friends is tunable for each run of the model. In extensions of the model, we include preferential attachment in the choice of friends, using the algorithm described by Newman [21]. Preferential attachment is tested with and without homophily on the basis of weight. In these cases, the probability of being selected as a friend is greater for nodes that already have more friends, in addition to any homophily effects.
We consider static simulations, in which the friendship networks are formed in the first period and do not change thereafter. (Such a situation applies when the rate of change of network ties is slow relative to the time-span under observation in the study, not merely to truly “static” networks for the entire life of the network.) We also consider dynamic networks, in which friendship networks change at tunable intervals (we set ours to every 30 time-steps).
In simulations in which network influence is present, each agent’s actual weight gain is a tunably weighted average of their own intrinsic weight gain rate and the difference between their current weight and the weight of their friends. Weight gain can be negative, if a given agent is friends with much lighter agents.
Having established the basis for each agent, a population of agents is then created and given initial values. At each simulation time-step, the agents’ weights change, including any network influences as specified by the parameters. In dynamic network models, friend nominations are made only after all agents’ weights have been updated. Agents are activated in a random order each time-step, but all are activated once and only once per round. Agents do not enter or exit the model during a run. After a user-defined number of time-steps (120 for us), a network data set is output, enumerating each agent, their current weight, and their current outgoing friend nominations at each time-step.
Initial values for parameters in the present case were set to model a potential study of weight gain in the U.S. Thus for the base case, we set each cohort size to 30; set a minimum intrinsic weight gain of 0.0 lbs and a maximum of 2.0 lbs pounds per time-step (“month”); each agent had one friend; and simulated cohort data collection were output for statistical analysis for each time-step (simulated month). In extensions, we replicated with cohort sizes of 1000 to test the extent to which the GEE’s performance varied across a range of feasible study sizes.
While we have explained this model in terms of weight and friendship, these models are not constructed so as to closely mimic physiology or some other application-specific characteristic. The agents, in fact, simply have one continuous-valued attribute with an intrinsic rate of growth of that attribute (which may be mean zero), and a propensity to develop relationships with other agents that may be based on that attribute.
This initial ABM model intentionally did not feature several additional complications that might be present in the real world—it was designed to examine the baseline performance of the GEE approach. Real world data would include missing data, random variation in weight gain and measurement of weight, heterogenous and variable numbers of friends, and other such complications in the data generating process. Our framework is readily extensible to such conditions, but for clarity they were not included in this first examination.
Statistical analysis using GEE
We simulated the collection of cohort data by examining the characteristics of agents and their network structure as a subset of time-steps—in our case, analyzing data from time-steps 24, 48, 72, 96, and 120 (as if biennial data collection). Since each analysis required lagged values, data from time-step 24 was used only to produce those lagged values.
The basic analytic framework estimated a dyadic-level GEE. The unit of analysis was an ego-alter pair for each wave of the survey, with the current and lagged weights of the ego and alter. Egos could appear multiple times in each survey wave, once with each alter to whom he or she was paired. Ties that had dissolved no longer contributed data. The following basic form was used for the estimation: (with subscripts indicating contemporaneous and lagged variable measurement, and regression coefficients suppressed for clarity).
$$ \mathrm{Ego}\ \mathrm{Weigh}{\mathrm{t}}_{\mathrm{T}} = \mathrm{Alter}\ \mathrm{Weigh}{\mathrm{t}}_{\mathrm{T}} + \mathrm{Ego}\ \mathrm{Weigh}{\mathrm{t}}_{\mathrm{T}\hbox{-} 1} + \mathrm{Alter}\ \mathrm{Weigh}{\mathrm{t}}_{\mathrm{T}\hbox{-} 1}+\upxi $$
These terms were interpreted as follows, following Christakis and Fowler [1, 2]. The coefficient on Alter Weight T was interpreted as the evidence of network influence. Alter Weight T-1 was interpreted as the homophily parameter. Ego Weight T-1 was interpreted as controlling for genetic endowments and the past history of the ego.
An exchangeable error structure was used in the GEE model to adjust the standard errors for the non-independence of Ego observations. Christakis and Fowler reported that, in general, their results were not sensitive to the particular form of the error structure that was specified [1]. In the original article on the network spread of obesity, weight was incorporated as a dichotomous variable for obese or not. In our analyses we used continuous variables.
We automated the process of analysis using Stata 10 [22]. Each data set was opened, GEE regressions were run, and coefficients were stored in a summary dataset for further analysis. As is conventional, we considered p < 0.05 statistically significant, and focused on the presence or absence of statistically significant findings rather than on the magnitude of the coefficients. Unless otherwise specified, all regressions controlled for the “survey wave” using a vector of indicator variables.
Availability for replication and extension
The following items are available, permanently archived at the Dryad Digital Repository (http://dx.doi.org/10.5061/dryad.v3s0k): the ABM generating code; each of the populations of simulated cohorts analyzed in this manuscript; and the Stata code used to implement the GEE model.