 Research
 Open access
 Published:
Detecting influential subjects in intensive longitudinal data using mixedeffects location scale models
BMC Medical Research Methodology volume 23, Article number: 237 (2023)
Abstract
Background
Collection of intensive longitudinal health outcomes allows joint modeling of their mean (location) and variability (scale). Focusing on the location of the outcome, measures to detect influential subjects in longitudinal data using standard mixedeffects regression models (MRMs) have been widely discussed. However, no existing approach enables the detection of subjects that heavily influence the scale of the outcome.
Methods
We propose applying mixedeffects location scale (MELS) modeling combined with commonly used influence measures such as Cook’s distance and DFBETAS to fill this gap. In this paper, we provide a framework for researchers to follow when trying to detect influential subjects for both the scale and location of the outcome. The framework allows detailed examination of each subject’s influence on model fit as well as point estimates and precision of coefficients in different components of a MELS model.
Results
We simulated two common scenarios in longitudinal healthcare studies and found that influence measures in our framework successfully capture influential subjects over 99% of the time. We also reanalyzed data from a health behavior study and found 4 particularly influential subjects, among which two cannot be detected by influence analyses via regular MRMs.
Conclusion
The proposed framework can help researchers detect influential subject(s) that will be otherwise overlooked by influential analysis using regular MRMs and analyze all data in one model despite influential subjects.
Background
In the past decades, intraindividual variability, also called withinsubject (WS) heterogeneity or level1 heterogeneity, of health behaviors and conditions has received increased attention [1,2,3]. Accordingly, new developments in statistical modeling, including mixedeffects location scale (MELS) models proposed by Hedeker et al. [4] and Nordgren et al. [5], allow researchers to model the mean, or location, and variability, or (square of the) scale of responses simultaneously.
Although these models can accommodate betweensubject (BS) heterogeneity, also called level2 heterogeneity, through random subject effects, there often exist subjects who are so different from the others that they need additional analyses and/or may affect model estimation. For example, in ecological momentary assessments (EMA) studies, a typical type of careless responses is that subjects give many consecutive items the same answer [6]. In this case, even if the data from these subjects do not deviate from any pattern in mean responses that subjects correctly responding to questions may show, they exhibit exceptionally low variability. Such abnormal behavior can distort the parameter estimates, especially the estimated variance of random scale effects. On the other hand, subjects who display exceptional consistency during behavioral studies, such as a study that aims at increasing subjects’ physical activities, might be of interest as such a pattern may indicate good adherence to the study protocol. Being able to identify such subjects is important in terms of informing personalized healthcare. Especially, there have been discussions about using mixedeffects regression models (MRMs) to find individuals who behave differently from others regarding their health conditions and outcomes [7, 8]. Since MELS models are an extension of standard MRMs, they can also perform similar functions and offer additional information about WS variability.
While there has been extensive literature discussing influential subjects and observations in MRMs [7, 9, 10] and software developed to implement these methods [11], to the best of our knowledge, there lack similar methods for the scale model of the response. As we will show in later sections, not considering WS variability in influence analysis can leave subject(s) with abnormal WS variability undetected. Hence, in the topic of influence analysis, WS variability deserves equal attention as the mean of the outcomes.
Therefore, we propose a framework for detecting influential subjects using MELS models. We examine each subject’s influence on model fit as well as point estimates of parameters and their efficiency. The proposed method enables researchers to identify subjects who exhibit dubious patterns in their WS variability and/or mean, thus facilitating research on intraindividual variability. Unlike the traditional casedeletion approach of influential data detection, we adopt the method proposed by Langford and Lewis [9]. If a subject is being examined for its influence, it is removed from the estimation of level2 effects and given subjectspecific fixed effects. We will demonstrate how the estimation of the leaveoneout models can be carried out in SAS and R. Also, a health behavior study will be used as an example to illustrate the proposed methodology. Finally, we will assess the performance of our method and demonstrate its benefits over influence analysis using MRMs via simulated examples.
Methods
Leaveoneout MELS model
In this section, we first briefly review MELS models developed by Hedeker et al. [4]. For simplicity, only random intercepts with scalar variances are included in both the location and the scale models described below. Nevertheless, more complicated models that include random slopes of timevarying covariates and/or have covariates influence the variances of the random effects are possible.
Suppose that subject i (\(i = 1, 2, \dots , N\)) is measured at visit j (\(j = 1, 2, \dots , n_i\)). The model for the response \(y_{ij}\) can be expressed as:
where \(x_{ij}\) is a \(p\times 1\) vector of timevarying covariates influencing the mean of \(y_{ij}\), and \(\nu _i\) is subject i’s random intercept, indicating subject i’s deviation from the fixed part of the model. \(\beta _0\) is the fixed intercept, i.e. average response when all covariates equal 0, and \(\beta '\) is a \(p \times 1\) vector of coefficients corresponding to \(x_{ij}\). \(\epsilon _{ij}\) is the level1 residual and follows a normal distribution with a mean of 0 and a variance of \(\sigma ^2_{\epsilon _{ij}}\). Later on, the notation \(\beta\) in influence measures represents a vector of which the first entry is \(\beta _0\) and the remaining entries come from \(\beta '\).
In the MELS model, a loglinear submodel is applied to the WS variance of the response to ensure a positive value:
where \(w_{ij}\) is an \(r \times 1\) vector of timevarying covariates influencing the scale of \(y_{ij}\), and \(\omega _i\) is the \(i^{th}\) subject’s deviation from the average log WS variance of \(y_{ij}\). \(\tau _0\) is the average log WS variance when every covariate in \(w_{ij}\) is 0, and \(\tau '\) is an \(r \times 1\) vector of coefficients of \(w_{ij}\). Likewise, \(\tau\) in influence measures refers to a vector of which the first entry is \(\tau _0\), and the remaining entries come from \(\tau '\).
The random location and scale effects can correlate with each other, and they are assumed to follow a bivariate normal distribution:
Given our focus on influence analysis at the subject level, also known as level 2 in longitudinal models, it’s important to pause here and clarify that level2 effects in MELS models comprise fixed effects of timeinvariant covariates (including the fixed intercepts) and all random effects. This distinction lays the foundation for our forthcoming exploration of influence analysis using MELS models. The following paragraphs present how the leaveoneout MELS models are formed, i.e., how each subject is separated from the random effects and subjectlevel fixed effects and given subjectspecific fixed effects, for subsequent influence analyses.
As suggested by Langford and Lewis [9], in order to separate the subject under evaluation, denoted as \(i^*\) \((i^* \in 1, 2, \dots , N\)), from level 2 of the location model, we exclude subject \(i^*\) from the estimation of the random effects and level2 fixed effects. In this case, the only level2 fixed effect is \(\beta _0\). Then, subject \(i^*\) is given a subjectspecific fixed effect denoted as \(c_{i^*}\). Namely, the location model becomes
where \(\mathbbm {1}(i = i^*)\) equals 1 for subject \(i^*\) and 0 for all other subjects, and \(\mathbbm {1}(i \ne i^*)\) equals 0 for subject \(i^*\) and 1 for all other subjects.
Note that for simplicity in illustrating leaveoneout MELS models, we assume that every covariate in \(x_{ij}\) and \(w_{ij}\) is timevarying. When some timeinvariant covariates are also present, subject \(i^*\) should be separated from the estimation of their associated coefficients as well.
To also separate subject \(i^*\) from level 2 of the scale model described in Eq. 2, the leaveoneout scale model is as follows:
Here, \(d_{i^*}\) is the \(i^{*th}\) subject’s subjectspecific fixed scale effect, and the variances of random location effects and random scale effects become \(\sigma ^2_{\nu (i^*)}\) and \(\sigma ^2_{\omega (i^*)}\) while their covariance is \(\sigma _{\nu \omega (i^*)}\) without subject \(i^*\). Note that the subscript \((i^*)\) here as well as in Eqs. 4 and 5 does not mean that subject \(i^*\) is entirely removed from modeling but instead only removed from the estimation of level2 effects.
Overall, the algorithm to estimate a leaveoneout MELS model includes the following steps: (1) create an indicator variable that equals 1 for subject \(i^*\) and 0 otherwise; (2) use the opposite of the indicator variable in (1) as the covariate for the random intercepts; (3) following step (2), subject \(i^*\) will not be included in the random effect estimation, and the fixed location effect for the indicator variable would be \(c_{i^*}\) while the fixed scale effect is \(d_{i^*}\). Our programming specifics and pertinent material will be explained in Section “Results”.
A separate leaveoneout model is estimated for every subject and compared with the model in which all subjects are treated the same. Besides allowing the detection of highly influential subjects, this leaveoneout structure can be viewed as a way to keep all subjects in the model despite influential subject(s). Using all available data to estimate the level1 fixed effects has the benefit of increasing statistical power.
Influence analysis
Influence on model fit
Given that the model described in Eqs. 1 and 2 are nested in the leaveoneout model described in Eqs. 4 and 5, likelihoodratio test was used to examine subject \(i^*\)’s influence on the model fit. The test statistic, difference in deviance, denoted as \(LR_{i^*}\) for subject \(i^*\), can be calculated as follows:
where \(\mathcal {L}_{(i^*)}\) represents the likelihood of the MELS model in which subject \(i^*\) is separated, and \(\mathcal {L}\) represents the likelihood of the naive MELS model. Since N tests are conducted simultaneously, the false discovery rate (FDR) procedure [12] is applied to adjust for multiple comparisons. This specific multiple testing correction method is chosen because of its ability to control for both Type I and Type II errors [13].
Influence on parameter estimates
Cook’s distance [14] is a wellknown measure for the influence of data on the point estimates of a group of parameter estimates. Based on the structure of MELS models, we calculate the Cook’s distances for three groups of parameters, namely, fixed location effects (\(\beta\)), fixed scale effects (\(\tau\)), and variances and covariance of random effects, denoted as \(\eta\) (\(\eta = [\sigma ^2_{\nu }, \sigma ^2_{\omega }, \sigma _{\nu \omega }]\)). Following the formula of Cook’s distance for multilevel models described by Snijder and Bosker [15], Cook’s distance can be calculated as
where \(\gamma\) can be \(\beta\), \(\tau\), or \(\eta\). Here, \(r_{\gamma }\) is the number of parameters being examined, and \(\Sigma _{\hat{\gamma }(i^*)}\) is the variancecovariance matrix of \(\hat{\gamma }\) after subject \(i^*\) is separated from the random effect estimation. A large Cook’s distance indicates a heavy influence on a specific group of parameter estimates.
Once a subject is determined to be influential on a particular group of parameter estimates, it is often of interest to investigate this subject’s influence on each specific parameter estimate within this group. In this case, DFBETAS can be used. DFBETAS is the difference between the estimate of a parameter obtained when all subjects are kept in the random effect and level2 effect estimation and the parameter estimate when a subject is separated, divided by the standard error of the parameter estimate [16]. Its mathematical representation is as follows:
where \(\theta\) can be any single parameter in the model. DFBETAS can have both positive and negative values. A highly positive value indicates that the inclusion of the influential subject creates a positive bias for \(\hat{\theta }\), and vice versa.
Influence on the precision of parameter estimates
Influential subjects can also affect the variances of parameter estimates besides their point estimates. COVTRACE proposed by Christensen et al. [10] and COVRATIO described by Belsley et al. [17] are often used to measure such changes.
\(\text {COVTRACE}^{\gamma }_{i^*}\) is calculated as the absolute value of the difference between the trace of the inverse ratio of variancecovariance matrix estimates with and without subject \(i^*\) in level2 model and the number of parameters under investigation:
The larger the COVTRACE value, the greater the influence of subject \(i^*\) on the precision of \(\hat{\gamma }\).
Meanwhile, \(\text {COVRATIO}^{\gamma }_{i^*}\) is the inverse ratio of the determinant of the estimated variancecovariance matrix of \(\gamma\) with and without subject \(i^*\) in level2 model:
Again, \(\gamma\) is one of \(\beta\), \(\tau\), and \(\eta\). The determinant of a variancecovariance matrix is also known as the generalized variance, a measure of multidimensional scatter [18]. The examples in the next two sections will focus on COVRATIO because it provides information on both the magnitude and the direction of the influence. A value of COVRATIO above 1 indicates a loss in precision when separating subject \(i^*\), and on the contrary, a value below 1 indicates a gain in precision.
All the procedures of influence analysis described in Section “Influence analysis” are summarized in Table 1.
Results
Application to health behavior study example
Data collected by Flueckiger et al. [19] for a study on the association between health behaviors and learning goal achievements are used as an example to illustrate the proposed method.
During the 32day span of the study, 72 students answered questions about their sleep quality (SQ), physical activity, positive and negative affect, learning goal achievement (LGA), and examination grades. The following analysis focuses on complete observations with values for all variables, so our analysis includes 1788 observations from 62 subjects. One of the major findings from the original study is that better SQ is positively associated with LGA. Hence, we chose SQ as the covariate for both the mean and the scale models. Namely, the location model for the example can be expressed as
and the scale model, in which the variance of \(\epsilon _{ij}\) is being modeled, is
where SQ was measured on a 4point Likert scale in which 1 means very bad, and 4 means very good. LGA was measured on a 5point Likert scale in which 0 represents having not achieved the goals at all and 4 represents having achieved the goals completely. We treated LGA and SQ as continuous, consistent with the approach taken in the original study. The approximate normal distribution of LGA has been validated through our exploratory analysis. Nevertheless, it is important for the readers to exercise caution when generalizing the results beyond the original LGA range.
The Cook’s distances and COVRATIOs of all subjects are visualized in Fig. 1. After applying the FDR procedure, subjects 7, 12, 49, and 69 are determined to have a statistically significant influence on the model fit based on likelihoodratio tests. Influence analysis results of these four subjects are summarized in Table 2. According to the Cook’s distances, we can see that subjects 7 and 12 have high influence on the fixed scale effects, subject 69 has high influence on the fixed location effects, while subject 49 has high influence on both. In particular, subject 7 has a large DFBETAS for \(\tau _{\text {SQ}}\), subject 49 has the largest DFBETAS for \(\beta _0\) and the smallest DFBETAS for \(\tau _0\), and subject 69 has the smallest DFBETAS for \(\beta _0\) and the \(2^{nd}\) largest DFBETAS for \(\beta _{\text {SQ}}\). All four subjects have high influence on the point estimates of the variances and covariance of the random effects. Specifically, separation of subjects 7, 12, and 49 shrinks \(\hat{\sigma }^2_{\omega }\); separation of subjects 49 and 69 shrinks \(\hat{\sigma }^2_{\nu }\); separation of subject 7 and 69 decreases \(\hat{\sigma }_{\nu \omega }\) while separation of subject 12 and 49 increases \(\hat{\sigma }_{\nu \omega }\). Moreover, excluding each of these subjects from the random effect estimation also shrinks the corresponding generalized variances of the groups of parameters that they have influence on the point estimates, suggesting that these four subjects have caused a loss in model precision in the original model.
We visualize the SQ and LGA of these four subjects in Fig. 2 to see how the data correspond with the influence analysis findings. While all except two of subject 7’s selfreported SQ ratings have a value of 4, the LGA of this subject was highly variable when SQ equaled 4. This is consistent with the influence analysis result that giving subject 7 subjectspecific fixed effects shrinks \(\hat{\tau }_{\text {SQ}}\) and \(\hat{\sigma }^2_{\omega }\). While subject 49 had consistently high LGA, subject 69 had all low LGA values at 0 and 1. Such patterns in the data correspond to subject 49’s large \(\text {DFBETAS}_{\beta _0}\) and subject 69’s small \(\text {DFBETAS}_{\beta _0}\). Given subject 12, 49, and 69 have such great influence on different components of the model and the fact that they have the lowest variability in LGA across all subjects, it could be worthwhile to investigate whether the similar answers for LGA is a result of careless responses or simply outstanding consistency in actual LGA.
Sensitivity analyses were conducted by excluding the scale model, i.e., using standard MRMs. The sensitivity analysis results still acknowledge the strong influence of subjects 49 and 69 on the location model but do not reveal significant influence from subjects 7 and 12. These findings are consistent with results in Section “Simulation study”, that is, subjects with influential data in terms of scale can often be neglected if influence analyses are conducted via MRMs only.
PROC NLMIXED in SAS OnDemand for Academics (SAS Institute Inc.) was used to estimate both the original and the leaveoneout MELS models. All the influence analyses using the results from SAS were carried out in R version 4.2.0 (R Core Team). Both the SAS codes and the R codes we used are included in the supporting information.
Simulation study
To further illustrate the advantages of conducting influence analyses using MELS models compared to analyses using MRMs with no scale components, we generated simulated examples in two different scenarios, with 500 datasets created for each scenario. Every dataset contains 50 subjects that follow the structure described in Eqs. 1 and 2 before designating some subject(s) to be influential. Both scenarios’ responses resemble daily physical activity time in minutes, so negative values were removed. Examples of one regular subject and one artificially influential subject simulated in each scenario are illustrated in Fig. 3.
The proposed method requires fitting a MELS model, for which estimation can be timeconsuming, for every subject in the dataset. Hence, all the MELS models involved in the simulation study were estimated using a fast estimation algorithm for MELS models, FastRegLS, developed by Gill and Hedeker [20]. The supporting information contains codes that utilize FastRegLS to carry out the influence analysis. One or more of the models failed to converge properly for 1 to 18 datasets (mean = 7.67) in the six simulation studies, and these datasets are not included in the result summary. One drawback of the FastRegLS algorithm is that it doesn’t offer loglikelihoods. Hence, the simulation studies primarily concentrate on point estimates, along with the variances and covariances of parameter estimates.
Simulation scenario 1
In the first scenario, the timevarying covariate, portions of fiber intake, is continuous and was simulated based on a \(\mathcal {N}(3, 1)\) distribution, after which absolute values were taken to ensure nonnegativity. The values of the parameters used to generate the simulated datasets are as follows: \(\beta = [100, 10]\); \(\tau = [7, 0.5]\); \(\eta = [20.09, 1, 0.45]\). In other words, the model used to generate data except for the influential one(s) is as follows:
There are 500 observations from each subject, and the covariate is standardized for influence analysis to facilitate model estimation. This scenario focuses on the case of careless responses, so we first designated subject 1 to have all responses at 150 and 155 and all covariates at 3 and 4 before standardization. The percentage of simulations in which each influence measure detects subject 1 to be the most influential are summarized in the first column of Table 3. In summary, subject 1 always has the smallest \(\text {COVRATIO}^{\tau }\) and the largest \(\text {DFBETAS}^{\sigma ^2_{\omega }}\). It also almost always has the largest \(\text {DFBETAS}^{\tau _0}\). The second column in Table 3 shows the percentage of detection when we specify the first three subjects to have such careless responses, and the three subjects still almost always have the smallest \(\text {COVRATIO}^{\tau }\)’s and the largest \(\text {DFBETAS}^{\sigma ^2_{\omega }}\)’s. Note that only all three influential subjects among the top three are counted as one detection.
Simulation scenario 2
For scenario 2, the covariate values were simulated to represent discrete time periods with values that range from 0 to 7, and each subject had 25 observations from each time period before removing negative values. The simulated datasets were generated according to the following structure:
The influential subject(s) are subject(s) with exceptional consistency throughout the behavioral study and are simulated with an intercept of 0 in the scale model. In the simulation study with the first subject like this, the subject almost always has the largest \(C^{\tau }\), the smallest \(\text {COVRATIO}^{\tau }\), the largest \(\text {DFBETAS}^{\tau _0}\), and the largest \(\text {DFBETAS}^{\sigma ^2_{\omega }}\). The same results remain in terms of \(\text {COVRATIO}^{\tau }\) and \(\text {DFBETAS}^{\sigma ^2_{\omega }}\) when the number of artificial influential subjects increases to three.
As shown in the third column of Table 3, using standard MRMs, none of the influence measures recognize the influences of the designated influential subjects by more than half of the time in either scenario. Therefore, in order to successfully identify an influential subject in these scenarios, it is essential to take the scale model into consideration.
Discussion
In this paper, we have discussed procedures to detect subject(s) influential in model fit and parameter estimates in MELS models. This approach allows researchers to identify subjects influential on the scale structure of the outcome being modeled in addition to the location structure. We hope that our method is able to help researchers, especially researchers interested in studying intraindividual variability, better identify interesting or troublesome subjects that they want to study further in an EMA study or a largescale longitudinal clinical trial. After determining which subjects are considered influential, researchers can keep these subjects in the analysis using our leaveoneout MELS model described in Section “Influence analysis”, and a summary of other common ways of dealing with influential data can be found in a recent work by Aguinis et al. [21]. The proposed method can also benefit analyses not carried out in MELS models by providing researchers with a better understanding of both the location and the scale structures of their data during exploratory analyses.
Our study has focused on subjectlevel influence analyses, so one possible extension is the detection of influential observations in MELS model. Given that there could be influential observation(s) within a noninfluential subject, estimating a separate model for each observation might be required. Because of the enormous number of observations in intensive longitudinal data, such methods can be extremely computationally intensive. This article has also focused on maximum likelihood estimates of 2level MELS models for normally distributed continuous outcomes. Further development can extend the proposed framework to accommodate models with 3 or more levels [22, 23], models on outcomes with more complicated structures [24], and Bayesian estimation approaches [25]. Future work will also extend to ordinal MELS models [26].
To plan for further data analysis based on the influence analysis results, we recommend readers go through all the results from our framework and use their domain knowledge to decide whether specific subject(s) are considered influential and need further analysis. However, we understand that one might want cutoffs to guide their judgment. Ruleofthumb cutoff values are \(\frac{4}{N}\) for Cook’s distances, \(1\pm 3(\frac{r_{\gamma }}{N})\) for COVRATIO, and \(\frac{2}{\sqrt{N}}\) for DFBETAS [17]. Also, a possible future direction of this work is to improve existing cutoff values to be more suitable for MELS models.
The simulation studies have showcased that at least some of the influence measures are able to capture all influential subjects in the case that multiple of them coexist. Nevertheless, not all measures perform the same, which might be attributed to the masking effect [27]. Therefore, we again recommend readers carefully examine all influence measures mentioned in the framework, and a future step of this study will be improving individual influence measures to overcome any possible masking effect.
Conclusion
The proposed influence analysis framework using MELS models enables detection of influential subjects on the scale structure and/or location structure of intensive longitudinal data. Thus, it can facilitate modeling that accounts for the abnormality of certain subject(s). Such benefits of the proposed methods are revealed in both the reallife example and the simulated examples.
Availability of data and materials
The dataset used in the health behavior study example are publicly available in Harvard Dataverse, https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/27388. Code used to generate data in the simulated study is included in the Supporting Information.
Abbreviations
 MELS:

Mixedeffects location scale
 MRMs:

Mixedeffects regression models
 BS:

Betweensubject
 WS:

Withinsubject
 EMA:

Ecological momentary assessments
 FDR:

False discovery rate
 SQ:

Sleep quality
 LGA:

Learning goal achievement
References
TudorLocke C, Ham SA, Macera CA, Ainsworth BE, Kirtland KA, Reis JP, et al. Descriptive epidemiology of pedometerdetermined physical activity. Med Sci Sports Exerc. 2004;36(9):1567–73.
MacDonald SW, Nyberg L, Bäckman L. Intraindividual variability in behavior: links to brain structure, neurotransmission and neuronal activity. Trends Neurosci. 2006;29(8):474–80.
Yu L, Wang T, Wilson RS, Leurgans S, Schneider JA, Bennett DA, et al. Common agerelated neuropathologies and yearly variability in cognition. Ann Clin Transl Neurol. 2019;6(11):2140–9.
Hedeker D, Mermelstein RJ, Demirtas H. An application of a mixedeffects location scale model for analysis of ecological momentary assessment (EMA) data. Biometrics. 2008;64(2):627–34.
Nordgren R, Hedeker D, Dunton G, Yang CH. Extending the mixedeffects model to consider withinsubject variance for Ecological Momentary Assessment data. Stat Med. 2020;39(5):577–90.
Meade AW, Craig SB. Identifying careless responses in survey data. Psychol Methods. 2012;17(3):437.
Shi L, Chen G. Detection of outliers in multilevel models. J Stat Plan Infer. 2008;138(10):3189–99.
Sharifi M, Marshall G, Goldman R, RifasShiman SL, Horan CM, Koziol R, et al. Exploring innovative approaches and patientcentered outcomes from positive outliers in childhood obesity. Acad Pediatr. 2014;14(6):646–55.
Langford IH, Lewis T. Outliers in multilevel data. J R Stat Soc Ser A Stat Soc. 1998;161(2):121–60.
Christensen R, Pearson LM, Johnson W. Casedeletion diagnostics for mixed models. Technometrics. 1992;34(1):38–45.
Nieuwenhuis R, Te Grotenhuis H, Pelzer B. Influence. ME: tools for detecting influential data in mixed effects models. R J. 2012;4(2):38–47.
Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B Methodol. 1995;57(1):289–300.
Verhoeven KJ, Simonsen KL, McIntyre LM. Implementing false discovery rate control: increasing your power. Oikos. 2005;108(3):643–7.
Cook RD. Detection of influential observation in linear regression. Technometrics. 1977;19(1):15–8.
Snijders TA, Bosker RJ. Multilevel analysis: An introduction to basic and advanced multilevel modeling. Thousand Oaks: Sage; 2011.
Fox J, Monette G. An R and SPlus companion to applied regression. Thousand Oaks: Sage; 2002.
Belsley DA, Kuh E, Welsch RE. Regression diagnostics: Identifying influential data and sources of collinearity. New York: John Wiley & Sons; 2005.
Wilks SS. Certain generalizations in the analysis of variance. Biometrika. 1932;24(3/4):471–94.
Flueckiger L, Lieb R, Meyer AH, Mata J. How health behaviors relate to academic performance via affect: An intensive longitudinal study. PLoS ONE. 2014;9(10):e111080.
Gill N, Hedeker D. Fast estimation of mixedeffects locationscale regression models. Stat Med. 2023;42(9):1430–44.
Aguinis H, Gottfredson RK, Joo H. Bestpractice recommendations for defining, identifying, and handling outliers. Organ Res Methods. 2013;16(2):270–301.
Li X, Hedeker D. A threelevel mixedeffects location scale model with an application to ecological momentary assessment data. Stat Med. 2012;31(26):3192–210.
Lin X, Mermelstein RJ, Hedeker D. A 3level Bayesian mixed effects location scale model with an application to ecological momentary assessment data. Stat Med. 2018;37(13):2108–19.
Blozis SA, McTernan M, Harring JR, Zheng Q. Twopart mixedeffects location scale models. Behav Res Methods. 2020;52(5):1836–47.
Rast P, Hofer SM, Sparks C. Modeling individual differences in withinperson variation of negative and positive affect in a mixed effects location scale model using BUGS/JAGS. Multivar Behav Res. 2012;47(2):177–200.
Hedeker D, Demirtas H, Mermelstein RJ. A mixed ordinal location scale model for analysis of Ecological Momentary Assessment (EMA) data. Stat Interface. 2009;2(4):391.
Atkinson A. Masking unmasked. Biometrika. 1986;73(3):533–41.
Acknowledgements
An earlier version of the abstract of this manuscript has been previously presented and published at the 2023 International Conference on Health Policy Statistics and the 2023 Midwest Biopharmaceutical Statistics Workshop. This work utilized the computational resources of the Center for Research Informatics’ Gardner HPC cluster at the University of Chicago (http://cri.uchicago.edu).
Funding
This project was supported by National Institute of Diabetes and Digestive and Kidney Diseases (grant number R01 DK125414). The content of this study is solely the responsibility of the authors and does not represent the official views of National Institute of Diabetes and Digestive and Kidney Diseases.
Author information
Authors and Affiliations
Contributions
DH and XZ developed the influential analysis framework and contributed to writing the manuscript. XZ analyzed and interpreted the health behavior data and simulated data. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Zhang, X., Hedeker, D. Detecting influential subjects in intensive longitudinal data using mixedeffects location scale models. BMC Med Res Methodol 23, 237 (2023). https://doi.org/10.1186/s12874023020469
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12874023020469