 Research article
 Open Access
 Published:
Comparing multiple statistical software for multipleindicator, multiplecause modeling: an application of gender disparity in adult cognitive functioning using MIDUS II dataset
BMC Medical Research Methodology volume 20, Article number: 275 (2020)
Abstract
Background
The multipleindicator, multiplecause model (MIMIC) incorporates covariates of interest in the factor analysis. It is a special case of structural equation modeling (SEM), which is modeled under latent variable framework. The MIMIC model provides rigorous results and becomes broadly available in multiple statistical software. The current study introduces the MIMIC model and how it can be implemented using statistical software packages SAS CALIS procedure, R lavaan package, and Mplus version 8.0.
Methods
In this paper, we first discussed the formulation of the MIMIC model with regard to model specification and identification. We then demonstrated the empirical application of the MIMIC model with the Midlife in the United States II (MIDUS II) Study (N = 4109) using SAS CALIS procedure, R lavaan package and Mplus version 8.0 to examine gender disparities in cognitive functioning. The input, output, and diagram syntaxes of the three statistical software packages were also presented.
Results
In terms of data structure, all three statistical programs can be conducted using both raw data and empirical covariance matrix. SAS and R are comprehensive statistical analytic packages and encompass numerous data manipulation capacities. Mplus is designed primarily for latent variable modeling and has far more modeling flexibility compared to SAS and R, but limited in data manipulation. Differences in model results from the three statistical programs are trivial. Overall, the results show that while men show better performance in executive function than women, women demonstrate better episodic memory than men.
Conclusions
Our study demonstrates the utility of the MIMIC model in its empirical application, fitted with three popular statistical software packages. Results from our models align with empirical findings from previous research. We provide coding procedures and examples with detailed explanations in the hopes of providing a concise tutorial for researchers and methodologists interested in incorporating latent constructs with multiple indicators and multiple covariates in their research projects. Future researchers are encouraged to adopt this flexible and rigorous modeling approach.
Background
This paper illustrates how to implement multipleindicator, multiplecause (MIMIC) modeling, a special case of structural equation model (SEM), under the latent variable modeling (LVM) framework using three statistical software packages: SAS CALIS procedure, Mplus, and R lavaan package. SAS is widely used in health sciences and Mplus is commonly used in social sciences for LVM. R has become popular in recent years because it is open source and free. In addition, all three packages generate path (or structural or causal) diagrams to help interpreting the output. This paper consists of two sections: 1) introduction of the MIMIC model, and 2) illustration of fitting the MIMIC model to cognitive function theory using the MIDUS II dataset [1].
Two types of variables are generally encountered in research: observed and unobserved. Observed variables are also referred to as manifest variables. For example, gender, age, responses to questions in surveys, and ranked observations by raters are examples of manifest variables. Unobserved variables can be regarded as latent constructs; examples include anxiety, quality of life, or sickness. To understand latent constructs, researchers rely on observed or measured variables. Therefore, the measured variables are also called measured indicators. For example, because of its multifaceted domains, the quality of life of an individual cannot be directly observed. Several indicators could describe quality of life. As a latent construct, it can be measured in different domains and by researchers’ values and perspectives on this latent concept. Measured aspects can include, but are not limited to, “the number of days in a week that one feels stressed”, “the number of days in a month one needs to worry about money”, or “the number of weeks this year one has to take care of parents.” Therefore, when it comes to deciding on indicators for measuring a latent construct, it is important that researchers have theoretical background knowledge to narrow down the range of perspectives and to focus on the definition of the construct and its use in a tobetested model. Researchers’ content knowledge is also essential in the model modification step, which will be discussed below.
SEM combines both measurement and structural considerations. It integrates psychometric concepts (i.e., measurement approaches) and the econometric ideas (structure approaches). The aforementioned examples in which the latent construct (i.e., health, sickness, quality of life, anxiety) is measured by indicators is regarded as the measurement approach to SEM. Measurement errors of indicators are taken into consideration. As for the structure approaches in SEM, path analysis is applied to estimate the relationships among latent constructs. The ability to combine these two analyses is one of the advantages of SEM. By specifying and describing the plausible relationships between latent concepts and manifest variables, associated measurement errors and proposed structural relationships among latent structures in SEM can effectively estimate parameters simultaneously, which mirror the fact that the variables coexist in reality [2].
Another advantage of SEM is that the measurement model in the latent variable approach takes into account potential measurement errors of the indicators. Traditional multiple regression analysis assumes that the independent variables included in the model are errorfree. However, if this assumption is not tenable, it will result in biased estimates of the regression coefficient and incorrect conclusions. SEM incorporates measurement errors during construction of the latent variables while simultaneously estimating the relationships among those latent variables, making this approach powerful and flexible.
Due to its broad capabilities and application in a diversity of fields, the emerging popularity of SEM has led to the development of statistical software packages for analysis based on SEM. Currently, available software packages include Mplus, STATA, LISREL, EQS, AMOS, lavaan package in R, and SAS CALIS procedure. Among these software, SAS is commonly used in biostatistics area and pharmaceutical companies; R is a free statistical computing language known for its rich packages; Mplus is especially designed for running latent variable models and is commonly used in social science field. Therefore, this paper focuses on using SAS CALIS procedure, R lavaan package, and Mplus. We begin with a brief introduction to the fundamental concepts of SEM, followed by a special case of SEM, the MIMIC model. The MIMIC model is popular in epidemiology on how to contextualize latent variables of interest. In this paper, we use a dataset of 4109 participants in MIDUS II to demonstrate how the MIMIC model can be used to examine gender disparity in cognitive functions via the three statistical software programs. Their input codes for generating the path diagram are also provided in this paper.
Since the purpose of this paper is to provide a tutorial of modeling procedure in fitting the MIMIC model with different software, answering the empirical questions in the development of the cognitive theory and gender disparity in cognitive function are not the focus of this paper. The measurement part of the MIMIC model in our empirical example is based on the factor analysis in Lachman, Agrigoroaei, Tun, & Weaver [3]. More detailed discussions on cognitive functioning and the psychometric properties of the instruments used in MIDUS II can be found in Lachman et al’s study [3]. Details of the technique of SEM (e.g., model estimation and model evaluation) in general, and MIMIC in particular, can be found in Bollen & Long [4]; Jöreskog & Goldberger [5]; O’Rourke & Hatcher [6]; and Wang & Wang [7].
Methods
In this section, we discussed the formulation of the MIMIC model with regard to model specification, model identification, and model fit. We started from introducing the components in onefactor MIMIC model and extended it to multiplefactor MIMIC model. Detailed assumptions of the MIMIC model and how to calculate the number of free parameters to construct a justidentified or overidentified model were elaborated. In the results section, we demonstrated the application of the twofactor MIMIC model with a real dataset from the Midlife in the United States II (MIDUS II) Study (N = 4109) using SAS CALIS procedure, R lavaan package and Mplus version 8.0 to examine gender disparities in cognitive functioning. The input, output, and diagram syntaxes of the three statistical software programs were also presented respectively. The parameter estimates results were compared, the interpretation, and the detailed reference of each software package were provided for practitioners.
MIMIC model
Model specification
The Multiple Indicators, Multiple Causes Model (MIMIC) is an extension of confirmatory factor analysis (CFA) with covariates. This model is commonly used to contextualize the latent variables of interest (e.g., quality of life, motor ability) using the demographic variables (e.g., age, gender), and it is assumed that those demographic variables are measured without error. By incorporating these covariates in CFA under the latent variable framework, the relationships between the demographic variables and the latent variables of interest are simultaneously estimated with the factor loadings in the measurement model. It is known for its advantage in simultaneous estimation of parameters. In addition, including direct paths from the demographic variables to the indicators of the latent variable would allow for examining differential item functioning effects for each indicator [8]. The modeling strategy provides an alternative way in the validation process of the examination when psychometric properties of the instrument/scale are the research interest, in which items with gender disparities or racial disparities can be identified [9,10,11]. The demographic variables are the ‘cause’ variables in the model. However, since there is no implication of causal effect in the model, it is also called exogenous manifest variables (in the field of econometrics), or predictors (in the field of psychology), to avoid confusion and to differentiate it from the indicators in the measurement model. Note that any type of covariates (i.e., continuous or categorical) can be included as the exogenous manifest variables in a MIMIC model.
Path diagrams are commonly used in LVM to depict the measurement and structural equations with a pictorial representation. In the diagram, we use squares to represent observed variables and circles to represent latent or unobserved variables. Arrows are used to indicate directed relationships. An arrow from X to Y represents a linear relationship in which Y is the dependent variable and X is the independent variable. The diagrams map out the relationships among constructs and covariates, facilitating the discussion and therefore, is one of the key features of the latent variable modeling.
A general linear structural relation (LISREL) model [12] consists of a set of linear structural equations with two parts: the measurement model and the structural model. The measurement model specifies how the unobserved latent constructs were measured by the indicators, and the structural model specifies the relationships between the latent variables. Since it allows for estimating measurement error variances for the measurement model and the disturbance variancecovariance matrix for the structural part, as well as the unknown coefficient among the structural relations [12], LISREL models gains great interest and attention from interdisciplinary fields. Latent variable modeling was later used interchangeably with LISREL to emphasize that the general LISREL model can also be used in modeling nonlinear relationships between unobserved and observed variables.
MIMIC model is a special case of SEM. For the sake of simplicity in this study, the model is first illustrated by a onefactor linear MIMIC model scenario (See Fig. 1). The measurement part is a general LISREL model for a p × 1 vector Y of endogenous variables followed by the structural part that incorporates the influence of exogenous variables, denoted by the q × 1 vector X, where p indicates the number of measured indicators, and q indicates the number of predictors. The single latent factor ties the two parts,
Y is also called measured indicators, and X referred to as predictors (i.e., exogenous manifest variables or cause variables). The p × 1 vector Λ and the q × 1 vector Γ are regression coefficients, or path coefficients. ϵ is the residual or measurement errors, and ζ is the disturbances. In other words, ζ is the error term for the regression of η on the covariates (X). The reduced form of (1) is expressed as
where Π = ΛΓ′ and ω = Λζ + ϵ is the composite error.
The standard assumptions of the MIMIC model are: (1) ϵ is uncorrelated with, and (2) ϵ, ζ are uncorrelated with X. These assumptions are sufficient to derive the covariance matrix of Y. From (2), \( \mathit{\operatorname{cov}}\left(\boldsymbol{Y}\right)=\mathbf{\prod}{\boldsymbol{\Theta}}_{\boldsymbol{X}}{\mathbf{\prod}}^{\prime }+\lambda {\lambda}^{\prime }{\sigma}_{\zeta}^2+{\boldsymbol{\Theta}}_{\boldsymbol{\epsilon}} \), where Θ_{X} = Cov(X), Θ_{ϵ} = Cov(ϵ), \( {\sigma}_{\zeta}^2= Var\left(\zeta \right) \). For model identification, Θ_{ϵ} is diagonal. The onefactor model has 2p + q + 1 parameters. Note that model specification does not include a mean structure.
A multiplefactor MIMIC model has m latent variables. Replace (1) by.
Let m be the number of latent variables, Λ is a p × m matrix of regression coefficients of Y on η, Γ is a m × q matrix of regression coefficients of η on X. The m × m matrix B are the regression coefficients of variables in η on other variables in η. The diagonal elements of B are zero, but (I − B) is assumed invertible.
The reduced form of (3) is Y = Λη + ϵ = Λ[I − B]^{−1}ΓX + Λ[I − B]^{−1}ζ + ϵ from which we get an expression for the covariance of Y, and the covariance between Y, X under the assumptions: (1) ϵ is uncorrelated with ζ and (2) ϵ, ζ are uncorrelated with X.
where Ψ = Cov(ζ). The input data set for estimation of parameters in (Λ,Γ,B) and in (Ψ, Θ_{ϵ}) is an empirical correlation or covariance matrix S for the manifest variables Y, X. A standard estimation method is maximum likelihood, which optimizes the distance between S and the modelinduced variancecovariance matrix: Σ. There are \( \frac{1}{2}\left(p+q\right)\left(p+q+1\right) \) data elements in Σ. The model has variancecovariance parameters in Σ, while have q × m regression coefficients. An example of a twofactor model is illustrated in the next section.
Model identification
To solve the scale indeterminacy issue in SEM, researchers can either set \( {\sigma}_{\zeta}^2=1 \) or one of the λ's equal to a certain value. Here MIMIC model is applied as an example.
Suppose the disturbance variable (ζ) is standardized. Distinct elements of parameters in the variancecovariance matrix of q exogenous variables (X) and p measured indicators (Y): \( \frac{\left(p+q\right)\times \left(p+q+1\right)}{2} \). In MIMIC model, the tobeestimated parameters include: 1) p factor loadings (λ_{1}…λ_{p})), 2) p nonzero residual variances \( {\sigma}_{\epsilon_1}^2 \), …, \( {\sigma}_{\epsilon_p}^2 \), 3) q × m regression coefficients (γ), where m is the number of latent constructs, 4) m latent construct residual variances (or the disturbance variable, ζ), which equals zero to solve for scale indeterminacy, 5) \( \frac{m\left(m1\right)}{2} \) covariance (Ψ) between residual variances (ζ) of the latent constructs, and 6) \( \frac{q\left(q+1\right)}{2} \) variances and covariances among exogenous variables (or predictors). Based on the counting rule, if the number of tobe estimated parameters (i.e., the number of free parameters) is equal to or less than the number of nonredundant elements in population covariance matrix Σ, then the model is called just identified or overidentified, respectively. Otherwise, the model can be not identified [13]. Based on the notations above, the one factor model can be extended to a multiplefactor model. In the following section, an example of a twofactor MIMIC model is illustrated.
Model fit
Model fit indicates the degree to which a model can reproduce the data. Though a goodfitting model is what a researcher pursues for, it is worth noting that a goodfitting model does not guarantee sensible and reasonable parameter estimates or a correctly specified model [14]. In other words, nonsensical results or poor validity evidence can be found from a goodfitting model. A reasonable model should consist of reasonable parameter estimates and goodfitting model fit. Even for a goodfitting model, model modification can still improve the model [14].
Latent variable modeling researchers originally used χ^{2} test statistic to measure/quantify model fit; however, it is sensitive to large sample size. Methodologists developed numerous fit indices to adjust the χ^{2} test statistics with the information in the model, such as degrees of freedom, sample size, and/or the number of variables. Depending on the elements in the formula, fit indices in latent variable models can be categorized into three types [15]: 1) relative fit indices (also called incremental fit index): BentlerBonett Index (NFI) [16], Tucker Lewis Index (TLI), and Comparative Fit Index (also called Bentler Comparative Fit Index, CFI) [17]. 2) absolute fit indices: Root Mean Square Error of Approximation (RMSEA), Standardized Root Mean Square Residual (SRMR), and 3) parsimony fit indices: Parsimony GoodnessofFit Index (PGFI) and Akaike information Criterion (AIC).
Although each index was developed to rectify the problems of other indices, considerable controversy about which fit indices and what cutoff criteria to use is substantial. For reporting purposes, the common practice is that unless the proposed model is close to saturated model or the purpose is to compare models, one should report χ^{2} test statistics and choose one index from each type: relative fit and absolute fit indices, to measure the model fit.
Real data set
The theoretical framework of cognitive function theory and its psychometric features in this paper are adapted from Lachman et al. [3] We use a dataset of 4109 subjects from the MIDUS (Midlife in the United States) study wave II collected between 2004 and 2006 [1]. The data that support the findings of this study are available in the Interuniversity Consortium for Political and Social Research at https://doi.org/10.3886/ICPSR25281.v6, reference number 25281. These data were derived from the following resources available in the public domain: http://midus.wisc.edu/index.php. We applied the same cognitive function framework for a measurement model, but adding a structural model with exogenous subject variables, age and gender to influence two latent constructs, Executive Function and Episodic Memory. The MIDUS (BTACT) Battery consists of seven tasks. The first latent construct, Executive Function is measured by five tasks: 1) Stop and Go Switch Task (SGST), 2) 30 Seconds and Counting Task (NmCorr), 3) Number Series (NmSr), 4) Category Verbal Fluency (UniItemF), and 5) Backward Digit Span (DgtSpan). The second latent construct, Episodic Memory is measured by two tasks: 1) Delayed Word List Recall (UniItemD), and 2) Immediate Word List Recall (UniItemI). These seven manifest variables Y are the indicators in the measurement model. The descriptive statistics of variables in the dataset is in Table 1.
For the structural model for influences, two manifest variables (X) age and gender (=0 for men, =1 for women) are included as the predictors (i.e., the causes). Relationships between X and the two latent constructs, and their indicators Y in the MIMIC model are presented in Fig. 2.
Note: Paths indicated by singleheaded arrows, alongside path coefficients. Errors (ϵ_{1}, ϵ_{2}, ϵ_{3}, ϵ_{4}, ϵ_{5}, ϵ_{6}, ϵ_{7}) associated with manifest endogenous variables, and disturbances (ζ_{1}, ζ_{2}) associated with endogenous latent variables.
Results
Empirical example
Model construction
The measurement model, Y = Λη + ϵ, are the following seven equations (see equation (1)):
where η = (F_{1}, F_{2})^{′} = (Executive Function, Episodic Memory)′. The matrix Λ of path coefficients has seven parameters. The seven parameters in the variance matrix, assumed diagonal, of the error ϵ. The structural model, η = Bη + ΓX + ζ is explicitly
where B indicated the variance covariance matrix among two latent constructs. In this model, there are predictors involved and the two latent constructs are the outcomes in this model. Therefore, B = 0, paths are not allowed between the two latent constructs. There are four path coefficients in Γ, and the covariance matrix of the disturbance ζ has a one covariance parameter Ψ_{12}. Because the disturbance variance terms ζ are given a fixed value of 1 for identification, Ψ_{12} is also the correlation between two latent constructs. The simplified specification of equation (4) is Cov(Y) = Λ(ΓΘ_{X}Γ^{′} + Ψ)Λ^{′} + Θ_{ϵ}, Cov(Y, X) = ΛΓΘ_{X}, Cov(X) = Θ_{X}. A total of 22 parameters in the specification (4) that must be estimated from the empirical covariance matrix S of (Y, X) that has 45 terms. The 22 tobeestimated parameters include seven factor loadings (λ), seven residual variances of the indicators (ϵ), four regression coefficients between two causes and two latent structures (γ), two latent construct residual variances (ζ), one covariance (Ψ) between residual variances (ζ_{1}, ζ_{2}) of the latent constructs, and one covariance between two predictors. Therefore, the degrees of freedom of the proposed model is 45–22 = 23.
For scale indeterminacy, either one of the factor loadings or the residual variance of the latent structures should be fixed at 1. The covariance Ψ_{12} between the two residuals ζ_{1} and ζ_{2} is used to demonstrate the relationship between two latent constructs. If one chooses to fix the residual variances, ζ_{1} and ζ_{2}, to 1, Ψ_{12} would be the correlation between the two constructs, like the example we demonstrate here.
In the following sections, we fit the proposed MIMIC model (shown in Fig. 2) to the MIDUS II dataset, which includes seven manifest variables (i.e., SGST, NmCor, NmSr, UnitItemF, DgtSpan, UnitItemD, and UnitItemI) and two predictors (or causes, i.e., age and gender). Explanations of the results as well as the input, output and diagram codes of SAS, R, and Mplus are provided. Applied researchers can choose the software packages they prefer and adapt the syntax to their own data set.
Software package comparison
SAS
In SAS, a MIMIC model can be fitted with the raw data set or the empirical covariance matrix S containing Cov(Y, X) using procedure CALIS. If one only has access to the S matrix, then the number of observation (NOBS option) needs to be specified after the name of the dataset in the syntax. In our example, if the data set is a covariance matrix, we specify “data = <<COVARIANCE MATRIX>>(type = cov) nobs = 4109” after PROC CALIS and before the modification option. The means of the variables are not needed. Both types of data input can produce the same results. Here, the SAS Input shows the syntax when raw data set is available.
SAS input (Fig. 3)
the statements PATH, PVAR and PCOV are included in the SAS syntax to specify the parameters to be estimated. If the parameters specified are not model parameters, SAS will estimate them, too. Factor loadings and regression coefficients are specified in the PATH statement with a onedirection arrow. For example, “F1 > SGST” indicates that F1 is measured by SGST. Since F1 is not in the variable list, SAS reads it as a latent construct, specified and named F1, and measured by SGST. This is also how the factor loading is specified in the PATH statement. “age > F1” means regressing age on F1, which is the regression coefficient of age on the first latent factor requested to be estimated. The terms after the equal signs are the userspecified names for the parameters, which are optional. The PVAR statement specifies the variance and residual variances terms to be estimated with supplied names or given their fixed value. An alternative way to request variance estimation is to specify doubleheaded paths, “<− −>” and the variable in the path statement, then the PVAR statement can be skipped. For example, the syntax “<> F1 1.0” in the PATH statement functions the same as “F1 = 1.0” in the PVAR statement, used to fix the residual variance of the F1 latent structure at 1. The syntax “<− > SGST” requests the residual variance estimate of SGST. The syntax “<− > age” requests the variance of the age variable. In SAS, all other variances will be estimated by default in this case the exogenous manifest variables age and gender.
The statement PCOV is used to specify the covariance terms and is the subsidiary model specification statement for the path model. For both PVAR and PCOV statements, the equal signs and the terms after the signs are the userspecified names for the parameters, which are optional. The PCOV statement, F1 F2 = covF1F2, names the covariance between F1 and F2 as “covF1F2”, and the parameter Ψ_{12} is to be estimated. Noted that the variance of age and gender requested in the PVAR statement and the covariance between age and gender requested in the PCOV statement are not model parameters. The three values in the output are directly from the covariance matrix S. Also noted, since two latent structures are used as the dependent variable in the structure model, when we specify the “F1 = 1.0” and “F2 = 1.0”, it is the residual variances of the latent constructs, \( {\sigma}_{\zeta_1}^2 \) and \( {\sigma}_{\zeta_2}^2 \) fixed at 1, rather than the variance of the latent constructs themselves. By default, PROC CALIS will estimate the following free parameters if not given a fixed value: error variances of all manifest or latent variables, and variances and covariance of all exogenous variables, manifest or latent. Note that although the variance and covariance of exogenous manifest variables are parameters to be estimated, they are not model parameters. The values showed in the result are directly from the sample variancecovariate matrix [13].
A PATHDIAGRAM statement can be used to request a diagram plot, and the parameter estimates can be shown in either standardized or unstandardized units. In factor analysis, standardized factor loadings are correlations. For example, \( \rho \left( SGST,{F}_1\right)=\frac{\lambda_{11} Var\left({F}_1\right)}{\sqrt{Var\left({F}_1\right) Var(SGST)}}=\lambda \sqrt{\frac{Var\left({F}_1\right)}{Var(SGST)}} \). But in MIMIC model, the standardized factor loadings are partial correlations between the latent structure and the indicators controlling for the exogenous variables in the model. The variances are derived from Cov(Y) = Λ(ΓΘ_{X}Γ^{′} + Ψ)Λ^{′} + Θ_{ϵ}, Cov(η) = ΓΘ_{X}Γ^{′} + Ψ.
Similarly, standardized variances and covariances of exogenous variables (i.e. X) are also correlations derived from Θ_{X}. However, standardized variances and covariances among errors ϵ and disturbances ζ are calculated as \( {\theta}_{ij}^{\ast }=\frac{\theta_{ij}}{\sqrt{\sigma_{ii}^2{\sigma}_{jj}^2}} \) for two terms (i, j) where (θ_{ij}) is the covariance and \( {\sigma}_{ii}^2 \) is the variance of the outcome corresponding to term i. The standardized variance is \( {\theta}_{ii}^{\ast }=\frac{\theta_{ii}}{\sigma_{ii}^2} \). Hence, \( {\theta}_{ij}^{\ast } \) is not a correlation, and \( {\theta}_{ii}^{\ast } \) does not necessarily equal 1. By default, \( {\sigma}_{ii}^2 \) called the total variance is reported for Y and η.
In our example SAS syntax, the diagram with unstandardized estimate was specified. One can replace it with “diagram = [initial unstandard standard]” to generate three diagrams available: the diagram with initial framework, the diagram with unstandardized estimate, and the diagram with standardized estimate. Noted that the additional option “exogcov” has to be specified right after the type of the diagram to have the covariance of exogenous variables shown on the diagram. Also, when the exogcov is specified, only one type of diagram can be generated. For simplicity, we only showed the diagram with unstandardized estimate for the demonstration purpose across three statistical software packages. Also, we only requested four commonlyseen indices in the plot for the purpose of simplicity. By default, SAS outputs seven absolute indices, nine parsimony indices, and six incremental indices (Fig. 4).
Fit statistics
Our model is estimated by maximum likelihood (ML). A likelihood ratio statistic comparing the fitted model (with 22 parameters) to the unconstrained saturated model with 45 parameters produces has a χ^{2} statistic of 509.1 with df = 23. The χ^{2} test is significant suggesting poor fit. However, this is often the case with large sample size. ML estimation is based of minimizing the objective function OBF = trace(SΣ^{−1}) − (p + q) = log (Σ) − log (S). The χ^{2} statistic is (N − 1) times the minimized objective function. Hence when the sample size N is large, the χ^{2} test is likely to be significant. Standardized root mean square residual (SRMR), Bentler Comparative Fit Index (CFI) are used to assess model fit. In this example, SRMR = .047 and CFI = .952, which are near the recommended cutoffs of < .08 and > .95, indicating that the model is a goodfitting model. Comparative indices, such as AIC and BIC, are used to compare competing models, that is, with different covariance structures. Since there is no competitive model in our example, AIC is not used in this case.
Mplus
Mplus is designed specifically for latent variable modeling, which is commonly used in social science and psychology to examine latent variable framework. Mplus will save a .dgm file every time it runs. Researchers can click the Diagram tab on the tool bar, and then choose ‘Open Diagrammer’ to open the Mplus Diagram module. The latent constructs, estimates, and indicators in the diagram can be repositioned by clicking and dragging (Fig. 5).
Mplus input consists of four sections: title, data, model, and output. The detailed options and choices for Mplus can be seen in its user guide [18]. Both raw data and covariance matrix data can be the input data. The model is specified in the model section. The ‘by’ statement is used to specify measurement model (see equation (1)). The term specified before the by statement is the name of the latent construct (η), and the variables specified after the by statement are the indicators (Y) that were used to measure the latent construct. The sign “@” is used to fix the parameter estimation. The statement “EpiMem @ 1” means we would like to fix the residual of the latent construct EpiMem at 1. Since EpiMem and ExeFun are the latent variables named by the researchers, the statements of fixing the residual variances at 1 have to be specified after the measurement model; otherwise, Mplus would not be able to know ExeFun and EpiMem are the latent constructs. The last command in the model section “ExeFun EpiMem on Age Gender” is to specify the structure model (see equation (2)), indicating the regression of latent constructs ExeFun and EpiMem on Age and Gender, or predicting ExeFun and EpiMem from gender and age (Fig. 6).
Mplus output provides five sections (see Supporting Information): 1) model specification, 2) univariate sample statistics, 3) model fit information, 4) model results, and 5) model modification indices. The first and second sections are provided for the researchers to verify that the model they specified in the input command and the data they read in are correct. Mplus is mainly designed for running latent variable models and has limited data manipulation functionality. Researchers are advised to clean the data before using the software. More details about Mplus, syntax and examples can be found on the website: statmodel.com.
R
R packages lavaan and semPlot are used in this paper. The former is used to run structural equation modeling, and the latter one is for generating the diagram. The format of the R output is very similar to Mplus output (Fig. 7).
In the model results (output is shown in the Supporting Information), two extra columns of the parameter estimates are printed at the end of the table. The Std.lv column reported the estimates when the latent variables “EpiMem” (Episodic Memory) and “ExeFun” (Executive Function) were standardized. The last column Std.all reported the parameter estimates when both the latent variables and the observed variables were standardized (also called the ‘completely standardized solution’).
The following code can be used to generate the diagram for the proposed model (Fig. 8).
The function semPaths is used to plot the SEM diagram. The solid arrow lines in the diagram are the 19 model parameters. The remaining three parameters shown as dashed arrow lines are 1) the residual variances of the latent constructs, which are fixed at 1 for scale indeterminancy, and 2) the covariance between age and gender that is given by the covariance matrix from the data set. Because the variance and covariance of the predictors are not model parameters and can be obtained from the covariance matrix, they were shown as dashed arrow lines in the diagram in R, for researchers’ information. Also, since the residual variances of the latent constructs are fixed at 1, they are shown as dashed arrows as well (Fig. 9).
Parameter estimates comparison
The results in Table 2 show the statistically significant model parameter estimates. Estimates from three statistical software packages are almost exactly the same. The differences are very trivial. The t test statistics of the parameter estimates across three packages are identical. For simplicity, only tvalue of SAS results are shown in Table 2. In this example, we provided evidence that age and gender can contextualize the cognitive functioning performance. Specifically, age and gender are significantly associated with each latent construct of cognitive functioning performance. Men have better scores on executive function compared to women, while women have better scores than men on episodic memory. The correlation between executive function and episodic memory of 0.44 is significant, but the covariance between age and gender is not (so is not included in the table).
Discussion
This paper aimed to provide a tutorial guideline to conducting the MIMIC model using SAS CALIS procedure, Mplus, and R lavaan package. We first provide the introduction of the model under LVM framework, then demonstrated the input commands for conducting the MIMIC model, generated diagrams with the three commonly used statistical software packages and illustrated the results and diagrams of the model.
The method section of this paper elaborated how the MIMIC model is specified and identified. We can see that a MIMIC model is a combination of path analysis and factor analysis, and that from a different perspective it can be seen as path analysis with latentconstructed outcome variables where the measurement errors are considered. The model has the following features. First, it is a onestep solution incorporating multiple indicators (the measurement model of SEM) and multiple causes (the structural model of SEM), while at the same time managing the inflated type I error rate that may arise from multiple testing. Second, the model can be utilized as a psychometric evaluation technique for differential item functioning (DIF) [8,9,10], measurement invariance [19], multiplegroup analysis [10, 11], and multidimensional measures [20]. Third, the factor scores extracted from the MIMIC model are the conditioned factor scores controlled for the demographic variables.
The MIMIC model is also referred to as factor analysis with covariates. Similarly, as a special case of SEM and also under the LVM framework, when the latent variables in the model are categorical, it is called a latent class analysis (LCA) with covariates, or latent class model with onestep procedure. Like the covariates in the MIMIC model, covariates in LCA have the same role of contextualizing the latent variables. Chang [21] extended the LCA model in multilevel contexts (e.g., students in after school programs) and used predictors at the student level and program level to contextualize the latent constructs. Similarly, future studies could extend MIMIC model to multilevel contexts or longitudinal scenarios. The transition of a subject on the latent factor (e.g., quality of life) at different time points can be modelled before and after an intervention. Meanwhile, the characteristics of groups of subjects with growth trajectories can be identified by incorporating their covariates in the model. In biostatistics, the path analysis part of MIMIC models can be extended to allow the indicators to be latent variables [22].
Conclusions
In this paper, we provided input code of three statistical software packages: SAS, Mplus, and R. Interpretation of the output and diagrams were also provided. We examined the effect of age and gender on cognitive function using 4109 participants in MIDUS II dataset. The results found that there is a significant gender disparity in cognitive functioning controlling for their age. Males tend to have better scores on executive functioning compared to females, while females have better scores on episodic memory compared to males. This result replicated Lechman et al’s study by fitting the data in the MIMIC model. The MIMIC model introduced in this paper incorporated the covariates of interest in the factor analysis, making the statistical modeling more approachable, the fitting procedure easier, and the results more rigorous. Since results were identical across three statistical software packages, application researchers can focus on constructing the MIMIC model of interest and the theoretical framework without concerning which software package should be used.
For simplicity, we used standard maximum likelihood estimation for the MIMIC model in this study since the sample size is large and the variables are continuous. All three statistical programs introduced in this study, the CALIS procedure in SAS, Mplus, and R lavaan package, have numerous estimation options available, such as maximum likelihood with robust standard error. Application researchers can choose an appropriate estimation method according to the type of the variables and dataset. More details about the performance of different estimation methods in latent variable models can be found in Li’s study [23].
The MIMIC model has been commonly utilized in applied research [24,25,26] but is mainly confined to the disciplines of mental health, social science, and education previously. In this paper, we provided syntax in three commonly used statistical software packages, explanation of the MIMIC model, and an empirical application to a real data set. We hope this paper can serve as a tutorial of MIMIC model and help facilitate the process of rigorous research for applied researchers in a diversity of fields.
Availability of data and materials
The datasets analyzed during the current study are available in the Interuniversity Consortium for Political and Social Research repository, https://www.icpsr.umich.edu/icpsrweb/ICPSR/studies/25281
Ryff CD, Lachman ME. Midlife in the United States (MIDUS 2): Cognitive Project, 2004–2006. 2017. doi:https://doi.org/10.3886/ICPSR25281.v6
Abbreviations
 SEM:

Structural Equation Modeling
 LVM:

Latent Variable Modeling
 MIMIC model :

multipleindicator, multiplecause model
 MIDUS II:

Midlife in the United States II
 SGST:

Stop and Go Switch Task (SGST)
 NmCorr:

30 Seconds and Counting Task
 NmSr:

Number Series
 UniItemF:

Category Verbal Fluency
 DgtSpan:

Backward Digit Span
 UniItemD:

Delayed Word List Recall
 UniItemI:

Immediate Word List Recall
 CFA:

confirmatory factor analysis
References
Ryff CD, Lachman ME. Midlife in the United States (MIDUS 2): cognitive project, 2004–2006; 2017. https://doi.org/10.3886/ICPSR25281.v6.
Todd D. Little. The Oxford handbook of quantitative methods. Oxford Libr Psychol. 2013;2: Statist:551. doi:https://doi.org/10.1017/CBO9781107415324.004.
Lachman ME, Agrigoroaei S, Tun PA, Weaver SL. Monitoring cognitive functioning: psychometric properties of the brief test of adult cognition by telephone. Assessment. 2014;21(4):404–17. https://doi.org/10.1177/1073191113508807.
Bollen KA, Long JS. Testing Structural Equation Models. Newbury Park: Sage Publications; 1993.
Jöreskog KG, Goldberger AS. Estimation of a model with multiple indicators and multiple causes of a single latent variable. J Am Stat Assoc. 1975;70(351):631–639.
O’Rourke N, Hatcher L. A StepbyStepApproach to Using SAS for Factor Analysis and Structural Equation Modeling. 2nd Ed. Cary, NC: SAS Institute; 2013.
Wang J, Wang X. Structural equation modeling: applications using Mplus. Hoboken: Wiley; 2012.
Finch H. The MIMIC model as a method for detecting DIF. 2005;29(4):278–295. doi:https://doi.org/10.1177/0146621605275728.
Woods CM, Grimm KJ. Testing for nonuniform differential item functioning with multiple indicator multiple cause models. Appl Psychol Meas. 2011;35(5):339–61. https://doi.org/10.1177/0146621611405984.
Woods CM. Evaluation of MIMICmodel methods for DIF testing with comparison to twogroup analysis. Multivariate Behav Res. 2009;44(1):1–27. https://doi.org/10.1080/00273170802620121.
Raykov T, Marcoulides GA, Lee CL, Chang C. Studying differential item functioning via latent variable modeling: a note on a multipletesting procedure. Educ Psychol Meas. 2013;73(5):898–908. https://doi.org/10.1177/0013164413478165.
Jöreskog KG, van Thillo M. LISREL: A general computer program for estimating a linear structural equation system involving multiple indicators of unmeasured variables. Princeton: Educational Testing Servicem; 1972.
Kaplan D. Structural equation modeling. Sage Publications, Inc; 2000.
Kenny DA. Measuring Model Fit. http://davidakenny.net/cm/fit.htm. Published 2020.
Hooper D, Coughlan J, Mullen MR. Structural equation modelling: Guidelines for determining model fit. Electron J Bus Res Methods. 2008;6(1):53–60. https://doi.org/10.21427/D79B73.
Bentler PM, Bonett DG. Significance tests and goodness of fit in the analysis of covariance structures. Psychol Bull. 1980;88(3):588–606. https://doi.org/10.1037/00332909.88.3.588.
Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107:238–46.
Muthén LK, Muthén BO. Mplus User’s Guide. 8th Ed. Los Angeles: Muthén & Muthén; 2018.
Masyn KE. Measurement invariance and differential item functioning in latent class analysis with stepwise multiple indicator multiple cause modeling. Struct Equ Model. 2017;24(2):180–97. https://doi.org/10.1080/10705511.2016.1254049.
Lee S, Bulut O, Suh Y. Multidimensional extension of multiple indicators multiple causes models to detect DIF. Educ Psychol Meas. 2017;77(4):545–69. https://doi.org/10.1177/0013164416651116.
Chang C. Nonparametric multilevel latent class analysis with covariates: an approach to classification in multilevel contexts [dissertation]. East Lansing: Michigan State University; 2016.
Tekwe CD, Zoh RS, Bazer FW, Wu G, Carroll RJ. Functional multiple indicators, multiple causes measurement error models. Biometrics. 2018;74(1):127–34. https://doi.org/10.1111/biom.12706.
Li CH. Confirmatory factor analysis with ordinal data: comparing robust maximum likelihood and diagonally weighted least squares. Behav Res Methods. 2016;48(3):936–49. https://doi.org/10.3758/s1342801506197.
Guan M. Measuring the effects of socioeconomic factors on mental health among migrants in urban China: a multiple indicators multiple causes model. Int J Ment Health Syst 2017;11(1):1–12. doi:https://doi.org/10.1186/s130330160118y.
Proitsi P, Hamilton G, Tsolaki M, et al. A multiple indicators multiple causes (MIMIC) model of behavioural and psychological symptoms in dementia (BPSD). Neurobiol Aging. 2011;32(3):434–42. https://doi.org/10.1016/j.neurobiolaging.2009.03.005.
Brailean A, Guerra M, Chua KC, Prince M, Prina MA. A multiple indicators multiple causes model of latelife depression in Latin American countries. J Affect Disord. 2015;184:129–36. https://doi.org/10.1016/j.jad.2015.05.053.
Acknowledgements
The authors thank Harlan McCaffery for the writing assistance. We wish to thank the editor and reviewers for their comments and suggestions for improving the paper.
Funding
The authors received no specific funding for this work.
Author information
Authors and Affiliations
Contributions
CC and JG conceived the study. JG and RH provided critical input regarding the model identification and model estimation. YLY provided critical input regarding theoretical framework of the empirical study and clinical and sociological aspects of the dataset. CC performed all the statistical analyses. CC wrote the first draft. JG revised it and wrote the second draft. All authors read, critically assessed, and approved the final version.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The data used in this study was secondary data; therefore, IRB review is exempted.
Consent for publication
Not Applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Chang, C., Gardiner, J., Houang, R. et al. Comparing multiple statistical software for multipleindicator, multiplecause modeling: an application of gender disparity in adult cognitive functioning using MIDUS II dataset. BMC Med Res Methodol 20, 275 (2020). https://doi.org/10.1186/s12874020011504
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12874020011504
Keywords
 MIMIC model
 MIDUS II
 Statistical software package comparison
 Cognitive functioning performance
 Latent variable framework
 Structural equation model
 R
 Mplus
 SAS