This article has Open Peer Review reports available.
Simultaneous evaluation of abstinence and relapse using a Markov chain model in smokers enrolled in a two-year randomized trial
© Yeh et al; licensee BioMed Central Ltd. 2012
Received: 15 October 2011
Accepted: 11 June 2012
Published: 7 July 2012
GEE and mixed models are powerful tools to compare treatment effects in longitudinal smoking cessation trials. However, they are not capable of assessing the relapse (from abstinent back to smoking) simultaneously with cessation, which can be studied by transition models.
We apply a first-order Markov chain model to analyze the transition of smoking status measured every 6 months in a 2-year randomized smoking cessation trial, and to identify what factors are associated with the transition from smoking to abstinent and from abstinent to smoking. Missing values due to non-response are assumed non-ignorable and handled by the selection modeling approach.
Smokers receiving high-intensity disease management (HDM), of male gender, lower daily cigarette consumption, higher motivation and confidence to quit, and having serious attempts to quit were more likely to become abstinent (OR = 1.48, 1.66, 1.03, 1.15, 1.09 and 1.34, respectively) in the next 6 months. Among those who were abstinent, lower income and stronger nicotine dependence (OR = 1.72 for ≤ vs. > 40 K and OR = 1.75 for first cigarette ≤ vs. > 5 min) were more likely to have relapse in the next 6 months.
Markov chain models allow investigation of dynamic smoking-abstinence behavior and suggest that relapse is influenced by different factors than cessation. The knowledge of treatments and covariates in transitions in both directions may provide guidance for designing more effective interventions on smoking cessation and relapse prevention.
clinicaltrials.gov identifier: NCT00440115
Smoking studies typically hypothesize intervention effects at a particular time point and analysis is performed in a cross-sectional manner using “time-naïve” approaches, e.g. the Pearson chi-square test or logistic regression. These approaches only use the outcome measures at the given time points, and appear to be appropriate when the research question focuses on these time point, or for short term smoking interventions with brief follow-up when smoking cessation is viewed as an acute, unidirectional problem (either smokers quit or they don’t). However, longitudinal studies often involve comparisons at multiple time points, and concern whether the intervention effects vary across time, which is beyond these “time-naïve” methods. Moreover, smoking cessation is increasingly being recognized as a dynamic process where people quit, relapse, and quit again, often with repeated cycles over years. Applying ‘time-naive’ approaches in these circumstances ignores this dynamic natural history of smoking cessation.
To address these concerns, researchers have proposed to use generalized estimating equations (GEE) and generalized linear mixed-effects models (GLMM) . Both GEE and GLMM use repeated outcome measures and take the intra-personal association into account, and provide a means to comparing intervention effects at each time point and to examining whether the effects vary over time . GEE provides population-averaged estimates, and the covariate effects can be interpreted as in standard logistic regression models. In contrast to GEE, GLMM is a subject-specific model. It estimates the probability that an individual would be abstinent (or smoking) at a given time point, allowing different propensity of abstinence (the random effects) among individuals. The two approaches also differ in other aspects, including the robustness or sensitivity to the assumptions of correlation structures and missing mechanisms . Despite the differences, these two methods are inherently uni-directional, in the sense that they focus on the outcome measure of abstinence (or smoking) across time from smoking at baseline. For instance, suppose GEE suggests 10% of the participants are not smoking at time 1 and also 15% at time 2, which suggest an increase in abstinence rate. However, these numbers do not indicate whether the 10% subjects at time 1 continued abstinent until time 2 and another 5% smokers at time 1 stopped smoking at time 2, or all the 10% subjects at time 1 regressed to smoking at time 2 and another 15% smokers at time 1 became abstinent at time 2, or some of the 10% at time 1 remained abstinent into time 2 and some of the 90% smokers at time 1 turned abstinent at time 2. Similarly, suppose GLMM suggest an individual’s chance of abstinence is 10% at time 1 and 15% at time 2. This information does not tell how likely this individual would have relapse because relapse refers to a condition that an individual was previously abstinent. In other words, neither GLMM nor GEE is capable of modeling relapse and estimating this conditional probability of regression to smoking given abstinent at the previous time point.
On the other hand, a third method known as transition models provides a means to simultaneously investigate transitions in both directions from smoking to abstinent and from abstinent to smoking. Transition models provide the capability of identifying factors that might work differently in one direction versus the other. Through characterizing factors associated with cessation and those with relapse, we may be able to design more effective interventions for both smoking cessation and relapse prevention. Transition models are a family of models that characterize transition patterns in longitudinal studies. In these methods, the outcome measures are often countable states (e.g. smoking and abstinent can be the only two states that an observation can take). Models developed to handle outcomes observed at a set of scheduled time points are known as discrete-time models . A common model is the discrete-time first-order Markov chain. It assumes that the future state will depend only on the current state but not the entire transition history, and transition probabilities from one state to another do not vary over time. With these two assumptions, we may summarize the transition of smoking status through the entire study window by a matrix with four probabilities: from smoking to smoking, from smoking to abstinent, from abstinent to abstinent, and from abstinent to smoking. Although the foundation of transition models is built on stochastic processes which tobacco researchers have seldom used, the implementation is straightforward when data are completely observed. It relies on counting the numbers of transitions from one state to another, and jointly models multiple logistic regression models, which most tobacco researchers are familiar with. See details in the Methods section.
In this article, we apply a non-homogeneous first-order Markov transition model to evaluate the transition of smoking behavior in a randomized smoking cessation trial KanQuit in which smoking was addressed as a chronic illness and smokers underwent repeated interventions over a two year period of time. We also identify potential factors associated with transitions from smoking to not smoking (abstinence) and the factors associated with the other direction (relapse). We first perform the analysis using the available data alone. We consider the non-responses, particularly those due to lost-to-follow or consent withdraws, might be related to the actual smoking status. In other words, non-responses might be more likely to be smoking. For such situation, assuming non-responses to be missing at random is not proper and may introduce bias. Instead of coding all non-responses as smoking or non-smoking, we treat the missing as nonignorable and apply the selection modeling approach  following the recommendation of Hall et al. . Parameter estimation is conducted by the expectation-maximization (EM) algorithm .
Ellerbeck et al.  developed the KanQuit program, a 2-year randomized trial for smoking cessation delivered to rural smokers who consumed at least 10 cigarettes per day. All participants provided written informed consent. The Human Subjects Committee at The University of Kansas Medical Center approvedthe study (HSC# 9196). In this study, 750 adult participants, regardless of whether they were interested in stopping smoking, were recruited from rural primary care clinics across Kansas and randomized to one of three intervention groups: pharmacotherapy management alone (PM), pharmacotherapy management plus 1 – 2 counseling calls every 6 month (moderate-intensity disease management, or MDM), or pharmacotherapy management supplemented with up to 6 counseling calls every 6 month (high-intensity disease management, or HDM). The primary outcome, the self-reported 7-day abstinence (defined as not having smoked a cigarette during the previous 7 days), was recorded every 6 months from baseline to 24 months. As in many longitudinal studies, participants did not always respond and only 76% (552 individuals) responded at every time point.
Baseline characteristics of 750 study participants
Total (n = 750)
PM (n = 250)
MDM (n = 249)
HDM (n = 251)
Age, mean (SD)
Female, counts (%)
Marital: Married/partners (vs. others), counts (%)
Education: High school or less, counts (%)
Annual income ≤ 40 K, counts (%)*
Child under 18 (vs. no), counts (%)
Child under 6 (vs. no), counts (%)
Cigarettes smoked per day (CPD), mean (SD)
Nicotine dependence (time to first cigarette): within 5 minutes after waking up, counts (%)
Number of friends that smoked: < 3 friends (vs. ≥ 3 friends), counts (%)
Partner smoked? counts (%)
Other smokers at home? counts (%)
Home smoking rules
Not allowed, counts (%)
Allowed some place, counts (%)
No rules (reference)
Motivation to quit score (MOT), mean (SD)
Confidence to quit score (CON), mean (SD)
Smoking Self-Efficacy total score, mean (SD)
Serious quit attempt (SQA), counts (%)
The Markov chain model
where the first row provides probabilities of transitions from the state of abstinence at the previous time point whereas the second row shows transition probabilities from smoking; indicates the probability of becoming abstinent (given being smoking previously), and the chance of relapse. Here the subscript i indicates an allowance for patient-specific transition probabilities.
The assumption of stationary in time may not be realistic in practice. A variant model, known as time-inhomogeneous Markov chain, relaxes the stationary assumption and allows transition probabilities to vary across time, and can be denoted as .
where is the vector from the design matrix for the i th individual at time t, and (1a) and (1b) model relapse and abstinence of smoking, respectively. Note that covariates may have different effects in the two models, and and need not be equal. For ease of communication, we call (1a) and (1b) as Model (1) thereinafter unless one of them is specifically discussed.
where can be any subset of and is the associated parameter vector. Thus, implies missing complete at random (MCAR), and indicate missing at random (MAR), and represents not missing at random (NMAR) .
and is the weight or the probability that the unobserved response has values given the covariates and the parameter estimates at the v-th iteration. In the M-step, is maximized with respect to by the Newton–Raphson algorithm. In practice, we start with initial values of to construct the conditional expectation which is maximized with respect to parameters to update (), and then repeat the E- and the M-steps iteratively until the parameter estimates converge. The standard error (SE) of parameter estimates are estimated by non-parametric bootstrapping  with 1,000 samples.
Analysis was performed on SAS version 9.2. PROC NLMIXED was used for analysis on available data. PROC IML was applied to conduct the selection modeling estimated by the EM algorithm, and a SAS macro was developed for bootstrapping (Additional file 1: Appendix).
Using available data alone (ignore missing mechanism)
Model (1a) Transition from smoking to abstinent
Transition model estimates of abstinence and relapse to smoking
Available data only§
Selection modeling 1
Selection modeling 2
From smoking to abstinent (abstinence model)
Cigarettes per day
Motivation to quit
Confidence to quit
Serious quit attempt
From abstinent to smoking (relapse model)
Income (> 40 K vs. ≤ 40 K)
First cigarette (≤ vs. > 5 min)
Model (1b) Transition from abstinent to smoking
Among the participants not currently smoking, those with lower income (OR = = 1.76, p = 0.029) and stronger nicotine dependence (i.e. first cigarette within 5 minutes after waking up) at baseline were more likely to have relapse (OR = = 1.80, p = 0.029) in the next 6 month as compared to their counterparts. Note the significant time effects ( = 4.89, p = 0.008), indicating that the relapse rates changed over time (Figure 1). After adjusting for temporal variation and income as well as nicotine dependence, the overall effects on preventing relapse were still not significant ( = 2.30, p = 0.100) though relapse seemed to be less likely in the MDM group as compared to the PM group (OR = = 0.49, p = 0.033).
In sensitivity analysis, we consider a couple of possible missing mechanisms and present only two of them in Table 2. In one analysis (Selection modeling 1), we assume missingness depends on whether the smoking status is observed at the previous time point (), the actual smoking status at previous () and current time point (); in the other analysis, we further consider time and treatment arms in the missing model. In either case, the conclusion of significance remains the same.
The missing mechanism model should be interpreted with caution. Apparently, missingness significantly depends on R t-1 (p < 0.0001) and treatment arms (p = 0.006 and 0.002 for the MDM vs. PM and HDM vs. PM, respectively), but not on Y t-1 or Y t in either sensitivity analysis, so the missing mechanism seems to be MCAR. However, when the missing process is jointly modeled with the repeated outcome measures, various parameter values may come to the same likelihood. In other words, even if the transition and the missing models are correctly specified, joint modeling can reduce bias in parameter estimates of the transition model, but may not do so in those of the missing model. The missing model parameters may take a wide range of values and the estimates may be biased. This issue is known as the “identifiability” problem (see e.g.  and ). Hence, some literature only reports the results for the main outcome model but not the missing model (see e.g.  and ).
In this work, we applied a Markov chain model to study the transitions from smoking to abstinence and from abstinence to smoking among the 750 patients in the KanQuit trial. Different factors associated with each type of transition were identified: gender, the baseline daily cigarettes consumption, the baseline motivation and confidence to quit, as well as having serious attempt to quit were associated with transitions from smoking to abstinent, whereas income and nicotine dependence were associated with relapse. The intervention effects were significant in promoting abstinence but not so in preventing relapse (in spite of the MDM group showing a trend of a lower relapse rate), which is probably due to the fact that too few observations of such transitions were available. The sensitivity analyses confirmed the conclusion.
Cox et al.  studied the predictors of smoking abstinence at Month 6 and Month 24, separately, using the 592 participants who completed the assessment at both time points. They identified male gender and lower baseline daily cigarettes consumption. They did not find intervention effects significant at either time point. Ellerbeck et al.  applied the GLMM to evaluate an individual’s odds in favor of abstinence in treatment arms (without covariates), with and without imputation (single imputation of coding all missing to be smoking and not smoking, respectively), and found a significantly higher odds in the HDM group than PM, but not between MDM and PM, or between the two intervention groups, except when all missing were coded as smoking. These findings seem to be consistent with the current work (the portion of transitions from smoking to abstinent or Model (1a)), but the interpretations are different. The GLMM (specifically the random intercept model) suggests that the odds in favor of abstinence for an individual in the HDM group was higher than the odds if the same individual were in the PM group, assuming the intervention effects were the same for all individuals. On the other hand, the transition model indicates that (1) among participants who were smoking at a given time point, the odds in favor of abstinence in the next 6-month was higher in the HDM than the PM group, in males than females, in those with higher motivation and confidence to quit, and in those having serious attempts to quit; and (2) among participants who were abstinent at a given time point, the odds of relapse in the next 6-month was greater at Month 18 (see Figure 1), and among those with lower income and stronger nicotine dependence. Note that relapse refers to turning from abstinence to smoking, thus analysis of relapse must confine to information of participants who ever quitted at some time point(s).
As mentioned in Introduction, GEE and GLMM may estimate the population abstinence rate or an individual’s chance of not smoking, respectively, but they do not provide the relapse rates which are conditional probabilities. Note that subtracting the abstinence rate or an individual’s chance of not smoking from 100% does not give the relapse rate because this number can include participants who ever quitted smoking and those who never. If a smoker continued smoking without abstinence until the observed time, we wouldn’t consider this observation as relapse. This limitation in GEE or GLMM is a strength of transition models. Another strength of Markov models is, when the outcome is multinomial, Markov models may estimate parameters more accurately and provide greater power in rejecting the null hypothesis than GEE .
Moreover, the transition models may serve as an intervention diagnostic tool, which can be particularly beneficiary for studies failing to show significant intervention effects in abstinence rates. An intervention may fail because either it does not motivate smokers to stop smoking, or it does motivate abstinence but fails to prevent relapse. This question can be answered by the transition patterns shown in Figure 1 and Markov models. Researchers may examine the factors associated with the transitions in both directions, and modify the intervention by incorporating these factors. In the KanQuit example, the results suggest that on the basis of the current disease management interventions, the intervention effects may be further improved by enhancing smokers’ motivation and confidence to quit and quit attempts, as well as reducing baseline daily cigarette consumption and/or nicotine dependence.
In the sensitivity analysis (Table 2), we note that the current and previous smoking status are not significant in the missing models. However, it is still premature to claim that missing was ignorable. In fact, even though missing is non-ignorable, the impact of missing values may be mild to moderate in some occasions. When observations are strongly correlated, we may borrow the information from the observed values at the neighboring time points to predict the missing values, and the bias due to the missing values can be reduced (, Section 2.5). Similarly, for the Markov chain models, a strong association among the repeated measures or strong dependence of current observations on previous observations may reduce bias in estimation due to non-ignorable missing . In this study, high probabilities of staying in a current state (from smoking to smoking and from abstinent to abstinent in Figure 1) suggest strong dependence of current states on the previous ones, which helped reduce the impact of non-ignorable non-responses.
As mentioned earlier, non-responses can be caused by various reasons and the true missing mechanism cannot be determined by the data at hand. If ignorable missing can be justified, an alternative approaches to handle missing values are to apply multiple imputation [17, 19] or the multi-step transition probabilities . The probabilities of transitions between two consecutive time points, as described in the Method section, are called the one-step probabilities. When missing values exist, one should consider all possible states for the missing values and sum up the one-step probabilities for all possible routes to obtain the multi-step transition probabilities. This approach may reduce SE and increase statistical power as the EM algorithm .
Another issue is about the validity of self-report smoking status. Ellerbeck et al.  showed deviation in abstinence rates between the self-reported status and the validated saliva cotinine levels (threshold of 15 ng/mL) among the 58% participants who provided saliva samples. When measurements are subject to errors, data are often analyzed by latent variable methods such as hidden Markov models  or latent transition analysis (LTA) [21–23]. We also attempted to fit the KanQuit data by the LTA . However, the LTA model did not seem to fit the data better than the conventional Markov model, and the results are contradictory to our understanding (nearly no chance of relapse and participants would have 14.8% chance of incorrectly reporting they were smoking when they were actually not). Therefore, the results are not presented in this work.
In this article, we discuss and demonstrate how a Markov chain model may use the information in randomized smoking cessation trials that the GEE or mixed-effects models do not utilize, and provide additional findings. With the Markov chain model, we are able to learn about the factors associated with relapse to smoking among those who are temporarily abstinent as well as the factors associated with abstinence. Therefore, to fully investigate longitudinal smoking cessation randomized trials, we encourage researchers to apply transition models together with either GEE or GLMM. GEE or GLMM compares treatment effects between intervention groups and examines the temporal profiles of the treatment effects; Markov models provide knowledge about transitions between abstinence and relapse in both directions. This knowledge may provide guidance in evaluating and designing more effective interventions for smoking cessation and relapse prevention. When non-response causes considerable missing values (e.g. 10% or more) and if non-ignorable missing is considered, sensitivity analysis based on a couple of missing mechanisms should be examined. The results of the missing model should be interpreted with caution.
The authors thank Mr. Niaman Nazir for preparing the data. The original KanQuit trial was sponsored by the National Institute of Health (NIH) grant R01CA101963. This work was supported by a CTSA grant from NCRR and NCATS awarded to the University of Kansas Medical Center for Frontiers: The Heartland Institute for Clinical and Translational Research # UL1TR000001 (formerly #UL1RR033179). The contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH, NCRR, or NCATS. The authors also thank three reviewers for their comments and suggestions that greatly improve this article.
- Diggle P, Liang KY, Zeger SL: Analysis of Longitudinal Data. 2002, New York: Oxford University Press, 2Google Scholar
- Hall SM, Delucchi KL, Velicer WF, Kahler CW, Ranger-Moore J, Hedeker D, Tsoh JY, Niaura R: Statistical analysis of randomized trials in tobacco treatment: longitudinal designs with dichomotous outcome. Nicotine Tob Res. 2001, 3: 193-202.View ArticlePubMedGoogle Scholar
- Ross SM: Stochastic Process. 1995, New York: John Wiley & Sons, 2Google Scholar
- Ellerbeck EF, Mahnken JD, Cupertino AP, Cox LS, Greiner KA, Mussulman LM, Nazir N, Shireman TI, Resnicow K, Ahluwalia JS: Effect of Varying levels of disease management on smoking cessation: a randomized trial. Ann Intern Med. 2009, 150: 437-446.View ArticlePubMedPubMed CentralGoogle Scholar
- Yang X, Li J, Shoptaw S: Imputation-based strategies for clinical trial longitudinal data with nonignorable missing values. Stat Med. 2008, 27: 2826-2849. 10.1002/sim.3111.View ArticlePubMedPubMed CentralGoogle Scholar
- Dempster AP, Laird NM, Rubin DB: Maximum likelihood from incomplete data via the EM algorithm. J Royal Stat Soc. B. 1977, 39: 1-38.Google Scholar
- Cox LS, Cupertino AP, Mussulman LM, Nazir N, Greiner KA, Mahnken JM, Ahluwalia JS, Ellerbeck EF: Design and baseline characteristics from the KAN-QUIT disease management intervention for rural smokers in primary care. Prev Med. 2008, 47: 200-205. 10.1016/j.ypmed.2008.04.013.View ArticlePubMedPubMed CentralGoogle Scholar
- Bhat UN, Miller GK: Elements of Applied Stochastic Processes. 2002, New York: John Wiley & Sons, 3Google Scholar
- Chen B, Yi GY, Cook RJ: Progressive multi-state models for informatively incomplete longitudinal data. J of Stat Plan Infer. 2011, 141: 80-93. 10.1016/j.jspi.2010.05.014.View ArticleGoogle Scholar
- Little RJ, Rubin DB: Statistical Analysis with Missing Data. 2002, New York: John Wiley & Sons, 2Google Scholar
- Efron B, Tibshirani RJ: An Introduction to the Bootstrap. 1994, New York: Chapman & Hall/CRCGoogle Scholar
- Fitzmaurice GM, Heath AF, Clifford P: Logistic regression models for binary panel data with attrition. J Royal Stat Soc. A. 1996, 159: 249-263. 10.2307/2983172.View ArticleGoogle Scholar
- Fitzmaurice GM, Lipsitz SR, Molenberghs G, Ibrahim JG: A protective estimator for longitudinal binary data subject to non-ignorable non-monotone missingness. J Royal Stat Soc. A. 2005, 168: 723-735. 10.1111/j.1467-985X.2005.00374.x.View ArticleGoogle Scholar
- Ibraham JG, Chen MH, Lipsitz SR: Missing responses in generalised linear mixed models when the missing data mechanism is nonignorable. Biometrika. 2001, 82: 551-564.View ArticleGoogle Scholar
- Cox LS, Wick JA, Nazir N, Cupertino AP, Mussulman LM, Ahluwalia JS, Ellerbeck EF: Predictors of early versus late smoking abstinence within a 24-month disease management program. Nicotine Tob Res. 2011, 13: 215-220. 10.1093/ntr/ntq227.View ArticlePubMedPubMed CentralGoogle Scholar
- Li YP, Chan W: Analysis of longitudinal multinomial outcome data. Biom J. 2006, 48 (2): 319-326. 10.1002/bimj.200510187.View ArticlePubMedGoogle Scholar
- Schafer J: Analysis of Incomplete Multivariate Data. 1997, New York: Chapman & Hall/CRCView ArticleGoogle Scholar
- Yeh HW, Chan W, Symanski E: Intermittent missing observations in discrete-time hidden markov models. Stat Commu Comp Simu. 2012, 41: 167-181. 10.1080/03610918.2011.581778.View ArticleGoogle Scholar
- Enders CK: A Premier on the use of modern missing-data methods in psychosomatic medicine research. Psychosom Med. 2006, 68: 427-436. 10.1097/01.psy.0000221275.75056.d8.View ArticlePubMedGoogle Scholar
- Yeh HW, Chan W, Symanski E, Davis BR: Estimating transition probabilities for ignorable intermittent missing data in a discrete-time Markov chain. Stat Commu Comp Simu. 2010, 39: 433-448.View ArticleGoogle Scholar
- Martin RA, Velicer WF, Fava JL: Latent transition analysis to the stages of change for smoking cessation. Addict Behav. 1996, 21: 67-80. 10.1016/0306-4603(95)00037-2.View ArticlePubMedGoogle Scholar
- Velicer WF, Martin RA, Collins LM: Latent transition analysis for longitudinal data. Addiction. 1996, 91: S197-S209. 10.1111/j.1360-0443.1996.tb02339.x.View ArticlePubMedGoogle Scholar
- Delucchi KL, Weisner C: Transitioning into and out of problem drinking: A 7-year, 5-wave study. Alcoh Clinic Exper Res. 2007, 31: 61A-61A. 10.1111/j.1530-0277.2006.00288.x.View ArticleGoogle Scholar
- Lanza ST, Collins LM: A new SAS procedure for latent transition analysis: transitions in dating and sexual risk behavior. Dev Psychol. 2008, 44 (2): 446-456. 10.1037/0012-16184.108.40.2066.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/12/95/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.