Time-varying SMART design and data analysis methods for evaluating adaptive intervention effects

Background In a standard two-stage SMART design, the intermediate response to the first-stage intervention is measured at a fixed time point for all participants. Subsequently, responders and non-responders are re-randomized and the final outcome of interest is measured at the end of the study. To reduce the side effects and costs associated with first-stage interventions in a SMART design, we proposed a novel time-varying SMART design in which individuals are re-randomized to the second-stage interventions as soon as a pre-fixed intermediate response is observed. With this strategy, the duration of the first-stage intervention will vary. Methods We developed a time-varying mixed effects model and a joint model that allows for modeling the outcomes of interest (intermediate and final) and the random durations of the first-stage interventions simultaneously. The joint model borrows strength from the survival sub-model in which the duration of the first-stage intervention (i.e., time to response to the first-stage intervention) is modeled. We performed a simulation study to evaluate the statistical properties of these models. Results Our simulation results showed that the two modeling approaches were both able to provide good estimations of the means of the final outcomes of all the embedded interventions in a SMART. However, the joint modeling approach was more accurate for estimating the coefficients of first-stage interventions and time of the intervention. Conclusion We conclude that the joint modeling approach provides more accurate parameter estimates and a higher estimated coverage probability than the single time-varying mixed effects model, and we recommend the joint model for analyzing data generated from time-varying SMART designs. In addition, we showed that the proposed time-varying SMART design is cost-efficient and equally effective in selecting the optimal embedded adaptive intervention as the standard SMART design. Electronic supplementary material The online version of this article (doi:10.1186/s12874-016-0202-7) contains supplementary material, which is available to authorized users.


Background
Sequential, multiple assignment, randomized trial (SMART) designs and their analysis are being used to construct high-quality adaptive interventions that can be individualized by repeatedly adjusting the intervention(s) over time on the basis of individual progress [1][2][3][4][5]. The SMART design was pioneered by Murphy, building on the work of Lavori and Dawson [6,7]. SMART designs involve an initial randomization of individuals to different intervention options, followed by re-randomization of some or all of the individuals to another set of available interventions at the second stage. At subsequent stages, the probability and type of intervention to which individuals are re-randomized may depend on the information collected from the previous stage(s) (e.g., how well the patient responded to the previous treatment; adherence to treatment protocol). Thus, there can be several adaptive interventions embedded within each SMART design. This allows for testing the tailoring variables and the efficacy of the interventions in the same trial. There are several practical examples of SMART studies that have been conducted (e.g., the CATIE trial [8] for antipsychotic medications in patients with schizophrenia, STAR*D for the treatment of depression, [9,10] and phase II trials at MD Anderson for treating cancer [11]). The goal of these studies is to optimize the longterm outcomes by incorporating the participant's characteristics and intermediate outcomes evaluated during the intervention [12,13].
An example of a two-stage SMART design is a study that characterized cognition in nonverbal children with autism [14]. To improve verbal capacity, participants were initially randomized to receive either a combination of behavioral interventions (Joint Attention Symbolic Play Engagement and Regulation (JASPER) + Enhanced Milieu Training (EMT)) or an augmented intervention (JASPER + EMT+ speech-generating device [SGD]). Children were assessed for early response versus slow response to the first-stage treatment at the end of 12 weeks. The second-stage interventions, administered for an additional 12 weeks, were chosen on the basis of the response status (only slow responders to JASPER + EMT were re-randomized to intensified JASPER + EMT or received the augmented JASPER + EMT + SGD; slow responders to JASPER + EMT + SGD received intensified treatment; all early responders continued on the same intervention). There were three pre-fixed assessment time points: at 12 weeks, 24 weeks and 36 weeks (follow-up), which were the same for all participants in the study. Compared to multiple, one-stage-at-a-time, randomized trials, SMART designs provide better ability to compare the impact of a sequence of treatments, rather than examining each piece individually. For example, a SMART allows us to detect possible delayed effects in which an intervention at a previous stage has an effect that is less likely to occur unless it is followed by a particular subsequent intervention option. The typical modeling approach for the SMART design as described by Nahum-Shani et al. includes the indicators of intervention at each stage as covariates and thus accounts for the delayed effects on the final response. In order to develop a sequence of best decision rules for each individual, various statistical learning methods of estimating the optimal dynamic treatment regimens have been proposed, among which Q-learning has been developed for assessing the relative quality of the intervention options and estimating the optimal (i.e., most effective) sequence of decision rules with linear regression. For a two-stage SMART, the Q-learning approach controls for the optimal second-stage intervention option when assessing the effect of the first-stage intervention, and reduces the potential bias resulting from unmeasured causes of both the tailored variables and the primary outcome. A similar approach for deriving the optimal decision rules for SMART is A-learning, which is more robust to model misspecification than Q-learning for consistent estimation of the optimal treatment regime [15]. Zhao et al. introduced the two learning methods of BOWL and SOWL, [16] which are based on directly maximizing over all dynamic treatment regimens (DTRs) a nonparametric estimator of the expected long-term outcome. As an alternative to the above learning approaches, Zhang et al. [17] proposed a robust estimation of the optimal dynamic treatment regimens for sequential treatment decisions, which maximizes a doubly robust augmented inverse probability weighted estimator for the population mean outcome over a restricted class of regimes. All these approaches model the outcomes of interest as dependent variables, and for the predictor variables, they model the main and interacting effects of the intervention options at each stage and the baseline individual characteristics. The amount of time an intervention is administered, however, is not explicitly modeled, although it can be used as a covariate in these regressions.
There are examples of SMART designs in which a participant is assessed at several pre-fixed time points during the first-stage treatment and once he/she meets an assigned criterion for response status, he/she is re-randomized to the second stage of treatment. Such a SMART design has been applied to develop a dynamic treatment regime for individuals with alcohol dependence using the medication naltrexone [2,18,19]. At the beginning of the study, patients were randomized to either a stringent or a lenient criterion for early non-response. Initially, all patients received naltrexone. Starting at the end of the second week, patients who showed early response were assessed weekly for eight weeks, and those who met the assigned criterion for non-response were assigned to the second stage randomization in that week; whereas the responders were re-randomized at week eight. Another example of using a SMART design to evaluate multiple, fixed time points is the study of pharmacological and behavioral treatments for children with ADHD, where children were assessed monthly for response or non-response [19][20][21][22][23]. In addition, Lu et al. [19] developed repeated-measures piecewise marginal models for comparing embedded treatments in such SMART designs with multiple evaluations at fixed time points. In these studies, subjects were assessed at fixed time points; thus, the time of treatment takes values along a finite set of time points.
Although, SMART designs with outcome assessments at fixed time points exist, there are advantages to administering a drug as soon as an individual achieves an intermediate response. For example, the smoking cessation drugs varenicline and bupropion can increase the risk of psychological side effects such as unusual changes in behavior, hostility, agitation, depressed mood and suicidal thoughts [24][25][26]. In addition, varenicline costs approximately $300 per month. Therefore, allowing the duration of treatment to vary among participants for one or more stages of the study may reduce the side effects and costs associated with the interventions. For such time-varying SMART designs, the duration of treatment plays an important role in decision making, and including it in the analysis may increase the power of the study and better serve our goal of analysis. To further extend the assignment strategies discussed in the above examples and utilize the information contained in the treatment duration, in this paper, we proposed a novel time-varying SMART design, which enables us to more efficiently assign different intervention options as soon as an individual achieves a set of intermediate response goals. Therefore, the time of treatment is a continuous random variable for each individual that can take any value on a subset of the positive real line, and is treated as an endogenous variable. The existing statistical methods are inappropriate for analyzing data obtained from such a time-varying SMART design. Therefore, to fully utilize the potential of this type of time-varying SMART design in making more efficient decisions, we also proposed two analytic approaches that can be used to analyze data from such a time-varying SMART design. The first approach is a linear mixed model with time-varying fixed effects [27,28], which is in fact a piecewise linear model. The second approach incorporates a joint modeling method in which a survival model is fitted jointly with the linear mixed model [29]. We performed simulations to evaluate the statistical properties of both methods. Our simulation results showed that both methods estimated the expected final outcome for each embedded adaptive intervention in such design accurately, but the joint-modeling method provided better estimates for certain parameters in the model.
To compare the power and cost efficiency of the timevarying SMART design to those of an analogous standard SMART design, we simulated two trials with identical sample sizes and intervention effects using (a) the timevarying SMART design and (b) the standard SMART design. These simulations showed that the time-varying SMART design is cost-efficient and has power similar to that of the standard SMART design in selecting the optimal embedded adaptive intervention.

Methods
Proposed time-varying SMART design Figures 1 and 2 illustrate the proposed time-varying designs. Both two-stage time-varying SMARTs were designed to provide data regarding how the intensity and combination of two types of interventions might be adapted to a subject's progress in a cost-and timeefficient manner.
In the first example (see Fig. 1), suppose medication (M) and behavioral intervention (B) are two initial intervention options for individuals who are heavy smokers (e.g., those who smoke more than or equal to 25 cigarettes per day). The number of cigarettes a subject smokes per day is the outcome of interest and is measured at the beginning of the study, at several intermediate time points and at the end of the study. Let Y 0 denote the number of cigarettes a subject smoked per day at the beginning of the study (t = 0). Subjects are randomly assigned to the medication or the behavioral interventions at the beginning of the study. Monitoring the outcome of interest begins at a pre-fixed time point (e.g. one week after the initial randomization and is denoted as t 00 ) after the initial intervention is implemented, and t 10 denotes the time point at which those who did not respond to a first-stage intervention are re- Fig. 1 Example of time-varying two-stage SMART design with equal probability allocation: each participant is randomized twice randomized. A subject is considered a responder to the first-stage intervention if there is a significant decrease in the number of cigarettes he/she smoked per day (e.g., the decrease in the number of cigarettes smoked per day is above a pre-fixed threshold, C) at an intermediate time point T 1 , before t 10 . Thus, T 1 is a random variable of time and varies among responders. A subject is classified as a non-responder if the decrease in the number of cigarettes he or she smoked per day by t 10 is below C. Therefore, all the non-responders are given the firststage intervention for a fixed time period of t 10 (e.g., the first month of initial interventions), which can be seen as the right-censored time point. Let Y 1 denote the number of cigarettes smoked per day at the end of the firststage intervention.
An indicator variable δ is defined as δ = I(T 1 < t 10 ), where I(⋅) is the indicator function that takes the value 1 if T 1 < t 10 (i.e., if the subject is a responder) and the value 0 if T 1 ≥ t 10 (i.e., if the subject is a nonresponder). A responder is re-randomized either to continue with the first-stage intervention (M or B) or to receive the first-stage intervention at a reduced intensity (M-or B-); whereas a non-responder is rerandomized to receive the first-stage option at an increased intensity (M+ or B+) or augmented with the other type of intervention (i.e., adding a behavioral intervention for those who started with medication or adding medication for those who started with a behavioral intervention). We let all the subjects in this design stay on their second-stage interventions for a fixed time period, Δt (e.g., one month). Therefore, for a subject whose first-stage intervention time is T 1 , the total study time is T 1 + Δt, which we denote as T 2 . For each participant, Y 2 is the final measurement of the number of cigarettes smoked per day at T 2 ; see Fig. 1). Fig. 2 is similar to that in Fig. 1 except that all the responders continue with their first-stage intervention options (i.e., each responder receives the same intervention after the response time point T 1 ) in the second stage (see Fig. 2).

The design illustrated in
The adaptive interventions that are embedded within the two SMART designs in Figs. 1 and 2 are listed in Additional file 1: Tables S1 and S2.

Analytic approach
Let A 1 and A 2 be the indicators of the first-and secondstage intervention options, respectively. For each individual, we observe the data (Y 0 , A 1 , T 1 , Y 1 , A 2 , Y 2 , T 2 , δ). The outcomes of interest are the longitudinal measurements Y 0 , Y 1 , and Y 2 , which are fitted with a linear mixed model, assuming they share the same random intercepts at the subject level. Because the intervention options and their durations change over time in this design, we first proposed a straightforward time-varying mixed effects model (TVMEM) to analyze the outcomes. In this approach, the duration of time a treatment is administered is used as a covariate in the model. Such an approach is better than the approaches that ignore the time component of the intervention (i.e., the duration of the intervention influences its effect). However, the time duration is a random variable and one may gain statistical efficiency by treating it as a dependent variable in the modeling. Therefore, we also proposed a jointmodeling approach that simultaneously postulates a linear mixed effects model for the longitudinal measurements Y = (Y 0 , Y 1 , Y 2 ) and a Cox model for the survival time T 1 . In particular, we fit a survival submodel for T 1 jointly with the previously mentioned TVMEM that will efficiently extract the information contained in T 1 . A linear TVMEM is fitted to the longitudinal outcomes, with interventions and their interactions and durations included as predictors. For each individual i in the study, we have where m i (t) is the unobserved true value of the longitudinal outcome at time point t, and b i is the subject-level random effects and is assumed to be normally distributed with a mean of zero and variance of σ b 2 ; Z i is a vector of the baseline covariates (e.g., age, sex, comorbidities, etc.) with a corresponding vector of the regression coefficients η(t); X i (t) is the vector of the first-stage and second-stage intervention options, their interactions, and duration of intervention with a corresponding vector of the regression coefficients β(t). Finally, ε i (t) is the error term at time t and is assumed to be normally distributed and independent of b i .
In our study design, we consider three time points at which the outcomes of interest are measured: t = 0, T 1i and T 2i , where T 1i and T 2i are the respective time points at which individual i completes the first-and secondstage interventions. Therefore, A 1i (t) takes the value of A 1i at times T 1i and T 2i and is equal to 0 at t = 0, and A 2i (t) takes the value of A 2i at T 2i and is equal to 0 at time points 0, and T 1i . In this way, η(t) and β(t) are piecewise linear fixed coefficients; therefore, model (1) at the three time points is equivalent to the following three linear mixed-effects submodels: where in equations (2) through (4), Y 0i , Y 1i and Y 2i are the outcome values at time 0, T 1i and T 2i , respectively; A 1i is the indicator of the first-stage intervention options (−1 for M and +1 for B), A 2i = (A 2Ri , A 2NRi ) is the indicator vector for the second-stage intervention options, where A 2Ri is the indicator for the second-stage intervention options for the responders to the first-stage intervention (1 = continue the initial intervention; −1 = reduce the intensity of the initial intervention) and A 2NRi is the indicator for the second-stage intervention options for the nonresponders (1 = increase the initial intervention; −1 = augment the initial intervention with the other type of intervention), with A 2Ri =0 for non-responders and A 2NRi =0 for responders. A 1i ⋅ A 2Ri and A 1i ⋅ A 2NRi are the interaction effects of the first-stage intervention and second-stage intervention among responders and non-responders, respectively, in the submodel of Y 2i (i.e., submodel (4)). Parameters η 0 , η 1 , η 2 and β 00 , β 01 , β 02 are the coefficients of the baseline covariates and intercepts at time points 0, T 1i and T 2i , respectively; submodel (2) includes only baseline covariates as predictors for the outcomes at the beginning of the study (i.e., Y 0i at t = 0); submodel (3) models the outcome of interest at the intermediate time point of the study (i.e., Y 1i at t= T 1i ) and includes covariates A 1i and T 1i , for which the corresponding coefficients β 11 and β 31 account for the direct effect of A 1i and indirect effects through T 1i on Y 1i ; submodel (4) includes all the main and interacting effects of the intervention options at each stage and the duration T 2i (T 2i = T 1i + Δt) as predictors, for which the coefficients β 12 and β 32 account for the delayed effect of A 1i and delayed indirect effects of A 1i through T 2i . The coefficients β 2 = (β 22 , β 23 ) and β 4 = (β 41 , β 42 ) account for the effects of the second-stage interventions and the effects of their interactions with the first-stage interventions on the final outcome Y 2i (measured at the end of the study, T 2i ).
The conditional expectations for models (1)-(4) are provided in Additional file 2. We also provided conditional expectations of the final outcomes for each of the eight embedded adaptive interventions in the SMART design of Fig. 1 and four embedded adaptive interventions in the SMART design of Fig. 2 [see Additional file 2].

Joint model
In addition to the TVMEM, we postulate a relative risk model for T 1i (time to the event of interest) as where W i is a vector of the baseline covariates, which could be different from vector Z i in model (1), and h 0 (⋅) is the baseline risk function. The underlying longitudinal measurement m i (0) at baseline (i.e., at time point t = 0), as approximated by the TVMEM, and at the first-stage intervention A 1i are included as predictors in model (5) because the time point at which an individual responds to the first-stage intervention (i.e., T 1i ) depends only on the type of first-stage intervention the subject received and the baseline characteristics. We jointly estimate the coefficients in models (1) and (5) by using the maximum likelihood estimation method. To define the joint distribution of the time-to-event and longitudinal outcomes, we assume that the random effect b i underlies both the longitudinal and survival processes for each subject. This means that the random effect accounts for both the association between the longitudinal and event outcomes and the correlation between the repeated measurements in the longitudinal process. We also assume that the longitudinal outcomes {Y 0i , Y 1i , Y 2i } are independent of the time T 1i conditional on the random effect b i . Therefore, the joint likelihood contribution for the ith subject can be formulated as where p{Y i (t ij )|b i ; β, η} is the univariate normal density for the longitudinal responses at time point t ij , which is the element from the vector is the normal density with standard deviation σ b for the random effects b i ; and p(T 1i , δ i |b i ; β, γ, α, η) is the likelihood for the time to the intermediate outcome and can be written as . Parameters in the model are estimated by maximizing the corresponding log-likelihood function with respect to (β, γ, α, η). We obtained the maximum likelihood estimates using the R package "JM" [30]. The parameters (β 12 , β 22 , β 23 , β 32 , β 41 , β 42 ) in submodel (4) (i.e., the model of final outcome Y 2 ) are of primary interest and were estimated using the two approaches described above.
The data organization and implementation of these methods is presented in Additional file 3.

Simulations
For the example illustrated in Fig. 1, we considered two simulation scenarios in which Y 0 and Y 1 were simulated using submodels (2) and (3), respectively, and Y 2 was simulated with and without the interaction terms (A 1i ⋅ A 2Ri and A 1i ⋅ A 2NRi ) in submodel (4). In both scenarios, we simulated 500 replicates of n = 1000 individuals, and randomly assigned subjects (with probability .5) to one of the two first-stage interventions (i.e., A 1 to be equal to 1 [behavioral intervention] or −1 [medication]). Responders and non-responders to the initial interventions were then re-randomized (with probability .5) to one of the corresponding second-stage intervention options (i.e., A 2R and A 2NR were randomly assigned to be 1 or −1 and A 2R =0 for non-responders and A 2NR =0 for responders; see Fig. 1). In both scenarios, the random effects {b i } i = 1 n for subjects i = 1, 2, …, n were generated from the normal distribution with a mean of 0 and a standard deviation of 5, and baseline outcomes {Y 0i } i = 1 n were simulated using submodel (2) with parameters β 00 = 10 and ε 0i~N (0, 4 2 ). The intermediate outcomes {Y 1i } i = 1 n were simulated using submodel (3) with parameters β 01 = 1, β 11 = 0.2, and β 31 = 0.1 in the first scenario; whereas outcomes {Y 1i } i = 1 n in the second scenario were simulated with β 01 = 1, β 11 = 0.6, and β 31 = 0.1, with a standard deviation of 5 (i.e., ε 1i~N (0, 5 2 ) in both scenarios and satisfying the The time points T 1i were generated from a lefttruncated Weibull distribution (truncated from t 00 =0.1, the start time for monitoring), with shape = 1 and scale= exp{γ 0 + γ 1 A 1i + αm i (0)}, where γ 0 = − 1.5, γ 1 = 0.4, and α = 0.25, and those for whom T 1i was greater than 1 (non-responders), were assigned T 1i = t 10 = 1 (the maximum time the first-stage intervention is administered [t 10 ]). The indicator of response status was then defined by δ i = I(T 1i < 1). The final outcomes Y 2i (i = 1, …, n) were generated using submodel (4), with ε 2i~N (0, 5 2 ). The values of the other parameters in submodel (4) are reported in Table 1 (without interactions) and Table 3 (with interactions).
For the intervention strategy depicted in Fig. 1, there are eight adaptive interventions imbedded in the design and represented by the three indicators A 1 , A 2R , and A 2NR . For example, in adaptive intervention (A 1 , A 2R , A 2NR ) = (−1, 1, 1), participants are initially randomized to the medication (A 1 = − 1); those who respond are re-randomized to continue on the medication (A 2R = 1) and those who do not respond are re-randomized to increased medication (A 2NR = 1). Another example of an adaptive intervention is (A 1 , A 2R , A 2NR ) = (1, 1, −1), in which participants are initially randomized to a behavioral intervention (A 1 = 1); those who respond are re-randomized to continue on the behavioral intervention (A 2R = 1), and those who do not respond are re-randomized to an augmented arm (M + B, A 2NR = − 1).
For the design in Fig. 2, only the non-responders are re-randomized in the second stage. Therefore, there are four embedded adaptive interventions in this design, which are represented by the vector of two indicators (A 1 , A 2NR ). For example (−1,−1) represents the adaptive intervention in which participants are initially randomized to medication (A 1 = − 1) and those who do not respond are re-randomized to the augmented arm (M + B, A 2NR = − 1), whereas responders continue on the medication arm.
Using this design, we also simulated the treatment of 1000 subjects. However, instead of using equal probability allocations as in Fig. 1, we used unequal probability allocations at both stages. Specifically, each of the 1000 subjects were initially assigned to either A 1 = − 1 (medication) or A 1 =1 (behavioral intervention) with probabilities 0.4 and 0.6, respectively. Then, the non-responders were re-allocated into either A 2NR = − 1 (augmented first-stage intervention, M + B) or A 2NR =1 (intensified first-stage intervention, M+ or B+) with probabilities 0.55 and 0.45, respectively; whereas all responders were continued on their initial interventions (therefore, A 2R =0) in their second stage. Random effects (b i ), errors (ε i ), and longitudinal outcomes (Y 0i , Y 1i (i = 1, …, n)) were generated as described for Fig. 1. The final outcomes, Y 2i (i = 1, …, n), were also generated using submodel (4), but without the variable A 2Ri , with the parameter values reported in Tables 5 and 7 for the two scenarios, respectively. In the first scenario, outcomes Y 2i (i = 1, …, n) were simulated without interaction terms and with the parameter values shown in Table 5; in the second scenario, outcomes Y 2i (i = 1, …, n) were simulated with interaction terms and with the parameter values shown in Table 7.

An alternate simulation approach
For the design illustrated in Fig. 1, we performed an alternate simulation approach that does not simulate Table 1 Simulation results for the design in Fig. 1  values for T 1i from the Weibull distribution. Instead, we considered a situation in which values of Y 1i are monitored and T 1i is the value for which the Y 1i crosses the pre-specified boundary condition for the first time. In this simulation approach, random effects {b i } i = 1 n and error terms ε 0 , ε 1 and ε 2 were all simulated the same way as described above. Baseline outcomes {Y 0i } i = 1 n are simulated using submodel (2) with β 00 = 2 and ε 0i~N (0, 2 2 ). Furthermore, we defined an individual i, as a responder if he/she had a certain percentage reduction in the intermediate outcome value, Y 1i , compared to his/her baseline value Y 0i . This may be a more appropriate definition of responders in some practical scenarios than a simple reduction by a fixed amount (e.g., C = 9) as was used in the previous simulations. In this simulation, those with a 40 % reduction from their baseline values were considered responders. The parameter values used for submodel (3) were β 01 = −2, β 11 = −0.5, and β 31 = 5 . For an individual i, we first simulated ε 1i~N (0, 2 2 ), and calculated T 1i * for which the β 01 + β 11 A 1i + β 31 T 1i * + b i + ε 1i equals the 40 % reduction from Y 0i , the baseline value. Therefore, we define T 1i = t 00 , if T 1i * < t 00 ; T 1i = T 1i * , if t 00 ≤ T 1i * ≤ t 10 ; and T 1i = t 00 , if T 1i * ≥ t 10 . Then, T 1i is substituted in the right side of equation (3) to obtain the value of Y 1i for the individual i(i = 1, …, n). The final outcomes Y 2i (i = 1, …, n) were generated using submodel (4), with ε 2i~N (0, 2 2 ). As previously, we simulated 500 replicates of n = 1000 individuals in each trial, and randomly assigned subjects (with probability 0.5) to one of the two first-stage interventions (i.e., A 1 to be equal to 1 [behavioral intervention] or −1 [medication]). Responders and non-responders to the initial interventions were then re-randomized (with probability 0.5) to one of the corresponding second-stage intervention options (i.e., A 2R and A 2NR were randomly assigned to be 1 or −1 and A 2R =0 for non-responders and A 2NR =0 for responders; see Fig. 1). Table 3 Simulation results for the design in Fig. 1: the estimated means, based on 500 replicates, are reported for coefficients in model (4)   We evaluated the performance of our two proposed analytic approaches in these simulated data sets by measuring the (a) means of the estimates of each of the adaptive interventions embedded in the design, (b) parameter estimates in the model, (c) mean squared error (MSE), (d) estimated coverage probability of the 95 % confidence interval, and (e) length of the confidence interval.
Using these simulations parameters, we simulated two trials with identical sample sizes: (a) the timevarying SMART design and (b) the standard SMART design. We evaluated the performance of the timevarying SMART design and an analogous standard SMART design by measuring the (a) power to select the optimal embedded intervention, and (b) associated cost.

Results
Tables 1-4 show the results of the two simulation scenarios based on the design shown in Fig. 1. Similarly, Tables 5-8 show the results of the two simulation scenarios for the design in Fig. 2.
For each of the eight embedded adaptive interventions in the design, Table 2 shows that both approaches accurately estimated the means of the final outcome, E[Y 2 |(A 1 , A 2R , A 2NR )]. For example, the simulated means of the adaptive interventions (A 1 , A 2R , A 2NR ) = (−1, −1, − 1), (−1, 1, 1), and (1, 1, 1) were 4.538, 5.536, and 6.564, respectively, and the estimated means were 4.543, 5.531, and 6.569, respectively, using the TVMEM and joint model. Tables 3 and 4 show results similar to those in Tables 1  and 2, respectively. In Table 3, the coefficient of interaction of the first-stage interventions and secondstage interventions among responders is denoted by β 41 , and the coefficient of interaction of the first-stage interventions and second-stage interventions among non-responders is denoted by β 42 . As shown in Table 3, both TVMEM and joint modeling accurately estimated parameters β 22 , β 23 , β 41 , and β 42 , with little difference in the MSE, estimated coverage probability, and length of the 95 % confidence interval. However, as in Table 1, the joint modeling approach estimated β 12 and β 32 more accurately than the TVMEM approach. For example, the true coefficient of T 2 was β 32 = 2.0, which was poorly estimated as 4.122 using the TVMEM and estimated as 1.626 using the joint model. Table 4 shows that the estimated means of the eight adaptive interventions obtained from both analytical approaches were identical and close to the simulated means up to the third decimal.
Similar trends were observed in Tables 5-8 for the two simulations of Fig. 2. β 12 and β 32 were better estimated using the joint modeling approach, whereas all the other parameters and the means of the final outcomes of the four adaptive interventions embedded in the design were accurately estimated using both approaches.
In Table 7 shows that the true parameters β 12 =0.40 and β 32 =2.0 were respectively estimated asβ 12 =0.298 andβ 32 =4.308 using the TVMEM, and asβ 12 =0.408 and β 32 =1.784 using the joint model. The other two parameters, β 23 and β 42 , were accurately estimated using both approaches. Table 8 shows that the means were accurately estimated using both approaches. Tables 9 and 10 show the results from the alternative simulation strategy. In Table 9, the true coefficient values of β 12 = −0.6 and β 32 = −1.5 were estimated asβ 12 = −0.534 andβ 32 = −2.367 using the TVMEM, and asβ 12 = −0.608 andβ 32 = −1.338 using the joint model. Coefficients β 22 , β 23 and the means of the final outcomes of the eight adaptive interventions embedded in the design were accurately estimated using both approaches (Table 10).

Comparison of power between the time-varying SMART design and the standard SMART design
We analyzed the time-varying SMART design's ability to select the most optimal embedded intervention and compared the associated power to that of the standard SMART design. We performed the comparison by conducing two trials with identical sample sizes and intervention effects using (a) the timevarying SMART design and (b) the standard SMART design. Figure 3 represents the standard SMART design that is analogous to the time-varying SMART design depicted in Fig. 1. The major difference between the two designs is that in the time-varying SMART design, a responder is re-randomized to the second-stage intervention at a random response time T 1 (< t 10 ); whereas in the standard SMART design, everyone is re-randomized at a fixed time point t 10 .
Responders are defined similarly in both designs. In our example, a subject is considered a responder to the first-stage intervention if there is a significant decrease in the number of cigarettes the person smoked per day. The second-stage intervention is identical for both designs.
For both designs, we calculated the percentage of times that the best embedded intervention is selected (i.e., the power of the design). We simulated six parameter scenarios: the true parameters for the coefficient of the first-stage interventions, β 21 ; coefficient of the second-stage intervention for responders, β 22 ; coefficient of the second-stage intervention for non-responders, β 23 ; coefficient of T 2 , the total time of the first-and second-stage interventions, β 32 ; coefficient of interaction of the firststage interventions and second-stage interventions among responders β 41 ; and the coefficient of interaction of the first-stage interventions and secondstage interventions among non-responders β 42 . The Table 5 Simulation results for the design in Fig. 2  simulated values of each of these parameters are reported in Tables 11 and 12. The simulation results are based on 500 replicates and are shown in Table 11 for comparing the two designs in Fig. 1 (time-varying SMART) and Fig. 3 (analogous standard SMART). Overall, both designs were equally effective in selecting the optimal embedded adaptive intervention. For example, when β 21 = 0.4, β 22 = 0.5, β 23 = 0.5 and β 32 = 2, using the joint model and implementing the time-varying SMART design showed 82.8 % power to select the optimal embedded adaptive intervention; whereas the power associated with the standard SMART design was 83.0 %. Similar results were obtained when comparing the time-varying SMART design in Fig. 2 and the standard SMART design in Fig. 4 (see Table 12).
Comparison of the cost associated with conducting the time-varying SMART design versus that associated with conducting the standard SMART design To assess the cost associated with the conducting trials using these two competing designs, we considered a linear cost function for both SMART designs.
Let c 1 and c 2 be the cost of the medication (M) and behavioral intervention (B), respectively. Additionally, we assumed that the reduced and increased intensity of the first-stage intervention are at half and twice the cost of the first-stage intervention, respectively, and that augmentation of the first-stage intervention in the second stage (M + B) has the cost c 1 + c 2 . Using these parameters, the cost for the time-varying SMART design in Fig. 1 is Δt 0 @ 1 A ; Table 7 Simulation results for the design in Fig. 2: the estimated means, based on 500 replicates, are reported for coefficients in model (4)   and the cost for the corresponding standard SMART in Fig. 3 is Similarly, the cost for the time-varying SMART in Fig. 2 is and the cost for the corresponding standard SMART in Fig. 4 is Note that in the above equations, T 1i = t 10 for non- A is the cost of the second stage for all the subjects assigned to the intervention M + B. Figure 5 shows the cost as a function of c 1 and c 2 , where red represents the cost of the time-varying SMART design and blue represents the cost of the standard SMART design. We can see that the cost of the time-varying SMART is less than the cost of the standard SMART in all scenarios. Table 13 shows the average costs and standard deviations calculated at select values of c 1 and c 2 based on 1000 replicates. For example, when the unit costs are c 1 = 2 and c 2 = 1 for medication and behavioral intervention, the average cost of the time-varying SMART in Fig. 1 is 3446.5, with standard deviation 49.87, while the average cost of the corresponding standard SMART is 3935.8, with standard deviation 41.47. Thus, the cost of the standard SMART is about 12 % higher than that of the time-varying SMART in this scenario.

Discussion
In the standard SMART design, the timing of allocating the intervention is generally ignored, which leads to a model of regression without the predictor of a time variable. Therefore, in this article, we proposed a timevarying SMART design that allows the re-randomization to the second-stage interventions to occur at different time points for different individuals. The two modeling approaches we proposed for analyzing data using such time-varying SMART designs provided good estimations of the means of the final outcomes of all the embedded interventions. However, the joint modeling approach provided more accurate parameter estimates and higher estimated coverage probability than the TVMEM, and thus we recommend the joint model for analyzing data generated from time-varying SMART designs.
In the examples illustrated in Figs. 1 and 2, a participant was defined as a responder if there was a significant decrease in the number of cigarettes the participant smoked per day. One may question the validity of rerandomizing individuals who have a quick response to the first-stage intervention because such a response indicates the effectiveness of the intervention. However, if significant adverse effects are associated with the intervention (e.g., radiation therapy for many types of cancer is commonly associated with skin damage [31], fatigue [32,33], diarrhea [34,35], and rectal bleeding [36]), it is  reasonable to shorten the duration of the intervention to avoid side effects. Therefore, the allocation strategy for the responders in the examples of the time-varying SMART design makes it more efficient than the standard SMART design. We proposed two approaches for analyzing the longitudinal outcomes obtained from the time-varying SMART design: the TVMEM and the joint model. According to the simulation results, the joint modeling approach better estimated the effects of the duration of the intervention (i.e., T 2 ) and the first-stage interventions (i.e., A 1 ) in model (4). More specifically, the joint modeling approach had more accurate estimates, smaller MSEs, higher estimated coverage probabilities, and smaller 95 % confidence intervals (i.e., smaller estimated standard deviations) for the coefficients of the effects of the first-stage intervention and the time of intervention. Because we wanted to illustrate the cost efficiency of the proposed time-varying SMART design and its ability to select the optimal embedded adaptive intervention, we implemented a rather simplified linear mixed-effects submodels (2)-(4) of the more general TVMEM in model (1). We showed that the joint model performs better than the TVMEM in analyzing the data collected from such time-varying SMART designs. The joint modeling approach extracts part of the information contained in the time of the response, which is a function of the first-stage treatment assignment. Also, the association between the longitudinal and event outcomes is accounted for by the random effect that underlies both the longitudinal and survival processes for each subject. Therefore, although complex, time-varying SMART designs may require more complicated models for time and an extra layer of joint modeling, and as such one would expect a better performance from joint modeling in general. Nevertheless, both modeling approaches performed well in estimating the other parameters and the mean of the final outcomes for each adaptive intervention embedded in the corresponding designs. Furthermore, equation (1) is a general form of TVMEM, and in our study is equivalent to equations (2)~(4) at time points t = 0,T 1i , T 2i for each subject i. T 1i is a subjectspecific random variable, and coefficients in equation (3) can also be subject-specific. However, in practice,  modeling coefficients to be subject-specific may lead to the estimation of too many parameters which, in some scenarios, may not be identifiable, particularly with small sample sizes. Therefore, as an initial attempt, we modeled T 1i as a subject-specific random variable and the coefficients as fixed parameters. For example, coefficients β 0 (t), β 1 (t), β 3 (t) in equation (1) are fixed coefficients β 01 , β 11 , β 31 in equation (3), as model (1) is equivalent to submodel (3) at time point T 1i . More complicated models such as subject-specific and timevarying coefficients in submodels (2)-(4) can be considered, if the sample sizes are large. We also illustrated the effectiveness of the joint modeling approach in accurately estimating the parameters even when no specific model was assumed for the duration of the first-stage intervention, T 1i . The conclusions were qualitatively similar as that in the simulation where Weibull model was assumed for the duration of the first-stage intervention.
In the scenarios we considered here, the time at which individuals were re-randomized was assessed only for responders to the first-stage intervention. However, one may also consider varying times for the non-responders and for the second-stage interventions. For example, a non-responder showing severe side effects or no trend towards achieving intermediate goals may be rerandomized sooner than t 10 . The analytic approaches for such designs would be similar to the joint or time-varying mixed effects models proposed in this manuscript, for example, with an extra submodel for the duration of the second-stage interventions.
Instead of randomization with certain pre-defined probabilities (e.g., in the first two simulation scenarios, randomization with probability 0.5 was used for both stages; in the last two scenarios, unequal randomization with probabilities 0.4(0.6) and 0.55(0.45) was used for the two stages, respectively), information concerning potential moderators could be used to tailor and assign the interventions. For example, the choice of the first-stage intervention options could depend on the severity of the subject's smoking habit at the beginning of the study; whereas the choice of the second-stage intervention option could depend on the subject's adherence to the first-stage intervention. The analysis of such a randomization scheme would require assigning weights for each subject [37].
We also compared the cost and power associated with selecting the optimal embedded adaptive intervention for the proposed time-varying SMART design versus that for the analogous standard SMART design. Our simulation results showed similar power for the two designs. We used a linear cost function to assess the cost efficiency of the proposed design and found that it can have substantially lower cost than the standard design. Several other forms of cost functions can be used to assess cost efficiency. However, as long as the cost is an  increasing function of time, the proposed time-varying SMART design will have lower cost than the standard SMART design. Therefore, the time-varying SMART design can be used to study how the intensity and combination of two types of interventions might be adapted to a subject's progress in a cost-and time-efficient manner.
In our study, we assume that there is no unmeasured confounder. As suggested by Chakraborty and Murphy [38], the assumption of "no unmeasured confounders" holds in a SMART design if the randomization probabilities of A 1 at most depend on the baseline covariates, and the randomization probabilities of A 2 at most depend on the baseline covariates, the intermediate outcome, and A 1 . We performed additional simulations to investigate the role of unmeasured confounders on the parameter estimations. From these simulations, we see that when the unmeasured confounders affect only T 1 and Y 1 , the parameter estimation is still accurate (Additional file 4: Table S4). However, when these unmeasured confounders affect Y 2 , there is bias in the estimation of T 2 (Additional file 4: Tables S5-S6).
In the ADHD SMART study discussed by Nahum-Shani et al. [20], a weighted average was applied to the final outcomes when their primary goal of the study was to compare the imbedded adaptive intervention options in the SMART. In our Time-Varying SMART study, we used regression-based methods to identify more efficient adaptive decision rules for each subject along with their longitudinal outcomes. Similar to the analytic process of the standard SMART design by Q-learning in which a regression model for the outcome is postulated at each decision as a function of the patient's information to that point, our TVMEM in equation (1) is equivalent to submodels (2)-(4) at three time points of longitudinal outcomes for each individual. Therefore, we did not include weights in this study of the time-varying SMART design. However, for increased complexity of time-varying SMART designs, weights may be incorporated into the analysis in a future study to develop more robust estimations and results.