 Research
 Open Access
 Published:
Comparison of statistical models for estimating intervention effects based on timetorecurrentevent in stepped wedge cluster randomized trial using open cohort design
BMC Medical Research Methodology volume 22, Article number: 123 (2022)
Abstract
Background
There are currently no methodological studies on the performance of the statistical models for estimating intervention effects based on the timetorecurrentevent (TTRE) in stepped wedge cluster randomised trial (SWCRT) using an open cohort design. This study aims to address this by evaluating the performance of these statistical models using an open cohort design with the Monte Carlo simulation in various settings and their application using an actual example.
Methods
Using Monte Carlo simulations, we evaluated the performance of the existing extended Cox proportional hazard models, i.e., the AndersenGill (AG), PrenticeWilliamsPeterson TotalTime (PWPTT), and PrenticeWilliamsPeterson Gaptime (PWPGT) models, using the settings of several event generation models and true intervention effects, with and without stratification by clusters. Unidirectional switching in SWCRT was represented using timedependent covariates.
Results
Using Monte Carlo simulations with the various described settings, in situations where interindividual variability do not exist, the PWPGT model with stratification by clusters showed the best performance in most settings and reasonable performance in the others. The only situation in which the performance of the PWPTT model with stratification by clusters was not inferior to that of the PWPGT model with stratification by clusters was when there was a certain amount of followup period, and the timing of the trial entry was random within the trial period, including the followup period. In situations where interindividual variability existed, the PWPGT model consistently underperformed compared to the PWPTT model. The AG model performed well only in a specific setting. By analysing actual examples, it was found that almost all the statistical models suggested that the risk of events during the intervention condition may be somewhat higher than in the control, although the difference was not statistically significant.
Conclusions
When estimating the TTREbased intervention effects of SWCRT in various settings using an open cohort design, the PWPGT model with stratification by clusters performed most reasonably in situations where interindividual variability was not present. However, if interindividual variability was present, the PWPTT model with stratification by clusters performed best.
Background
A cluster randomised trial (CRT) is a randomised trial design in which a cluster of regions or sites is used when it is not possible or appropriate to assign an intervention to an individual patient, like a randomised controlled trial (RCT) [1, 2]. The stepped wedge CRT (SWCRT) is a type of CRT, in which multiple randomization procedures are enforced temporally to switch into intervention, and all clusters are sequentially transferred (unidirectional switch) from the control condition to the intervention condition [3, 4]. For the sake of simplicity, it is assumed that the intervention effect will persist from the time the control condition is switched to the intervention condition until the end of the trial.
There are three main types of SWCRT design: (i) continuous recruitment short exposure, (ii) closed cohort, and (iii) open cohort [5]. In the open cohort design, each subject is assessed repeatedly at a series of measurement points or at a subjectspecific time point, such as the occurrence of an event. In this design, subjects may get enrolled or censored the trial at any time during the trial period based on prespecified eligibility criteria. Thus, some subjects are exposed to both control and intervention conditions during the trial, while others are only exposed to one.
The INSPIRED trial, which is the actual example used in this study, was a multicentre SWCRT that examines whether a model of care that provides specialist palliative care interventions in residential care homes (i.e. the intervention condition) leads to fewer (acute care) hospitalisations and shorter lengths of stay in hospital for care home residents, when compared to usual care (i.e. the control condition) [6]. A schematic representation of actual example is presented in Fig. 1. It is an open cohort design as all the residents in each facility at the start of the trial and allnew enrolments to the facility after the start of the trial were included. Many residents were exposed to both the control and intervention conditions, as they remained in their residences continuously unless they died or were discharged from the care home. The primary outcome was the length of the hospital stays, and the secondary outcomes were the number of hospitalisations and the cost.
Some residents never experienced hospitalisation, while others were repeatedly hospitalised in the actual example. The same event repeatedly occurs over time to the same individual, such as the hospitalisation in the actual example, is called a recurrent event [7]. A common way to analyse recurrent event is the recurrence rate (average number of recurrences per unit time), which corresponds to the number of hospitalisations per facilitymonth as a secondary outcome in the actual example. This analysis requires the assumption that the incidence of hospitalisation is always constant in the interval per facilitymonth which is generally a strong assumption. In addition, even if the number of hospitalisations per facilitymonth are the same, there may be differences in the time it takes for each hospitalization to occur, and this is called the time to hospital admission (TTHA) and may represent the effects of the intervention. Since admission and discharge data are collected for each hospitalisation in the actual example, that is, the TTHA is measured repeatedly, it may be useful to evaluate hospitalisations as recurrent events within the framework of a timetoevent (TTE) analysis.
When assessing the impact of a covariate on the TTE with a hazard ratio (HR), the Cox proportional hazard (CoxPH) model is most often used [8], and it assumes that the event is a onetime terminal event. When the CoxPH model is applied to recurrent events, only timetofirstevent (TTFE) can be included in the analysis. Against this background, the extension of the CoxPH model to recurrent events has been actively pursued, especially in the 1980s [9,10,11], and it has mainly been used to evaluate the timetorecurrentevent (TTRE) in RCTs.
Methods to analyse TTE in SWCRT are currently unclear [12]. SWCRT using an open cohort design, by its nature, must deal with subjects who are exposed to both the control and intervention conditions (observed across the unidirectional switch). When estimating the intervention effects based on the TTFE, if the change in the timedependent covariate is independent of TTE, then the unidirectional switch in the CoxPH model can be explained using the timedependent covariate [13, 14], and methodological studies on the performance in the context of SWCRT have previously been conducted [15]. In TTRE, the existing extended CoxPH model with timedependent covariates possibly apply to SWCRT with unidirectional switching [16,17,18]. In addition, CRT is known to have a problem with cluster effects when the outcomes of individuals in the same cluster become similar for various reasons. In other words, it means that there will be an increase in the variability of regression coefficients among clusters, which is also a concern in SWCRT. When estimating intervention effects based on TTFE, the cluster effect in SWCRT can be treated using the CoxPH model stratified by clusters [15], as it assumes that each cluster’s baseline hazard function is different. For TTRE, the existing extended CoxPH model stratified by clusters possibly be used.
To our knowledge, there are currently no methodological studies on the performance of the statistical models for estimating intervention effects based on TTRE in SWCRT with an open cohort design, or examples of its application to actual studies. Investigating the performance of the statistical models used to estimate intervention effects based on TTRE in SWCRT using an open cohort design in various settings, may contribute to the selection of statistical models for the actual planning and analysis of SWCRT.
The purpose of this study was to evaluate “which statistical models resulted in better performance estimating intervention effects using TTRE in SWCRT with an open cohort design” with the Monte Carlo simulation (hereafter, simulation) in various settings. We also applied each statistical model to hospital admission data to test the actual example and interpreted the results based on the simulation results.
Methods
Actual example
Details of the trial design, interventions, resident background information, and efficacy results of the INSPIRED trial have been published previously [6]. The trial included 1700 residents from 12 care homes in Australia, of which 1089 (64.1%) were residents at the start of the trial, and the remaining 611 (35.9%) became residents after the start of the trial. There were 1149 hospitalisations during the trial, of which 943 hospitalizations of more than 24 h (> 24 h) were used for the primary outcome, length of stay in hospital. Of the residents, 377 had only one hospitalization of > 24 h, while 211 had multiple hospitalizations of > 24 h (137 had two, 45 had three, 11 had four, and 18 had four or more). The number of residents who died during the trial period was 534 (31.4%). The secondary outcome, number of hospitalizations > 24 h per facilitymonth, was 5.6 in the control condition and 4.3 in the intervention condition, a decrease of approximately 23% (no adjustment by covariates or comparison by estimation/statistical testing was performed).
Basic notation
The timing of the unidirectional switch (henceforth, switch) in each cluster of the SWCRT is called a step, and here, we consider SWCRT with \(m\) clusters and \(s\) steps. For simplicity, we assume that the number of clusters to be switched from the control condition to the intervention condition in one step is one (\(s=m\)). In the \(i\) th cluster (\(i=1,\:\dots ,\:m\)), \({n}_{i}\) is the number of subjects observed during the entire trial duration.
Assuming that the start of the test is \({t}_{S}\) and the end of the last step period is \({t}_{E}\), the timing of the switch in each cluster is calculated as follows: \({W}_{i}={t}_{S}+i*({t}_{E}{t}_{S}) / (m+1)\), and the distance between switches is calculated as follows: \({W}_{d}={W}_{i+1}{W}_{i}={W}_{i}{W}_{i1}=({t}_{E}{t}_{S}) / (m+1)\). Let \({d}_{ij}\) be the time point at which the \(j\) th subject (\(j=1,\: \dots ,\: {n}_{i}\)) in the \(i\) th cluster entered the trial. The distance \({w}_{ij}\) to the switch for each subject from the trial entry is defined as follows:
Suppose the starting point of the second and subsequent TTREs is the time of the previous event, and the actual TTRE used in the analysis based on the kth recurrence is \({T}_{ijk}\). Considering the starting point of each recurrence, the distance \({w}_{ijk}\) to the switch for each subject is defined as follows:
where \({h}_{ijk}\left(t\right)\) is the hazard function of the \(k\) th recurrence of the \(j\) th subject in the \(i\) th cluster at time \(t\), and \({h}_{0ik}\left(t\right)\) is the baseline hazard function of the \(k\) th recurrence of the \(i\) th cluster at time \(t\). No specific distribution is assumed for the baseline hazard function. \({Y}_{ijk}\left(t\right)\) is the indicator variable for the \(k\) th recurrence of the \(j\) th subject in the \(i\) th cluster at time \(t\), and this is 1 if the subject is at risk of recurrence and under observation, and 0 if not. \({X}_{ijk}\) is a vector of timeindependent covariates for the \(k\) th recurrence of the \(j\) th subject in the \(i\) th cluster, and \({\beta }_{ik}\) is a vector of fixed parameters for the timeindependent covariates of the \(k\) th recurrence of the \(i\) th cluster. \({Z}_{ijk}\left(t\right)\) is the intervention indicator as a timedependent covariate for the \(k\) th recurrence of the \(j\) th subject in the \(i\) th cluster, which is 0 for \({t<{w}_{ij} \mathrm{\:or\: }w}_{ijk},\) and 1 for \({t\ge {w}_{ij} \mathrm{\:or\:} w}_{ijk}\) (changes before and after the switch). \({\beta }_{tik}\) is the parameter for the intervention effect for the \(k\) th recurrence of the \(i\) th cluster. The subscript \(i\) is omitted if it is assumed that each cluster has a common effect. The subscript \(k\) is omitted if it is assumed that each recurrence has a common effect.
Statistical models
The first model considered was the CoxPH model [8, 14]. The hazard of the \(j\) th subject in the \(i\) th cluster at time \(t\) is expressed as follows:.
As was previously mentioned, applying the CoxPH model to recurrent events would result in a loss of information because only the TTFE of each subject can be included in the analysis, and the second and subsequent events are ignored. Taking recurrent events into account should theoretically improve the efficiency of estimating the effects of interventions [19]. Since the purpose of this study is to evaluate the performance of the statistical model in estimating the intervention effect using TTRE, no performance evaluation on the CoxPH model will be conducted. In the following, we present an extended CoxPH model that allows for the inclusion of TTRE in the analysis.
The Andersen and Gill (AG) model assumes a common baseline hazard function for all events, independent of the number of previous recurrences, and it is considered beneficial when investigating the overall intervention effect on the occurrence of recurrent events [9]. The hazard for the \(j\) th subject in the \(i\) th cluster at time \(t\) is expressed as follows:
In the usual CoxPH model, a subject who has experienced one event is no longer at risk for that event. In contrast, the AG model assumes that subjects who have experienced at least one event remain at risk unless they drop out of the trial. In the AG model, multiple events that occur within the same subject are considered to be independent. However, because they may not be independent in reality, it is advised that robust variance is used to handle the correlation within the subject when inferring the parameter vector [20, 21].
The PrenticeWilliamsPeterson (PWP) model assumes a different baseline hazard function for each recurrence and accounts for correlation by stratifying by the number of prior recurrences. Therefore, it is considered beneficial when the risk of repeat events differs between recurrences [17]. The hazard \({h}_{ijk}\left(t\right)\) for the \(k\) th recurrence is defined by the history of the covariates and the number of recurrences up to time \(t\). Conditionally, it is assumed that the (\(k1\))th recurrence is independent of the \(k\) th recurrence. Furthermore, it assumes that the subject is not at risk for the \(k\) th recurrence until the (\(k1\))th recurrence, so that \({Y}_{ijk}\left(t\right)\) is 0 until the (\(k1\))th recurrence and 1 after that.
The PWP model can be broadly divided into two models depending on the treatment of the time points. First, the PWP totaltime (PWPTT) model uses the time from the start of the followup to each recurrence. The hazard of the \(k\) th recurrence of the \(j\) th subject in the \(i\) th cluster at time \(t\) is expressed as follows:
The second is the PWP gaptime (PWPGT) model, which uses the time from the occurrence of the previous recurrence to each recurrence. The hazard of the \(k\) th recurrence for the \(j\) th subject in the \(i\) th cluster at time \(t\) is expressed as:
As the number of recurrences increases in the PWP model, the number of subjects at risk becomes relatively small. This would make the estimates unstable, so limiting the data to a specific number of recurrences is usually necessary [22]. Due to these characteristics, the PWP model is helpful in situations where the number of recurrences per subject is small [17]. Our study assumes that each recurrence has a common effect when estimating parameters using the PWP model.
For each of the statistical models described so far, there are two analysis policies: (i) with stratification by clusters, which assumes that the baseline hazard function is different for each cluster, and (ii) without stratification by clusters, which assumes that the baseline hazard function is the same for each cluster.
The performance of each statistical model in the simulation was evaluated in terms of bias, mean square error (MSE), and coverage probability (CP). Bias is the mean difference across simulated replicates of the parameters of the intervention effect based on each statistical model and the true intervention effect \({\beta }_{t}\), where a positive value indicates underestimation and a negative value indicates overestimation; MSE is the sum of bias squared and variance of the estimated intervention effect based on each statistical model, with smaller values indicating better performance. CP is the proportion of the 95% confidence interval (CI) for the HR obtained by each statistical model that includes the HR based on the true intervention effect \({\beta }_{t}\). The closer the CI is to 0.95, the better the performance.
Data generation process
For the time point \({d}_{ij}\) of the \(j\) th subject in the \(i\) th cluster to enter in the trial, we use \({t}_{S}\) at the beginning of the trial and \({t}_{E}\) at the end of the last step period already mentioned, and generate them randomly within the interval of \({t}_{S}+(\left({t}_{E}{t}_{S}\right)*e)/E\) or \({t}_{S}+(\left({t}_{F}{t}_{S}\right)*e)/E\). From this point, the TTFE at least, always occurs starting from \({d}_{ij}\). Here, \(e\) is a pseudorandom number generated from a uniform distribution, \(e\sim U(0, 1)\).
\({t}_{F}\) indicates the end of the trial and is expressed as \({t}_{F}={t}_{E}+({W}_{d}*F)\) using the distance \({W}_{d}\) between \({t}_{E}\) and the switch at the end of the last step period, as described above. \(F\) is a coefficient that specifies the followup period that may be set after the end of the last step period. When \(F=0\), there is no followup period, and \({t}_{F}={t}_{E}\). If \(F=X(>1)\), there is a followup period of \(X\) step after the end of the last step period. In the actual example, as shown in Fig. 1, each step is set every two months, and there is a followup period of 5 months (= 2.5 steps) after the end of the last step period. Based on the purpose and setting of the trial, other SWCRT have adopted a similar design [23,24,25].
In the actual simulation, three policies are considered: (i) no followup period and \({d}_{ij}={t}_{S}+(\left({t}_{E}{t}_{S}\right)*e)/E\); (ii) there is a followup period and \({d}_{ij}={t}_{S}+(\left({t}_{F}{t}_{S}\right)*e)/E\) (allow trial entry until the end of the followup period; illustrated in Fig. 2a); (iii) there is a followup period but \({d}_{ij}={t}_{S}+(\left({t}_{E}{t}_{S}\right)*e)/E\) (terminate trial entry at the end of the last step period; illustrated in Fig. 2b).
In addition, \(E\) is a coefficient that specifies the timing of the trial entry. If \(E=1\), the subject enters the trial randomly between \({t}_{S}\) and \({t}_{E}\) or \({t}_{F}\), which reflects the open cohort design in that the subject may enter in the trial at any time. If \(E\) is greater than 1, it reflects a situation where the entry of the trial is concentrated at an earlier stage of the trial (illustrated in Fig. 2c). In the actual example, 64.1% of the residents entered at the start of the trial. Depending on the purpose and setting of the trial, other SWCRT show similar situations [26, 27].
In the actual simulation, policies (i) to (iii) above regarding the followup period and the time of trial entry can be taken for \(E=1\) and \(E>1\), respectively. Our study adopts only policy (iii) instead of (ii) at \(E>1\) (illustrated in Fig. 2d).
To compare our results with the secondary outcome of the actual example, number of hospitalisations > 24 h per facilitymonth, we decided to treat only hospitalizations > 24 h as a TTE in this study. It was previously published [6] that the number of residents repeatedly hospitalised more than four times was very small. Therefore, in our study, the maximum number of recurrent events generated in the simulation was three.
The relative performance of the statistical models used in TTRE, which are based on bias and variability, depend on the event generation model used in the simulation, and it is thus recommended that simulations based on multiple event generation models be considered [28]. Therefore, in this study, three types of event generation model were used.
The first is the Poisson process, which generates TTEs based on exponential distributions independent of each other, not only between subjects but also within subjects. The exponential distribution consists only of scale parameter. The starting point of all TTEs is \({d}_{ij}\) at the time of trial entry, and the hazard of a TTE is always constant, regardless of the time and number of recurrences (illustrated in Fig. 3a).
The second model uses the same Poisson process as the first one, but adopts the exponential distribution with different scale parameters between the subjects using random effect (i.e., interindividual variability exists). It is referred to as the MixedPoisson process.
The third is the Weibull model, where the starting point of the first TTE is \({d}_{ij}\), as in the Poisson process, but the starting point of the second and subsequent TTEs is the time of the previous event (illustrated in Fig. 3b). Then, a Weibull distribution was assumed for the time between events within each subject. In addition to a scale parameter similar to an exponential distribution, the Weibull distribution contains the shape parameter. The Weibull distribution allows the hazard to vary with time depending on the setting of the shape parameter. As this model adopts a Weibull distribution with a common parameter from the first to the third TTE (i.e. the way the hazard changes are common from the first to the third TTE), we refer to it as the Weibull model (constant).
The fourth model uses the same Weibull model as the second one, but adopts the Weibull distribution with different parameters between the “first TTE” and the “second and third TTE” (i.e., the way the hazard changes is different between the first and second and third TTEs), and so it is referred to as the Weibull model (change).
In a simple RCT situation where an intervention effect exists, previous studies with timeindependent covariates have shown that both the AG and PWPTT models perform well for the Poisson process. On the other hand, it has been shown that only the PWPTT model performs well for the Weibull model (constant), and only the AG model performs well for the MixedPoisson process [28].
To generate TTREs that can account for unidirectional switching, which is assumed to be estimating intervention effects using the CoxPH model and several extended CoxPH models, we use a data generation process for the CoxPH model with timedependent covariates, based on the three event generation models previously described [29]. If the generated TTRE exceeds \({t}_{E}\) or \({t}_{F}\), it is treated as rightcensored at \({t}_{E}\) or \({t}_{F}\).
In the generation of TTRE in the Poisson process and the MixedPoisson process, three pseudorandom numbers were generated independently from the uniform distribution \(U(0, 1)\) and sorted in increasing order, \({u}_{1},\: {u}_{2},\: {u}_{3}\) in turn (\({u}_{k},\: k=1,\: 2,\: 3\)). If the scale parameter of the exponential distribution is \(\lambda\), the baseline hazard function is \(\lambda\), which is always constant regardless of the time or number of recurrences. The \(k\) th TTRE of the \(j\) th subject in the \(i\) th cluster, when the starting point is not considered, is as follows:
where \({\tau }_{i}\) and \({\tau }_{j}\) is the random effect on the variations between clusters and between subjects, \({\tau }_{i}\sim N\left(0, {\sigma }^{2}\right)\) and \({\tau }_{j}\sim N(0, {\sigma }_{s}^{2})\). \({\sigma }_{s}^{2}\) is 0 for the Poisson process and > 0 for the MixedPoisson process.
As already mentioned, \({\beta }_{tik}\) is the parameter of the intervention effect on the \(k\) th recurrence of the \(i\) th cluster, and \({w}_{ij}\) is the distance to switch for each subject from the trial entry. For simplicity, we omitted the \({\beta }^{^{\prime}}x\) for the timeindependent covariates in the simulation. The TTRE, which is used in the analysis considering the starting point, is represented by \({T}_{ijk}={d}_{ij}+{T}_{ijk}^{*}\).
In the generation of TTRE in the Weibull model, three pseudorandom numbers were generated independently from the uniform distribution \(U(0, 1)\), \({u}_{1},\: {u}_{2},\: {u}_{3}\) in the order in which they are generated (\({u}_{k},\: k=1,\: 2,\: 3\)). Let the scale parameter of the Weibull distribution for each recurrence be \({\lambda }_{k}\), and the shape parameter be \({\nu }_{k}\). The baseline hazard function is \({\lambda }_{k}{\nu }_{k}{t}^{{\nu }_{k}1}\) and it is allowed to vary with time. The \(k\) th TTRE of the \(j\) th subject in the \(i\) th cluster, when the starting point is not considered, is as follows:
\({\tau }_{i}\), \({\beta }_{tik}\), \({w}_{ijk}\), and \({\beta }^{^{\prime}}x\) were explained in the previous sentence. The TTRE that is actually used for the analysis considering the starting point is:
The parameters are \({\lambda }_{1}={\lambda }_{2}={\lambda }_{3}, {\nu }_{1}={\nu }_{2}={\nu }_{3}\) for the Weibull model (constant), and \({\lambda }_{1}\ne {\lambda }_{2}={\lambda }_{3}, {\nu }_{1}\ne {\nu }_{2}={\nu }_{3}\) for the Weibull model (change).
In the actual example, 31.4% of the residents died during the trial period. Therefore, in our simulation, we considered the timetoterminalevent (TTTE) as independent of the distance to switch and TTRE. If the generated TTTE does not exceed \({t}_{E}\) or \({t}_{F}\) and it is before the third TTRE, it is treated as midtrial rightside censoring at the occurrence of the terminal event. The scale parameter of the Weibull distribution for the terminal event is \({\lambda }_{c}\), and the shape parameter is \({\nu }_{c}\). Without considering the starting point, the TTTE of the \(j\) th subject in the \(i\) h cluster, \({C}_{ij}^{*}\), can be expressed using the probability density function as follows:
The TTTE used in the actual analysis considering the starting point is expressed as \({C}_{ij}={d}_{ij}+{C}_{ij}^{*}\).
Parameter settings
The scale parameter for the exponential distribution in the generation of the TTRE by the Poisson process was set to \(\lambda =0.003281\). This parameter was estimated based on the TTHA up to the third of the actual example, with all starting points set to zero. In addition, the interindividual variability of the scale parameter in the generation of the TTRE by the MixedPoisson process was set to \({\sigma }_{s}^{2}=0.3455\). For this parameter, we used an estimate of the standard deviation of the normal distribution for the scale parameter based on the TTHA.
The scale and shape parameters of the Weibull distribution in the generation of TTRE using the Weibull model (constant) were set to \({\lambda }_{1}={\lambda }_{2}={\lambda }_{3}=0.004703,\: {\nu }_{1}={\nu }_{2}={\nu }_{3}=1.1219\). These parameters were estimated based on the TTHA, up to the third of the actual example. The starting point of the second and subsequent TTHA was the time of the previous hospitalisation.
The scale and shape parameters of the Weibull distribution in the generation of the TTRE using the Weibull model (change) were set to \({\lambda }_{1}=0.003599,\: {\lambda }_{2}={\lambda }_{3}=0.009910,\: {\nu }_{1}=1.5122,\: {\nu }_{2}={\nu }_{3}=0.9108\). These parameters were estimated based on the “first TTHA” and the “second and third TTHA” of the actual example, respectively. The starting point of the second and subsequent TTHAs was the time of the occurrence of the previous hospitalisation.
The scale and shape parameters of the Weibull distribution in the generation of TTTE as midtrial rightside censoring were set to \({\lambda }_{c}=0.003674\) and \({\nu }_{c}=1.7191\). These parameters were estimated based on the time to death in the actual example.
Two parameters were set for the true intervention effect. The first is \({\beta }_{tik}={\beta }_{t}=0.264\), which was calculated as \(\mathrm{ln}(4.3/5.6)\) based on the secondary outcome of the actual example, number of hospitalisations per facility month. The second is \({\beta }_{tik}={\beta }_{t}=0\), a setting used in previous studies on event generation models: HR = 1, which indicates that there is no difference in the risk of event occurrence between the control and intervention conditions. In a simple RCT situation where there is no intervention effect, both the AG and PWPTT models have been shown to perform well, regardless of the type of event generation model [28].
Simulation setup
For all simulations, we fixed \({t}_{S}=0\) at the beginning of the trial, \({t}_{E}=360\) at the end of the last step period, and the total sample size per simulation (total number of subjects per trial) \(N=2000\). These settings were based on the fact that the actual example lasts for 12 months from the start of the trial to the end of the final step period; if one month is considered to be approximately 30 days, the trial period can be calculated as 12 × 30 = approximately 360 days, and the total number of subjects was 1700. Unless otherwise noted, the basic settings for each simulation scenario are as follows: the number of simulations is 1000, the event generation model consists of three types (Poisson process, Weibull model (constant), Weibull model (change)), the parameters of the true intervention effect are two ways (\(0.264,\: 0\)), and \(s\left(=m\right)=5,\: {n}_{i}=n=N/m=400,\: {W}_{d}=({t}_{E}{t}_{S}) / (m+1)=60,\: {\sigma }^{2}=0,\: E=1,\: F=0\). The setting of \(s=m=5\) is in reference to the fact that the number of steps in the actual example is five (Fig. 1).
Each simulation scenario is listed below. Scenario II applied two policies for each statistical model: (i) with stratification by clusters and (ii) without stratification by clusters. In all scenarios, except for scenario II, only (i) was applied.
In Scenario I, the number of steps (clusters) varied as \(s\left(=m\right)=2, 4, 5, 8, 10, 20\) to investigate how the performance of each statistical model changed as the number of steps (clusters) increased. As the number of steps changes, it becomes \(n=N/m=1000, 500, 400, 250, 200, 100,\: {W}_{d}=120, 72, 60, 40, 33, 17\). The results based on \(s\left(=m\right)=5,\: n=400,\: {W}_{d}=60\) in this scenario were used as a reference throughout the simulations in our study.
In Scenario II, we varied the variance with respect to the random effect \({\tau }_{i}\), which represents the variation among clusters, as \({\sigma }^{2}=0.25,\: 0.5,\: 1\), and investigated how the performance of each statistical model changed as the variation between clusters increased.
In Scenario III, the followup period varied as follows, \(F=1, 2, 3, 4\) to investigate how the performance of each statistical model changed as the followup period increased. The setting of \(F\) is based on the followup period of 2.5 steps in the actual example (Fig. 1). In this scenario, the time point of the trial entry point was \({d}_{ij}={t}_{S}+(\left({t}_{F}{t}_{S}\right)*e)/E\), and the subject was allowed to enter until the end of the followup period.
In Scenario IV, the followup period was changed to \(F=1, 2, 3, 4\) to investigate how the performance of each statistical model changed as the followup period increased. In this scenario, the time point of the trial entry point was \({d}_{ij}={t}_{S}+(\left({t}_{E}{t}_{S}\right)*e)/E\), and entry was terminated at the end of the final step period.
In Scenario V, we varied the timing of the trial entry as follows, \(E=1.5,\: 2,\: 4,\: 6\) to investigate how the performance of each statistical model changed as trial entry was concentrated at an earlier stage of the trial.
In Scenario VI, the time of trial entry varied as follows, \(E=1.5,\: 2,\: 4,\: 6\), and the followup period was changed to \(F=1, 2, 3, 4\), to investigate how the performance of each statistical model changed in a situation where trial entry was concentrated in an earlier stage of the trial, and there was a followup period. In this scenario, for convenience, we used \({d}_{ij}={t}_{S}+(\left({t}_{E}{t}_{S}\right)*e)/E\) as the time point for trial entry.
Analysis of an actual example
The timeindependent covariates employed in the model analysis for the primary outcome in the actual example (age, sex, medical power of attorney, health directive, advance care plan/statement of choices, primary diagnosis, ageadjusted Charlson comorbidity index, and fidelity) were used for adjustment, when analysing hospitalization > 24 h repeatedly occurred with the TTRE in the actual example using each statistical model.
Two policies were applied to each statistical model: (i) with stratification by clusters and (ii) without stratification by clusters. Fidelity is a percluster variable and was employed only with policy (ii), as it is not available for adjustment in (i). The unidirectional switch from the control condition to the intervention condition in each cluster was expressed using the intervention indicator as a timedependent covariate.
In the usual TTRE analysis, continuous risk intervals were employed. However, in reality, they are not exposed to the risk of further hospitalisation during their hospital stay. Therefore, in this study, we adopted a discrete risk interval [30]. Thus, for example, if a resident was hospitalised, subsequent exposure to the risk of new hospitalisation would be from the day of discharge.
The results of the analysis were evaluated using HR and its 95% CI and pvalue. In addition, parameter estimates and standard error (SE) were evaluated for the intervention effects.
Software and code
All statistical analyses, including simulations, were performed using SAS, version 9.4 (SAS Institute, Cary, NC, USA). The PROC PHREG of SAS was used to analyse the TTRE. For the generation of pseudorandom numbers by SAS, the RANUNI function was used to generate the time point of trial entry and TTRE, the RANNOR function was used to generate the cluster effect, and the RAND function was used to generate the TTTE. For information on simulation codes, see Availability of data and materials.
Results
Simulation
The reference results for Scenario I with \(s\hspace{1.5em}\left(=m\right)=5,\:n=400,\:W_d=60\) are shown in Table 1. These results were used as a reference for all the other simulations assessed in this study, as the setting \(s \left(=m\right)=5\) references the fact that the number of steps in the actual example is five (Fig. 1).
From the reference results for \({\beta }_{t}=0.264\), the MSE under the Poisson process and the MixedPoisson process was smaller for the AG and PWPTT models, and slightly larger for the PWPGT model; the CP performances of the AG and PWPTT models were similar, but the bias was much smaller for the PWPTT model. The PWPGT model performed very well in both the Weibull model (constant) and Weibull model (change) but showed much lower performance in the MixedPoisson process. Under the Weibull model (change), the performance of the AG model was found to be very poor. In reference to the results for \({\beta }_{t}=0\), the overall performance was higher than that of \({\beta }_{t}=0.264\). The AG model under the Weibull model (change) tended to overestimate CP. In all event generation models, the PWPTT and PWPGT models showed similar results, but the bias of the PWPGT model in the MixedPoisson process was slightly larger than all other combinations.
The results for Scenario I when the parameter for the true intervention effect is \({\beta }_{t}=0.264\) are shown in Table 2, and the results when \({\beta }_{t}=0\) are shown in Additional File (S.1). Regardless of the setting for \({\beta }_{t}\), the overall MSE increased slightly as the number of steps (clusters) increased, but this did not substantially impact on the performance comparison between the statistical models.
The results for Scenario II when the parameter for the true intervention effect is \({\beta }_{t}=0.264\) are shown in Table 3, and the results when \({\beta }_{t}=0\) are shown in Additional File (S.2). Regardless of the setting of \({\beta }_{t}\), the performance of policy (ii) without stratification by clusters decreased as intercluster variation increased. At \({\sigma }^{2}=0.25\), the lowest variance in the setting, the decrease in performance was already apparent, especially for CP, as the performance was very poor. The reference results where policy (i) with stratification by clusters was performed in the absence of intercluster variation were similar to the results when (i) with stratification by clusters was performed in this scenario where intercluster variation was present.
The results for Scenario III, when the parameter for the true intervention effect is \({\beta }_{t}=0.264\) are shown in Table 4, and the results when \({\beta }_{t}=0\) are shown in Additional File (S.3). When \({\beta }_{t}=0.264\), the performance of the AG and PWPTT models under the Weibull model (constant) and the PWPTT model under the Weibull model (change) improved as the followup period increased, when the trial entry was allowed until the end of the followup period. In particular, for CP, the performance was comparable to that of the PWPGT model under the respective event generation model. On the other hand, the performance of the MixedPoisson process tended to be less than or equal to that of the reference results.
The results for Scenario IV when the parameter for the true intervention effect was \({\beta }_{t}=0.264\) are shown in Table 5, and the results when \({\beta }_{t}=0\) are shown in Additional File (S.4). When \({\beta }_{t}=0.264\), the performance of the AG and PWPTT models under the Weibull model (constant) and the PWPTT model under the Weibull model (change), improved as the followup period increased, given the policy of terminating trial entry at the end of the final step period. However, none of them reached the same level of performance as the PWPGT model in their respective event generation models. In contrast, the performance of the PWPGT model under the Poisson process tended to decrease as the followup period increased. In addition, the performance of the MixedPoisson process tended to be less than or equal to that of the reference results.
The results for Scenario V when the parameter for the true intervention effect is \({\beta }_{t}=0.264\) are shown in Table 6, and the results when \({\beta }_{t}=0\) are shown in Additional File (S.5). Regardless of the setting of \({\beta }_{t}\), there was a tendency for the overall MSE to increase as the trial entry was more concentrated at the beginning of the trial. When \({\beta }_{t}=0.264\), for the PWPGT model under the Poisson process, the AG and PWPTT models under the Weibull model (constant), and the PWPTT model under the Weibull model (change), CP always performed poorly when compared to the reference results, regardless of the value for \(E\). On the other hand, in the MixedPoisson process, Bias tended to decrease in the AG and PWPTT models and increase in the PWPGT model as the trial entry was more concentrated at the beginning of the trial.
The results for Scenario VI, when the parameter for the true intervention effect is \({\beta }_{t}=0.264\), are shown in Additional File (S.6), and the results when \({\beta }_{t}=0\) are shown in Additional File (S.7), respectively. The results are similar to those of Scenario V, regardless of the setting of \({\beta }_{t}\) or the value of \(F\).
Actual example
The results summarising only the intervention indicators as timedependent covariates are shown in Table 7. The overall results, including the timeindependent covariates used for adjustment, are shown in Additional File S.8.
The HR for the intervention indicator shows the relative risk of the intervention condition when compared to the control. Except for the PWPTT model under policy (i) with stratification by clusters, the overall HR was slightly above 1, suggesting that the risk of events in the intervention condition may be higher than in the control, although the difference was not statistically significant. Reviewing the results of the statistical model, under policy (ii) without stratification by clusters, the HR tended to be larger, and the range of the SE and 95% CI was smaller than under policy (i) with stratification by clusters.
The results of the covariates other than the intervention indicator, showed that the primary diagnosis of “dementia and Parkinson’s disease”, and the ageadjusted Charlson comorbidity index were statistically significant for all statistical models. Residents with dementia and Parkinson’s disease had a lower risk of event occurrence than those without dementia and Parkinson’s disease, suggesting that the risk of event occurrence may increase with the severity of comorbidities.
Discussion
In this study, we have conducted comparative simulations to identify the statistical model’s whose performance for estimating intervention effects based on TTRE in SWCRT using an open cohort design were superior and could effectively be applied to actual clinical trial data.
The results of the simulations show that the performance under policy (ii) without stratification by clusters was worse when compared with policy (i) with stratification by clusters, in both the statistical models and settings. As SWCRT is implemented at the cluster level, it is necessary to consider that “cluster effects may exist” in any situation. Furthermore, even if there is no variation among the clusters, there is no difference in performance with and without stratification by clusters, so (i) with stratification by clusters should always be adopted in the estimation of intervention effects based on TTRE in SWCRT when using an open cohort design.
The results of the simulations, in a situation where there is no followup period, and the timing of the trial entry tends to be random, showed that Poisson processes were similar to those of previous studies in settings that did not include timedependent covariates [28]. We found that the performance of the PWPTT model decreases for the Weibull model (constant) and increases for the MixedPoisson process, which is somewhat different from previous studies. This is a tendency that is considered to be specific to SWCRT with an open cohort design.
In realworld SWCRT, there may be situations in which a followup period is established, or trial entry is concentrated in the early period, due to the nature of the study objectives and target clusters. The simulation results are important because they show that the performance of the statistical models against TTRE depends not only on the true intervention effects and event generation model, but also on the trial design of SWCRT (the presence of a followup period and the timing of trial entry).
The MixedPoisson process is an eventgenerating model that induces interindividual variability in the Poisson process. Overall, in simulations in the presence of intervention effects, the bias in all statistical models was positively larger in the MixedPoisson process than the Poisson process; that is, it tended to underestimate the intervention effects. This is a similar result to simulations in previous studies [22]. In addition, the simulation as a whole tended to significantly degrade the performance of PWPGT, especially in the MixedPoisson process with interindividual variability, compared to the Poisson process and Weibull model without interindividual variability. Since PWPGT is the only one that is assumed to be “gaptime independent”, the result that the MixedPoisson process with no gaptime independence degrades performance is natural.
The event generation model used in our study was only a simulation assumption. The primary analysis methods in the clinical trials usually need to be specified in advance in the study protocol or statistical analysis plan. If the policy is to adopt a statistical model for the primary analysis, and it needs to determine a statistical model in the early phase of trial planning, it would be desirable to adopt one that shows reasonable performance in various settings, rather than one that performs well only in a particular event generation model. In our study, through simulations based on various settings, the PWPGT model with stratification by clusters showed the best performance in most settings and reasonable performance in other settings, in situations where interindividual variability did not exist. On the other hand, the PWPGT model with stratification by clusters consistently underperformed compared to the PWPTT model with stratification by clusters, in situations where interindividual variability existed. Therefore, if the policy is to adopt a statistical model as the primary analysis, and this needs to be determined in the early phase of the trial planning, the PWPTT model with stratification by clusters should be adopted if the interindividual variability is known to be high from previous studies, and the PWPGT model with stratification by clusters should be adopted if it is not.
Under the Weibull model (change), the overall performance of the AG model tended to be very low when intervention effects were present, and the CP of the AG model tended to be excessive when there were no intervention effects. The AG model assumes a common baseline hazard function for all events, independent of the number of previous recurrences. In contrast, in the Weibull model (change), the hazard clearly changes between the first event and the second and subsequent events. Therefore, it is to be expected that the performance of the AG model degrades under the Weibull model (change), theoretically. However, we believe that further research is needed on the cause of the terrifically low performance of CP. Anyway, considering the possibility that the actual event generation model is a Weibull model (change), it is challenging to adopt the AG model during the early phase of trial planning.
Under conditions where interindividual variability does not exist, the only situation in which the performance of the PWPTT model with stratification by clusters is not inferior to that of the PWPGT model with stratification by clusters is when there is a certain amount of followup period, and the timing of the trial entry tends to be random within the trial period, including the followup period. Therefore, in this situation, it may be acceptable to adopt the PWPTT model with stratification by clusters during the early phase of the trial planning, instead of the PWPGT model with stratification by clusters. In our study, the performance of the PWPTT model was particularly good when the followup period was more than three steps. In addition, considering that the original trial period consisted of six steps (\(s=m=5\)), it may be possible to think of it as a rough guide that “a certain amount of followup period” as “a followup period that is more than half the length of the original trial period”. The results presented in Additional File (S.9) indicate that it can be assumed that the same is true for different numbers of steps (clusters). The choice of which statistical model to use depends on the nature of the intervention, the characteristics of the subjects, and the clinical interpretability of the analysis results. In our study, for the sake of comparability, we estimated only the overall effects based on the PWP model, assuming that each recurrence had a common effect. However, in an actual analysis, it is possible to estimate eventspecific effects. The PWPTT model is appropriate when one wants to know the effect of each recurrence since the start of the subject’s followup. On the other hand, the PWPGT model is suitable for understanding the effect of recurrence, in relation to the previous occurrence.
A previous study on the CoxPH model in the context of SWCRT showed a tendency for the MSE to decrease as the number of steps (clusters) increased. However, the simulations in our study showed an opposite trend. This difference is not apparent, but it is thought to be due to the differences in the various settings during the simulation. For example, in the previous study, the true intervention effect was set to 1, whereas in our study, it was set to − 0.264 or 0.
The observed numbers of recurrence per subject and the censoring proportions will be different depending on the scenario. Also, a previous study that evaluated the performance of the CoxPH model in SWCRT mentioned that the controltointervention ratio (the ratio of the total time in the control condition to the total time in the intervention condition) is related to the estimation accuracy [15]. A summary of this information for each scenario is given in Additional file (S.10) when the true intervention effect parameter is \({\beta }_{t}=0.264\), and a summary for \({\beta }_{t}=0\) is given in Additional file (S.11). It may be helpful to take this information into account to interpret the simulations’ results for each scenario.
There is a followup period in the actual example, and trial entry is concentrated early in the trial period. Therefore, based on the results of the simulations, if the policy is to adopt a statistical model for estimating intervention effects based on TTRE against the actual example as the primary analysis, and this needs to be determined in the early phase of trial planning, the PWPTT model with stratification by clusters should be adopted if the interindividual variability is known to be high from previous studies, and the PWPGT model with stratification by clusters should be adopted if it is not. However, considering that the parameter estimates are close to zero for any statistical models, if one model is adopted as the primary analysis, the other might be adopted as the exploratory analysis. The number of hospitalizations per facilitymonth, was evaluated as a secondary outcome in the actual example and showed an obvious decrease in the intervention condition when compared to the control, which is a substantial deviation from the results from the TTRE analysis of our study. One possible reason for this is that the analysis of the number of hospitalizations per facilitymonth ignores that residents are exposed to both the control and intervention conditions. The purpose of our study was to provide a different perspective to the existing evaluations. Therefore, it does not negate the conclusions of the actual example, which have previously been published.
Our study had several limitations. First, all of the statistical models employed treat a terminal event before the third TTRE as a midtrial censoring event. However, if a death occurs, for instance, in actual example, the possibility of a subsequent hospitalisation is lost. An event such as a death in such a situation is called a competing risk [31], but in our study, we did not account for terminal events as competing risks.
Second, we assumed noninformative censoring for the terminal event, which was treated as midtrial censoring. This assumes that censoring occurs independently due to causes unrelated to the TTRE. However, if, for example, repeated hospitalisations occur in an actual example, the risk of death is likely to increase. In such situations, it is possible to use an approach that considers the terminal event as informative censoring and corrects for it, but this was not applied [32, 33].
Third, the simulation in our study employed continuous risk intervals as it has been adopted in many previous studies [19, 22, 34]. However, we believe that simulations for discontinuous risk intervals (adopted in the analysis of the data from actual example) should be considered in the future.
Fourth, for simulation simplicity, we assumed that the number of clusters moving from the control condition to the intervention condition in one step was one (\(s=m\)). However, in actual example, two or three care homes are included in one cluster that transitions in one step. If the intervention effects can be assumed to be common among multiple care homes within a cluster, this is not an issue. If they cannot, they should be considered in the analysis, but we were not able to do this in our study.
Fifth, we adopted only “stratification by clusters” to handle the cluster effect. In our study, we were more interested in the differences in performance due to the differences in the design of SWCRT itself rather than the differences in performance due to the way the cluster effect is handled. In a previous study that evaluated the performance of the CoxPH model in SWCRT, both “stratified by cluster” and “frailty” were used for the cluster effect, and no difference in performance was found [15]. Based on the results, the “frailty” method, which is computationally expensive and takes a lot of time, was not included in our study from the beginning. However, stratification does not allow us to infer the effects of clusterlevel variables (such as “fidelity” in actual example). Also, frailty is more appropriate when the clusters we are dealing with are considered to be drawn from a larger population of clusters. Therefore, examination using frailty is an issue that needs to be addressed in the future.
Conclusions
Our objective was to evaluate which of the AG, PWPTT, and PWPGT models performed best in estimating the intervention effects using TTRE in SWCRT with an open cohort design. The performance was evaluated by Bias, MSE, and CP based on different event generation models and true intervention effects and several scenarios involving the SWCRT design itself. The simulation results showed that the PWPGT model with stratification by clusters showed the most reasonable performance in situations where interindividual variability was not present, especially when evaluated by CP, regardless of the presence of cluster effects. However, if interindividual variability was present, the PWPTT model with stratification by clusters performed best.
Availability of data and materials
Simulation codes supporting the conclusions of this article are available from a GitHub repository at https://github.com/soyamada/TimeToRecurrentEventInSteppedWedge.
Abbreviations
 RCT:

Randomized controlled trial
 CRT:

Cluster randomized trial
 SWCRT:

Stepped wedge cluster randomized trial
 TTHA:

Timetohospital admission
 TTE:

Timetoevent
 TTFE:

Timetofirstevent
 TTRE:

Timetorecurrentevent
 TTTE:

Timetoterminalevent
 CoxPH:

Cox proportional hazard
 AG:

Andersengill
 PWPTT:

Prenticewilliamspeterson totaltime
 PWPGT:

Prenticewilliamspeterson gaptime
 HR:

Hazard ratio
 MSE:

Mean square error
 CP:

Coverage probability
 CI:

Confidence interval
 SE:

Standard error
References
Eldridge S, Kerry S. A practical guide to cluster randomised trials in health services research. 1st ed. US: John Wiley & Sons Inc; 2012.
Meurer WJ, Lewis RJ. Cluster randomized trials: evaluating treatments applied to groups. JAMA. 2015;313(20):2068–9.
Ellenberg SS. The steppedwedge clinical trial: evaluation by rolling deployment. JAMA. 2018;319:607–8.
Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis and reporting. BMJ. 2015;350:h391.
Copas AJ, Lewis JJ, Thompson JA, Davey C, Baio G, Hargreaves JR. Designing a stepped wedge trial: three main designs, carryover effects and randomisation approaches. Trials. 2015;16:352.
Forbat L, Liu WM, Koerner J, Lam L, Samara J, Chapman M, et al. Reducing time in acute hospitals: A steppedwedge randomised control trial of a specialist palliative care intervention in residential care homes. Palliat Med. 2020;34:571–9.
Cook RJ, Lawless JF. The Statistical Analysis of Recurrent Events. 2nd ed. NY: Springer; 2010.
Cox DR. Regression Models and LifeTables. J Royal Stat Soc B. 1972;34(2):187–220.
Andersen PK, Gill RD. Cox’s regression model for counting processes: a large sample study. Ann Stat. 1982;10:1100–20.
Prentice RL, Williams BJ, Peterson AV. On the regression analysis of multivariate failure time data. Biometrika. 1981;68:373–9.
Wei LJ, Lin DY, Weissfeld L. Regression analysis of multivariate incomplete failure time data by modeling marginal distributions. J Am Stat Assoc. 1989;84:1065–73.
Zhan Z, de Bock GH, van den Heuvel ER. Statistical methods for unidirectional switch designs: past, present, and future. Stat Meth Med Res. 2018;27:2872–82.
Cox DR, Oakes D. Analysis of Survival Data, Monographs on statistics and applied probability. 1st ed. London: Chapman & Hall; 1990.
Fisher LD, Lin DY. Timedependent covariates in the cox proportionalhazards regression model. Annu Rev Public Health. 1999;20:145–57.
Zhan Z, de Bock GH, Wiggers T, van den Heuvel E. The analysis of terminal endpoint events in stepped wedge designs. Stat Med. 2016;35:4413–26.
Kalbfleisch JD, Prentice RL. The Statistical Analysis of Failure Time Data. 2nd ed. New York: Wiley; 2002.
Amorim LD, Cai J. Modelling recurrent events: a tutorial for analysis in epidemiology. Int J Epidemiol. 2015;44:324–33.
Lemeshow S, May S, Hosmer SW. Applied Survival Analysis: Regression Modeling of TimetoEvent Data. 2nd ed. US: John Wiley & Sons Inc; 2008.
Therneau TM, Grambsch PM. Modeling Survival Data: Extending the Cox Model. New York: SpringerVerlag; 2010.
Lin DY, Wei DJ. The robust inference for the Cox proportional hazards model. J Am Stat Assoc. 1989;84:1074–8.
Therneau TM, Hamilton SA. RhDNase as an example of recurrent event analysis. Stat Med. 1997;16:2029–47.
Kelly PJ, Lim LLY. Survival analysis for recurrent event data: an application to childhood infectious diseases. Stat Med. 2000;19:13–33.
Bouwsma EVA, Huirne JAF, van de Ven PM, Noordegraaf AV, Schaafsma FG, Koops SES, et al. Effectiveness of an internetbased perioperative care programme to enhance postoperative recovery in gynaecological patients: Cluster controlled trial with randomised steppedwedge implementation. BMJ Open. 2018;8:e017781.
Kerber KA, Damschroder L, McLaughlin T, Brown DL, Burke JF, Telian SA, et al. Implementation of evidencebased practice for benign paroxysmal positional vertigo in the emergency department: a steppedwedge randomized trial. Ann Emerg Med. 2020;75(4):459–70.
Freeman CR, Scott IA, Hemming K, Connelly LB, Kirkpatrick CM, Coombes I, et al. Reducing Medical Admissions and Presentations Into Hospital through Optimising Medicines (REMAIN HOME): a stepped wedge, cluster randomised controlled trial. Med J Aust. 2021;214(5):212–7.
Leontjevas R, Gerritsen DL, Smalbrugge M, Teerenstra S, VernooijDassen MJ, Koopmans RT. A structural multidisciplinary approach to depression management in nursinghome residents: a multicentre, steppedwedge clusterrandomised trial. Lancet. 2013;381:2255–64.
Halek M, Reuther S, MullerWidmer R, Trutschel D, Holle D. Dealing with the behaviour of residents with dementia that challenges: A steppedwedge cluster randomized trial of two types of dementiaspecific case conferences in nursing homes (FallDem). Int J Nurs Stud. 2020;104:103435.
Metcalfe C, Thompson SG. The importance of varying the event generation process in simulation studies of statistical methods for recurrent events. Stat Med. 2006;25:165–79.
Austin PC. Generating survival times to simulate Cox proportional hazards models with timevarying covariates. Stat Med. 2012;31(29):3946–58.
Braga JR, Tu JV, Austin PC, Sutradhar R, Ross HJ, Lee DS. Recurrent events analysis for examination of hospitalizations in heart failure: insights from the Enhanced Feedback for Effective Cardiac Treatment (EFFECT) trial. Eur Heart J Qual Care Clin Outcomes. 2018;4:18–26.
Austin PC, Lee DS, Fine JP. Introduction to the Analysis of Survival Data in the Presence of Competing Risks. Circulation. 2016;133:601–9.
Wang MC, Qin J, Chiang CT. Analyzing recurrent event data with informative censoring. J Am Stat Assoc. 2001;96(455):1057–65.
Liu L, Wolfe RA, Huang X. Shared frailty models for recurrent events and a terminal event. Biometrics. 2004;60:747–56.
Twisk JWR, Smidt N, de Vente W. Applied analysis of recurrent events: a practical overview. J Epidemiol Community Health. 2005;59:706–10.
Acknowledgements
We thank and acknowledge Liz Forbat (University of Stirling) and WaiMan Liu (Australian National University) for providing electronic data (anonymized) collected in an actual example and advice on the content of our study. We would like to thank Editage (www.editage.com) for English language editing.
Funding
Not applicable.
Author information
Affiliations
Contributions
SO, SC and TY participated in the design of the study. SO carried out the simulation study and the statistical analysis of an actual example data, and drafted the manuscript. SC and TY participated in a discussion about statistical aspects. All the authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
In the INSPIRED trial (actual example of our study), consent to run the trial was gained at the site, rather than individual resident, level given the impracticalities of gaining informed consent from a large population. This follows national guidelines for Australia from the National Health and Medical Research Council (NHMRC). Since our study is only an analysis based on simulations and precollected data, we did not obtain additional consent from the individual resident. For the INSPIRED trial group to provide us with the electronic data (anonymized) collected in the INSPIRED trial, we obtained the approval of the Ethics Review Committee of the Tohoku University Graduate School of Medicine for the study protocol. (Reception No.: 2020–11180) Our study follows Ethical Guidelines for Medical and Biological Research Involving Human Subjects (Japanese).
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Oyamada, S., Chiu, SW. & Yamaguchi, T. Comparison of statistical models for estimating intervention effects based on timetorecurrentevent in stepped wedge cluster randomized trial using open cohort design. BMC Med Res Methodol 22, 123 (2022). https://doi.org/10.1186/s12874022015526
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12874022015526
Keywords
 Steppedwedge
 Cluster randomized trial
 Open cohort design
 Recurrent event
 Timetoevent
 Statistical model
 Timedependent covariate
 Simulation
 Comparison