Evaluating the performance of copula models in phase I-II clinical trials under model misspecification

Braun copula

An analogous conditional probability of Y _T |Y _E can also be derived.

This is a key point that must be considered during dose finding.

Gumbel copula

The conditional probability of Y _T given Y _E can be expressed in an analogous fashion.

An advantage of this model is that both π _E and π _T retain their original interpretations as the marginal probabilities of efficacy and toxicity, respectively. This can be easily seen by summing P(Y _E =1,Y _T =1) and P(Y _E =1,Y _T =0) from (4). Unlike the Braun Copula, the correlation parameter for the Gumbel copula, ψ ₂, does not have a straight-forward interpretation.

Independent model

which is of course what we get by setting ψ ₁=0.5 and ψ ₂=0 in the Braun and Gumbel copulas, respectively. While it is unlikely that this model accurately reflects the true association between Y _T and Y _E , this model may still be useful because the sample size in phase I-II oncology trials is limited and we may lack the sample size to precisely estimate ψ ₁ and ψ ₂. If the likelihood contains very little information about these parameters, it may be that we do not lose much with respect to our ability to identify the optimal dose by assuming independence instead of a more complicated model.

Likelihood and priors

where π(Y _T ,Y _E |z) is defined using either (3), (4) or (5) and $\overrightarrow{\beta }=({\beta }_{0,T},{\beta }_{1,T},{\beta }_{0,E},{\beta }_{1,E},{\beta }_{2,E},{\psi }_k)$ with k=1,2 for the two copula models and $\overrightarrow{\beta }=({\beta }_{0,T},{\beta }_{1,T},{\beta }_{0,E},{\beta }_{1,E},{\beta }_{2,E})$ for the independence model.

We must specify a prior distribution for each regression and association parameter, to complete a Bayesian analysis. We specify the following normal priors for the two intercept terms and the quadratic term for efficacy: β _0,T ∼N(−3,s d=3), β _0,E ∼N(−1,3), and ${\beta }_{2,E}\sim N\left(0,\frac{1}{4}\right)$. The priors for β _0,T and β _0,E correspond to a prior belief of P(Y _T =1|z=1)=0.05 and P(Y _E =1|z=1)=0.27 but provide sufficient support over all plausible values for β _0,T and β _0,E and represent only mildly informative priors. The prior for β _2,E is chosen to reflect a strong belief against a quadratic relationship but allows the model flexibility should there be drastic departures from a linear relationship. Gamma priors were set for β _1,T and β _1,E with mean 1 and standard deviation 2, corresponding to a $\mathit{\text{Gamma}}(\frac{1}{4},\frac{1}{4})$. Assuming Gamma priors for β _1,T and β _1,E implies that the marginal probability of toxicity will be monotonically increasing but the same is not true for the marginal probability of efficacy due to the inclusion of a quadratic term for the marginal probability of efficacy. Finally, we specify non-informative uniform priors for the association parameters: U n i f o r m(0,1) for ψ ₁ and U n i f o r m(−1,1) for ψ ₂.

Dose-finding algorithm

For our simulation study, we follow the dose-finding algorithm proposed by Thall and Cook [4]. These authors identify a set of acceptable doses by defining a maximum acceptable probability of toxicity assuming 100% efficacy, a minimum acceptable probability of efficacy assuming no toxicity and define a desirability index to identify the optimal dose from the set of acceptable doses.

where q is defined by identifying a probability of toxicity and probability of efficacy pair, $\left({\pi }_T^{\ast },{\pi }_E^{\ast }\right)$, that is equally desirable to $\left({\overline{\pi }}_T,1.0\right)$ and $\left(0,{\underline{\pi }}_E\right)$, plugging $\left({\pi }_T^{\ast },{\pi }_E^{\ast }\right)$ into (7) and solving for q when D(z) equals 0. Larger values of D(z) are considered more desirable and the optimal combination, (0.0,1.0), has D(z) equal to 1 regardless of q.

Scenarios

For each scenario, we simulated data from both copula models and fit the data using either the correct copula model, the incorrect copula model or the independence model. The association parameters, ψ ₁ and ψ ₂, were varied to determine the impact of the correlation of Y _T and Y _E on model performance under model misspecification. The Braun association parameter, ψ ₁, in (3) ranges from 0.5 to 0.9 by increments of 0.2; this corresponds to an odds ratio between toxicity and efficacy ranging from 1 to 9. The Gumbel association parameter, ψ ₂ in (4) ranges from 0 to 0.8 by increments of 0.4. Recall that efficacy and toxicity are independent if ψ ₁ equals 0.5 for the Braun model and ψ ₂ equals 0 for the Gumbel model. In these cases, the independence model is the correct model and the Braun and Gumbel models are unnecessarily trying to estimate a correlation parameter when the two endpoints are actually independent.

Results

Table 2 displays results for Scenario 1. Concentrating first on the results when data are generated from the Braun model, we see that both copula models perform similarly well in their ability to select the correct dose and in the number of patients treated at the optimal dose regardless of the true correlation. The probability of correctly identifying the optimal dose differs by less than 0.024 and the number of patients treated at the optimal dose differs by less than 1 across the three correlation conditions. Surprisingly, the independence model also performs very well regardless of the true correlation and has the highest probability of identifying the optimal dose in two of the three correlation conditions. Although, the differences are small and the independence model has essentially the same performance as the two copula models.

Similar results are observed when data are generated from the Gumbel model. Both copula models performed similarly with respect to the probability of accurately identifying the optimal dose and the number of patients treated at the optimal dose. The independence model again exhibits good performance across the three scenarios, which is surprising because the independence model lacks the flexibility to model the correlation between the two outcomes. Across the three correlation scenarios, the probability of correctly identifying the optimal dose differed by less than 0.05 and the average number of patients treated at the optimal dose differed by less than 1.5 between the three models.

Results for Scenarios 2 and 3 are found in Tables 3 and 4, respectively. The results for Scenarios 2 and 3 are similar to the results for Scenario 1. The probability of correctly identifying the optimal dose and the average number of patients treated at the optimal dose are similar for both copula models regardless of how the data are generated and the independence model provides similar performance even though the independence model is unable to appropriately model the correlation between Y _T and Y _E .

Scenario 4 (Table 5), represents the scenario where multiple dose levels are acceptable, dose levels 3 and 4, but dose level 3 is optimal. This scenario represents one of the primary motivations for phase I-II designs as there is a dose level where further escalation results in a greater probability of toxicity but relatively little efficacy benefit. The results for Scenario 4 are consistent with our previous results: there is little difference between the three models in the probability of correctly identifying the optimal dose and the average number of patients treated at the optimal dose regardless of the correlation between endpoints and how the data are generated. Finally, the results for Scenario 5 can be found in Table 6. In this scenario, all dose levels are safe but have unacceptable efficacy and the correct decision is to terminate for futility. The Gumbel and independence models exhibit similar performance across all scenarios but we do observe that the Braun model is more likely to terminate for futility when Y _T and Y _E are correlated and the data are generated for the Braun model. Although, the differences are small in both cases.

The simulations results presented in Tables 2, 3, 4, 5 and 6 indicate that specifying the correct copula model has little impact on the operating characteristics of Phase I-II clinical trials but are dependent on a number of factors, including the sample size and priors specified for the logistic regression models used for efficacy and toxicity. In order to account for these factors, we completed additional simulations to evaluate the robustness of our conclusions to changing the sample size and prior distributions. Figure 1 presents simulation results to evaluate the impact of sample size on our conclusions. Presented are the simulated probabilities of correctly identifying the optimal dose for Scenario 1 for maximum sample sizes of 30, 45, 60 and 75 subjects, with data simulated from both models and various levels of correlation. As expected, the probability of correctly identifying the optimal dose increases as the maximum sample size increases. More importantly, increasing the sample size appears to have no impact on our primary conclusion that incorrectly specifying the copula model has little impact on the probability of correctly identifying the optimal dose regardless of the level of correlation between the two endpoints and the model from which the data are generated.

Figure 2 presents simulation results to evaluate the impact of changing the priors for the parameters of the logistic regression models for efficacy and toxicity. We considered four prior specifications. Prior specification 1 is the original priors: β _0,T ∼N(−3,s d=3), β _0,E ∼N(−1,3), ${\beta }_{1,T}\sim \mathit{\text{Gamma}}\left(\frac{1}{4},\frac{1}{4}\right)$ and ${\beta }_{1,E}\sim \mathit{\text{Gamma}}\left(\frac{1}{4},\frac{1}{4}\right)$, in prior specification 2, we increase the prior variances for the intercept parameters: β _0,T ∼N(−3,5) and β _0,E ∼N(−1,5), in prior specification 3, we increase the prior variances for the slope parameters but not the intercept parameters: ${\beta }_{1,T}\sim \mathit{\text{Gamma}}\left(\frac{1}{25},\frac{1}{25}\right)$ and ${\beta }_{1,E}\sim \mathit{\text{Gamma}}\left(\frac{1}{25},\frac{1}{25}\right)$ and in prior specification 4, we increase the prior variances of both the slope and intercept parameters: β _0,T ∼N(−3,5), β _0,E ∼N(−1,5), ${\beta }_{1,T}\sim \mathit{\text{Gamma}}\left(\frac{1}{25},\frac{1}{25}\right)$ and ${\beta }_{1,E}\sim \mathit{\text{Gamma}}\left(\frac{1}{25},\frac{1}{25}\right)$. Presented are the simulated probabilities of correctly identifying the optimal dose for Scenario 1 for the four prior specifications, with data simulated from both models and various levels of correlation. We see that increasing the prior variance of the intercept terms has little impact on the probability of correctly identifying the optimal dose but increasing the variance of the slope decreases the probability of correctly identifying the optimal dose considerably. More importantly, we see that the independence model performs as well or better than the other model in all cases. In addition, we see that the Braun model performs worse than the other two models when we increase the prior variance of the slope parameters with a larger effect observed with moderate and high correlation. This effect is similar regardless of how the data are generated and, in fact, appears slightly larger when the Braun model is actually the correct model.