Performance comparison of first-order conditional estimation with interaction and Bayesian estimation methods for estimating the population parameters and its distribution from data sets with a low number of subjects

Background Exploratory preclinical, as well as clinical trials, may involve a small number of patients, making it difficult to calculate and analyze the pharmacokinetic (PK) parameters, especially if the PK parameters show very high inter-individual variability (IIV). In this study, the performance of a classical first-order conditional estimation with interaction (FOCE-I) and expectation maximization (EM)-based Markov chain Monte Carlo Bayesian (BAYES) estimation methods were compared for estimating the population parameters and its distribution from data sets having a low number of subjects. Methods In this study, 100 data sets were simulated with eight sampling points for each subject and with six different levels of IIV (5%, 10%, 20%, 30%, 50%, and 80%) in their PK parameter distribution. A stochastic simulation and estimation (SSE) study was performed to simultaneously simulate data sets and estimate the parameters using four different methods: FOCE-I only, BAYES(C) (FOCE-I and BAYES composite method), BAYES(F) (BAYES with all true initial parameters and fixed ω 2), and BAYES only. Relative root mean squared error (rRMSE) and relative estimation error (REE) were used to analyze the differences between true and estimated values. A case study was performed with a clinical data of theophylline available in NONMEM distribution media. NONMEM software assisted by Pirana, PsN, and Xpose was used to estimate population PK parameters, and R program was used to analyze and plot the results. Results The rRMSE and REE values of all parameter (fixed effect and random effect) estimates showed that all four methods performed equally at the lower IIV levels, while the FOCE-I method performed better than other EM-based methods at higher IIV levels (greater than 30%). In general, estimates of random-effect parameters showed significant bias and imprecision, irrespective of the estimation method used and the level of IIV. Similar performance of the estimation methods was observed with theophylline dataset. Conclusions The classical FOCE-I method appeared to estimate the PK parameters more reliably than the BAYES method when using a simple model and data containing only a few subjects. EM-based estimation methods can be considered for adapting to the specific needs of a modeling project at later steps of modeling. Electronic supplementary material The online version of this article (10.1186/s12874-017-0427-0) contains supplementary material, which is available to authorized users.

Methods: In this study, 100 data sets were simulated with eight sampling points for each subject and with six 23 different levels of IIV (5%, 10%, 20%, 30%, 50%, and 80%) in their PK parameter distribution. A stochastic simulation 24 and estimation (SSE) study was performed to simultaneously simulate data sets and estimate the parameters using 25 four different methods: FOCE-I only, BAYES(C) (FOCE-I and BAYES composite method), BAYES(F) (BAYES with all true 26 initial parameters and fixed ω 2 ), and BAYES only. Relative root mean squared error (rRMSE) and relative estimation 27 error (REE) were used to analyze the differences between true and estimated values. A case study was performed 28 with a clinical data of theophylline available in NONMEM distribution media. NONMEM software assisted by Pirana, 29 PsN, and Xpose was used to estimate population PK parameters, and R program was used to analyze and plot the results. 30 Results: The rRMSE and REE values of all parameter (fixed effect and random effect) estimates showed that all four 31 methods performed equally at the lower IIV levels, while the FOCE-I method performed better than other EM-based 32 methods at higher IIV levels (greater than 30%). In general, estimates of random-effect parameters showed significant bias 33 and imprecision, irrespective of the estimation method used and the level of IIV. Similar performance of the estimation 34 methods was observed with theophylline dataset.
146 where C ij indicates the j-th observations of i-th individual, 147 C pred, ij indicates the model-predicted C ij , and ε ij indicates 148 the proportional residual error.

149
The following equations [Eqs. (5) and (6)] describe the 150 rate of change in drug amount in a one-compartment system:  Table   T1 1.

170
The FOCE-I method is a classical estimation method  In this study, true parameter values, i.e., the parameter 189 values used in the simulation step, were established as 190 initial estimates in all estimation methods. In NON-191 MEM, convergence criteria for a FOCE-I are based only 192 on the parameter estimation gradient and are tested by 193 default. The number of significant digits for the estima-194 tion of each parameter was set to three (SIG = 3) for the 195 FOCE-I method. In the BAYES estimation method, the 196 convergence test type was set to 3 (CTYPE = 3), where 197 changes in objective function value, THETAs, OMEGAs, 198 and SIGMAs, are accessed. The number of significant 199 digits to which the objective function was evaluated was 200 set to 8 (SIGL = 8). In the BAYES methods, the max-201 imum number of iterations for which to perform the 202 burn-in phase was set to 4000 (NBURN = 4000), and the 203 number of iterations for which to perform the stationary 204 distribution for BAYES analysis was set to 10,000 205 (NITER = 10,000), both of which are default values in 206 NONMEM. The former option ensured that all parame-207 ters and objective functions did not appear to move in a 208 specific direction, but appeared to instead move around 209 a stationary region, and the latter provides a large set 210 (10,000) of likely population parameters. 211 Assessment and comparison of estimation methods 212 The estimation methods were assessed by relative root 213 mean squared error (rRMSE) and relative estimation 214 error (REE) for fixed-effect as well as random-effect pa-215 rameters to calculate and visualize the magnitude of dif-216 ferences between the true value and the estimated value. 217 The rRMSE [Eq. (8)] provides a combined measure of 218 bias and precision.

239
The rRMSE values of the estimated parameters (fixed-ef-   The REE of both fixed-effect and random-effect pa-258 rameters versus the estimation methods, stratified by dif-259 ferent levels of IIV, are shown in Fig.   F3 3. The plots were 260 adjusted to include ±100% REE for the purpose of clar-261 ity. In general, all estimation methods overestimated 262 fixed-effect parameters to some extent. At a lower level 263 of IIV (5-10%), all estimation methods estimated fixed-264 effect parameters with negligible bias and reasonable 265 precision. However, the bias as well imprecision  The overall stability of estimations were high with a 287 100% success rate of minimization and covariance step 288 for BAYES(C), BAYES(F), and BAYES methods. For the 289 FOCE-I method, the minimization step had a 100% 290 success rate, but the rate of the successful covariance 291 step was 52% at 5% IIV while other estimations had a 292 successful covariance step close to 100%.

293
The THEO data set used as a case study had 132 294 observations from 12 subjects, 11 observations per indi-295 vidual after an oral dose of 320 mg theophylline. A one-296 compartment PK model with first order absorption 297 described the data well and it was used as a final model.

298
The PK parameters from the THEO data set were: CL/F Relative estimation error (%) 5% CV

Relative estimation error (%)
20% CV For an estimation method, the most desirable features 323 are a low bias and high precision. In this study, we used 324 rRMSE and REE to evaluate these features. The rRMSE 325 provides a single value that indicates both bias and pre-326 cision. Moreover, rRMSE provides a way to compare 327 performance across parameters and models. However, 328 the REE allows for comparison of different parameters 329 with varying magnitudes in a single plot while acknow-330 ledging bias and precision. For an estimation method to 331 be unbiased and precise, the REE should have a normal 332 distribution with a median of 0 and a narrow range of 333 values.

334
The FOCE-I method performed better among the four 335 methods tested based on the overall rRMSE. This per-336 formance was supported by the REE plot, which did not 337 show any significant bias for any fixed effect parameters 338 at any given level of IIV. The median REE values for the 339 random-effect parameters were not greater than −17% at 340 any given level of IIV. A resembling result of negative 341 bias was observed with the FOCE-I algorithm in a simi-342 lar studies comparing different estimation methods [9]. 343 The FOCE-I method has been shown to work suffi-344 ciently well for simple models when compared to other 345 EM based algorithms in previous studies. Furthermore, 346 when the IIV was low, the performance of classical esti-347 mation methods and EM based methods were very close. 348 Similar results were observed in a previous study for 349 such simple model (1-compartment model), where the 350 performance of those estimation methods were found to 351 be nearly equal [5].

517
This work was also supported by research funds from Chungnam National 518 University.