Skip to main content

Table 3 Area under the curve (AUC) for comparison of mean logarithmic score LS ¯ ( r ) of true vs. wrong modeling strategy ( r  = 1,…,100 iterations per simulation scenario)

From: Bayesian model selection techniques as decision support for shaping a statistical analysis plan of a clinical trial: An example from a vertigo phase III study with longitudinal count data as primary endpoint

k, degree of   AUC
overdispersion Model n  = 20 n  = 50 n  = 100
0.5 Poi 1 1 1
0.5 ZIP 1 1 1
0.5 arcsinh 0.796 0.781 0.953
0.5 ZINB 0.498 0.498 0.499
1 Poi 1 1 1
1 ZIP 1 1 1
1 arcsinh 0.683 0.831 0.883
1 ZINB 0.497 0.501 0.499
5 Poi 0.999 1 1
5 ZIP 0.999 1 1
5 arcsinh 0.673 0.796 0.877
5 ZINB 0.526 0.513 0.509
10 Poi 0.903 0.988 1
10 ZIP 0.909 0.990 1
10 arcsinh 0.733 0.826 0.913
10 ZINB 0.562 0.526 0.515
20 Poi 0.675 0.831 0.920
20 ZIP 0.686 0.837 0.925
20 arcsinh 0.773 0.902 0.966
20 ZINB 0.628 0.564 0.548
50 Poi 0.526 0.567 0.644
50 ZIP 0.548 0.582 0.658
50 arcsinh 0.799 0.914 0.985
50 ZINB 0.742 0.667 0.512
  1. True model: negative binomial GLMM; competing modeling strategies: Poisson, ZIP, NMM for arcsinh-transformed counts, ZINB. Sample size: n = 20, 50, 100 patients per group; degree of overdispersion: k  = 0.5, 1, 5, 10, 20, 50. AUC can be interpreted as a summary measure for the goodness of discrimination between the true negative binomial model generating the longitudinal data and rival models which should be taken into consideration in practice. For k, the difference between a negative binomial and the alternative Poisson model dissolves because of convergence in distribution; therefore, AUC 0.5 and both model alternatives approximate with respect to their forecasting ability.