Fig. 7

Pairwise agreement in statistical significance of estimates of p-value comparisons for level change. In the top triangle, boxes are divided into 16 cells with p-values categorised using a fine gradation of statistical significance, namely, p-value ≤ 0.01, 0.01 < p-value ≤ 0.05, 0.05 < p-value ≤ 0.1, p-value > 0.1. In the bottom triangle, boxes are divided into four cells with p-values categorised at the 5% level of statistical significance (i.e. ≤ 5%, > 5%). Each cell within a box contains the percentage of datasets falling within the row and column defined statistical significance levels. The colour bands surrounding the left/right and top/bottom side of the plot indicate the two methods being compared. Concordant results are shown in blue. Discordant results are shown as either white (0–5% discordance), orange (5–10% discordance), red (10–20% discordance) or purple (over 20% discordance). For example, within the box comparing ARIMA and OLS in the bottom triangle, in 12% of the datasets the ARIMA method yields a p-value > 0.05 while the OLS method yields a p-value ≤ 0.05 (bottom right cell). Numbers may not add to 100 due to rounding. Abbreviations: ARIMA, autoregressive integrated moving average; OLS, ordinary least squares; NW OLS with Newey-West standard error adjustments; PW, Prais-Winsten; REML, restricted maximum likelihood; Satt, Satterthwaite adjustment