Table 2 Differences in coverage of the treatment effect on the outcome of interest by the 95% confidence interval and difference in mean of standardised absolute biases of the treatment effect on the outcome of interest. Default empirical calibration was performed with five negative controls and using the all-controls systematic error model
From: Assessing the effectiveness of empirical calibration under different bias scenarios
Difference Coverage | Difference in Mean of Standardised Absolute Bias | |||||
|---|---|---|---|---|---|---|
|  | Ideal—Suitable negative controls | Random— Suitable negative controls | Random—Unsuitable negative controls | Ideal—Suitable negative controls | Random—Suitable negative controls | Random—Unsuitable negative controls |
No calibration vs default empirical calibration | ||||||
 Unmeasured confounder | 0.48 | 0.60 | 0.05 | −0.46 | −0.23 | 0.00 |
 Model misspecification Quadratic term | 0.24 | 0.27 | 0.08 | −0.05 | 0.03 | −0.01 |
 Model misspecification Interaction term | −0.09 | 0.04 | 0.02 | −0.19 | −0.09 | − 0.02 |
 Non-positivity | 0.02 | 0.00 | 0.01 | −0.02 | 0.07 | −0.01 |
 Measurement error | 0.01 | 0.01 | NA | −0.02 | −0.02 | NA |
Default empirical calibration vs empirical calibration with the NULL systematic error model | ||||||
 Unmeasured confounder | 0.09 | −0.06 | −0.04 | 0.55 | −1.56 | 0.00 |
 Model misspecification Quadratic term | −0.09 | −0.13 | − 0.08 | 0.20 | 0.35 | 0.02 |
 Model misspecification Interaction term | −0.52 | −0.06 | − 0.03 | 0.29 | 0.00 | 0.01 |
 Non-positivity | −0.02 | −0.01 | − 0.03 | −0.01 | 0.09 | −0.01 |
 Measurement error | −0.04 | −0.02 | NA | 0.02 | 0.01 | NA |
Default empirical calibration (5 negative controls) vs empirical calibration with 30 negative controls | ||||||
 Unmeasured confounder | 0.05 | 0.06 | 0.01 | 0.85 | −8.31 | −0.36 |
 Model misspecification Quadratic term | 0.00 | 0.00 | −0.05 | 0.04 | −0.11 | 0.08 |
 Model misspecification Interaction term | 0.02 | 0.02 | 0.02 | 0.06 | 0.06 | −0.60 |
 Non-positivity | 0.02 | 0.03 | 0.00 | −0.02 | 0.43 | 0.12 |
 Measurement error | 0.01 | −0.01 | NA | −0.11 | −0.06 | NA |