Table 2 Brief descriptions of the quantitative bias analysis software applicable for a linear regression analysis

treatSens Bias parameters: \(\phi =(\zeta ^Y,\zeta ^X)\) where \(\zeta ^Y\) and \(\zeta ^X\) represent the coefficients of U from regressions Y|X, C, U and X|C, U, respectively. Benchmarks: For each covariate \(C_j\) of C, the benchmarks for \(\zeta ^Y\) and \(\zeta ^X\) are the coefficients of \(C_j\) from regressions Y|X, C and X|C, respectively. Method: For prespecified values of \(\phi =(\zeta ^Y,\zeta ^X)\), simulates U from model for joint distribution X, Y, U|C and then fits linear regression Y|X, C, U to the observed data and the simulated U to obtain \(\hat{\beta }_{Y|X,C,U(\phi )}\) and its standard error. Output: Contour plot of \(\hat{\beta }_{Y|X,C,U(\phi )}\) for different combinations of \(\phi =(\zeta ^Y,\zeta ^X)\) with added benchmark values and indications of the values of \(\phi =(\zeta ^Y,\zeta ^X)\) at the tipping points. Tabular outputs of the (1) values of \(\phi =(\zeta ^Y,\zeta ^X)\) at the tipping points, (2) \(\hat{\beta }_{Y|X,C,U(\phi )}\) and its standard error for prespecified values of \(\phi =(\zeta ^Y,\zeta ^X)\), and (3) benchmark values for \(\phi =(\zeta ^Y,\zeta ^X)\). Other: Standardises continuous variables to facilitate comparison between \(\phi =(\zeta ^Y,\zeta ^X)\) and their benchmark values.

causalsens Bias parameters: \(\phi =(R_\alpha ^2)\) where the magnitude of \(R_\alpha ^2\) represents the proportion of unexplained variance in the potential outcomes of Y (to non-exposure and exposure) that is explained by U and the sign of \(R_\alpha ^2\) represents the direction of bias due to unmeasured confounding (e.g., if U explains \(5\%\) of the unexplained variance, set \(R_\alpha ^2=\pm 5\%\) to allow for bias towards and away from the null). Benchmarks: Based on partial \(R^2\) of Y with each measured covariate. Method: Generates a modified outcome, \(Y^{adj}_\phi\), adjusted for the bias due to unmeasured confounding using: (1) estimated probabilities \(Pr(X=1|C)\) and a user-defined function (called the “confounding function”) for the average difference in the potential outcomes of Y between the exposure groups. Exposure effect results from regression \(Y^{adj}_\phi |X,C\) are \(\hat{\beta }_{Y|X,C,U(\phi )}\), and its confidence interval (CI). Output: Line plot of \(\hat{\beta }_{Y|X,C,U(\phi )}\) and its CI for prespecified values of \(R_\alpha ^2\) with each benchmark added as a positive and negative value (e.g., \(\pm 5\%)\).

sensemakr Bias parameters: \(\phi =(R^2_{X \sim U|C},R^2_{Y \sim U|X,C})\) where \(R^2_{X \sim U|C}\) is the proportion of the variance of X, not explained by C, that is explained by U and \(R^2_{Y \sim U|X,C}\) is the proportion of the variance of Y, not explained by X and C, that is explained by U. Benchmarks: Calculates “benchmark bounds” for \(R^2_{X \sim U|C}\) and \(R^2_{Y \sim U|X,C}\) based on partial \(R^2\) values of each measured covariate with X and Y, respectively. Method: (1) Formulae to calculate summary measures\(^a\) for the point estimate and its t-value (called “robustness measures”). (2) Formulae to estimate \(\hat{\beta }_{Y|X,C,U(\phi )}\) and its t-value for prespecified values of \(\phi =(R^2_{X \sim U|C},R^2_{Y \sim U|X,C})\). Output: (1) Robustness values for the point estimate and its t-value at prespecified tipping points. (2) Contour plots of \(\hat{\beta }_{Y|X,C,U(\phi )}\) and its t-value for different combinations of \(\phi =(R^2_{X \sim U|C},R^2_{Y \sim U|X,C})\). Plots indicate values of \(R^2_{X \sim U|C}\) and \(R^2_{Y \sim U|X,C}\) that correspond to tipping points and (multiples of) the benchmark bounds. Also, outputs a table of bias-adjusted results, \(\hat{\beta }_{Y|X,C,U(\phi )}\) and its CI, when \(R^2_{X \sim U|C}\) and \(R^2_{Y \sim U|X,C}\) equal (or equal multiples of) their benchmark bounds.

EValue Bias parameters: \(\phi =(RR_{XU},RR_{UY})\) where (for binary X, Y and single, binary U), \(RR_{XU}\) and \(RR_{UY}\) denote risk ratios for X on U and U on Y (conditional on C), respectively. Benchmarks: None provided. Method: Formulae to calculate summary measures\(^a\) (called “E-values”) for the point estimate and CI limit. Output: Line plots indicating the combinations of \(\phi =(RR_{XU},RR_{UY})\) at which \(\hat{\beta }_{Y|X,C,U(\phi )}\) and its CI limit equate to their tipping point values with the E-values added to these plots. Other: Applicable for effect measures other than the risk ratio [52] and \(\phi =(RR_{XU},RR_{UY})\) is defined for a single or multiple unmeasured confounders of type continuous, categorical or mixed [77].

konfound Bias parameters: \(\phi =(r_{X \sim U|C},r_{Y \sim U|C})\) where \(r_{X \sim U|C}\) and \(r_{Y \sim U|C}\) denote the partial correlation of U with X and Y, respectively, conditional on C. Benchmarks: Based on partial correlations of each covariate \(C_j\) of C with X and Y given the remaining covariates. Method: Formulae to calculate: (1) “percent bias” the minimum percentage of \(\hat{\beta }_{Y|X,C}\) explained by U at which the P-value for \(\hat{\beta }_{Y|X,C,U(\phi )}\) equals statistical significance. (2) Summary measure\(^a\) (called “impact threshold”). Output: Reports percent bias depicted as a bar graph and impact threshold depicted as a causal-type diagram. Stata command outputs a table of benchmark values for \(\phi =(r_{X \sim U|C},r_{Y \sim U|C})\). Other: Percent bias and impact threshold evaluate how much unmeasured confounding would be needed to invalidate inference (i.e., change from statistically significant \(\hat{\beta }_{Y|X,C}\) to statistically insignificant \(\hat{\beta }_{Y|X,C,U(\phi )}\)) or sustain inference (i.e., change from statistically insignificant \(\hat{\beta }_{Y|X,C}\) to statistically significant \(\hat{\beta }_{Y|X,C,U(\phi )}\)).

a: For \(\phi =(\phi _1,\phi _2)\), summary measure represents the minimum value of \(\phi\) (when \(\phi _1=\phi _2\)) at the prespecified tipping point value.