Table 3 Results of the RCT replication experiments. Pseudo MSE and MAE are respectively the pseudo mean squared error and the pseudo mean average error obtained by replacing the unknown ground truth with the RCT estimate. Estimate agreement is the percentage of RCT \(95\%\) confidence intervals that contain ATT estimation. Regulatory agreement is the percentage of time the cutoff \(P<0.05\) obtained from the non-randomized experiments agrees with the RCT result about \(P<0.05\)
| Â | Pseudo MSE(x1000) | Pseudo MAE(x100) | C.I. Width(x100) | Estimate Agreement | Regulatory Agreement |
|---|---|---|---|---|---|
Unadjusted | 7.94 | 7.30 | 25.1 | 84.2% (16/19) | 73.7% (14/19) |
PSM | 4.51 | 6.15 | 29.0 | 89.5% (17/19) | 73.7% (14/19) |
IPTW | 5.75 | 5.86 | 28.5 | 89.5% (17/19) | 78.9% (15/19) |
G-computation | 3.26 | 4.68 | 25.9 | 100% (19/19) | 78.9% (15/19) |
DDML | 4.70 | 5.60 | 31.3 | 100% (19/19) | 84.2% (16/19) |