Comparing g-computation, propensity score-based weighting, and targeted maximum likelihood estimation for analyzing externally controlled trials with both measured and unmeasured confounders: a simulation study

Table 2 Performance of different methods for estimating the treatment effects (n = 200)

Outcome/Method	Scenario 1 (small and blocked confounding paths)				Scenario 2 (medium and blocked confounding paths)				Scenario 3 (one large unblocked confounding path)
Outcome/Method	Bias	RMSE	Coverage	Width	Bias	RMSE	Coverage	Width	Bias	RMSE	Coverage	Width
Outcome Y_a
Raw	-0.800	0.856	0.264	1.205	-1.086	1.145	0.164	1.429	-1.461	1.502	0.016	1.393
GC	-0.001	0.351	0.950	1.376	0.022	0.408	0.948	1.617	-0.443	0.600	0.809	1.613
PS_IPTW	-0.066	0.447	0.831	1.190	-0.138	0.500	0.843	1.417	-0.618	0.784	0.548	1.382
PS_OW	-0.006	0.354	0.899	1.164	0.024	0.416	0.902	1.384	-0.458	0.616	0.696	1.356
PS_SMR	-0.072	0.521	0.766	1.169	-0.111	0.568	0.778	1.392	-0.625	0.878	0.513	1.370
TMLE	-0.039	0.429	0.928	1.514	-0.054	0.488	0.922	1.726	-0.548	0.731	0.734	1.724
Outcome Y_b
Raw	-0.267	0.405	0.854	1.208	-0.326	0.443	0.796	1.177	-0.432	0.524	0.687	1.157
GC	0.014	0.355	0.938	1.367	0.014	0.349	0.934	1.328	-0.133	0.376	0.917	1.332
PS_IPTW	0.037	0.443	0.836	1.223	0.000	0.430	0.848	1.196	-0.179	0.470	0.794	1.176
PS_OW	0.037	0.381	0.894	1.211	0.041	0.372	0.891	1.192	-0.126	0.394	0.867	1.174
PS_SMR	0.000	0.500	0.777	1.200	-0.028	0.497	0.773	1.183	-0.222	0.573	0.703	1.169
TMLE	0.036	0.432	0.974	1.885	0.015	0.423	0.974	1.869	-0.166	0.462	0.958	1.857
Outcome Y_c
Raw	0.181	0.248	0.792	0.662	0.201	0.261	0.773	0.658	0.237	0.290	0.695	0.650
GC	-0.002	0.190	0.926	0.717	-0.006	0.189	0.932	0.725	0.048	0.196	0.922	0.724
PS_IPTW	0.017	0.237	0.934	0.865	0.015	0.241	0.937	0.867	0.059	0.243	0.923	0.852
PS_OW	-0.026	0.208	0.948	0.796	-0.035	0.212	0.941	0.808	0.023	0.211	0.942	0.806
PS_SMR	-0.015	0.234	0.929	0.838	-0.031	0.238	0.922	0.847	0.033	0.245	0.925	0.862
TMLE	0.146	0.317	0.901	1.191	0.135	0.302	0.908	1.173	0.170	0.312	0.902	0.139

Bias was the average difference between the true value (simulated) and its estimate across the simulation replicates using the original scale for the continuous outcome and the log-transformed scale for the binary and time-to-event outcomes, such as log (HR). RMSE was the square root of the mean squared error (MSE) that is the average squared difference between the true value and its estimate across the simulation replicates. Coverage was the proportion of times the 95% confidence interval of the estimate contained the true value. Width was the average difference between the upper and lower bounds of 95% confidence interval of estimate
GC G-computation, RMSE Root mean squared error, PS- propensity score-based, IPTW Inverse probability of treatment weighting, SMR Standardized mortality or morbidity ratio, OW Overlap weighting, TMLE Targeted maximum likelihood estimation

ISSN: 1471-2288