Skip to main content

Table 1 Performance (type 1 error rates) of the tests in simulation 1 under H 0,C (i.e., both H 0,A and H 0,B are true) and yet f (Y|Y ≤ τ ∩ X = 1) is radically different from f (Y|Y ≤ τ ∩ X = 0) (see Figure 3 for details of simulation).

From: Testing for differences in distribution tails to test for differences in 'maximum' lifespan

Test

Sample Size (N) Per Group

 

50

80

100

 

α = .05

α = .01

α = .05

α = .01

α = .05

α = .01

Tests of H 0, A (Wang-Allison tests)

QT3 with Ï„ set to 130.

0.032 (.027, .036)#

0.008 (.005, .011)

0.041 (.036, .046)

0.006 (.003, .009)

0.040 (.035, .045)

0.006 (.003, .009)

QT3 with Ï„ set to sample 90th percentile

0.026 (.022, .030)

0.026* (.020, .032)

0.080 (.072, .088)

0.007 (.004, .010)

0.040 (.035, .045)

0.010 (.006, .014)

QT4 with Ï„ set to 130.

0.038 (.033, .043)

0.008 (.005, .011)

0.051 (.045, .057)

0.009 (.006, .012)

0.047 (.041, .053)

0.007 (.004, .010)

QT4 with Ï„ set to sample 90th percentile.

0.026 (.022, .030)

0.026 (.020, .032)

0.083 (.075, .091)

0.026 (.020, .032)

0.040 (.035, .045)

0.010 (.006, .014)

Tests of H 0, B

Wilcoxon-Mann-Whitney** with Ï„ set to 130.

0.041 (.036, .046)

0.017 (.012, .022)

0.044 (.038, .050)

0.008 (.005, .011)

0.046 (.040, .052)

0.008 (.005, .011)

Wilcoxon-Mann-Whitney with Ï„ set to sample 90th percentile.

0.049 (.043, .055)

0.014 (.010, .018)

0.065 (.058, .072)

0.015 (.011, .019)

0.080 (.072, .088)

0.018 (.013, .023)

Permutation test with Ï„ set to 130.

0.050 (.036, .064)

0.009 (.001, .017)

0.050 (.036, .064)

0.011 (.002, .020)

0.064 (.049, .079)

0.015 (.005, .025)

Permutation test with Ï„ set to sample 90th percentile.

0.077 (.060, .094)

0.016 (.006, .026)

0.078 (.061, .095)

0.022 (.010, .034)

0.083 (.066, .100)

0.019 (.008, .030)

Tests of H 0, C (new tests)

Wilcoxon-Mann-Whitney with Ï„ set to 130.

0.042 (.036, .048)

0.007 (.004, .010)

0.049 (.043, .055)

0.010 (.006, .014)

0.051 (.045, .057)

0.008 (.005, .011)

Wilcoxon-Mann-Whitney with Ï„ set to sample 90th percentile.

0.055 (.049, .061)

0.015 (.011, .019)

0.060 (.053, .067)

0.015 (.011, .019)

0.061 (.054, .068)

0.015 (.011, .019)

Permutation test with Ï„ set to 130.

0.052 (.038, .066)

0.015 (.005, .025)

0.047 (.034, .060)

0.009 (.001, .017)

0.057 (.043, .071)

0.007 (.000, .014)

Permutation test with Ï„ set to sample 90th percentile.

0.045 (.032, .058)

0.017 (.006, .028)

0.062 (.047, .077)

0.011 (.002, .020)

0.068 (.053, .084)

0.018 (.007, .029)

  1. #2-tailed 95% confidence interval.
  2. *The bolded values are those simulated type I error rates which are significantly higher than the nominal α at the 2-tailed 95% confidence level (i.e., the lower bound of the interval is higher than α). Note that for the permutation tests we used 1000 replicated datasets and for other tests we used 5000 replicated datasets.
  3. **In all the simulation studies (Tables 1-5), we used Wilcoxon-Mann-Whitney exact test.
Back to article page

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us