Skip to main content

Table 2 Approach to multiplicity due to multiple outcomes

From: Approaches to multiplicity in publicly funded pragmatic randomised controlled trials: a survey of clinical trials units and a rapid review of published trials

a) Review: multiplicity approach taken
  Formal adjustment Hierarchical testing Other approach None
PRIMARY OUTCOMES
All outcomes to be declared effective 0/8 2/8a 0/8 6/8
One or more outcomes to be declared effectiveb 4/20c 1/20d 2/20e 13/20
SECONDARY OUTCOMES
Secondary outcomes 14/136f (10%) 1/136g (1%) 2/136h (1%) 119/136 (88%)
b) Survey: responses to posed scenarios
  Yes No Unsure
PRIMARY OUTCOMES
Consider a parallel group trial with two primary outcomes. Would you adjust for multiplicity in the following scenarios?    
The trial hypotheses require both null hypotheses to be rejected? 9/27 (33%) 16/27 (59%) 2/27 (7%)
The trial hypotheses require either null hypothesis to be rejected? 16/27 (59%) 8/27 (30%) 3/27 (11%)
SECONDARY OUTCOMES
Would you adjust for multiplicity arising from multiple secondary outcomes? 3/27 (11%) 20/27 (74%) 4/27 (15%)
Would the type of outcomes (efficacy, safety, cost-effectiveness) have an impact on your response to the above question? 9/27 (33%) 17/27 (63%) 1/27 (4%)
  1. Notes: aBoth trials sequentially tested two outcomes
  2. b Includes 13 trials that stated multiple primary outcomes in the methods section, and seven that stated only one outcome but made multiple comparisons
  3. c Two trials performed a Holm correction, and two implemented a graphical multiple testing procedure
  4. d Trial sequentially tested non-inferiority then superiority
  5. e One trial recommended that p-values between 0.025 and 0.05 were considered to have borderline significance; one trial performed post-hoc analysis of the primary outcomes with one-sided 97.5% confidence intervals
  6. f Three trials performed a Bonferroni correction, five a Holm correction, one a Hochberg correction, three used a 1% threshold for significance and two used a graphical method
  7. g Formal hypothesis testing was only performed for secondary outcomes if the primary efficacy outcome was statistically significant
  8. h Formal hypothesis testing was only performed for a small number of key secondary outcomes, other secondary outcomes were just presented descriptively
Back to article page

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us