Ronan Conroy, Royal College of Surgeons in Ireland
10 August 2005
A distinction needs to be made between tests of multiple hypothesis and multiple tests of the same hypothesis. If a study of treatment effectiveness uses multiple endpoints, then finding a significant effect for any of the endpoints will lead to the conclusion that the treatment is effective. For this reason, protection against a false positive conclusion has to be based on keeping the family-wise error rate to 5%.
However, many studies simultaneously test several hypotheses. I see no reason why we should penalise these designs. Replication, or failure to replicate, seem to me to offer sufficient security in these situations.
What are multiple hypothesis tests?
10 August 2005
A distinction needs to be made between tests of multiple hypothesis and multiple tests of the same hypothesis. If a study of treatment effectiveness uses multiple endpoints, then finding a significant effect for any of the endpoints will lead to the conclusion that the treatment is effective. For this reason, protection against a false positive conclusion has to be based on keeping the family-wise error rate to 5%.
However, many studies simultaneously test several hypotheses. I see no reason why we should penalise these designs. Replication, or failure to replicate, seem to me to offer sufficient security in these situations.
Competing interests
None