Methodological standards in non-inferiority AIDS trials: moving from adherence to compliance: Rebuttal
Jean-Jacques Parienti, INSERM, Université Pierre et Marie Curie, Côte de Nacre University Hospital
21 March 2007
We would like to thank Dr van Leth and collaborators for their interest in our study [1], which reviewed the methodology of 18 non-inferiority trials in the field of antiretroviral therapies. In their correspondence, the authors claimed that we did not report adequately the study they published in the Lancet [2]. Raising doubts about one single study may compromise the validity of our work overall. Thus, we believe their comment deserves a rebuttal.
In addition to the independent reviewing process of their correspondence, we confirm that the wording of the 2NN abstract’s conclusion (which did not distinguish nevirapine once-a-day and twice-a-day): “nevirapine or efavirenz showed similar efficacy” is ambiguous. Moreover, it is our view that this conclusion was not supported by the data presented.
First, two patients’ death were the consequence of nevirapine toxicities. The outcome would not have been “similar” for those individuals if they had received efavirenz, because no death was attributed to efavirenz.
Second, comparison of the primary outcome between nevirapine (twice-a-day) and efavirenz was not statistically significant (P=0.091) in their intent-to-treat (ITT) population. Concluding similarity from a non-significant superiority test is an exercise of self-deception and should be avoided [3], regardless of the superiority or non-inferiority/equivalence hypothesis.
Third, the authors performed an additional sensitivity analysis (excluding patients who never started treatment), possibly to assess to robustness of their result. Of note other studies in this setting, such as the CNAAB3005 or NEFA, used this “modified ITT” to draw conclusions about efficacy. For this comparison between nevirapine (twice-a-day) and efavirenz, the authors found a P=0.03, not statistically significant for superiority because of the Bonferroni’s correction (P<0.0125). However, the 95% confidence interval of the primary outcome difference (from +0.8 to +14.6) did not contain zero, suggesting the inferiority of nevirapine as compared with efavirenz in terms of efficacy among antiretroviral naïve patients. Dr van Leth and collaborators [2] stated that the results were “similar”, again.
Because some randomized patients were excluded from analysis, we used the term “on-treatment” instead of “modified ITT” in our Table 2 [1], inappropriately. In fact, the authors did not provide a formal on-treatment analysis, as shown in our Figure 2 [1]. Our definitions of the confidence intervals used in non-inferiority and equivalence designs are from the CONSORT statement, as referenced in our article [1]. We did try to contact authors, when we felt some additional data were necessary to assess the conclusion of the paper, as stated in our methods’ section [1]. Clearly, this was not the case for the 2NN study.
We agree that the merit of one drug over another drug also depends of secondary outcomes (particularly in non-inferiority/equivalence trials) such as long term complications, as we point out at the end of the discussion in our paper [1]. Regarding these two specific drugs, it appears that long term complications are different, not similar [4, 5].
We are also concerned by the credibility of published trials by clinical practitioners. In the first place, authors, helped by reviewers and editors, should avoid ambiguousness in the articles they are responsible for. Finally, we hope our work will contribute to improve the quality of future studies by the investigators and their critical appraisal by the readers, in a setting larger than the AIDS therapy research.
Université Pierre et Marie Curie, Paris6, UMR-S707, Paris F-75012 France;
Côte de Nacre University Hospital, Biostatistics and clinical research and infectious diseases departments, Caen, F-14033 France
References
1. Parienti JJ, Verdon R, Massari V: Methodological standards in non-inferiority AIDS trials: moving from adherence to compliance.
BMC Med Res Methodol 2006, 6:46.
2. van Leth F, Phanuphak P, Ruxrungtham K, Baraldi E, Miller S, Gazzard B, Cahn P, Lalloo UG, van der Westhuizen JP, Malan DR, Johnson MA, Santos BR, Mulcahy F, Wood R, Levi GC, Reboredo G, Squires K, Cassetti I, Petit D, Raffi F, Katlama C, Murphy RL, Horban A, Dam JP, Hassink E, van Leeuwen R, Robinson P, Wit FW, Lange JMA: Comparison of first-line antiretroviral therapy with regimens including nevirapine, efavirenz, or both drugs, plus stavudine and lamivudine: a randomised open-label trial, the 2NN Study.
Lancet 2004, 363(9417):1253-63.
3. Altman DG, Bland JM: Absence of evidence is not evidence of absence.
BMJ 1995, 311(7003):485.
4. van Leth F, Phanuphak P, Stroes E, Gazzard B, Cahn P, Raffi F, Wood R, Bloch M, Katlama C, ohn J. P. Kastelein JJP, Schechter M, Murphy RL, Horban A, Hall DB, Lange JMA, Reiss P: Nevirapine and efavirenz elicit different changes in lipid profiles in antiretroviral-therapy-naive patients infected with HIV-1.
PLoS Med 2004, 1(1):e19. Erratum in: PLoS Med 2004, 1(3):e73
5. Parienti JJ, Massari,V, Rey D, Poubeau P, Verdon R; for the SIROCCO study team: Efavirenz to nevirapine switch in HIV-1 infected patients with dyslipidemia: A randomized controlled study.
Methodological standards in non-inferiority AIDS trials: moving from adherence to compliance: Rebuttal
21 March 2007
We would like to thank Dr van Leth and collaborators for their interest in our study [1], which reviewed the methodology of 18 non-inferiority trials in the field of antiretroviral therapies. In their correspondence, the authors claimed that we did not report adequately the study they published in the Lancet [2]. Raising doubts about one single study may compromise the validity of our work overall. Thus, we believe their comment deserves a rebuttal.
In addition to the independent reviewing process of their correspondence, we confirm that the wording of the 2NN abstract’s conclusion (which did not distinguish nevirapine once-a-day and twice-a-day): “nevirapine or efavirenz showed similar efficacy” is ambiguous. Moreover, it is our view that this conclusion was not supported by the data presented.
First, two patients’ death were the consequence of nevirapine toxicities. The outcome would not have been “similar” for those individuals if they had received efavirenz, because no death was attributed to efavirenz.
Second, comparison of the primary outcome between nevirapine (twice-a-day) and efavirenz was not statistically significant (P=0.091) in their intent-to-treat (ITT) population. Concluding similarity from a non-significant superiority test is an exercise of self-deception and should be avoided [3], regardless of the superiority or non-inferiority/equivalence hypothesis.
Third, the authors performed an additional sensitivity analysis (excluding patients who never started treatment), possibly to assess to robustness of their result. Of note other studies in this setting, such as the CNAAB3005 or NEFA, used this “modified ITT” to draw conclusions about efficacy. For this comparison between nevirapine (twice-a-day) and efavirenz, the authors found a P=0.03, not statistically significant for superiority because of the Bonferroni’s correction (P<0.0125). However, the 95% confidence interval of the primary outcome difference (from +0.8 to +14.6) did not contain zero, suggesting the inferiority of nevirapine as compared with efavirenz in terms of efficacy among antiretroviral naïve patients. Dr van Leth and collaborators [2] stated that the results were “similar”, again.
Because some randomized patients were excluded from analysis, we used the term “on-treatment” instead of “modified ITT” in our Table 2 [1], inappropriately. In fact, the authors did not provide a formal on-treatment analysis, as shown in our Figure 2 [1]. Our definitions of the confidence intervals used in non-inferiority and equivalence designs are from the CONSORT statement, as referenced in our article [1]. We did try to contact authors, when we felt some additional data were necessary to assess the conclusion of the paper, as stated in our methods’ section [1]. Clearly, this was not the case for the 2NN study.
We agree that the merit of one drug over another drug also depends of secondary outcomes (particularly in non-inferiority/equivalence trials) such as long term complications, as we point out at the end of the discussion in our paper [1]. Regarding these two specific drugs, it appears that long term complications are different, not similar [4, 5].
We are also concerned by the credibility of published trials by clinical practitioners. In the first place, authors, helped by reviewers and editors, should avoid ambiguousness in the articles they are responsible for. Finally, we hope our work will contribute to improve the quality of future studies by the investigators and their critical appraisal by the readers, in a setting larger than the AIDS therapy research.
Jean-Jacques Parienti
INSERM, U707, Epidémiologie, Systèmes d'informations, Modélisation, Paris, F-75012 France;
Université Pierre et Marie Curie, Paris6, UMR-S707, Paris F-75012 France;
Côte de Nacre University Hospital, Biostatistics and clinical research and infectious diseases departments, Caen, F-14033 France
References
1. Parienti JJ, Verdon R, Massari V: Methodological standards in non-inferiority AIDS trials: moving from adherence to compliance.
BMC Med Res Methodol 2006, 6:46.
2. van Leth F, Phanuphak P, Ruxrungtham K, Baraldi E, Miller S, Gazzard B, Cahn P, Lalloo UG, van der Westhuizen JP, Malan DR, Johnson MA, Santos BR, Mulcahy F, Wood R, Levi GC, Reboredo G, Squires K, Cassetti I, Petit D, Raffi F, Katlama C, Murphy RL, Horban A, Dam JP, Hassink E, van Leeuwen R, Robinson P, Wit FW, Lange JMA: Comparison of first-line antiretroviral therapy with regimens including nevirapine, efavirenz, or both drugs, plus stavudine and lamivudine: a randomised open-label trial, the 2NN Study.
Lancet 2004, 363(9417):1253-63.
3. Altman DG, Bland JM: Absence of evidence is not evidence of absence.
BMJ 1995, 311(7003):485.
4. van Leth F, Phanuphak P, Stroes E, Gazzard B, Cahn P, Raffi F, Wood R, Bloch M, Katlama C, ohn J. P. Kastelein JJP, Schechter M, Murphy RL, Horban A, Hall DB, Lange JMA, Reiss P: Nevirapine and efavirenz elicit different changes in lipid profiles in antiretroviral-therapy-naive patients infected with HIV-1.
PLoS Med 2004, 1(1):e19. Erratum in: PLoS Med 2004, 1(3):e73
5. Parienti JJ, Massari,V, Rey D, Poubeau P, Verdon R; for the SIROCCO study team: Efavirenz to nevirapine switch in HIV-1 infected patients with dyslipidemia: A randomized controlled study.
Clin Infect Dis 2007, in press.
Competing interests
see [1].