Results from this study strongly suggest that most RCTs of anti-infectives consistently employ ITT or mITT, which should provide the most conservative estimate of effect size (with the possible exception of non-inferiority trials). Few studies (4 %) described PP as their primary approach. This is in harmony with current best practices and recommendations for appropriate conduction of RCTs, recommending a multi-faceted approach to data analysis that centers on use of ITT [4]. This is also an improvement compared to previous evaluations of musculoskeletal and ear, nose, and throat (ENT) RCTs, where fewer studies (68 % and 12 %, respectively) used ITT or mITT as the primary data analysis population [6, 8]. While use of ITT is not specifically recommended in CONSORT, the relatively consistent use of ITT and mITT supports CONSORT Item 13, which advocate for clarity in describing the numbers of patients analyzed for each outcome, and providing reasons for patient exclusions [1].
Although the overall results regarding use of ITT in this sample of clinical trials were positive, a substantial minority of studies (17 %) failed to clearly define their primary population of analysis as recommended in CONSORT [1]. This failure decreases study transparency and puts the onus on the reader to determine whether an appropriate population was used when interpreting results. Misinterpretation of study group size may cause significant ramifications to study applicability, particularly in subspecialties such as infectious diseases where sample size might be lower than ideal. Additionally, considerable inconsistency between the primary population defined in methods and the one in used to illustrate results was anecdotally observed. These types of discrepancies can be particularly concerning when ITT is described in the methods, but less conservative mITT, or even PP, analyses are described in the study results. This could contribute to systematic overestimation of observed effect size of anti-infectives.
The most common modification used in mITT analyses required analyzed patients to receive at least one dose of medication (48 %). This minor modification would not be expected to greatly alter the observed effect size compared to ITT, and the finding is similar to a previous study where this was the most common modification (56 %) [15]. From a clinician’s perspective, a mITT population, depending on the modification, may provide a more representative picture of actual clinical practice. Practically though, increasing the number of modifications can introduce bias in the form of changing the overall study into a per-protocol analysis. Study authors and readers alike must balance these competing interests when designing and reading this type of manuscript. To the authors’ knowledge, there is no standard recommendation for the number or types of modifications considered appropriate for a mITT analysis. On a related note, it was observed that mITT was frequently miscategorized as ITT, which could be misleading to the reader depending on the degree of modification. This underscores previously reported findings that as few as 42 % of published RCTs claiming to use ITT actually assessed all randomized patients in their primary analysis [5].
A problem with ITT trials that is somewhat unique to infectious diseases research is the issue of how to account for patients who are randomized to a treatment group, but never positively have an established infection. Their inclusion into the study population while true to an ITT model provides increased noise, with limited benefit in terms of determining the overall effectiveness of an intervention. Simply excluding this population can also be viewed in a different light as a deviation from standard practice and less “real-world” in extrapolation. Our results suggest that microbiological confirmation of infectious is less commonly used as a mITT modification.
The overwhelming majority of RCTs that used ITT or mITT did not describe how investigators accounted for missing patient data due to missed visits or dropouts. These results were worse than prior studies where this information was provided in 58 to 65 % of RCTs [6, 16]. To the investigators’ knowledge, there is no current, rigorous standard regarding an ideal approach to this practice. The European Medicines Agency (EMA) has acknowledged lack of consensus in this area and suggests preferential use of more conservative approaches to handling missing data [17]. Some commonly used approaches to missing data (e.g., best observation carried forward [BCOF], LCOF) could easily result in a more “best case” result that might not truly reflect patient outcomes [4]. Other approaches, such as assuming treatment failure may be preferable as they would result in a more conservative estimation of effect size, especially in subspecialties such as infectious diseases, where dichotomous clinical endpoints (70 % in this study) are commonly employed [4, 17, 18]. Guidelines from EMA suggest assuming treatment failure may be most appropriate for clinical trials investigating response-type of endpoints, which comprised the majority in this study [17]. A positive finding was that the most commonly described approach was assuming treatment failure (13 %).
ITT or mITT was employed in the majority (97 %) of non-inferiority trials. However, it has been recommended that non-inferiority trials, and possibly all RCTs, should describe both ITT and PP results, and facilitate the reader’s focus on the more conservative findings [4, 13, 19–22]. Results from this study suggested that about half of non-inferiority trials of anti-infectives currently report both ITT and PP. This is another opportunity for improvement in transparency of study reporting that could yield more clinically applicable results.
Having focused on infectious disease medications, future studies could examine additional therapeutic classifications, such as cardiovascular medications. Additionally, subsequent work could focus on non-inferiority trials on a more global scale. A recent cross-sectional study of non-inferiority trials identified serious shortcomings in terms of reporting and justifying non-inferiority margins, and other information needed to interpret non-inferiority [23]. A broader assessment of non-inferiority trial reporting focused on data analysis populations could identify additional concerns. Along with a broader scope focusing on different populations, a more longitudinal approach spanning more years could also be constructed to evaluate not only the prevalence of the use of ITT over the years, but also how this modification to the effect size of studies changes with different approaches to RCTs.
Results from this study will be applicable for medical professionals in health systems, especially those with a role in antimicrobial stewardship or direct patient care. However, it should be cautioned that the consistent use of ITT in this sample may not be extrapolated to other subspecialties. Similarly, the proportion of non-inferiority trials used in infectious diseases is somewhat higher than expected in other disciplines, which further limits generalizability to other therapeutic areas [7, 24]. Another potential limitation is that the search purposefully focused on identifying and evaluating the most impactful infectious diseases RCTs published in highly regarded journals, and these results themselves may represent a “best case” scenario.
Results from this study highlight key information to consider when evaluating study methods and results, and a general lack of transparency in study reporting. Without a thorough understanding of the types and appropriateness of data being presented in a study, decisions about the use of a medication for a patient or its addition to the hospital formulary become dependent on the study authors’ presentation and conclusions. Lack of transparency could contribute to misinterpretation of important findings or use of an agent outside of its intended target population. Furthermore, the important shortcomings identified in this study will also be important for journal editors and peer reviewers to consider when evaluating RCTs, particularly those of anti-infectives, for publication.