Assessing transparency practices in dental randomized controlled trials

Abstract

Background

To evaluate transparency practices in randomized controlled trials (RCTs) in dentistry.

Methods

This meta-research study included RCTs in dentistry regardless of topic, methods, or level of detail reported. Only studies in English were considered. We searched PubMed for RCTs in dentistry published in English from December 31, 2016, to December 31, 2021. The screening was performed in duplicate, and data extracted included journal and author details, dental specialty, protocol registration, data and code sharing, conflict of interest declaration, and funding information. A descriptive analysis of the data was performed. We generated maps illustrating the reporting of transparency items by country of the corresponding author and a heat table reflecting reporting levels by dental specialty.

Results

A total of 844 RCTs were included. Only 12.86% of studies reported any information about data and code sharing. Protocol registration was reported for 50.36% of RCTs. Conflict of interest (83.41%) and funding (71.68%) declarations were present in most studies. Conflicts of interest and funding were consistently reported regardless of country or specialty, while data and code sharing had a low level of reporting across specialties, as well as low dissemination across the world. Protocol registration exhibited considerable variability.

Conclusions

Considering the importance of RCTs for evidence-based dentistry, it is crucial that everyone who participates in the scientific production and dissemination process actively and consistently promotes adherence to transparent scientific standards, particularly registration of protocols, and sharing of data and code.

Introduction

The primary objective of healthcare research is to enhance the quality of patient care. Achieving this necessitates the translation of study results, particularly those of randomized controlled trials (RCTs), into clinical practice by the biomedical community [1, 2]. Well-planned, well-executed, and well-reported RCTs, which embrace transparent and open science practices, enhance the likelihood that results are accurate, reliable, and applicable [2, 3]. However, the literature contains studies with questionable methodologies, missing or incomplete data, false or exaggerated effect measures, and many studies do not mention funding or potential conflicts of interest [4,5,6,7,8]. All these factors can erode clinicians’ and patients’ confidence in health research, prompting concerns about the costs and societal risks associated with RCTs [9].

As a potential solution, researchers can employ practices such as reporting data and code sharing, making protocol registration available, and disclosing conflicts of interest and funding which make research more transparent. Protocol registration also has the role of preventing research misconduct, reduce the potential for incomplete or selective reporting of results and minimize unintentional duplication of studies. By sharing data and code, additional analyses (which can generate new hypotheses and future research) are enabled, as well as data verifiability [10,11,12]. Reporting funding and conflicts of interest helps prevent bias; reinforces researchers’ commitment to conducting research impartially; and enables readers, reviewers, and editors to better assess results by understanding potential external influences [13, 14].

Studies indicate that, although many authors agree on the importance of these transparent practices, they are often neglected [6, 7]. Some authors cite concerns about the confidentiality of data from research participants, the inappropriate use of secondary data, and lower rewards for conducting original research [15, 16]. Consequently, recent years have seen intensified efforts to promote transparent and open science practices in biomedical research. Several major clinical research funders and biomedical journals have adopted policies supporting or mandating the use of reporting guidelines such as CONSORT [10, 17,18,19], which include encouraging protocol registration, conflict of interest disclosure, and funding. Secondary users of RCT data, such as the Cochrane Collaboration, advocate for stronger data-sharing policies to increase access to clinical trial data with the aims of testing the reliability of medical evidence and improving evidence-based practice [20].

These initiatives, along with increased discussions about transparency in biomedical research, may have contributed to improvements. However, evidence is scarce in dentistry, and there are opportunities for discoveries and the evolution of research practices. Therefore, this study aimed to evaluate the transparency characteristics reported by authors of RCTs in dentistry in recent years.

Methods

The protocol for this study was registered on the Open Science Framework platform [https://osf.io/qbg9n/]. The current meta-research study aimed to evaluate transparency characteristics of RCTs in dentistry.

Eligibility criteria

RCTs in dentistry were considered as described by Friedman et al. [21]. RCTs needed to be related to the evaluation, diagnosis, prevention, and/or treatment of diseases, disorders, and/or conditions of the oral cavity, maxillofacial and/or adjacent areas, or associated structures. RCTs that discussed educational aspects of dentistry were also included. Studies were included regardless of the methods used or their level of reporting. However, due to a lack of funding to translate articles, studies published in languages other than English were excluded.

Search

We searched for reports of RCTs in dentistry indexed from December 31, 2016, to December 31, 2021. The search strategy was developed based on MeSH terms from PubMed. A specific filter for RCTs was used. The search strategy can be found in the supplementary material.

Screening

In Microsoft Excel, we randomly selected 20 references from the search to perform a pilot test of screening. The screening of studies for eligibility criteria was conducted in duplicate and independently using the online review software DistillerSR (Evidence Partners Incorporated, Ontario, CA). Two reviewers screened all titles and abstracts without consulting each other during the decision process. Retrieved records were classified as “include,” “exclude,” or “uncertain.” Subsequently, the same two researchers independently analyzed the full-text articles of the included and uncertain records. Discrepancies in screening titles, abstracts, or full texts were resolved through discussion between the two reviewers, and if necessary, the opinion of a third reviewer was solicited.

Sample

The sample in this study is part of a larger project that evaluated women’s participation in science [22]. A sample size calculation was performed considering the estimated identification of approximately 2500 RCTs (533 RCTs were indexed in PubMed in 2017) [17], using the OpenEpi software. The estimated minimum sample size to find associations, considering a probability of error of 5% (α = 0.05), power (1-β) of 80%, equal proportion of exposed and unexposed (women and men), and an estimated effect size of odds ratio (OR) = 1.5 based on a previous study on the contribution of female teams [23], was 844 studies. An Excel list of random numbers containing all articles classified as included was used to randomly select the 844 studies. The selection of the 844 studies also considered the proportion of articles indexed per year (i.e., if 10% of the studies were indexed in 2021, then 84 studies were selected from that year). In the case of multiple reports from the same study, the most recent report was used.

Data extraction

Data extraction was performed using the same screening software (DistillerSR). Initially, we conducted pilot data extraction on a random sample of 20 included RCTs, obtained using a list of random numbers in Excel. The pilot test was conducted through discussion between the reviewers to ensure consistency in the interpretation of items. Subsequently, two reviewers extracted data from half of the included articles each, and a third reviewer verified the consistency of the data. Data were re-extracted in cases of doubt or inconsistency.

The following data were extracted: journal data (name, publication model, and impact factor for the year 2022), number of authors, country of the corresponding author, subject of the article (based on dental specialties recognized by the Federal Council of Dentistry of Brazil) [24], and total number of citations. We extracted the following data on transparency practices: report of protocol registration (in the case of reporting it was classified as reported or reported not registered), report of data and code sharing (if reported, this was classified as available, upon request, not available, or unclear), conflict of interest declaration (when declared, it was assessed whether or not the conflict existed), and statement of financial support. The funding type, when a funding statement was present, was classified as a non-profit sponsor, a for-profit sponsor, mixed, no funding, or unclear.

Data analysis

The analyses were conducted using Excel. All descriptive analyses used frequency for categorical data and median and interquartile range for continuous data. For the analyses presented below, the reporting of transparency items was dichotomized into “reported” and “not reported,” regardless of what was mentioned in the report. For example, RCTs that reported not registering the study protocol were classified as “reported” for the category, even if in fact registration was not carried out.

Excel was also employed to generate maps illustrating the number of RCTs reporting transparency items, categorized by the corresponding authors’ countries. Countries depicted in white did not have any RCTs assigned to them. Countries shown in gray produced RCTs that did not report the transparency items assessed. Countries that produced RCTs that reported items are color-coded in shades of purple that reflect the level of reporting. The darker a country’s color on the map, the higher the reporting of the items in the RCTs attributed to that country.

Additionally, a heat table was created to demonstrate the reporting of transparency characteristics by dental specialty. Studies were excluded if the specialty (1) had fewer than five studies, (2) could not be clearly identified, or (3) was categorized as “other.” In this table, the higher the percentage of reporting of the transparency items, the closer the cell’s color to green. A red cell represents low reporting of the observed item. A total value of the reporting of transparency items was generated for each specialty. This total value refers to the average reporting of the four items evaluated for each specialty.

Results

A total of 5,557 studies were identified in PubMed through the search strategy developed. Of these, 3,512 met the eligibility criteria, and 844 were included, as indicated by the sample size calculation. Further elucidation of the screening process outcomes is available in a prior publication [22]. A table containing all 844 RCTs included is available in the supplementary material.

Table 1 presents the characteristics and transparency practices of the included RCTs. The studies had a median of five citations. A small portion of studies (47, 5.56%) received a high number of citations (25 or more). The impact factor of most journals that published the included RCTs ranged between 2 and 3.999 (52.36%). Furthermore, a few journals were fully open access (26.46%), while the majority provided hybrid access (62.91%).

Table 1 Basic characteristics and practices of transparency in the randomized controlled trials included

As shown by the transparency attributes outlined in Table 1, only 12.68% of RCTs reported any information about sharing data and codes. Among those that reported such sharing, availability upon request was the predominant response (80, 9.48%). In contrast, most studies reported registering the RCT protocol (50.36%), declared information about conflicts of interest (83.41%), and disclosed their financial support (71.68%). A large portion of the studies that reported funding mentioned being funded by a non-profit (37.32%).

Figure 1 presents maps delineating the transparency reporting levels of the included dental RCTs, categorized by the country of the corresponding author. In the map illustrating the reporting of data and code sharing, we observe a low level of reporting and a limited diffusion of this practice globally. Countries that showed a higher level of reporting for this practice (darker shades of purple) maintained the same pattern for other evaluated items. On the protocol registration map, light and dark shades of purple predominate, indicating that the reporting of this item is still quite variable worldwide. The protocol registration report gained more prominence in the Americas and European countries, in addition to those already visible on map A. In the remaining two maps, darker shades of purple predominate, indicating a high level of reporting of conflicts of interest and funding sources in virtually all countries with RCTs included in our analysis. Notably, we observed the absence of RCTs attributed to African countries and various South American countries. The percentages of RCTs reporting the transparency items by country can be found in the supplementary material.

Fig. 1

Map of the reporting level of transparency practices considering the country of the corresponding author of the included dental RCTs. Countries in white did not have RCTs assigned to them; countries in gray did not report the transparency items evaluated and; countries in shades of purple reported the evaluated transparency items at different levels (the darker the shade of purple, the higher the indicator reporting)

Figure 2 shows transparency practices according to dental specialty. The most frequently reported transparency practices across all specialties were the declaration of conflicts of interest and funding, with percentages above 67.24% and 54.55%, respectively. The declaration of conflicts of interest was reported in 100% of studies in the specialties of dentistry for patients with special needs and jaw and facial orthopedics. Data reporting and code sharing were poor across all specialties, with percentages below 23.08%. The reporting of protocol registration exhibited considerable heterogeneity across specialties, ranging from 84.62% for dentistry for special needs patients to 25.00% for jaw facial orthopedics. In general, the specialty that reported the fewest transparency items in the RCTs analyzed was pediatric dentistry (48.28%), and the specialty that most employed these practices was dentistry for special patients (72.24%).

Fig. 2

Report of transparency practices according to the dental specialties identified in the RCTs. Each cell is colored according to the reporting level of the indicators: red - low reporting (0%), yellow - intermediate reporting (50%) and green - high reporting (100%). *The total value refers to the average report of the four items evaluated for each specialty

Discussion

This is one of the first studies to assess transparency practices in dental RCTs. Previous studies on this topic focused their assessments on specific dental specialties, such as pediatric dentistry [25], or on specific transparency practices, such as data and code sharing [26]. Our study provides a global overview of the dental field as well as addresses different transparency practices. Our analysis reveals that the reporting of items such as conflicts of interest and funding is well established regardless of the dental specialty or country associated with RCTs. However, reporting of items such as data and code sharing remains suboptimal. Adhering to transparency standards is essential because the safety; benefits; and social, academic, and scientific value of an RCT depend on these factors.

Our findings align with previous biomedical and dental research, which reported the presence of more articles with statement of conflict of interest and funding than data and code sharing and protocol registration [27,28,29,30]. In dental research, concerns regarding conflicting interests and potential financial sponsorship, particularly from industry, have persisted for decades [31]. These factors can affect RCTs from conception to the reporting of results and conclusions [13, 32, 33]. Higher percentages of reports of conflicts of interest and funding disclosures, consistently across the specialties and countries observed, may be attributed to the journals’ stricter policies that reinforce the obligation of these declarations compared to other transparency practices.

The reporting of protocol registration exhibited great variation among countries and specialties. However, compared to dental research involving other types of studies [29, 34], our analysis revealed the highest value for the reporting of protocol registration (50.36%). This value aligns with another study on pediatric dentistry RCTs [25]. This discrepancy among studies could be attributed to the nature of the RCTs evaluated since platforms such as ClinicalTrials.gov have advocated for RCT protocol registration since 2000. However, protocol registration is applicable to and recommended for all studies and is supported by various platforms, such as the Open Science Framework [3, 29, 35,36,37]. Endorsement of the use of the CONSORT statement [38, 39] has been observed for many years and is prevalent in many journals. Considering that the statement encourages both the registration of studies and the declaration of funding and conflicts of interest, we can expect higher reporting of these characteristics compared to the sharing of data and codes.

Data and code sharing has been more recently emphasized in the literature, as also noted in our study by the limited uptake in RCTs worldwide. Several biomedical journals already mandate data sharing as a prerequisite for publishing clinical trials (e.g., British Medical Journal and PLOS Medicine). However, few dental journals impose this practice as mandatory; 60% only recommend it [39]. Siebert et al. (2020) [40] emphasized that an undemanding editorial policy results in a lack of adherence on the part of researchers. When examining the reporting of transparency items in each specialty, we identified that the lowest reporting rate for all specialties was data and code sharing. Prior analyses have shown that dental articles seldom employ multiple transparency practices simultaneously [28], and data and code sharing has been the least reported aspect in many studies [25,26,27,28,29,30, 41]. Given the varying degrees of endorsement of open science practices among dental journals [39], it is now imperative for dental journals to adopt similar requirements to ensure balanced adherence to transparency standards.

We acknowledge certain limitations of our study. First, we included only articles in English and from a single database, potentially restricting the generalizability of our findings. Additionally, data extraction was not conducted in duplicate; however, a pilot test and the involvement of a third reviewer were implemented to ensure data consistency, both within the study and between reviewers. Finally, the scientific production of RCTs varies between countries and specialties. Thus, some countries, such as Russia, South Africa and Indonesia, only had one ECR assigned to them. This fact limits the percentage of adherence to the evaluated practices to extremes, 100% in the case of reporting a certain evaluated item or 0% in the case of the item not being reported.

Transparent and reproducible science offers well-established benefits: enhanced research reliability and credibility, increased applicability of results, and reduced bias and waste of resources [10,11,12,13,14]. While improvements in transparency rates have been documented, there remains room for improvement, particularly, protocol registration, and data and code sharing. Funders, universities, research authorities, reviewers, and journal editors should prioritize standardizing and uniformly demanding these practices. All participants in the scientific production process are responsible for actively promoting transparent scientific practices.

Conclusion

Our findings highlight an imbalance in the reporting of transparency items. Regardless of country or specialty, there was consistent and comprehensive reporting of conflicts of interest and financial support. However, the reporting of data and code sharing was deemed suboptimal, and protocol registration exhibited considerable variability. Given the critical role of RCTs in evidence-based dentistry, all stakeholders in the scientific production and dissemination process must advocate for active and uniform adherence to transparent scientific standards. This promotion should focus on protocol registration, and the sharing of data and code.