The endorsement of general and artificial intelligence reporting guidelines in radiological journals: a meta-research study

Background Complete reporting is essential for clinical research. However, the endorsement of reporting guidelines in radiological journals is still unclear. Further, as a field extensively utilizing artificial intelligence (AI), the adoption of both general and AI reporting guidelines would be necessary for enhancing quality and transparency of radiological research. This study aims to investigate the endorsement of general reporting guidelines and those for AI applications in medical imaging in radiological journals, and explore associated journal characteristic variables. Methods This meta-research study screened journals from the Radiology, Nuclear Medicine & Medical Imaging category, Science Citation Index Expanded of the 2022 Journal Citation Reports, and excluded journals not publishing original research, in non-English languages, and instructions for authors unavailable. The endorsement of fifteen general reporting guidelines and ten AI reporting guidelines was rated using a five-level tool: “active strong”, “active weak”, “passive moderate”, “passive weak”, and “none”. The association between endorsement and journal characteristic variables was evaluated by logistic regression analysis. Results We included 117 journals. The top-five endorsed reporting guidelines were CONSORT (Consolidated Standards of Reporting Trials, 58.1%, 68/117), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, 54.7%, 64/117), STROBE (STrengthening the Reporting of Observational Studies in Epidemiology, 51.3%, 60/117), STARD (Standards for Reporting of Diagnostic Accuracy, 50.4%, 59/117), and ARRIVE (Animal Research Reporting of In Vivo Experiments, 35.9%, 42/117). The most implemented AI reporting guideline was CLAIM (Checklist for Artificial Intelligence in Medical Imaging, 1.7%, 2/117), while other nine AI reporting guidelines were not mentioned. The Journal Impact Factor quartile and publisher were associated with endorsement of reporting guidelines in radiological journals. Conclusions The general reporting guideline endorsement was suboptimal in radiological journals. The implementation of reporting guidelines for AI applications in medical imaging was extremely low. Their adoption should be strengthened to facilitate quality and transparency of radiological study reporting. Supplementary Information The online version contains supplementary material available at 10.1186/s12874-023-02117-x.


Introduction
Complete reporting is essential for translating results of clinical research into scientifically robust evidence to support decision-making in daily practice [1].Reporting guidelines are developed to serve as useful tools to enhance the quality and transparency of clinical research [2,3], while it is still a long-standing and widespread issue that the reporting quality is suboptimal [4,5].It is no wonder that a series of studies have repeatedly stressed the unsatisfied endorsement of reporting guideline in clinical journals [6][7][8][9][10][11][12][13][14][15][16][17][18][19], since the journals' endorsement of reporting guidelines may raise the stakeholders' awareness of them and is related to the completeness of reporting in medical journals [20].
The reporting quality of diagnostic accuracy tests and systematic reviews in Radiology, European Radiology, and Korean Journal of Radiology has been improved since it became mandatory to use the corresponding reporting guidelines [21][22][23].It is reasonable that the change of the editorial policy of one specific journal significantly influences the reporting quality.On the other hand, the reporting quality of the abstracts of randomized controlled trials in the field of interventional radiology did not show obvious improvement after the update of corresponding guidelines [24].One of the potential reasons for this difference is that the level of endorsement of reporting guidelines are different among the 61 interventional radiological journals that had been investigated.Therefore, it is necessary to investigate the endorsement of reporting guidelines, and sequent call those journals with low endorsement to make it mandatory.The earlier use of reporting guidelines during research process is considered to have greater impact on the final manuscript in radiological journals, suggesting that there is a need for enhanced education on the use of reporting guidelines [25].The reporting guidelines should be introduced to the researchers to educate them about the benefits, be used by the reviewers during their review process, and be endorsed by journals to guarantee the adherence.
The artificial intelligence (AI) has engendered a rapid increasing medical application, especially in medical imaging [26][27][28][29][30].The clinical translation of these academic research on AI should be based on scientific publication with enough details to allow readers to determine the rigor, quality, and generalizability.These publications of AI research in medical imaging calls for specific reporting guidelines for transparent and organized AI research reporting [31][32][33][34][35].In addition to general reporting guidelines, the endorsement of these AI reporting guidelines has potential to improve the reporting quality and transparency of AI research, and reshape the current radiological practice.So far, the endorsement of AI reporting guidelines has not been assessed.
Therefore, this study aimed to investigate the endorsement of general reporting guidelines and those for AI applications in medical imaging in radiological journals, and explore associated journal characteristic variables.

Study design
Our study is a meta-research study, and corresponding reporting guideline is currently not available [36][37][38][39][40][41]. Ethical approval or written informed consent were not required for this study because no human or animal subjects have been included in this study.We did not register the study protocol since there were no appropriate platform.However, we have drafted a protocol for this crosssectional meta-research study (Supplementary Note S1).The sample selection, data extraction, and guideline endorsement assessment were conducted by two independent reviewers (JYZ and YX).The statistical analysis was performed by one reviewer (JYZ) under supervision of a methodologist (JJL).Any discrepancies will be resolved by discussion or consulting with the review group.Our study group is consisted of reviewers with diverse background and knowledge, including radiologists, and health professionals from multiple disciplines.All the reviewers have experience in manuscript drafting and publishing.Some of the reviewers have served as reviewers for radiological journals and evidence-based medicine journals, as well as editorial board members for radiological journals.

Sample journals
The journals from the Radiology, Nuclear Medicine & Medical Imaging category, Science Citation Index Expanded of the 2022 Journal Citation Reports were identified via Clarivate website [42], and assessed for eligibility.The exclusion criteria were (1) journals not publishing original research; (2) journals published in non-English languages; and (3) journals lacking instructions for authors.

Data extraction
The following bibliometrics information was directly downloaded via Clarivate: journal name, 2022 Journal Impact Factor (JIF), and the JIF quartile (Q1, Q2, Q3, Q4).The following journal characteristics were extracted: publication region, publication institution/ publisher, publication language, publication frequency, type of access, whether the journal is only in the Radiology, Nuclear Medicine & Medical Imaging category, and whether the journal is owned by an academic society [43,44].The official website address of each journal was recorded.The search and data extraction via Clarivate was carried out on 20 July 2023.
The modified 5-level tool rated the endorsement into: (1) "active strong": journal requires completed checklist and/or flow diagram with article submission; (2) "active weak": journal encourages to flow a specific guideline; (3) "passive moderate": journal only requires the abstract to follow a specific guideline; (4) "passive weak": journal encourages to prepare manuscripts according to EQUA-TOR Network website [70] or the International Committee of Medical Journal Editors document [71]; and (5) "none": journal does not mention any reporting guideline [19].The assessment on journal's endorsement of reporting guidelines were based on their documents on submission (instructions for authors, submission guideline, editorial policies, etc.).The endorsement of reporting guidelines was assessed from 22 July 2023 to 23 July 2023.

Statistical analysis
The statistical analysis was conducted by using R language version 4.1.3within RStudio version 1.4.1106[72][73][74][75].The endorsement types of "active strong", "active weak", "passive moderate", and "passive weak" were considered as a positive outcome, while the type of "none" was treated as a negative outcome.The journal was considered as positive if at least one reporting guideline was positive; otherwise, the journal was treated as negative.The journal characteristics between the positive and negative groups were compared.The journal characteristics were further evaluated by logistic regression analysis to tell whether they are associated with the reporting guidelines endorsement.The factors associated with the endorsement of general reporting guidelines and those for AI applications in medical imaging were assessed together, because the endorsement of reporting guidelines for AI applications in medical imaging were very low.All of the statistical tests were two-sided.The alpha level for statistically significance is set at 0.05, if not stated otherwise.

Journal inclusion
There were 135 journals in the Radiology, Nuclear Medicine & Medical Imaging category, Science Citation Index Expanded of the 2022 Journal Citation Reports.We excluded 10 journals that only publish reviews, 7 non-English journals, and 1 journal without available website.Finally, 117 radiological journals were included (Fig. 1).

Journal characteristics
The mean ± standard deviation, median (range) of 2022 JIF of included journals was 3.7 ± 2.8, 2.9 (0.6 to 19.7) (Table 1 2).The characteristics of each journal are presented in Supplementary Tables S1 and S2.The original data for analysis are presented in Supplementary Data Sheet.
The most implemented reporting guidelines for AI applications in medical imaging was CLAIM (1.7%, 2/117), while other nine artificial intelligence reporting guidelines were not mentioned in documents on submission of radiological journals.The examples for five levels of endorsement are presented in Supplementary Table S3.The endorsement of reporting guidelines of each journal is presented in Fig. 3 and Supplementary Table S4.

Factors associated with the endorsement of reporting guidelines
The journal characteristics did not show difference between the positive and negative groups, except for the distribution of publisher (Table 1).The multivariable logistic regression analysis showed that JIF quartile and publisher were associated with the endorsement of reporting guidelines in radiological journals (Table 3).The JIF Q2 journals were more likely to endorse the reporting guidelines than JIF Q1 journals (odds ratio 7.83, 95% confidence interval 1.70-36.20,P = 0.008).The journals that are published by academic societies (0.17, 0.04-0.64,P = 0.009), Lippincott Williams & Wilkins Fig. 1 Flowchart for radiological journal inclusion (0.13, 0.03-0.68,P = 0.016), and other publishers (0.03, 0.002-0.33,P = 0.004) were less likely to endorse the reporting guidelines than those published by Springer.

Discussion
The endorsement of general reporting guidelines was lowest for SPRIT, PRISMA-P, AGREE, RIGHT, and COREQ (all 32.5%) in radiological journals, and highest for CONSORT (58.1%).Only two journals suggested to implant the CLAIM (1.7%), while the other nine evaluated reporting guidelines for AI applications in medical imaging were not mentioned.The JIF quartile and publisher were associated with the endorsement of reporting guidelines in radiological journals.
The CONSORT statement was one of the most early developed reporting guidelines for randomized clinical trials, followed by a brunches of reporting guidelines published covering main study types.In accordance to the previous studies, the CONSORT statement has been most commonly endorsed by journals in varying specialties [6][7][8][9][10][11][12][13][14][15][16][17][18][19].The methodology of randomized clinical trials was less difference among journals from different specialties, and the CONSORT statement met the requirements for most of the journals.Likewise, the reporting guidelines for systematic reviews,

Table 1 Characteristics of included radiological journals
The endorsement types of "active strong", "active weak", "passive moderate", and "passive weak" were considered as a positive outcome, while the type of "none" was treated as a negative outcome.The independent t test was used for comparing the 2022 JIF between positive and negative groups, while the chi-square tests were used for comparing other characteristics between positive and negative groups observational studies, and animal studies were also with relatively high endorsement, because their methodology were also of less variety.The radiological journals further recommend to use STARD statement, since the diagnostic accuracy test plays an important role in radiology [21,22].However, the TRIPOD statement has not been widely accepted by the radiological journals.It is possible that the study type of multivariable prediction model is less conducted than the diagnostic accuracy test.The reporting guidelines for other study types were less endorsed by radiological journals.It is possible that the protocol, and case report are not an acceptable study type for some of the journals, and guidelines, qualitative research, qualitative research, quality improvement, and economic evaluations were not common study types for radiological journals.
In addition to general reporting guidelines, the reporting guidelines for AI applications in medical imaging were also evaluated in our study.Unfortunately, the endorsement of investigated AI reporting guidelines was extremely low in radiological journal.Only European Radiology and Journal of the American College of Radiology recommended CLAIM for AI studies, although the CLAIM was published on the Radiology: Artificial Intelligence [65].The MAIC-10 and CLEAR were two guidelines recently published on Insights into Imaging for AI studies and radiomics studies [66,69], and the IBSI statement for radiomics was introduced by Radiology [68].However, they have not been widely implanted by radiological journals even the journals published them.In contrast to the low endorsement in radiological journals, these reporting guidelines were usually applied for systematic reviews [76][77][78][79][80][81][82][83][84][85].These systematic reviews found that the adherence rate of RQS, IBSI, and CLAIM were suboptimal for radiomics and AI studies for medical imaging.It is not weird since most of the radiological journals did not endorse these reporting guidelines.The reporting transparency has been improved after the introduction of reporting guidelines [20][21][22][23].It is expectable to make the reporting guidelines for AI applications in medical imaging mandatory to improve the awareness and application of them, in order to achieve transparent AI and radiomics study reporting.The AI and radiomics community should understand the importance of proper self-reporting, and encourage researchers, journals, editors, and reviewers to take action to ensure the proper usage of checklists [82][83][84][85].We plan to investigate the influence of the endorsement of the reporting guidelines for AI applications in medical imaging on the quality of study reporting.
We found that the JIF quartile and publisher were associated with the endorsement of reporting guidelines in radiological journals.The journals with higher JIF were generally more likely to endorse the reporting guidelines than lower ones [6,15,17,18].It is reasonable that the journals with higher impact factors have higher endorsement of reporting guidelines, since they were considered to have higher quality.However, our study showed that the JIF Q2 journals were more likely to endorse the reporting guidelines than JIF Q1 ones in radiological journals.The underlying reason is not yet clear.The journals that were published by Springer showed higher endorsement of reporting guidelines in both surgery and radiological journals [17].We infer that the higher endorsement was benefited by the unified editorial policy of Springer that recommended the specific reporting guidelines and EQUATOR Network website.The surgical journals from United Kingdom and Europe were more likely to endorse the reporting guidelines than those from North America [17], but we did not find the influence of publication region in radiological journals.
The requirement of adherence to reporting guidelines may improve reporting quality [20][21][22][23], and the agreement among journals on the endorsement of reporting guidelines could improve the quality of research publishing [86].Rather than prioritizing additional studies on the poor quality of health research reporting, interventions are needed to improve reporting [87].However, only a limited number of tools has been raised for this purpose, and a smaller number of them has been evaluated [88].
There was the only one that showed a statistically significant effect on reporting quality was a Consort-based WEB tool [89,89].This tool supports adherence at the manuscript writing stage.The earlier the reporting guidelines were used, the more impact on the final manuscript and higher perceived value [25].Therefore, it has been repeatedly suggested to enhanced education on the use of these guidelines.It seemed to be most pivotal to support journals to include hard-wiring adherence to reporting guidelines into their editorial policy.It may be an effective way to make the reporting guidelines mandatory, and ask the reviewers to use the reporting guidelines during the review process.The reporting quality has been improved if the journal required authors to incorporate section headings that reflected CONSORT items into their manuscripts [90].Although it has not been found  that these interventions are effective in radiological journals, the authors, reviewers, and editors may actively use the reporting guideline to guide themselves in the study design, conduction, drafting, reviewing, and revision.Further survey is need to identify the obstacles against the endorsement of reporting guidelines in journals.The potential reasons for suboptimal endorsement of reporting guidelines includes the insufficient resources for mandatory use, the needs to reduce barriers to submission and review, the unique editorial perspective, or the use of alternative to improve the study quality [16,17].As the current study may aware the radiological community on the issue of suboptimal endorsement of reporting guidelines.We plan to re-evaluate the endorsement in the future to find out whether the publication of radiological research is reshaped.
Our study has limitations.First, our study only included the radiological journals in the Science Citation Index Expanded.We selected these radiological journals as a representative sample of the high-quality journals of this field.Our results may over-estimate the journals' endorsement of the reporting guideline.Second, our study is a cross-sectional study only presenting the current endorsement of reporting guidelines in radiological journals is suboptimal.An update study should be conducted in the future.Third, we relied on online documents to assess the endorsement without verifying the additional instructions potentially appear during the manuscript submission and review process.
Although we collected and cross-checked the journal information, the journal websites may be update afterwards.Changes in editorial policies may not be timely updated online, which may influence on our conclusion.Fourth, we only investigated a limited number of general reporting guidelines and those for AI applications in medical imaging.There are a lot of reporting guidelines developed for more specific purpose and are available on EQUATOR Network website.Although our study did not cover all the available guidelines, we considered that the included reporting guidelines can at least represent the usually-used ones.We did find many extra AI guidelines which was not included in our study [91][92][93][94][95].It would be interesting to investigate the endorsement of AI reporting guidelines which is not specifically designed for AI application medical imaging.Finally, our study only emphasized the

Conclusion
As a summary, our study found that the general reporting guideline endorsement are suboptimal in radiological journals.The implementation of artificial intelligence reporting guidelines was extremely low.Radiological journals may consider making general and artificial intelligence reporting guidelines mandatory to improve their awareness and application, in order to achieve high-quality and transparent radiological study reporting.
• fast, convenient online submission • thorough peer review by experienced researchers in your field • rapid publication on acceptance • support for research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations maximum visibility for your research: over 100M website views per year

Fig. 3
Fig. 3 Endorsement of each reporting guideline according to radiological journals.The left part presents the endorsement of 15 general reporting guidelines.The right part presents the endorsement of 10 reporting guidelines for AI applications in medical imaging.Abbreviation: Q1 to Q4 = the first to the forth Journal Impact Factor quartile 92. Collins GS, Dhiman P, Andaur Navarro CL, Ma J, Hooft L, et al.Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence.BMJ Open.2021;11(7): e048008.93.Sounderajah V, Ashrafian H, Rose S, Shah NH, Ghassemi M, Golub R, et al.A quality assessment tool for artificial intelligence-centered diagnostic test accuracy studies: QUADAS-AI.Nat Med.2021;27(10):1663-5.94.Olczak J, Pavlopoulos J, Prijs J, Ijpma FFA, Doornberg JN, Lundström C, et al.Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal.Acta Orthop.2021;92(5):513-25.95.Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, et al; DECIDE-AI expert group.Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI.Nat Med.2022;28(5):924-933.
Abbreviation: SD standard deviation, JIF journal impact factor, LWW Lippincott Williams & Wilkins

Table 2
Endorsement levels of reporting guidelines in radiological journals
•At BMC, research is always in progress.