There is still room for improvement in the completeness of abstract reporting according to the PRISMA-A checklist: a cross-sectional study on systematic reviews in periodontology

Background To evaluate the completeness of reporting abstracts of systematic reviews (SRs) before and after the publication of the PRISMA-A checklist in 2013 and to assess if an association exists between abstract characteristics and the completeness of reporting. Methods A systematic search of the literature was conducted in the PubMed and Scopus databases in March 2020. The search focused on the SRs of evaluations of interventions published since 2002 in the field of periodontology. The abstracts of the selected SRs were divided into two groups before and after publication of the PRISMA-A checklist in 2013, and compliance with the 12 items reported in the checklist was evaluated by three calibrated evaluators. Results A set of 265 abstracts was included in the study. The total score before (mean score, 53.78%; 95% CI, 51.56–55.90%) and after (mean score, 56.88%; 95% CI, 55.39–58.44%) the publication of the PRISMA-A statement exhibited a statistically significant improvement (P = 0.012*). Nevertheless, only the checklist items included studies and synthesis of the results displayed a statistically significant change after guideline publication. The total PRISMA-A score was higher in the meta-analysis group and in articles authored by more than four authors. Conclusions The impact of the PRISMA-A was statistically significant, but the majority of the items did not improve after its introduction. The editors and referees of periodontal journals should promote adherence to the checklist to improve the quality of the reports and provide readers with better insight into the characteristics of published studies. Supplementary Information The online version contains supplementary material available at 10.1186/s12874-021-01223-y.


Background
Systematic reviews (SRs) and meta-analyses (MAs) are located at the top of the scientific evidence pyramid, offering the highest level of scientific evidence on clinical protocols and diagnostic procedures [1]. They are one of the most cited published items and are used by research stakeholders to appraise, synthetize, and apply evidence [2]. Notably, SRs employ a study design that attracts more online attention and accrue great interest outside academia [3]. Scientific production has experienced a fast increase in recent decades, with a current production of more than 2.5 billion articles, doubling every 9 years [4]. The number of published systematic reviews displayed a similar trend and increased sharply in recent decades in both dentistry and medicine [5,6]. A behaviour analysis of average PubMed users highlighted how most of them only looked at titles. When an abstract was requested and viewed, the user moved on and retrieved the corresponding full-text article only 29% of the time. Indeed, after the title, the abstract is the most read part of a biomedical article [7]. An abstract can either be the only part of a published item accessed by the reader or be the article section used to determine whether to review the full text. The abstracts of SRs should therefore provide a structured summary that allows a quick appraisal of the validity and applicability of the review and an easy retrieval after an electronic search. In 2013, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) steering group published consensus-based reporting guidelines as an extension to the PRISMA statement to improve the quality of reporting of SRs in journals and conference abstracts. The PRISMA for Abstracts (PRISMA-A) checklist provides the authors with a framework to summarize the essential sections of their systematic review to meet the needs of the readers [8]. Although the effect of the PRISMA-A has been evaluated in the medical literature, this was not done in periodontal literature, and its effect on the reviews published in this branch of dentistry is still unclear. The aims of this research were to evaluate and compare the quality of reporting of abstracts of SRs before and after the publication of the PRISMA-A in 2013 and to evaluate whether an association exists between the abstract characteristics and the completeness of reporting, as measured by the PRISMA-A checklist.

Eligibility criteria
Articles on a periodontics-related topic with the term "systematic review" or "meta-analysis" in their title, abstract, or keywords were included. The search was limited to items published in the English language from 2002 to 2020. This time window was chosen considering that 2009 was the year of publication of the PRISMA statement [9], whereas in 2013, the PRISMA for abstracts guidelines was published [8].
Systematic reviews of evaluations of interventions were included. Systematic reviews incorporating the aetiology, diagnostics, test accuracy, or prognosis were excluded. All primary studies were also excluded. Cochrane reviews were excluded because they adhered to their own reporting standards [10].

Search strategy
A systematic search of the literature was conducted independently and in duplicate by 2 authors (DG and RA) using the PubMed and Scopus electronic databases in March 2020.
First, the search was performed in both databases using MeSH terms related to the field of periodontics and provided by the United States National Library of Medicine A second search was then performed to find the systematic reviews and meta-analyses published in periodontics journals listed in the 2018 edition of the Journal Citation Report (JCR) in the category "Dentistry Oral Surgery and Medicine".
The journals included in the search were The Journal of Clinical Periodontology, the Journal of Periodontology, Periodontology 2000, the Journal of Periodontal Research, the Journal of Periodontal & Implant Science, and the Journal of Periodontics and Restorative Dentistry. The journals with a specific focus on periodontology were targeted by a specific search because, according to our previous experience in the bibliometric field, performing such a search can improve overall search efficiency [2,3,11].
The search strings used in the advanced search tool of the PubMed and Scopus databases are reported in Table 1.
Three researchers (DG, RA, and FS) screened, as a team during two sessions, the retrieved SRs and MAs according to the inclusion and exclusion criteria. The same team of reviewers extracted the information regarding the retrieved items, and in case of disagreement, any conflict was resolved through discussion. The following data were saved on an   (TITLE-ABS-KEY ("dental prophylaxis" OR "dental scaling" OR "periodontal debridement" OR "gingivectomy" OR "gingivoplasty" OR "guided tissue regeneration, periodontal" OR "periodontal dressings" OR "periodontal index" OR "periodontal prosthesis" OR "periodontal splints") AND TITLE-ABS-KEY ("systematic review" OR "meta-analysis")) AND PUBYEAR > 2001 AND (LIMIT-TO (LANGUAGE, "English")) Date; 18.03.2020 519 Items  Identify the report as a systematic review (+/− meta-analysis) or both.
Objectives 2 Indicate the research question, intervention, comparator and outcomes.

Eligibility criteria 3
Include study characteristics used as criteria for eligibility.

Information sources 4
List the key databases searched and the search dates.

Risk of bias & applicability 5
Indicate the methods of assessing risk of bias and applicability.

Included studies 6
Number and type of included studies and the participants and relevant characteristics of studies.
Synthesis of results 7 Results for main outcomes (benefits and harms), preferablyindicating the number of studies and participants for each.If meta-analysis was done, include summary measures and confidence intervals.
Description of the effect 8 Direction of the effect (i.e., which group is favoured) and size ofthe effect in terms meaningful to clinicians and patients.

Strengths and limitations 9
Provide a brief summary of the strengths and limitations of the evidence.

Interpretation 10
Provide a general interpretation of the results and the important implications.

Funding 11
Indicate the primary source of funding for the review. The impact of the review type (SR or MA), the number of authors, the country of origin, the article subject, and the abstract structure on the completeness of the report were explored.

Assessment of reporting completeness
The abstracts of the selected articles were evaluated using the 12 items of the PRISMA-A checklist [8] ( Table 2). Each item was evaluated on a scale from 0 to 2, where 0 meant that the item was not reported at all, 1 meant that the item was only partially or inadequately reported, and 2 meant the item was fully reported. A higher score, therefore, denoted higher quality reporting, and the highest possible score was 24 points. The reference guide used for abstract scoring is provided in Additional file 2.
According to the PRISMA-A guidelines, both SRs and MAs were scored according to the same items, but in the case of MAs in item 7 (synthesis of results), summary measures and confidence intervals were be reported, and in item 9 (strengths and limitations of evidence), heterogeneity were to be discussed. Failing to report these data in an MA abstract had a negative impact on the completeness of the report [8].

Training and calibration
Abstracts were assessed independently by three reviewers who had no prior experience with the PRISMA-A (DG, RA, and FS) and who underwent a calibration process prior to the start of the screening. First, they were involved in theoretical training sessions in which the abstracts of 10 SRs were reviewed according to the checklist with the help of supporting documents [8] and the support of other members of the research group (RR) who had previous experience with the PRISMA-A and authored several SRs. The abstracts used for training were not included in the selected pool, and any disagreements were resolved through consensus. Calibration sessions were repeated with sets of 10 abstracts assessed twice with a week between each assessment until excellent intra-and inter-operator reliability was achieved. An intraclass correlation coefficient (ICC) higher than 0.90 was considered enough to define the method error as low and obtain high intra-and inter-operator reliability.

Statistical analysis
Descriptive models were used to describe ordinal and continuous variables (mean, standard deviation, range, and median). Categorical variables were described by means of absolute and relative frequencies. Due to the large sample size, a 95% confidence interval was obtained for score means. To determine the distribution of the data, the Kolmogorov-Smirnov test was applied for quantitative variables. Nonnormally distributed data were found, and nonparametric tests, which included the Mann-Whitney U test and Kruskal-Wallis test, were used. The reference level of significance was designated as up to 5% (α = 0.05). Post hoc power was calculated, and the final abstract pool provided 97.1% of the statistical power in detecting an effect size d = 0.5 (medium) between groups using Mann-Whitney's test and assuming a confidence level of 95%.

Results
In total, 2005 published items were initially retrieved. After the removal of duplicates, 925 articles were retained for further screening. After applying the inclusion and exclusion criteria, a final set of 265 abstracts was included in the study. The included articles were published by 19 journals from 2002 to 2013 and by 43 journals from 2014 to 2020. Only eight journals published and included SRs in both periods, as reported in Tables 3 and 4. Further details of the search and screening process are shown in the flow diagram in Fig. 1. The list of references of the included studies is available in Additional file 3. The results of the quality assessment of the included SRs are available in Additional file 4.
As outlined in Table 5, the mean general PRISMA-A score was 55.72% (95% CI, 54.46-56.79%). The total score before (mean score, 53.78%; 95% CI, 51.56-55.90%) and after (mean score, 56.88%; 95% CI, 55.39-58.44%) the publication of the PRISMA-A statement revealed a statistically significant improvement (P = 0.012*). The most recent pool of abstracts presented a better score. The improvement in the overall score after 2013 was mainly due to the improvement in reporting item number 6 "included studies" (p = 0.004) and item number 7 "synthesis of the results" (p = 0.025) of the PRISMA-A checklist. The lowest scores in both groups were those in the "funding and conflict of interest report" and "registration" sections, followed by "risk of bias" and "strength and limitation of evidence" sections.
The distribution of compliance with items in the PRISMA-A checklist according to the publication period is graphically presented in Fig. 2.
As highlighted in Table 6, only the International Journal of Dental Hygiene displayed a significant improvement in the PRISMA-A score when comparing the two studied periods, whereas top-cited journals such as the Journal of Clinical Periodontology or the Journal of Periodontology did not show any improvement.
Half of the selected articles were systematic reviews without meta-analysis (51.3%). This percentage dropped over time (64% before 2014 and 43.6% from 2014 to 2020). As reported in Table 7, the total PRISMA-A score was significantly higher in the meta-analysis group. The mean number of authors increased over time, from 3.9 authors prior to 2014 to 4.9 authors in SRs/MAs published after 2013. The total PRISMA-A score was significantly higher in the articles authored by more than four authors.
European institutions were the most prevalent affiliation of the first author (47.2%), followed by North America (15.5%), Asia (15.1%), and Latin America (14.3%). Latin America and North America turned out to be the most compliant areas in terms of abstract reports, and these are the areas that increased their relative prevalence more after 2013 (Additional file 5). A structured abstract was present in 87.2% of the cases and was related to a higher PRISMA-A score when compared to nonstructured abstract items. The difference was not statistically significant.

Discussion
To the best of our knowledge, this is the first study to address the completeness of abstract reporting in SRs in periodontology using the PRISMA-A checklist, since previously published studies about abstract completeness in periodontology and oral implantology were performed prior to the publication of the checklist in 2013 with different assessment methods of the included studies [12,13].
The present study provided analyses of a larger data pool compared to previous investigations in the same branch of dentistry. A similar article was recently published in Orthodontics, including 389 abstracts [14] of SRs on intervention and non-intervention procedures. The PRISMA-A checklist was designed with a special focus on systematic reviews of evaluations of interventions in which one or more meta-analyses were conducted [8]. If SRs are performed on non-intervention protocols or on questions about aetiology or diagnostic test accuracy, there may be a need to modify items or include others to improve the quality of the report. In the present study, non-intervention SRs were excluded to reduce possible bias in the assessment process.
The primary objective of this study was to assess the impact of the PRISMA-A on the completeness of abstract reporting in SRs. The results highlighted how the total score presented a statistically significant improvement after guideline publication.
However, a discussion of the results based only on the total score could be misleading since it could omit important information. Despite the global improvement, the analysis of the scores obtained by each of the checklist items highlights how 10 out of 12 items did not exhibit a statistically significant change after guideline publication. The leading periodontal journals, such as the Journal of Clinical Periodontology or the Journal of Periodontology, did not show any improvement in the completeness of the report, and although they require authors of SRs to comply with the PRISMA guidelines, the journals do not seem to have implemented the PRISMA-A checklist and guidelines effectively.
Other authors in different fields of dentistry also considered compliance with the guidelines to be poor [14,15]. Faggion et al. in 2013 suggested how the authors of SRs with meta-analyses should improve the report of the limitations of evidence, the risk of bias, and the measures of heterogeneity among the included primary studies to enable the readers to understand the strengths and weaknesses of the findings and allow for better clinical application of the evidence. These findings were consistent with those of our study that outlined how "risk of bias" and "strength and limitation of evidence" were some of the less reported items. In 2019, in an appraisal of the SRs published in top-ranked nursing journals, Wang et al. found no difference in completeness before and after the PRISMA-A guideline release [16]. Maticic et al., in the field of anaesthesiology, and Bigna et al., in the field of general medicine, reported poor or no improvement after checklist publication [17]. In agreement with our results, other authors in different fields of medicine and dentistry reported how information about registration is often missing in SRs´abstracts [13][14][15][16]18]. The lack of registration reports is surprising, especially in the SRs published after 2013, taking into account the wide acceptance of registration databases such as PROSPERO, which from its launch in 2011 to 2017 registered more than 30,000 SRs [19]. It is important to note that even if the SRs were not registered, this should have been reported in the abstract. A similar consideration may also be applied to information about funding, which in our pool was not reported in any abstract. According to our findings, a higher number of authors is related to a better quality report, and the same result was also highlighted by previous research [13,14,16,18]. In contrast, Bigna et al. (2016), in top-rated medical journals, reported no association between the number of authors and the quality of the report [20]. However, the authors discriminated between articles with more or fewer than nine authors since the number of co-authors is higher in medicine than in dentistry. The correlation between the quality of the report and the number of authors could be due to the nature of SRs. Writing a systematic review involves a lengthy and elaborate process following strict adherence to search, screening, and selection protocols. From this perspective, relying on a larger study group can ensure better support and collaboration, favouring higher quality research output. According to the data presented by previous reports, the MAs group displayed better compliance with the PRIS MA-A checklist than the SRs group [13,14,18]. It should be noted that the number of MAs was higher among the most recently published items that generally presented higher scores. According to what was reported by other authors in bibliometric studies about SRs [13,16,18,21], the majority of first authors belonged to European institutions. However, other articles, such as that by Wasiak et al. [22], had a higher prevalence of North American first authors, whereas the most prevalent authors were from Latin America in Bassani et al. [23]. It should be noted that the report by Bassani et al. [23] only included SRs published during 2017, and their findings cannot be generalized to all SRs throughout time.
Among the 265 abstracts included in the assessment, 34 were not structured, and the prevalence was higher than that previously reported by other authors [13,18]. Structured abstracts have been strongly recommended by the International Committee of Medical Journal Editors [24], and they are supposed to enhance the quality of the report [25].
The accuracy of the search strategy, the high number of included abstracts, and the accurate training and calibration process of the research group to maximize the accuracy and reliability of the assessment can be considered strengths of this study. A number of limitations of this study should also be acknowledged. The impact of the abstract word count was not explored. In theory, a larger abstract could give more room for a detailed report of the required items. Nevertheless, the relationship of this factor with the quality of the report remains controversial [26]. The full texts of the selected articles were not systematically assessed and thus were not used to detect which items were not reported due to their absence (e.g., funding, conflict of interest or registration) or to a lack of compliance in the report. The lack of a full-text assessment could have also hindered the articles in which the word meta-analysis was not mentioned in the title, abstract, or keywords, in cases in which a meta-  Fig. 2 The distribution of compliance with items on the PRISMA for abstracts checklist according to the publication periods. Score: 0 = not reported, 1 = partially or inadequately reported, and 2 = fully reported analysis was performed as a statistical method. Moreover, in the search strategy, a specific screening was performed among the periodontal journals indexed in the 2018 edition of the JCR, not including the journals indexed in larger databases such as Scopus.

Conclusions
The statistically significant improvement in the overall PRISMA-A score was largely driven by improvements in only 2 of the 12 checklist items. The reporting of the majority of items did not improve after the introduction of the PRISMA-A checklist in 2013. There is definitely room for further improvement, and efforts should be made to reach the required standards. In particular, "risk of bias", "strength and limitation of evidence", "registration" and "funding" are the items that displayed the major need for improvement. The leading periodontal journals did not show any improvement in the completeness of the report during the studied period.
The editors and referees of periodontal journals should promote adherence to the PRISMA-A checklist to increase the quality of the reports and provide readers with better insight into the study outcomes.