Exploring reporting quality of systematic reviews and Meta-analyses on nursing interventions in patients with Alzheimer’s disease before and after PRISMA introduction

Background Systematic reviews (SRs) and meta-analyses (MAs) are distillation of current best available evidence, but are potentially prone to bias. The bias of SRs and MAs comes from sampling bias, selection bias and within study bias. So, their reporting quality is especially important as it may directly influence their utility for clinicians, nurses, patients and policy makers. The SRs and MAs on nursing interventions in patients with Alzheimer’s disease (AD) have been increasingly published over the past decade, but the reporting quality of article has not been evaluated after the introduction of Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) Statement. Methods According to the inclusion and exclusion criteria, we searched the databases including PubMed, EMBASE and The Cochrane Library from inception through October 16th 2018. Two reviewers independently selected articles and extracted data. The PRISMA checklist was adopted to evaluate reporting quality. Comparisons were made between studies published before (2001–2009) and after (2011–2018) its introduction. Results A total of 77 eligible articles, 18 (23.4%) were published before the PRISMA Statement and 59 (76.6%) were published afterwards. There was higher score after publication of the PRISMA Statement than before (20.83 ± 3.78 vs 17.11 ± 4.56, P <  0.05). There was an improvement in the following items after the PRISMA statement was released (P <  0.05): title (item 1, 50.0% vs 74.6%, OR = 3.10, 95CI%: 1.00–9.61), search (item8, 27.8% vs 57.6%,OR = 3.25, 95CI%: 1.14–9.28), study selection (item 9, 44.4% vs 81.4%,OR = 6.28, 95CI%: 1.93–20.37), Data collection process (item 10, 50.0% vs 76.3%,OR = 3.45, 95CI%:1.10–10.84), risk of bias in individual studies (item 12, 50.0% vs 83.1%, OR = 5.78, 95CI%:1.71–19.52), risk of bias across studies (item15, 5.6% vs 28.8%,OR = 3.60, 95CI%:1.04–12.43), study characteristics (item 18, 77.8% vs 98.3%, OR = 28.13, 95CI%:3.35-236.19), risk of bias with studies (item 19, 50.0% vs 83.1%, OR = 5.78, 95CI%:1.71-19.52), results in individual studies (item 20, 72.2% vs 94.9%, OR = 11.09, 95CI%:1.99–61.82), conclusions (item 26, 77.8% vs 98.3%, OR = 28.13, 95CI%:3.35–236.19). After controlling for the confounding factors, there were higher PRISMA score for systematic reviews including meta-analyses, protocol or registration, can’t answer of RCT, journal source of SCI (Science Citation Index), manuscript length > 13 page and funding support. Conclusion Since the publication of the PRISMA Statement, there has been an improvement in the quality of reporting of SRs and MAs on nursing interventions in patients with AD. More endorsement by journals of the report guideline for SRs/MAs may improve articles reporting quality, and the dissemination of reliable evidence to nurses. We recommend authors, readers, reviewers, and editors to become more acquainted with and to more strictly adhere to the PRISMA checklist.


Background
Systematic reviews (SRs) and meta-analyses (MAs) keep up to date with developments in modern medicine. SRs and MAs can provide evidence of the value and feasibility on nursing interventions in patients with Alzheimer's disease (AD), and to help clinicians, nurses, and policy makers to inform clinical decision-making. Despite their strengths, SRs have shown varying quality. As such, high quality of reporting is crucial to ensure reliable, transparent and accurate interpretation of evidence. Thus, some checklists have been published in an attempt to improve the reporting quality of systematic reviews and meta-analyses.
In an attempt to ensure validity of evidence and improve both the quality and completeness of reporting of SRs and MAs, a 27-item PRISMA statements have been published to assess the reporting quality of SRs in 2009, which was a successor to the original Quality Of Reporting Of Meta-analysis (QUOROM) guidelines [1,2]. In recent years, a few of studies in various medical fields have been conducted for assessment of the qualities of SRs and MAs based on the fulfillments of PRISMA [3][4][5]. Cullis et al. [3] investigated that compliance with the PRISMA guidelines was poorer and the reporting quality of SRs and MAs in the published paediatric surgical literature needs to be improved. Panic et al. [4] found that the quality of reporting quality of SRs and MAs in journals in the field of gastroenterology and hepatology have significantly increased after PRISMA endorsement. Wasiak et al. [5] demonstrated that the reporting quality of SRs in burn care management is suboptimal and requires further improvement with stricter adherence by authors to the PRISMA checklist. Recently, more journals encourage the adoption of PRISMA include mandatory submission of reporting checklists and integration of PRISMA into the peer-review process [6]. Hence, an assessment of compliance before and after introduction of the PRISMA Statement will play an increasingly important role.
Alzheimer's disease (AD) is a disease that causes significant functional impairments even in its early stage [7]. It is estimated that there will be 131.5 million people living with dementia globally by 2050. There are currently no treatments to reverse the course of AD, more by nursing intervention [8]. However, certain treatments, both pharmacological and psychotherapeutic, do achieve a slowing of the impairment of AD, especially regarding to psychological treatments [9]. So far, research on reporting quality evaluation of SRs and MAs on nursing interventions in patients with AD has not been available. Thus, this study evaluated the reporting quality of SRs and MAs on nursing interventions in patients with AD according to PRISMA checklists, and hoping to provide reference for authors, readers, reviewers, and journal editors. The secondary aim was to find that whether some predictive factors especially the publication of the PRISMA statement was associated with an improvement in reporting.

Search strategy
The article that met inclusion criteria was identified by searching in PubMed, EMBASE and The Cochrane Library up to October 16th 2018. The language was limited to English. The search strategy included the use of Title/Abstract related to: ("Alzheimer's disease" or "Senile dementia" or "dementia" or "AD") and ("systematic review" or "meta-analyses" or "overview" or "systematic literature review" or "meta-analysis" or "review" or "synthesize review" or "integrated review" or "comprehensive review") and ("nursing" or "nursing intervention" or "nurse" or "care"). Combined with Google Scholar and Baidu Scholar, we also scanned the reference lists manually reviewed from included articles to identify additional relevant studies. The search strategy for PubMed is outlined in Fig. 1.

Study selection
To be eligible for inclusion, the reports have the following inclusion criteria: (1) articles being identified as SRs or MAs; (2) published in English language; (3) an article of nursing interventions in patients with Alzheimer's disease. The exclusion criteria were as follows: (1) scoping review, traditional literature review and evidence-based commentaries; (2) an article of non-nursing interventions in patients with Alzheimer's disease. Two investigators independently screened the titles and abstracts of all the retrieved articles using the inclusion criteria.
Full-text articles were retrieved and reviewed independently in duplicate by two authors for potentially eligible articles. Any discrepancy was resolved by discussion or by asking a third reviewer if consensus could not be reached.

Data extraction
Data extraction was performed independently on included articles by two investigators (S.X; L.H.H). Discrepancies and unobtainable data were resolved by group discussion between at least three investigators. General characteristics of SRs or MAs were extracted, including year of publication, author origin, the name and type of the journal, the authors' affiliations, the number of affiliations, the article type, whether an RCT was identified, impact factor (IF), number of times cited, funding support, followed PRISMA guideline, international collaborative authorship, the number of included studies, topics of intervention, word count and protocol or registration, number of authors, journal source of SCI, manuscript length. Comparisons were made between manuscripts published before (2001)(2002)(2003)(2004)(2005)(2006)(2007)(2008)(2009)) and after (2011-2018) introduction of the PRISMA Statement to access the reporting quality. Since all three articles published in 2010 were submitted in 2009, we considered them as pre-published manuscripts.

Quality assessment
According to the detailed per-item descriptions of the PRISMA Statement, appraisal of reporting was performed by two authors (S.X; L.H.H). The PRISMA statement covers seven modules with 27-items, ranging from 0 to 27. Each item was rated as yes, no or cannot answer, not applicable. A score of 1 was assigned to a "yes" answer, and a score of 0 was assigned to all other answers except "not applicable". Items that were not applicable, such as those that only applied to MAs (ie, PRISMA items 14, 16 and 23), were scored 1. Therefore every study had an overall PRISMA score rated out of a maximum score of 27.

Data analysis
Initially, descriptive analysis on the characteristics of the included SRs/MAs published before (2001-2009) and after (2011-2018) was conducted. Differences in categorical variables and comparison overall compliance in each PRISMA item were explored using the χ 2 test (≤ 2009 vs. ≥2011). Comparisons of the PRISMA score between manuscripts published before (2001-2009) and after (2011-2018) were conducted using the t-test and covariance of analysis for adjusting confounding factors. A p < 0.05 was considered significant on statistical analyses. All the statistics analyses were conducted by SPSS 19.0 and RevMan software version 5.3.

Search results
A total of 2413 articles were identified in initial search, whilst hand searching captured an additional 20 articles for potential inclusion. After removing duplicate articles, reviewing titles and abstracts, reviewing the full-text, there are a total of 77 articles to be included in this research finally. The process of literature retrieval was shown in a flow diagram (Fig. 2). The agreement between data extractors was moderate (kappa = 0.79, P < 0.01) for the PRISMA statement, which indicates a good level of agreement between scorers. All of the discrepancies were resolved by consensus between the two extractors.  Statement than before (20.83 ± 3.78 vs 17.11 ± 4.56, P < 0.05). The characteristics of the 77 articles are shown in Table 1.

Reporting quality of included reviews
To determine whether the publication of the PRISMA statement was associated with an improvement in the quality of reporting, the period ≤2009 was compared with ≥2011 in each of the PRISMA criteria (Fig. 3). The results showed that there was an improvement in the following items after the PRISMA statement was released, which was significant difference:

Univariable and covariance analysis on publication time
Univariable and covariance analysis of study demographics using PRISMA score as dependent variables identified several significant trends showed in Table 2 After controlling for the confounding factors, there were higher PRISMA score for systematic reviews

Discussion
The number of SRs/MAs about AD has been accumulating at an increasing rate in recent years. Although the quality of the reporting was suboptimal, the reporting quality of SRs and MAs of nursing interventions in patients with AD had been an improvement in the quality of reporting after the PRISMA statement was released. On average, PRISMA score with before and after publication of the PRISMA Statement was 17.11 and 20.83, respectively. Similarly, there was significant difference in compliance with some items of the PRISMA Statement before and after its introduction. The articles [4,10,11] have evaluated compliance with the PRISMA Statement in other medical disciplines.  [10]. An assessment SRs and MAs of diagnostic tests published by in China revealed that there had been some statistically significant improvement in total compliance for 9 PRISMA items after the PRISMA was released [11]. Like our study, the study demonstrated that several potential predictive factors for superior quality of urological meta-analyses were identified, including funding support, following PRISMA guideline, and "a priori" design [12]. There are discrepancies in evaluation quality of SRs among various medical specialties.
The main strengths of this study include the focused search and selection of SRs/MAs, comprehensive assessment, and planned linear regression analyses. For selection of SRs/MAs, we not only included SRs but also MAs because our study has to be more comprehensive   and objective. In addition, some of the assessment items only applied to meta-analyses, such as PRISMA items 14, 16, 23. To minimize bias against systematic review, we identified these items as not applicable and considered the item to be fulfilled when the total number of completed items was compiled. Thus, they have a more consistent score criterion. Since the PRISMA statement was published in 2009, we did not use this guidance to assess the quality of SRs/MAs published in 2010, which was sufficient to allow adequate uptake by authors and editors before assessment. Also, our results would have more credibility and our conclusions would have a more specific implication. It is worth noting that a number of PRISMA items (item 1, item 8, item 9, item 10, item 12, item 15, item 18, item 19, item 20 and item 26) have increased in reporting after PRISMA Statement introduction. Prevalence of reporting of these items has gone up 3.10 to 28.13 times after PRISMA Statement introduction. The majority of included studies were described as SRs and/ or MAs within the title (item 1). In order to identify instantly by the reader and to improve indexing, the title of the paper should include the PICOS approach (participants, intervention, comparison, outcome and study design), which will make readers know potential high level of evidence and help them decide whether the paper is worth reference [13]. The study selection processes (item 9) perform that authors identify records from their search and sequentially exclude records according to eligibility criteria. It protects the rigor and science of the SRs and/or MAs. Of greater importance was the high rate of reporting of study characteristics (item 18). For readers to gauge the validity and applicability of a SR's results, authors need to report information (such as PICOS, questions related to specific patients, interventions, comparisons,) about the included studies [14].
It is generally believed that PRISMA statement is significant for authors of SRs/MAs to follow to improve their reporting quality and transparency. But the poorly reported items need to further improvement. Whether it is before or after publication, five items with compliance less than 50% include items related to describing the review protocol and registration, describing the risk of bias, describing summary measures, and describing synthesis of results (items 5, 15, 22, 13, and 21, respectively). Based on our data, the generally lower compliance was reported and reflected a lack of awareness regarding protocols and registration (items 5) and risk of bias across studies (items 15 and 22) among authors as to their importance and potential implications. The use of protocol and review registration is important because a protocol makes more accurate for authors' researches and supports better transparency during the review process. As for SRs, only Cochrane reviews require the authors to publish a peer-reviewed protocol before conducting the review. Previous studies have shown that Cochrane reviews appear to have higher reporting quality than SRs or MAs published in paper-based journals [3,5,15]. The registration resources such as PROSPERO [16] and Systematic Reviews [17] can make authors easily use to obtain information regarding protocol and registration. Risk of bias across studies is typically performed by testing for publication bias [18]. Publication and outcome reporting biases are common in the literature and are likely to affect summary effect estimate [19][20][21]. To explore the publication biases, funnel plots and tests for funnel plot asymmetry should be provided in meta-analyses with more than 10 included studies [22]. Therefore, authors should carefully consider whether their results are affected by these common biases by the examination of funnel plots or the performance of statistical tests [23].
Covariance analysis was conducted to explore characteristics associated with overall reporting quality. Overall, SRs including MAs score higher with regards PRISMA scores, than SRs alone. Compared with the traditional descriptive review, SRs including MAs can make validity and reliability of evidences. The reporting quality of SCI articles was better compared with non-SCI articles. For more than 50 years, the SCI database has been continuously developed and has become the most important large-scale database in the contemporary world. It is one of the important indicators to evaluate the academic level of a country, a scientific research institution, an institution of higher learning, a journal, and even a researcher. Manuscript length was also associated with higher compliance with the PRISMA statements. Manuscript length limits imposed by some journals are one of the major obstacles and these findings showed that reporting quality is compromised when inadequate space is provided in all sections [24]. We also found evidence that funding support with better reporting quality were more likely to call for further research. The impact of funding on author conclusions has long been recognized in randomized trials and MAs of clinical trials [25,26]. It is necessary to improve awareness among journal reviewers, editors, funders, institutions, and readers. We call for journals to make the PRISMA checklist mandatory for the electronic submission of SRs. There are some limitations to this study. Firstly, this study was restricted to SRs/MAs examining nursing interventions in Alzheimer's disease only, thus excluding a large number of reviews that may have produced a different response. Secondly, we were restricted to electronic databases, which may not have been representative of all indexed studies. And only articles written in English were enrolled for analysis.

Conclusion
Overall, some improvements in compliance with some items of the PRISMA Statement were found SRs and MAs articles of nursing interventions in patients with Alzheimer's disease after the publication of the PRISMA statement, even the overall quality of reporting. After controlling for the confounding factors, there were higher PRISMA score for systematic reviews including meta-analyses, can't answer of RCT, protocol or registration, can't answer of RCT, journal source of SCI, manuscript length > 13 page and funding support. But clinicians, nurses and investigators should critically appraise all reports of SRs/MAs before considering the results. More endorsement by journals of the report guideline for SRs/MAs may improve articles quality, and the dissemination of reliable evidence to nurses. We recommend readers, reviewers, and editors to become more acquainted with to the PRISMA checklist. In addition, authors should stricter adhere to the PRISMA checklist when conducting a SR/MA.
Abbreviations AD: Alzheimer's disease; IF: Impact factor; MAs: Meta-analyses; PRISMA: Preferred Reporting Items for Systematic Reviews and Meta Analyses; SRs: Systematic reviews