A scoping review on the methodological and reporting quality of scoping reviews in China

Background Scoping reviews have emerged as a valuable method for synthesizing emerging evidence, providing a comprehensive contextual overview, and influencing policy and practice developments. The objective of this study is to provide an overview of scoping reviews conducted in Chinese academic institutions over the last decades. Method We conducted a comprehensive search of nine databases and six grey literature databases for scoping reviews conducted in Chinese academic institutions. The reporting quality of the included reviews was assessed using the Preferred Reporting Items for PRISMA-ScR checklist. We performed both quantitative and qualitative analyses, examining the conduct of the scoping reviews and exploring the breadth of research topics covered. We used Chi-squared and Wilcoxon rank-sum tests to compare methodological issues and reporting quality in English and Chinese-language reviews. Results A total of 392 reviews published between 2013 and 2022 were included, 238 English-reported reviews and 154 Chinese-reported reviews, respectively. The primary purposes of these reviews were to map and summarize the evidence, with a particular focus on health and nursing topics. 98.7% of reviews explicitly used the term “scoping review”, and the Arksey and O’Malley framework was the most frequently cited framework. Thirty-five English-reported scoping reviews provided a protocol for scoping review. PubMed was the most common source in English-reported reviews and CNKI in Chinese-reported reviews. Reviews published in English were more likely to search the grey literature (P = 0.005), consult information specialists (P < 0.001) and conduct an updated search (P = 0.012) than those in Chinese. Reviews published in English had a significantly high score compared to those published in Chinese (16 vs. 14; P < 0.001). The reporting rates in English-reported reviews were higher than those in Chinese reviews for seven items, but lower for structured summary (P < 0.001), eligibility criteria (P < 0.001), data charting process (P = 0.009) and data items (P = 0.015). Conclusion There has been a significant increase in the number of scoping reviews conducted in Chinese academic institutions each year since 2020. While the research topics covered are diverse, the overall reporting quality of these reviews is need to be improved. And there is a need for greater standardization in the conduct of scoping reviews in Chinese academic institutions. Supplementary Information The online version contains supplementary material available at 10.1186/s12874-024-02172-y.


Introduction
Along with the increased production of primary research, the conduct and publication of evidence synthesis have also increased over time [1].To address various questions for policymakers and other stakeholders, different types of reviews have emerged [2], with 48 types of reviews identified by 2019 [3].One of the review types is the "scoping review", also known as a "scoping study", or "mapping review" [4].Scoping reviews are a valuable approach to synthesizing emerging evidence, providing a comprehensive overview of the context and the potential to influence policy and practice developments [5,6].Scoping reviews have been widely conducted in various fields, including health, technology and social sciences over the past decades [7].
Scoping reviews follow similar processes to systematic reviews in terms of identifying and analyzing relevant literature on a specific topic [8].Scoping review characteristically involves the development, assimilation, and synthesis of a broad base of evidence derived from a diverse range of published research and grey literature research [9].It aims to clarify the key concepts and characteristics that underpin a research area, determine a precise volume of literature and studies available, or can be a precursor to a systematic review [10].Unlike systematic reviews, scoping reviews do not require a quality assessment of individual studies or the integration of evidence from different studies [11].While a scoping study requires a framework to investigate existing literature, it does not involve assessing the weight of evidence for particular interventions or policies [12].
Scoping review is widely used to answer board research questions now.The concept was initially proposed by Mays in 2001 [13] and later used by Arksey and O'Malley, who provided the first guidance on conducting scoping review in 2005 [14].In 2014, the JBI and JBI collaboration published their guidance on scoping review [15] and updated it in 2020 [16,17].As a result, the publication of scoping reviews has significantly increased.To improve the methodological and reporting quality of scoping reviews, the Preferred Reporting Items for Systematic Reviews extension for Scoping Review [18] (PRISMA-ScR) was published in 2018.
The research community involved in scoping reviews, mainly from Canada, the United States, the United Kingdom, and Australia, shows a steady increase in literature and maintains a relatively high growth rate [4].China is currently in the early stages of introducing and familiarizing itself with the scoping review methodology, with few documented examples of its practical application in the field of medicine [19].Additionally, some review articles in China, although conceptually similar to scoping reviews, did not follow the standardized methodology and therefore could not be classified as scoping reviews [20,21].This may be attributed to a lack of awareness and a delay in the adoption of scoping reviews by Chinese academic authors.
In light of the above situation and the growing utilization of scoping reviews in China, our study aims to (1) examine volume, scope and distribution of scoping reviews conducted in Chinese academic institutions; (2) summarize the purpose, topics, and methodological issues in these scoping reviews; (3) explore the extent to which scoping review adheres to reporting guidelines.

Methods and analysis
We used Arksey and O'Malley's framework [14] on scoping review to guide our study.This protocol was registered on the open science framework (https://osf.io/f9u6q/).

Identify the research questions
Our research questions were as follows.
1) What is the volume, scope and distribution of scoping reviews conducted in Chinese academic institutions?
2) What are the purposes, topics and methodological issues in the included scoping reviews?
3) To what extent do these scoping reviews adhere to the PRISMA ScR reporting guidelines?

Eligibility criteria
We included all scoping reviews that met the following criteria: (1) utilized a scoping review of the literature approach with a description of the synthesis method used and (2) focused on the field of health/medicine.We excluded studies that (1) did not synthesize literature, such as complete scoping of surveillance or administrative databases; (2) primarily described scoping review methods or guidelines and (3) full text was not available.We defined scoping reviews conducted in Chinese academic institutions as the corresponding author's affiliations located in mainland China, Hong Kong, Macau or Taiwan.
Grey literature (e.g.thesis and dissertation, newspaper, conference paper) and some web search engines (e.g.Google, Google Scholar, Baidu, Baidu Scholar) were also searched.The search strategy was not restricted by study design and an expert information specialist collaborated with the research team.The search strategies were listed in Appendix 1 We also cross-checked the included studies and references of a relevant scoping review.

Study selection
Inter-rater agreement for study inclusion was calculated using percent agreement.If the agreement exceed 75% among the team members, we proceeded to the next stage.All title and abstract screening and full-text screening were performed independently by at least two review authors (Xue XY, Tang XT, Liu SS, Yu T) using a pre-defined form.Any discrepancies were resolved through consensus or involving a third reviewer (Yu JJ) when necessary.

Charting the data
The following general information was collected from each eligible study: published year, regions, affiliations, journal name, study population, number of studies included, and funding source.We also collected details on the purpose of the scoping review (e.g., identified evidence gaps, future research opportunities, implications for policy or practice), as well as the topic addressed.We collected methodological information including the definition of "scoping review", utilization of established methodological guidance (e.g., Arksey and O'Malley, Levac, Joanna Briggs Institute, or others), protocol and registration, research question, inclusion criteria, eligible study design (e.g., primary studies, secondary studies, both, or others), search strategy, databases searched, additional search resources (e.g.explore breadth/extent of evidence, grey literature, consulted experts, crosscheck references), title and abstract screening, full-text screening, pre-defined charting form, flow diagram, result presentation (tables and/or diagrams), the implication for research and practice [22].
We assessed the reporting quality of eligible scoping reviews using the PRISMA-ScR checklist [18], which includes 20 items and 2 optional items for critical appraisal of individual studies.Each item was determined with the option of "yes" or "no", allocating 1 point if the study met the requirement for a specific item and 0 if not.A total score ranging from 0 to 22 was developed.
Data extraction and reporting quality from each eligible study were conducted by four reviewers, a pilot study was performed before formal extraction, and the interrater agreement percentage needed to be > 75%.Discrepancies were resolved by consensus or the involvement of a third reviewer.

Collating and summarizing results
We conducted both quantitative and qualitative analyses on the scoping reviewsThe quantitative analysis involved examining the distribution of reviews, methodological issues, and reporting quality.For quantitative analysis, frequencies and proportions were calculated for the categorical variables and mean (SD), median (range) or median (IQR) were used to analyze the continuous variables.Word clouds were generated using the online program WordClouds to visualize the synthesis topics (Zygomatic, 2022) (https://www.wordclouds.com).
We performed a comparative analysis of methodological issues and reporting quality between eligible scoping reviews published in English and Chinese.Either X 2 or Fisher's exact tests were used for the analysis of categorical variables and the Wilcoxon rank-sum test was used for continuous data with a non-normal distribution.For qualitative analysis, two reviewers independently categorized the key components, and the results were subsequently discussed by the research team.

Search and selection of scoping review
A total of 2958 citations relevant to scoping review were searched, and 2046 studies were included for screening after duplication.After reading 589 potentially relevant full-text papers, 392 articles were finally included (Fig. 1).The interrater agreements among the four reviewers were good, with agreement rates of 92.2% at the title and abstract screening, 94.8% at the full-text reading, and 95.3% at the table extraction.

Study characteristics of including scoping reviews
The scoping reviews included in our study were published between 2013 and 2022, with a significant majority (n = 340, 86.7%) published after 2020 (Fig. 2).The median number of studies included in these scoping reviews was 29 (Range: 5-6430).Among the included reviews, 238 (60.7%) were reported in English.A total of 217 journals were involved in publishing the scoping review, with the Chinese Journal of Nursing (n = 14, 3.6%) and the International Journal of Environmental Research and Public Health (n = 10, 2.6%) being the most commonly published Chinese and English journals, respectively (Table 1).Most of the reviews were conducted in Beijing (n = 66, 16.8%), Hong Kong (n = 59, 15.1%), and Shanghai (n = 44, 11.2%) (Appendix 2).The reviews originated from 166 institutions, primarily universities and hospitals (Table 1).274 (69.9%) received funding support, with majority (98.5%) being publicly sponsored.Notably, the distribution of these characteristics was different between Chinese reviews and English reviews (Table 1).
The study population for the included scoping review primarily consisted of patients (n = 200, 51.0%) and healthcare professionals (n = 60, 15.3%) (Appendix 3).The main purposes of the reviews were to map and summarize existing evidence (n = 230, 58.7%), followed by to identify and/or address knowledge gaps (n = 111, 28.3%) (Appendix 4).The topics covered in the reviews were diverse, with health and nursing being the common topics (Fig. 3).

Method characteristics of including scoping reviews
Of the 392 scoping reviews, the majority (387, 98.7%) explicitly identified themselves as "scoping review", while 122 (31.1%) provided a definition of the term 'scoping review' .The most frequently cited framework for conducting scoping review was Arksey and O'Malley framework (2005), referenced in 157 reviews (40.1%).The research question was clearly stated in 39.8% of the reviews, while 75.8% clearly reported their inclusion criteria.
Thirty-four (8.7%) English-reported scoping reviews provided a protocol of scoping review, and 20 (5.1%) were registered on the Open Science Framework (OSF).The majority of reviews (369, 94.1%) searched more than one database, with PubMed (n = 184, 77.3%) being the most common source for English-language reviews and CNKI (n = 137, 89.0%) for Chinese-language reviews.In terms of additional search, English-reported reviews were more likely to search the grey literature (70 vs. 39, P = 0.005), consult information specialists (18 vs. 3, P < 0.001) and conducted an updated search (16 vs. 2, P = 0.012) compared to Chinese-reported reviews (Table 2).
Approximately 80% of reviews screened title/abstract and full-text articles with more than two reviewers.A predefined abstraction form was used in 30.4% of the reviews and data extraction involved more than two reviewers in 57.7% of reviews.Among the included reviews, 56 reviews (14.3%) assessed the quality of the studies and six reviews (1.5%) conducted a meta-analyses.More than 75% of included reviews provided a study flow chart, while the difference was significant between reviews in English language and Chinese (212 vs. 90, P < 0.001).Additionally, nearly 90% of the reviews presented their results in tabular form, while 31.1% used graphical representation.In terms of discussion, a higher proportion of scoping reviews published in English journals compared to Chinese journals mentioned the limitations of their studies (193 vs. 50, P < 0.001) as well as their strengths (68 vs. 22, P = 0.001) (Table 2).

Reporting quality of including scoping reviews
Of the 22 items, six (27.3%) were adequately reported: identification of the report as a scoping review in the title (93.1%), statement of eligibility criteria for included evidence (90.6%), description of all sources of information used in the search (100.0%),process for synthesizing the results (88.3%), methods of summarizing the evidence (100.0%) and presentation of a conclusion (97.4%).On the other hand, four items were reported less than 50% of the scoping reviews, including description of the rationale for the review (44.6%), accessibility of a protocol and registration information (8.2%), critical appraisal of individual sources of evidence (13.8%), and critical appraisal within sources of evidence (12.5%) (Fig. 4).

Discussion
We conducted a comprehensive scoping review of 392 Chinese-authored scoping reviews over the last decade.Our findings revealed a significant increase in the number of scoping reviews in China, particularly since 2020, with 207 scoping reviews published in the past year alone, accounting for approximately half of the total number published in the previous decade.The increase in the number of scoping review publications after 2020 could be attributed to several possible reasons.First, the COVID-19 pandemic in 2020 may have led to a broader literature review in the fields of health sciences and medicine to better understand and respond to the crisis.Second, scoping reviews have gained recognition within the academic and research community, leading to an increasing number of studies adopting scoping reviews for literature synthesis.Third, as research fields continue to evolve and expand, study topics become increasingly complex.Scoping reviews offer a flexible approach covering a wide range of literature, helping researchers to gain a more comprehensive understanding of the current state of research.Finally, some academic institutions and publishers may actively promote the use of scoping review methodology.
The majority of scoping reviews in our studies were conducted in hospitals and universities, with nursing and health being the predominant topics.This trend can be attributed to the education and dissemination efforts of organizations such as Joanna Briggs Institute (JBI) and its collaboration.Additionally, most studies provided a clear definition of scoping reviews and followed the framework by Arksey and O'Malley' framework [14].However, few studies mentioned the dissemination of findings [22], which may be due to the optional nature in the Asksey and O'Malley framework.
In general, scoping reviews published in English demonstrated higher methodological and reporting quality compared to those published in Chinese.However, some key items recommended by JBI guidance were also poorly reported, including protocol registration, search strategy and data presentation.The significance of the protocol has been emphasized in both the JBI guidance and the PRISMA-ScR checklist (Peters et al., 2022).Surprisingly, less than 10% of the included scoping reviews in our study provided information about the protocol.It is worth noting that various platforms such as Figshare, Open Science Framework, ResearchGate, and Research Square allow protocol registration, and it is encouraged to include full protocols with available information for preregistration purposes.
Unsimilar to systematic review [23] and other evidence synthesis approaches, scoping reviews have the flexibility to include various types of literature including grey literature, newspapers, websites and social media, to address the question of "what has been done before" [24].However, we found a limited number of included scoping reviews that conducted the additional search or consulted with information specialists.Furthermore, among those reviews performed database searches, only half of them provided a comprehensive search strategy for at least one database.It is important to consider conducting a more comprehensive search during the planning phase of a scoping review.
The scoping reviews included in our study showed a preference for presenting results in tables rather than images, resulting in a lack of diversity in data presentation.To enhance the interpretability of scoping review findings, various engaging methods, such as bubble charts, infographics, and Wordless, are available.We recommend that researchers, journal editors, and peer reviewers undergo additional training courses or access online resources (Stern et al., 2018) to improve the methodological quality of scoping reviews conducted by Chinese academic institutions.Furthermore, journal editors  should require authors submitting scoping reviews to adhere to the PRISMA-ScR checklist before final submission.Additionally, we encourage research management agencies to promote the practice of conducting a scoping review in their respective fields prior to the initiation of research projects, helping researchers gain a better understanding of their research background and reduce research waste (Khalil et al., 2022).
Our findings are broadly consistent with the two previous scoping reviews of scoping reviews [7,25], demonstrating considerable variability in the purpose, topics, and methodological aspects.However, our study finds certain improvements in specific areas, such as a better understanding of the distinctions between scoping review and systematic review [26][27][28], as well as the adoption of descriptive conclusions instead of definitive conclusions for practice.
To identify all relevant scoping reviews conducted by Chinese academic institutions, we conducted a comprehensive search that encompassed grey literature and web search engines.Based on our findings, we provide suggestions to researchers, journal editors, and administrators.However, our study also has some limitations.We may have partially missed some reviews conducted by Chinese academic institutions if the author's name was not a traditional Chinese form.Additionally, we only included studies in Chinese and English, which may have excluded reviews published in other languages by authors from Chinese academic institutions.

Conclusion
The annual number of scoping reviews conducted by Chinese academic institutions has shown a significant upward trend since 2020, encompassing a wide range of research topics, particularly within the realms of nursing and health.Scoping reviews are increasingly employed in practical applications, such as research preparation and identification of research questions.However, there remains a notable deficiency in the methodological rigor and reporting quality of scoping reviews conducted by Chinese academic institutions.Future research should prioritize enhancing the transparency of search and screening processes, diversifying data presentation techniques, and promoting standardization in reporting practices.

Fig. 1
Fig. 1 Study selection Details the flow of information through the different phases of the review; map out the number of records identified, included and excluded, and the reasons for their exclusion

Fig. 2
Fig. 2 Published year The annual number of scoping reviews conducted by Chinese academic institutions is visually represented in the form of a line graph

Fig. 3
Fig. 3 Word cloud of topicsThe most common topic in the 392 scoping reviews is displayed, with the size of the topics in the word cloud corresponding to the frequency of their show

Table 1
Study characteristics of included reviews

Table 2
Method characteristics of included reviews Xue et al.BMC Medical Research Methodology

Table 2 (
continued) * STROBE: Strengthening the reporting of observational studies in epidemiology n: number of studies † X 2 test, except ‡ Wilcoxon rank-sum test