Skip to main content
Download PDF

The relationship of publication language, study population, risk of bias, and treatment effects in acupuncture related systematic reviews: a meta-epidemiologic study

BMC Medical Research Methodology volume 23, Article number: 96 (2023) Cite this article

Abstract

Background

There are debates in acupuncture related systematic reviews and meta-analyses on whether searching Chinese databases to get more Chinese-language studies may increase the risk of bias and overestimate the effect size, and whether the treatment effects of acupuncture differ between Chinese and non-Chinese populations.

Methods

In this meta-epidemiological study, we searched the Cochrane library from its inception until December 2021, and identified systematic reviews and meta-analyses with acupuncture as one of the interventions. Paired reviewers independently screened the reviews and extracted the information. We repeated the meta-analysis of the selected outcomes to separately pool the results of Chinese- and non-Chinese-language acupuncture studies and presented the pooled estimates as odds ratios (OR) with 95% confidence interval (CI). We calculated the Ratio of ORs (ROR) by dividing the OR of the Chinese-language trials by the OR of the non-Chinese-language trials, and the ROR by dividing the OR of trials addressing Chinese population by the OR of trials addressing non-Chinese population. We explored whether the impact of a high risk of bias on the effect size differed between studies published in Chinese- and in non-Chinese-language, and whether the treatment effects of acupuncture differed between Chinese and non-Chinese population.

Results

We identified 84 Cochrane acupuncture reviews involving 33 Cochrane groups, of which 31 reviews (37%) searched Chinese databases. Searching versus not searching Chinese databases significantly increased the contribution of Chinese-language literature both to the total number of included trials (54% vs. 15%) and the sample size (40% vs. 15%). When compared with non-Chinese-language trials, Chinese-language trials were associated with a larger effect size (pooled ROR 0.51, 95% CI 0.29 to 0.91). We also observed a higher risk of bias in Chinese-language trials in blinding of participants and personnel (97% vs. 51%) and blinding of outcome assessment (93% vs. 47%). The higher risk of bias was associated with a larger effect estimate in both Chinese-language (allocation concealment: high/unclear risk vs. low risk, ROR 0.43, 95% CI 0.21 to 0.87) and non-Chinese-language studies (blinding of participants and personnel: high/unclear risk vs. low risk, ROR 0.41, 95% CI 0.23 to 0.74). However, we found no evidence that the higher risk of bias would increase the effect size of acupuncture in Chinese-language studies more often than in non-Chinese-language studies (the confidence intervals of all ROR in the high-risk group included 1, Table 3). We further found acupuncture appeared to be more effective in Chinese than in non-Chinese population (Table 4).

Conclusions

The findings of this study suggest the higher risk of bias may lead to an overestimation of the treatment effects of acupuncture but would not increase the treatment effects in Chinese-language studies more often than in other language studies. The difference in treatment effects of acupuncture was probably associated with differences in population characteristics.

Trial registration

We registered our protocol on the Open Science Framework (OSF) (https://doi.org/10.17605/OSF.IO/PZ6XR).

Peer Review reports

Background

Acupuncture is a popular complementary alternative treatment and is widely used both in China and elsewhere, in over 100 countries [1]. Acupuncture can be used to treat various diseases such as chronic pain, urinary incontinence, stroke, arthritis, and insomnia [2,3,4,5,6]. According to the recent collection of articles on acupuncture in the BMJ, over two thousand systematic reviews and meta-analyses (SR/MAs) of acupuncture therapies were published over the past 20 years, and many of them have been used to support recommendations in clinical guidelines [2, 7,8,9].

Acupuncture is a traditional Chinese form of therapy. The number of publications on acupuncture from Chinese institutions is increasing: according to a survey, studies from China accounted for 47% of the 13,320 acupuncture-related publications indexed in PubMed from 1995 to 2014 [10]. To avoid missing potentially relevant Chinese-language literature on acupuncture, many researchers have begun to search and include acupuncture studies in Chinese in their SR/MAs [11,12,13]. Based on a prior survey we performed, more than half of Cochrane acupuncture reviews have included Chinese-language studies, and many of them also searched Chinese databases to include more Chinese-language literature [14].

There are however some concerns. First, there have been fears that the inclusion of Chinese-language studies could increase the risk of bias in meta-analyses, and leading to misleading conclusions [15, 16]. Second, given the long application history of acupuncture in China, some people believe that acupuncture might be more effective in the Chinese population than elsewhere [17,18,19].

However, to our knowledge, no study has so far explored whether the treatment effects of acupuncture differ between Chinese and non-Chinese population, and whether the inclusion of Chinese-language studies would influence the effect size due to the increased risk of bias. To address these questions, we conducted this meta-epidemiologic study. The Cochrane Library has published more than 80 full reviews of acupuncture with rigorous methodology [14], which provided an optimal sample for this study.

Methods

Registration and protocol

We registered our protocol on the Open Science Framework (OSF) (https://doi.org/10.17605/OSF.IO/PZ6XR). Based on the feedback from the peer reviewers, we conducted additional analyses investigating the differences between populations, which were not included in the original protocol.

Data sources and selection

We searched the Cochrane Databases of Systematic Reviews from their inception until December 2021 using the keywords “acupuncture”, “needling*”, “acupressur*”, and “electro-acupuncture” (Appendix 1) [20]. Two reviewers (JL and XH) independently screened the studies in two stages: (1) titles and abstracts, and (2) full texts of potentially eligible studies. We included SR/MAs that included acupuncture as one of the interventions. Reviews were excluded if they (1) were withdrawn, (2) did not include any trials, (3) were protocols, or (4) were outdated versions.

Data extraction

Two reviewers (JL and XH) obtained the following details from each included review using a standardized pre-designed data extraction form: (1) basic information on the review including the first author, year, country, disease, retrieved databases, and language restrictions for searching the literature; (2) information on the included trials such as the number of trials, total sample size, risk of bias (ROB), the language (we accessed the full text of each trial to identify the publication language as Chinese or other), the target population (Chinese or non-Chinese population), and (3) data on binary outcomes including the number of events and sample size. From each review, we selected for analysis one primary binary outcome (as defined by the Cochrane Collaboration) that met the following conditions (applied in the following hierarchical order): (1) the forest plot included both Chinese- and non-Chinese-language studies; (2) the outcome had the largest number of trials; and (3) the outcome had the largest sample size. If no primary outcome met the criteria, we selected a secondary binary outcome according to the same criteria. Disagreements were resolved by discussion.

Data analysis

We presented the numbers of eligible Chinese-language studies and participants in acupuncture SR/MAs that searched versus did not search Chinese databases. Referring to the previous methodological studies [21,22,23,24], we used Review Manager 5.4 to repeat the meta-analyses of the selected outcomes using the same method as reported in the original reviews (fixed or random model, and the application of any correction for zero events) to separately pool the results of Chinese- and non-Chinese-language acupuncture studies, and presented the pooled estimates as odds ratios (OR) with 95% confidence intervals (CI). If necessary, we transformed the direction of the ORs so that OR < 1 was defined as favoring acupuncture.

We then calculated the Ratio of OR (ROR) for each outcome by dividing the OR of the Chinese-language trials by the OR of the non-Chinese-language trials, and the ROR by dividing the OR of trials addressing Chinese populations by the OR of trials addressing non-Chinese populations. To pool overall ROR or ROR in different groups, we used random-effects model, inverse variance method and the metafor package (metagen command) in R 4.1.3 software. If the 95% CI of ROR included 1 (null effect), we concluded that there was not enough evidence neither in favour nor against the Chinese-language acupuncture trials; if the 95% CI was completely below 1, Chinese-language studies were associated with larger effect size than non-Chinese-language studies.

Statistical heterogeneity among studies were measured by the I2 statistic and Q test [25]. For the I2 test we used the following definitions: [1] 0–40%, heterogeneity not necessarily important; [2] 30–60%, possibly moderate heterogeneity; [3] 50–90%: substantial heterogeneity; and [4] 75–100%: considerable heterogeneity. Subgroup analysis was conducted according to the types of outcomes (primary vs. secondary outcome) and threshold (high/unclear risk vs. low risk) of each ROB domain. We tested subgroup differences and considered P < 0.05 as statistically significant [26].

In each domain of ROB (randomization, concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting), we explored whether studies of each domain with the higher risk of bias overestimated the treatment effect of acupuncture in Chinese- and non-Chinese-language studies. ROR less than 1 implies that the treatment effects are overestimated in the trials with the domain with a high risk of bias.

We also explored, in studies with the higher risk of bias (including high and unclear risk of bias), whether the impact of the higher risk of bias on the treatment effect of acupuncture differ between the Chinese- and non-Chinese-language studies. ROR less than 1 implied that the contribution of the higher risk of bias to an overestimation of the treatment effects of acupuncture was greater in Chinese-language than non-Chinese-language studies. Furthermore, we explored whether the treatment effect of acupuncture differed between the Chinese and non-Chinese population. This analysis was restricted to studies with a low risk of bias to avoid confounding from the higher risk of bias. ROR less than 1 implied that acupuncture is more effective in Chinese population than in non-Chinese population.

Results

Characteristics of acupuncture reviews

Our study identified 84 Cochrane acupuncture reviews (Appendix Fig. 1, Appendix Table 1) involving 33 Cochrane groups. None of the reviews restricted the language except one [27] that only searched English-language literature. Among the 84 acupuncture reviews, 31 (37%) searched Chinese databases in addition to the standard databases recommended by Cochrane systematic review hanbook, such as MEDLINE, PUBMED, EMBASE, CENTRAL) (see details in Appendix Fig. 3).

As shown in Table 1, 45 acupuncture reviews (28 of which also searched Chinese databases) included studies in both Chinese and in other languages (331/352 = 94% of which were in English). Twenty-one [27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47] of these reviews that included only Chinese-language trials and nine reviews [48,49,50,51,52,53,54,55,56] that did not include any Chinese-language trials in the forest plots of binary outcomes were excluded from the quantitative analysis (Appendix Table 2). The remaining 15 [20, 57,58,59,60,61,62,63,64,65,66,67,68,69,70] acupuncture reviews included both Chinese- and non-Chinese-language studies in the forest plots of binary outcomes and were included in the quantitative analysis. Appendix Table 2 presents the selected binary outcomes for each review based on the pre-defined criteria (including nine primary and six secondary outcomes).

Table 1 The proportion of reviews, Chinese-language trials, and participants in Cochrane acupuncture reviews searching versus not searching Chinese databases
Full size table

Of the 31 reviews that searched Chinese databases, 28 (90%) included Chinese-language studies; of the 53 reviews that did not search Chinese databases, 17 (32%) included Chinese-language studies. The probability of including Chinese-language studies was thus approximately 1.8 times higher if Chinese databases were searched versus if not (90% vs. 32%, Table 1).

The 31 Cochrane acupuncture reviews that searched Chinese databases included a total of 264 Chinese-language trials (54% of all included trials) with 22,818 (40% of all included participants) participants (Table 1). The 53 Cochrane acupuncture reviews that did not search Chinese databases included a total of 55 (15%) Chinese-language trials (identified from standard databases) with 7,154 (15%) participants. Searching versus not searching Chinese databases in the acupuncture reviews, on average, resulted in a 2.6 times higher contribution of Chinese-language studies (54% vs. 15%, Table 1), and 1.7 times higher contribution of participants from Chinese-language studies (40% vs. 15%, Table 1) in the meta-analysis.

Risk of bias in chinese- and non-chinese-language acupuncture studies

The 45 eligible acupuncture reviews included a total of 319 Chinese-language trials (264 from reviews that searched Chinese databases and 55 from reviews that did not search any Chinese databases) and 352 non-Chinese-language trials (191 from reviews that searched Chinese databases and 161 from reviews that did not any search Chinese databases; Appendix Fig. 2). Figure 1 presents the risk of bias in the acupuncture studies by the language of publication, which suggested the Chinese-language acupuncture studies had a higher risk of bias regarding the blinding of participants and personnel (97% vs. 51% having high or unclear risk) and blinding of outcome assessment (93% vs. 47%) than studies written in languages other than Chinese (Fig. 1, Appendix Table 4, Appendix Table 5, Appendix Table 6).

Fig. 1
figure 1

The risk of bias in Chinese- and non-Chinese-language acupuncture trials

Full size image

Overall discrepancies in effect sizes between chinese- and non-chinese-language acupuncture studies

Chinese-language acupuncture studies tended to have a larger effect size than non-Chinese-language studies (pooled ROR 0.51, 95% CI 0.29 to 0.91, Fig. 2 and Appendix Fig. 4).

Fig. 2
figure 2

Discrepancies of effect sizes between Chinese- and non-Chinese-language acupuncture trials in the selected binary outcomes

Full size image

We found that the higher risk of bias was associated with larger effect size in both Chinese-language studies (allocation concealment: ROR 0.43, 95% CI 0.21 to 0.87, Table 2), and non-Chinese-language studies (blinding of participants and personnel: ROR 0.41, 95% CI 0.23 to 0.74, Table 2).

Table 2 Discrepancies of effect sizes of high/unclear risk versus low risk in each risk of bias domain in Chinese- and non-Chinese-language acupuncture trials
Full size table

We found no evidence suggesting that the higher risk of bias would increase the effect size in Chinese-language studies more often than in the non-Chinese-language studies (all confidence intervals of RORs in the higher risk of bias group included 1, Table 3).

Table 3 Discrepancies of treatment effects of Chinese- versus non-Chinese-language acupuncture trials stratified by ROB domains with high/unclear risk of bias
Full size table

Discrepancies in treatment effects between chinese and non-chinese populations

When restricting to low risk of bias studies, we found that acupuncture was associated with larger treatment effects when applied to Chinese than other populations (low risk in random sequence generation, Chinese vs. non-Chinese population: ROR 0.23, 95% CI 0.08 to 0.65; low risk in blinding of participants and personnel ROR 0.46, 95% CI 0.26 to 0.82; low risk in blinding of outcome assessment, ROR 0.50, 95% CI 0.28 to 0.87; and low risk in incomplete outcome data, ROR 0.54, 95% CI 0.32 to 0.93, Table 4).

Table 4 Discrepancies of treatment effects of Chinese versus non-Chinese population# in studies with a low risk of bias in each domain
Full size table

Discussion

Principal findings

Our findings suggest that an acupuncture related systematic review can increase the number of eligible studies and provide more information on the outcomes by extending the search to Chinese databases. Although the higher risk of bias could lead to an overestimation of treatment effects of acupuncture, we found no evidence that the higher risk of bias would increase the effect size of acupuncture in Chinese-language studies more often than in non-Chinese-language studies. We further tested the difference of treatment effects of acupuncture between different population groups and found acupuncture appearing to be more effective in Chinese populations than in other populations.

Strengths and limitations

Strengths of our study include rigour of data abstraction and analysis, and the exploration of the influence of different types of risk of bias on the effect size of outcomes between Chinese- and non-Chinese-language acupuncture studies. We also compared the effect sizes between Chinese and non-Chinese populations. The study provides evidence for confirming the benefits and harms of including Chinese-language studies in acupuncture SR/MAs. We analyzed SR/MAs from Cochrane Library which is the database of largest collection of high-quality SR/MAs worldwide. The findings from this study are likely to be generalizable to acupuncture SR/MAs published elsewhere as well.

One limitation of this study is that we did not conduct our own detailed assessment of risk of bias to verify the authors’ ratings. Another limitation is that, in some Cochrane acupuncture reviews, the small number of studies in one of the language groups led to a very wide confidence interval of the ROR and made it challenging to interpret the results. To minimize this bias, we separately pooled the overall RORs for primary and secondary outcomes. Finally, we found a high risk of bias in Chinese-language studies than in studies published in other languages, however, we could not determine whether this risk was different between studies only indexed in Chinese versus international databases.

Comparison with other studies

One prior study investigated 32 Cochrane reviews on acupuncture and suggested that searching Chinese databases did not increase the odds of positive conclusions in meta analyses (OR 0.51, 95% CI 0.11 to 2.44) when compared with not searching, and the odds of positive conclusions were also similar in Chinese-language acupuncture trials than in trials published in other languages (OR 1.29, 95% CI 0.26 to 6.49) [71]. Another previous study found that 60% of Cochrane acupuncture reviews did not search any Chinese database and inconclusive findings were more common in reviews that did not search any Chinese database [72]. In contrast to our study, none of the prior studies addressed the impact of risk of bias and population difference to the treatment effect of acupuncture.

Implications

Our findings suggest that investigators should balance the benefits and harms when deciding about the inclusion of Chinese-language studies indexed in Chinese databases for an acupuncture SR/MA. Although searching Chinese databases can substantially increase the number of eligible studies and sample size in acupuncture reviews, the potentially higher risk of bias is an argument that needs to be considered in the inclusion of Chinese-language studies. Patients, investigators, and guideline panels should be cautious when adopting evidence from acupuncture reviews where studies with a high risk of bias contributed with a high weight to the meta-analysis.

We also observed larger treatment effects of acupuncture in Chinese-language studies than in studies published in other languages. Although the treatment effects of acupuncture tended to be greater in studies with a high risk of bias, this potential overestimation did not differ between studies published in Chinese and in other languages. In other words, the larger treatment effects in Chinese-language studies cannot be explained by a high risk of bias. Furthermore, our study found acupuncture to be more effective in Chinese populations than in other populations, which could at least partly explain the larger treatment effects observed in Chinese-language studies.

Conclusion

Including Chinese-language trials in SR/MAs on acupuncture can increase the statistical power and provide more outcome data. A high risk of bias may lead to an overestimation of the treatment effects of acupuncture, but the degree of overestimation did not differ between Chinese-language studies and studies published in other languages. The difference in treatment effects of acupuncture was probably rather associated with differences in the study populations. Our findings can thus help understand the relationship between the risk of bias, language of publication, study population, and treatment effects in acupuncture related systematic reviews.

Data Availability

The datasets analysed during this study are available from the corresponding author on reasonable request.

Abbreviations

SRs:

Systematic reviews

MAs:

Meta-analyses

ROR:

Ratio of odds ratio

CI:

Confidence interval

OSF:

Open Science Framework

ROB:

Risk of bias

References

  1. World Health Organization. WHO global report on traditional and complementary medicine 2019 2019 [Available from: https://apps.who.int/iris/handle/10665/312342.

  2. Xu S, Yu L, Luo X, Wang M, Chen G, Zhang Q et al. Manual acupuncture versus sham acupuncture and usual care for prophylaxis of episodic migraine without aura: multicentre, randomised clinical trial. BMJ. 2020;368.

  3. Zhao L, Li D, Zheng H, Chang X, Cui J, Wang R, et al. Acupuncture as adjunctive therapy for chronic stable angina: a Randomized Clinical Trial. JAMA Intern Med. 2019;179(10):1388–97.

    Article  PubMed  PubMed Central  Google Scholar 

  4. He Y, Guo X, May BH, Zhang AL, Liu Y, Lu C, et al. Clinical evidence for Association of acupuncture and acupressure with Improved Cancer Pain: a systematic review and Meta-analysis. JAMA Oncol. 2020;6(2):271–8.

    Article  PubMed  Google Scholar 

  5. Lee IS, Chae Y. A bibliometric analysis of acupuncture research trends in acupuncture in Medicine. Acupunct medicine: J Br Med Acupunct Soc. 2019;37(6):375–7.

    Article  CAS  Google Scholar 

  6. Lu L, Zhang Y, Tang X, Ge S, Wen H, Zeng J, et al. Evidence on acupuncture therapies is underused in clinical practice and health policy. BMJ. 2022;376:e067475.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Zhang Y-Q, Lu L, Xu N, Tang X, Shi X, Carrasco-Labra A, et al. Increasing the usefulness of acupuncture guideline recommendations. BMJ. 2022;376:e070533.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Madsen MV, Gøtzsche PC, Hróbjartsson A. Acupuncture treatment for pain: systematic review of randomised clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups. BMJ. 2009;338:a3115.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Zhang Y-Q, Jiao R-M, Witt CM, Lao L, Liu J-P, Thabane L, et al. How to design high quality acupuncture trials—a consensus informed by evidence. BMJ. 2022;376:e067476.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Ma Y, Dong M, Zhou K, Mita C, Liu J, Wayne PM. Publication Trends in acupuncture research: a 20-Year bibliometric analysis based on PubMed. PLoS ONE. 2016;11(12):e0168123.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Chi Y, Barth J, Wang M, Robinson N, Li Z-H, Liu J-P. Eye acupuncture for pain conditions: a scoping review of clinical studies. BMC Complement Med Ther. 2021;21(1):101.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Wang Y. [Bibliometric study of randomized controlled trial of acupuncture in chinese journals and influencing factors of positive results]. Beijing University of Chinese Medicine; 2014.

  13. Chen Y, Zhang X, Fang Y, Yang J. Analyzing the study of using acupuncture in delivery in the past ten years in china. Evid Based Complement Alternat Med. 2014;2014:672508.

    PubMed  PubMed Central  Google Scholar 

  14. Lu Z, Lao Y, Yao L, Yan P, Hui X, Han X et al. Current Cochrane acupuncture reviews might be biased without searching Chinese, Korean and Japanese databases. In: Advances in Evidence Synthesis: special issue Cochrane Database of Systematic Reviews. 2020;9(Suppl 1).

  15. Luo S, Long Y, Xiao W, Wang X, Chen R, Guo Q, et al. Risk of bias assessments and reporting quality of systematic reviews and randomized controlled trials examining acupuncture for depression: an overview and meta-epidemiology study. J evidence-based Med. 2020;13(1):25–33.

    Article  Google Scholar 

  16. Higgins J, Thomas J, Chandler J, C M, L T. MJ. P, Cochrane Handbook for Systematic Reviews of Interventions. Chapter 7: Considering bias and conflicts of interest among the included studies Cochrane 2021 [Available from: https://training.cochrane.org/handbook/current/chapter-07.

  17. Chen FP, Kung YY, Chen TJ, Hwang SJ. Demographics and patterns of acupuncture use in the Chinese population: the Taiwan experience. Journal of alternative and complementary medicine (New York, NY). 2006;12(4):379 – 87.

  18. Lam CN, Murnane D, Ruth C, Kim Y, Chou CP, Black DS. Demographic and Clinical Factors Associated with Patient Adherence to Acupuncture Treatment. Journal of alternative and complementary medicine (New York, NY). 2019;25(5):522-5.

  19. Naito C. Ethnic factors in the acceptability of Foreign Clinical Data. Drug Inform J. 1998;32(1suppl):1283S–92S.

    Article  Google Scholar 

  20. Armour M, Ee CC, Hao J, Wilson TM, Yao SS, Smith CA. Acupuncture and acupressure for premenstrual syndrome.Cochrane Database of Systematic Reviews. 2018(8).

  21. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408–12.

    Article  CAS  PubMed  Google Scholar 

  22. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001;135(11):982–9.

    Article  CAS  PubMed  Google Scholar 

  23. Savovic J, Turner RM, Mawdsley D, Jones HE, Beynon R, Higgins JPT, et al. Association between Risk-of-Bias assessments and results of randomized trials in Cochrane Reviews: the ROBES Meta-epidemiologic study. Am J Epidemiol. 2018;187(5):1113–22.

    Article  PubMed  Google Scholar 

  24. Dechartres A, Atal I, Riveros C, Meerpohl J, Ravaud P. Association between publication characteristics and treatment effect estimates. Ann Intern Med. 2018;169(6):385–93.

    Article  PubMed  Google Scholar 

  25. Higgins J, Thomas J. Cochrane Handbook for Systematic Reviews of Interventions (Version 6.2, 2021)-Chap. 10.10: Analysing data and undertaking meta-analyses-Heterogeneity 2021 [Available from: https://training.cochrane.org/handbook/current/chapter-10#section-10-10.

  26. Richardson M, Garner P, Donegan S. Interpretation of subgroup analyses in systematic reviews: a tutorial. Clin Epidemiol Global Health. 2019;7(2):192–8.

    Article  Google Scholar 

  27. Xie Y, Wang L, He J, Wu T. Acupuncture for dysphagia in acute stroke.Cochrane Database of Systematic Reviews. 2008(3).

  28. Sinopoulou V, Gordon M, Akobeng AK, Gasparetto M, Sammaan M, Vasiliou J, et al. Interventions for the management of abdominal pain in Crohn’s disease and inflammatory bowel disease. Cochrane Database Syst Rev. 2021;11:Cd013531.

    PubMed  Google Scholar 

  29. Zhu X, Proctor M, Bensoussan A, Wu E, Smith CA. Chinese herbal medicine for primary dysmenorrhoea.Cochrane Database of Systematic Reviews. 2008(2).

  30. Yang A, Wu HM, Tang JL, Xu L, Yang M, Liu GJ. Acupuncture for stroke rehabilitation.Cochrane Database of Systematic Reviews. 2016(8).

  31. Wong V, Cheuk DKL, Lee S, Chu V. Acupuncture for acute management and rehabilitation of traumatic brain injury.Cochrane Database of Systematic Reviews. 2013(3).

  32. Wang Y, Zhishun L, Peng W, Zhao J, Liu B. Acupuncture for stress urinary incontinence in adults.Cochrane Database of Systematic Reviews. 2013(7).

  33. Teixeira LJ, Valbuza JS, Prado GF. Physical therapy for Bell’s palsy (idiopathic facial paralysis).Cochrane Database of Systematic Reviews. 2011(12).

  34. Smith CA, Collins CT, Levett KM, Armour M, Dahlen HG, Tan AL et al. Acupuncture or acupressure for pain management during labour.Cochrane Database of Systematic Reviews. 2020(2).

  35. Manheimer E, Cheng K, Wieland LS, Min LS, Shen X, Berman BM et al. Acupuncture for treatment of irritable bowel syndrome.Cochrane Database of Systematic Reviews. 2012(5).

  36. Lim CED, Ng RWC, Cheng NCL, Zhang GS, Chen H. Acupuncture for polycystic ovarian syndrome.Cochrane Database of Systematic Reviews. 2019(7).

  37. Kong DZ, Liang N, Yang GL, Zhang Z, Liu Y, Yang Y et al. Acupuncture for chronic hepatitis B.Cochrane Database of Systematic Reviews. 2019(8).

  38. Kim TH, Lee MS, Kim KH, Kang JW, Choi TY, Ernst E. Acupuncture for treating acute ankle sprains in adults. Cochrane Database of Systematic Reviews. 2014(6).

  39. Green S, Buchbinder R, Barnsley L, Hall S, White M, Smidt N et al. Acupuncture for lateral elbow pain.Cochrane Database of Systematic Reviews. 2002(1).

  40. Deare JC, Zheng Z, Xue CCL, Liu JP, Shang J, Scott SW et al. Acupuncture for treating fibromyalgia.Cochrane Database of Systematic Reviews. 2013(5).

  41. Cui Y, Wang Y, Liu Z. Acupuncture for restless legs syndrome. Cochrane Database of Systematic Reviews. 2008(4).

  42. Crepinsek MA, Taylor EA, Michener K, Stewart F. Interventions for preventing mastitis after childbirth. Cochrane Database Syst Rev. 2020;9(9):Cd007239.

    PubMed  Google Scholar 

  43. Coyle ME, Smith CA, Peat B. Cephalic version by moxibustion for breech presentation.Cochrane Database of Systematic Reviews. 2012(5).

  44. Choi GH, Wieland LS, Lee H, Sim H, Lee MS, Shin BC. Acupuncture and related interventions for the treatment of symptoms associated with carpal tunnel syndrome.Cochrane Database of Systematic Reviews. 2018(12).

  45. Cheuk DKL, Wong V, Chen WX. Acupuncture for autism spectrum disorders (ASD).Cochrane Database of Systematic Reviews. 2011(9).

  46. Cheng K, Law A, Guo M, Wieland LS, Shen X, Lao L. Acupuncture for acute hordeolum.Cochrane Database of Systematic Reviews. 2017(2).

  47. Cao H, Yang G, Wang Y, Liu JP, Smith CA, Luo H et al. Complementary therapies for acne vulgaris.Cochrane Database of Systematic Reviews. 2015(1).

  48. Yang J, Chen J, Yang M, Yu S, Ying L, Liu GJ et al. Acupuncture for hypertension.Cochrane Database of Systematic Reviews. 2018(11).

  49. Trinh K, Graham N, Irnich D, Cameron ID, Forget M. Acupuncture for neck disorders.Cochrane Database of Systematic Reviews. 2016(11).

  50. Smith CA, Armour M, Dahlen HG. Acupuncture or acupressure for induction of labour.Cochrane Database of Systematic Reviews. 2017(10).

  51. Natale P, Ruospo M, Saglimbene VM, Palmer SC, Strippoli GFM. Interventions for improving sleep quality in people with chronic kidney disease.Cochrane Database of Systematic Reviews. 2019(5).

  52. Mitchell C, Bowen A, Tyson S, Butterfint Z, Conroy P. Interventions for dysarthria due to stroke and other adult-acquired, non‐progressive brain injury.Cochrane Database of Systematic Reviews. 2017(1).

  53. Manheimer E, Cheng K, Wieland LS, Shen X, Lao L, Guo M et al. Acupuncture for hip osteoarthritis.Cochrane Database of Systematic Reviews. 2018(5).

  54. Lan L, Zeng F, Liu GJ, Ying L, Wu X, Liu M et al. Acupuncture for functional dyspepsia.Cochrane Database of Systematic Reviews. 2014(10).

  55. Kwan I, Wang R, Pearce E, Bhattacharya S. Pain relief for women undergoing oocyte retrieval for assisted reproduction.Cochrane Database of Systematic Reviews. 2018(5).

  56. Franco JVA, Turk T, Jung JH, Xiao YT, Iakhno S, Garrote V et al. Non-pharmacological interventions for treating chronic prostatitis/chronic pelvic pain syndrome.Cochrane Database of Systematic Reviews. 2018(5).

  57. Griffiths JD, Gyte GM, Popham PA, Williams K, Paranjothy S, Broughton HK, et al. Interventions for preventing nausea and vomiting in women undergoing regional anaesthesia for caesarean section. Cochrane Database Syst Rev. 2021;5(5):Cd007579.

    PubMed  Google Scholar 

  58. Xu M, Li D, Zhang S. Acupuncture for acute stroke.Cochrane Database of Systematic Reviews. 2018(3).

  59. White AR, Rampes H, Liu JP, Stead LF, Campbell J. Acupuncture and related interventions for smoking cessation.Cochrane Database of Systematic Reviews. 2014(1).

  60. Thomas LH, Coupe J, Cross LD, Tan AL, Watkins CL. Interventions for treating urinary incontinence after stroke in adults.Cochrane Database of Systematic Reviews. 2019(2).

  61. Smith CA, Armour M, Zhu X, Li X, Lu ZY, Song J. Acupuncture for dysmenorrhoea.Cochrane Database of Systematic Reviews. 2016(4).

  62. Smith CA, Armour M, Lee MS, Wang LQ, Hay PJ. Acupuncture for depression.Cochrane Database of Systematic Reviews. 2018(3).

  63. Shen X, Xia J, Adams CE. Acupuncture for schizophrenia. Cochrane Database of Systematic Reviews. 2014(10).

  64. Lee A, Chan SKC, Fan LTY. Stimulation of the wrist acupuncture point PC6 for preventing postoperative nausea and vomiting.Cochrane Database of Systematic Reviews. 2015(11).

  65. Kim KH, Lee MS, Kim TH, Kang JW, Choi TY, Lee JD. Acupuncture and related interventions for symptoms of chronic kidney disease. Cochrane Database of Systematic Reviews. 2016(6).

  66. Kim KH, Lee MS, Choi TY, Kim TH. Acupuncture for symptomatic gastroparesis.Cochrane Database of Systematic Reviews. 2018(12).

  67. Ju ZY, Wang K, Cui HS, Yao Y, Liu SM, Zhou J et al. Acupuncture for neuropathic pain in adults.Cochrane Database of Systematic Reviews. 2017(12).

  68. Huang T, Shu X, Huang YS, Cheuk DKL. Complementary and miscellaneous interventions for nocturnal enuresis in children.Cochrane Database of Systematic Reviews. 2011(12).

  69. Cheuk DKL, Wong V. Acupuncture for epilepsy.Cochrane Database of Systematic Reviews. 2014(5).

  70. Bath PM, Lee HS, Everton LF. Swallowing therapy for dysphagia in acute and subacute stroke.Cochrane Database of Systematic Reviews. 2018(10).

  71. Jiao S, Tsutani K, Haga N. Review of Cochrane reviews on acupuncture: how chinese resources contribute to Cochrane reviews. Journal of alternative and complementary medicine. (New York NY). 2013;19(7):613–21.

    Google Scholar 

  72. Lui S, Smith EJ, Terplan M. Heterogeneity in search strategies among Cochrane acupuncture reviews: is there room for improvement? Acupuncture in medicine. J Br Med Acupunct Soc. 2010;28(3):149–53.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This study is supported by the Major Project of the National Social Science Fund of China: Research on the Theoretical System, International Experience, and Chinese Path of Evidence-based Social Science (NO.: 19ZDA14).

Author information

Author notes

  1. Jing Li is first author. Xu Hui and Liang Yao are co-first authors.

Authors and Affiliations

  1. Health technology Assessment Centre, Evidence Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, China

    Jing Li, Xu Hui & Kehu Yang

  2. Evidence Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China

    Jing Li, Xu Hui, Janne Esill & Kehu Yang

  3. Health Research Methodology, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada

    Liang Yao & Qi Wang

  4. Second Clinical Medical College of Lanzhou University, Lanzhou, China

    Anya Shi, Xiaowen Feng & Yifan Qiao

  5. West China school of public health, Sichuan university, Chengdu, China

    Peijing Yan

  6. First Clinical Medical College of Lanzhou University, Lanzhou, China

    Yuan Yao, Jingxi Wu & Wenhao Zhang

  7. Health Policy PhD Program and McMaster Health Forum, McMaster University, Hamilton, ON, Canada

    Qi Wang

  8. School of Chinese Medicine, Hong Kong Chinese Medicine Clinical Study Center, Hong Kong Baptist University, Hong Kong, China

    Yanfang Ma & Huijuan Li

  9. Chinese EQUATOR Centre, Hong Kong, China

    Yanfang Ma

  10. Institute of Environmental Medicine, Karolinska Institute, Stockholm, Sweden

    Dang Wei

  11. School of Acupuncture, Moxibustion and Massage, Chengdu University of Traditional Chinese Medicine, Chengdu, China

    Lei Lan

  12. Faculty of Medicine and Health, The Back Pain Research Team, Sydney Musculoskeletal Health, The Kolling Institute, University of Sydney, Sydney, NSW, Australia

    Lingxiao Chen

  13. Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China

    Lijiao Yan

  14. Guangzhou University of Chinese Medicine, Guangzhou, China

    Fang Fang

  15. Institute of Global Health, University of Geneva, Geneva, Sweden

    Janne Esill

  16. The First Hospital of Lanzhou University, Lanzhou, China

    Chengdong Qiao

  17. Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou, China

    Kehu Yang

Authors
  1. Jing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xu Hui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liang Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anya Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peijing Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuan Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Qi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yanfang Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Dang Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Lei Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Lingxiao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Lijiao Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Fang Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Huijuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Xiaowen Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Jingxi Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Yifan Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Wenhao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Janne Esill
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Chengdong Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Kehu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conception and design: Jing Li, Xu Hui, Liang Yao, Kehu Yang.

Analysis and interpretation of the data: Jing Li, Xu Hui, Liang Yao, Kehu Yang, Chengdong Qiao.

Drafting of the article: Jing Li, Xu Hui, Liang Yao.

Critical revision of the article for important intellectual content: Jing Li, Xu Hui, Liang Yao, Anya Shi, Peijing Yan, Qi Wang, Yanfang Ma, Dang Wei, Lingxiao Chen, Lijiao Yan, Fang Fang, Lei Lan, Huijuan Li, Janne Esill.

Final approval of the article: Jing Li, Xu Hui, Liang Yao, Kehu Yang, Chengdong Qiao.

Statistical expertise: Peijing Yan, Dang Wei.

Obtaining of funding: Kehu Yang.

Administrative, technical, or logistic support: Kehu Yang, Chengdong Qiao.

Collection and assembly of data: Jing Li, Anya Shi, Xu Hui, Yuan Yao, Xiaowen Feng, Jingxi Wu, Yifan Qiao, Wenhao Zhang.

Corresponding authors

Correspondence to Chengdong Qiao or Kehu Yang.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing Interest

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Hui, X., Yao, L. et al. The relationship of publication language, study population, risk of bias, and treatment effects in acupuncture related systematic reviews: a meta-epidemiologic study. BMC Med Res Methodol 23, 96 (2023). https://doi.org/10.1186/s12874-023-01904-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12874-023-01904-w

Keywords

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us