Identifying competing interest disclosures in systematic reviews of surgical interventions and devices: a cross-sectional survey
BMC Medical Research Methodology volume 20, Article number: 260 (2020)
A competing interest is an important source of bias in research and disclosure is frequently employed as a strategy to manage it. Considering the importance of systematic reviews (SRs) and the varying prevalence of competing interests in different research fields, we conducted a survey to identify the range of competing interests in SRs assessing surgical interventions or devices and explored the association between the competing interest disclosures and authors’ conclusions.
We retrieved SRs of surgical interventions and devices published in 2017 via PubMed. Information regarding general characteristics, funding sources, and competing interest disclosures were extracted. We conducted a descriptive analysis of the studies’ characteristics and compared them between Cochrane SRs (CSRs) and non-Cochrane SRs using the Chi-square test. Results were expressed as odds ratio and their 95% confidence interval.
One hundred fifty-five SRs published in 2017 were included in the study. More than half of the SRs (58.7%) reported their funding sources and 94.2% reported authors’ competing interest disclosures. Among 146 SRs that stated competing interest disclosures, only 35 (22.6%) SRs declared at least one author had a competing interest. More than 40 terms were used to describe competing interests. Cochrane SRs (CSRs) were more likely to provide a detailed description of competing interests compared to those in non-CSRs (48.0% versus 25.4%, P = 0.023). No association between positive conclusions and competing interest disclosures was found (P = 0.484, OR = 0.43, 95%CI: 0.08, 2.16). In the subgroup analyses, SRs stating no competing interest disclosure were more likely to report positive conclusions than those stating at least one type of competing interest, but the difference is not significantly different (P = 0.406, OR = 1.38, 95%CI: 0.64, 2.98).
In surgical SRs, there is a high percentage of competing interest disclosures but without detailed information. The identification and statement of competing interests with a detailed description, particularly the non-financial ones, needs improvement. Some efficient and effective methods/tools for identifying, quantifying, and minimizing potential competing interests in systematic reviews remains valuable.
A competing interest is defined as “a set of circumstances which create a risk that professional judgment or actions regarding a primary interest will be unduly influenced by a secondary interest” [1, 2] and considered to be “ubiquitous and inevitable in academic life” [3, 4]. A full disclosure is acknowledged as an important method for reporting and managing competing interests and serves to highlight the potential for bias [5,6,7,8]. Since 1988, the International Committee of Medical Journal Editors (ICMJE) has required authors to disclose financial or other relationships that might lead to a competing interest , and hundreds of other biomedical journals put similar policy into practice .
The number of systematic reviews (SR) increased rapidly in the recent year for providing end-users with a comprehensive, critical, and up to date method to synthesize available evidence. Meanwhile, studies regarding competing interests of systematic reviews increased for the concerns about suboptimal decisions made by end-users for potential competing interests [11,12,13,14,15,16]. Surgery, with its wide use of medical devices and strong personal preference for interventions, could be a field with high potential for competing interests . In 2017, the JAMA published a collection to discuss issues relating to competing interests and stated that transparent disclosure is increasingly emphasized as an essential component in the reporting of both clinical trials and systematic reviews [18, 19]. Although numerous published studies addressed competing interests in medical research, there is a significant disproportion in the number of studies assessing competing interests, especially non-financial interests in surgical interventions and devices research [20,21,22].
Considering the importance of SRs and the varying prevalence of competing interests in different fields . We, therefore, performed a survey to identify the range of competing interests in systematic reviews assessing surgical interventions and devices and proposed a potential checklist for competing interest disclosures in surgical research. We also explored the impact of competing interest disclosures on the study conclusions.
A study was included if it was described as a systematic review or a meta-analysis and only included RCTs assessing at least one surgical intervention or device. Network meta-analyses, methodological systematic reviews or systematic reviews reported as conference abstracts and research letters were excluded. A systematic review was defined as described in the Cochrane handbook (a. a clearly stated set of objectives with an explicit, reproducible methodology; b. a systematic search that attempts to identify all studies that would meet the eligibility criteria; c. an assessment of the validity of the findings of the included studies, such as through the assessment of risk of bias; and d. systematic presentation and synthesis of the characteristics and findings of the included studies)  and the definitions of a surgical intervention and device have been described elsewhere . A competing interest disclosure was defined as whether a competing interest disclosure was stated or not.
Data resource and study procedures
Systematic reviews were identified through PubMed, we limited our search to literature published in the English language, and between 1 January 2017 and 31 December 2017 due to a massive number of published surgical studies. The search strategy was based on MeSH terms and their variants and developed in collaboration with an experienced librarian (Additional file 1). Two reviewers (JY and GS), trained in trial and systematic review methods, used predefined, pilot-tested forms independently to screen abstracts and full texts for eligibility.
Two teams of reviewers (JY and GS, YZ and ZY) independently collected data from eligible studies. Twenty systematic reviews met eligibility criteria were enrolled for a pilot study before the formal screening and data extraction. We iteratively adjusted the screening rules to improve the consistency between reviewers. Any conflicts not resolved by discussion between the reviewers were referred to the study team. Coher’s κ statistic was used to assess the concordance of reviewers, and a value of 0.75 or greater was chosen for satisfactory agreement.
The following general information was collected for each eligible study: (1) number of authors; (2) country of the corresponding author; (3) number of trials included; (4) total number of participants involved; (5) review type (Cochrane SR or non-Cochrane SR); (6) adherence to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses); (7) type of journal (general or surgical journal); (8) type of control (surgical intervention, non-surgical intervention or both); (9) involvement of methodologist (i.e. statistician, epidemiologist). We judged that a systematic review involved a methodologist if any of the authors were affiliated with a department of epidemiology, statistics, or evidence-based medicine, or listed in the acknowledgement section.
We extracted the funding sources and classified them as industry funding (e.g. funding from the device industry; company; insurance company), non-industry funding (e.g. funding from the academic institution; university; hospital; government bodies; foundation; charity et al.), no funding and not reported. We extracted the statement of competing interests verbatim and then categorized them based on their characteristics and nature.
The authors’ conclusion was labelled “positive” when the overall results favoured intervention over control or both groups were equal but the interventional group had minor advantages. A “negative” conclusion was defined if the results favoured control or both groups were equal .
We conducted a descriptive analysis of study characteristics of included systematic reviews. For all descriptive analyses, frequencies (and percentages) were used for dichotomous variables, and mean (and standard deviation) or median (and range) for continuous variables. We compared the funding sources and competing interest disclosures between Cochrane SR and non-CSR using Fisher’s exact test for dichotomous variables.
We categorized the studies into two groups based on whether the investigators disclosed competing interests or not. If they did, we then divided the competing interest disclosure group into two subgroups (at least one type of competing interest was disclosed and no competing interest disclosure group). The differences in the authors’ conclusions were compared between groups using the Chi-square test. Results were expressed as odds ratios and their 95% confidence interval. All analyses were performed by IBM SPSS Statistics 26.
A total of 6256 systematic reviews were identified from PubMed. After the title and abstract screening, 479 were found to be potentially eligible, of which 155 were finally included based on the full-text article review (Fig. 1). The interobserver agreement on data collection was good (κ = 0.84). A full list of the included SRs is presented in Additional file 2.
General information from the included SRs
The general characteristics are listed in Table 1. The median number of included trials was 6 (range 2 to 21), 25 (16.1%) were Cochrane SRs and 89 (57.4%) mentioned PRISMA in their studies. 109 (70.3%) were published in general medical journals; 23 (16.1%) involved researchers with affiliations with either epidemiology or statistics departments, and 132 (85.2%) tested alternative surgical interventions.
Funding information from the included SRs
Of the 155 selected SRs, 42 (27.1%) reported review funding from governmental agencies and academic institutions, three (1.9%) reported review funding from pharmaceutical or devices company, 45 (29.0%) stated the reviews were not funded; and 64 (41.3%) did not provide the funding information (Table 2).
Competing interest disclosures concerning the included SRs
One hundred and forty-six studies (94.2%) disclosed the information of authors’ competing interests. However, of the 146 systematic reviews that provided competing interest disclosures, only 35 (22.6%) SRs declared at least one author had one competing interest. More than 40 terms were used to describe competing interests, and we categorized them into 12 terms that were the most common descriptions of competing interests, including grants/fellowship, honoraria, consulting, employment/salary, patent/copyright, equity/stocks/bonds, non-monetary support, service in other affiliations, founder or other leadership in a company, intellectual beliefs, experience, and personal relationship.
Of the 35 SRs that stated competing interests, Cochrane SRs were more likely to provide a detail description of the competing interests in comparison to those that are non-CSRs (48.0% versus 25.4%, P = 0.023). The most frequently reported types was “grant/fellowship” (n = 17, 11.0%), followed by “honoraria” (n = 15, 9.7%), “consulting” (n = 14, 9.0%), “non-monetary support” and “services in other affiliations” (n = 8, 5.2%) (Table 2).
We found no significant difference between SRs that had a competing interest disclosure and those that did not (P = 0.484, OR = 0.43, 95%CI: 0.08, 2.16). In the subgroup analyses, SRs stating no competing interest disclosure were more likely to report a positive conclusion than those stating at least one type of competing interest. However, the difference is still not significant (P = 0.406, OR = 1.38, 95%CI: 0.64, 2.98).
Our survey shows that 94.2% of surgical systematic reviews published in 2017 reported authors’ competing interest disclosures. This result is in line with the strict journal requirement for more than 95% of medical journals have a policy, that refers to competing interest disclosure as an essential part of biomedical studies . However, of the147 studies that declared their competing interests, only 35 (22.6%) provided a detailed disclosure.
There are several possible reasons for the low rate of detailed competing interest disclosures. One of the reasons is, most researchers are unaware of the fact that behaviours which include receiving support for food and beverages or academic competition are also competing interests and could affect their research [9, 27]. Another potential possibility is the policies regarding some competing interests are regularly obscure, particularly non-financial competing interests [16, 28,29,30]. We reviewed the “instruction to authors” on journals’ websites and found that the instructions of competing interests are limited, with most of them only involving a few types, such as a grant, consulting fee and employment. We have summarized a checklist of competing interests based on our results to assist researchers in identifying potential competing interests. Journals should also consider to include this checklist in their policies or instructions (Table 3).
Several methodological studies have been conducted to assess the reporting of competing interest disclosure, and there is substantial variability in the reporting rates among different study designs and specialties (17.0 to 71.2%) [16,17,18,19, 21, 22, 31,32,33,34]. Generally, authors of randomized controlled trials and cohort studies more likely to provide detailed information rather than those in systematic reviews. Our finding on the rate of competing interest disclosure is consistent with other studies assessing SRs [16,17,18] and no significant difference was found between trials in the drug and surgical field.
A competing interest in biomedical research has often been associated with a positive conclusion [19, 20, 35,36,37]; however, we did not have a similar finding in our study. Considering Cochrane’ policy regarding funding and competing interest is stricter than other journals , we also conducted sensitivity analyses that only included non-CSRs and the result was the same. Notably, our findings should be cautiously interpreted for the low percentage of SRs sponsored by industries in comparison to other published studies [19, 35].
We used a systematic method for data screening and extraction and provided a potential checklist of competing interests to readers and editors based on the results. However, our study also has some limitations. First, the restriction of our search to one database and year could compromise the generalizability of findings. Second, these disclosures have all been reported by authors, and we have limited methods to confirm or verify them. Thirdly, we only assessed SRs that included randomized controlled trials which could have underestimated the rate of competing interest disclosures.
In summary, 94% of SRs assessing surgical interventions and devices stated their competing interest disclosures but without detailed information. The identification and statement of competing interests, particularly non-financial interests, remains challenging for researchers, reviewers, and editors. The International Committee of Medical Journal Editors (ICMJE) and other communities need to continue to develop more efficient and effective methods/tools for identifying, quantifying, and minimizing potential competing interests in systematic reviews.
Availability of data and materials
The dataset supporting the conclusions of this article are included within the article and its additional files.
International Committee of Medical Journal Editors
Cochrane systematic review
Schunemann HJ, Al-Ansary LA, Forland F, Kersten S, Komulainen J, Kopp IB, et al. Guidelines international network: principles for disclosure of interests and management of conflicts in guidelines. Ann Intern Med. 2015;163:548e53.
Field MJ, Lo B. Conflict of interest in medical research, education, and practice. Washington, DC: National Academies Press; 2009.
DeAngelis CD, Fontanarosa PB, Flanagin A. Reporting financial conflicts of interest and relationships between investigators and research sponsors. JAMA. 2001;286(1):89–91.
Fontanarosa PB, Flanagin A, DeAngelis CD. Reporting conflicts of interest, financial aspects of research, and role of sponsors in funded studies. JAMA. 2005;294(1):110–1.
Jureidini JN, McHenry LB. Conflicted medical journals and the failure of trust. Account Res. 2011;18(1):45–54.
Eliades T, Turpin DL. Conflict of interest: always report it, and if in doubt, ask. Am J Orthod Dentofac Orthop. 2008;134(3):327–8.
Cooper RJ, Gupta M, Wilkes MS, Hoffman JR. Conflict of interest disclosure policies and practices in peer-reviewed biomedical journals. J Gen Intern Med. 2006;21(12):1248–52.
Lipton S, Boyd EA, Bero LA. Conflicts of interest in academic research: policies, processes, and attitudes. Account Res. 2004;11(2):119–39.
International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. BMJ. 1991;302(6772):338–41.
Drazen JM, Van der Weyden MB, Sahni P, et al. Uniform format for disclosure of competing interests in ICMJE journals. N Engl J Med. 2009;122(1305):12–4.
Forsyth SR, Odierna DH, Krauth D, Bero LA. Conflicts of interest and critiques of the use of systematic reviews in policymaking: an analysis of opinion articles. Syst Rev. 2014;3:122.
Hakoum MB, Anouti S, Al-Gibbawi M, et al. Reporting of financial and non-financial conflicts of interest by authors of systematic reviews: a methodological survey. BMJ Open. 2016;6(8):e011997.
Bou-Karroum L, Hakoum MB, Hammoud MZ, et al. Reporting of financial and non-financial conflicts of interest in systematic reviews on health policy and systems research: a cross sectional survey. Int J Health Policy Manag. 2018;7(8):711–7.
Klanica K. Conflicts of interest in medical research: how much conflict should exceed legal boundaries? Ann Intern Med. 2006;144(3):225.
Kaestner V, Prasad V. Financial conflicts of interest among editorialists in high-impact journals. Blood Cancer J. 2017;7(9):e611.
Roseman M, Turner EH, Lexchin J, Coyne JC, Bero LA, Thombs BD. Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study. BMJ. 2012;345:e5155.
Patel SV, Yu D, Elsolh B, Goldacre BM, Nash GM. Assessment of conflicts of interest in robotic surgical studies: validating Author's declarations with the open payments database. Ann Surg. 2018;268(1):86–92.
Roseman M, Milette K, Bero LA, et al. Reporting of conflicts of interest in meta-analyses of trials of pharmacological treatments. JAMA. 2011;305(10):1008–17.
Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289(4):454–65.
Okike K, Kocher MS, Mehlman CT, Bhandari M. Conflict of interest in orthopaedic research. An association between findings and funding in scientific presentations. J Bone Joint Surg Am. 2007;89(3):608–13.
Agha RA, Barai I, Rajmohan S, et al. Support for reporting guidelines in surgical journals needs improvement: a systematic review. Int J Surg. 2017;45:14–7.
Probst P, Huttner FJ, Klaiber U, Diener MK, Buchler MW, Knebel P. Thirty years of disclosure of conflict of interest in surgery journals. Surgery. 2015;157(4):627–33.
Adie S, Ma D, Harris IA, Naylor JM, Craig JC. Quality of conduct and reporting of meta-analyses of surgical interventions. Ann Surg. 2015;261(4):685–94.
Cochrane Handbook for Systematic Reviews of Interventions (version 6, 2019). https://training.cochrane.org/handbook/current. Accessed 15 Aug 2019.
Yu J, Li X, Li Y, Sun X. Quality of reporting in surgical randomized clinical trials. Br J Surg. 2017;104(3):296–303.
Kjaergard LL, Als-Nielsen B. Association between competing interests and authors’ conclusions: epidemiological study of randomised clinical trials published in the BMJ. BMJ. 2002;325(7358):249.
Cochrane Community. Disclosure of Potential Conflicts of Interest form: Instructions https://community.cochrane.org/help/tools-and-software/archie/resources-archie/disclosure-potential-conflicts-interest-form-instructions. Accessed 15 Jan 2020.
Grundy Q, Mayes C, Holloway K, Mazzarello S, Thombs BD, Bero L. Conflict of interest as ethical shorthand: understanding the range and nature of “non-financial conflict of interest” in biomedicine. J Clin Epidemiol. 2019;120:1–7.
Hakoum MB, Jouni N, Abou-Jaoude EA, et al. Authors of clinical trials reported individual and financial conflicts of interest more frequently than institutional and nonfinancial ones: a methodological survey. J Clin Epidemiol. 2017;87:78–86.
Shawwa K, Kallas R, Koujanian S, et al. Requirements of clinical journals for authors? Disclosure of financial and non-financial conflicts of interest: a cross sectional study. PLoS One. 2016;11(3):e0152301.
Lerner TG, Miranda Mda C, Lera AT, et al. The prevalence and influence of self-reported conflicts of interest by editorial authors of phase III cancer trials. Ann Intern Med. 2012;156(11):809–16.
Hakoum MB, Jouni N, Abou-Jaoude EA, et al. Characteristics of funding of clinical trials: cross-sectional survey and proposed guidance. BMJ Open. 2017;7(10):e015997.
Cherla DV, Olavarria OA, Bernardi K, et al. Investigation of financial conflict of interest among published ventral hernia research. J Am Coll Surg. 2018;226(3):230–4.
Bridoux V, Moutel G, Schwarz L, Michot F, Herve C, Tuech JJ. Disclosure of funding sources and conflicts of interest in phase III surgical trials: survey of ten general surgery journals. World J Surg. 2014;38(10):2487–93.
Lundh A, Barbateskovic M, Hrobjartsson A, Gotzsche PC. Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue - cohort study. PLoS Med. 2010;7(10):e1000354.
Cherla DV, Viso CP, Olavarria OA, et al. The impact of financial conflict of interest on surgical research: an observational study of published manuscripts. World J Surg. 2018;42(9):2757–62.
Fabbri A, Gregoraci G, Tedesco D, et al. Conflict of interest between professional medical societies and industry: a cross-sectional study of Italian medical societies’ websites. BMJ Open. 2016;6(6):e011124.
We would like to express our deep appreciation for Dr. Mudathir (Maimonides Medical Center, Southeast University) for his linguistic assistance during the preparation of this manuscript.
The first author (JY) was supported by a grant from the “National Natural Science Foundation of China” (project No: 71704120), “National Science and Technology Major Projects” (project No: 2018ZX09201009–005-003), and International visiting program for Excellent Young Scholars of SCU. The funders had no involvement in study design, collection, analysis, or interpretation of data, writing the manuscript, and the decision to submit the manuscript for publication.
Ethics approval and consent to participate
Consent for publication
All authors declare no competing interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Yu, J., Su, G., Hirst, A. et al. Identifying competing interest disclosures in systematic reviews of surgical interventions and devices: a cross-sectional survey. BMC Med Res Methodol 20, 260 (2020). https://doi.org/10.1186/s12874-020-01144-2