A scoping review of comparisons between abstracts and full reports in primary biomedical research
BMC Medical Research Methodology volume 17, Article number: 181 (2017)
Evidence shows that research abstracts are commonly inconsistent with their corresponding full reports, and may mislead readers. In this scoping review, which is part of our series on the state of reporting of primary biomedical research, we summarized the evidence from systematic reviews and surveys, to investigate the current state of inconsistent abstract reporting, and to evaluate factors associated with improved reporting by comparing abstracts and their full reports.
We searched EMBASE, Web of Science, MEDLINE, and CINAHL from January 1st 1996 to September 30th 2016 to retrieve eligible systematic reviews and surveys. Our primary outcome was the level of inconsistency between abstracts and corresponding full reports, which was expressed as a percentage (with a lower percentage indicating better reporting) or categorized rating (such as major/minor difference, high/medium/low inconsistency), as reported by the authors. We used medians and interquartile ranges to describe the level of inconsistency across studies. No quantitative syntheses were conducted. Data from the included systematic reviews or surveys was summarized qualitatively.
Seventeen studies that addressed this topic were included. The level of inconsistency was reported to have a median of 39% (interquartile range: 14% - 54%), and to range from 4% to 78%. In some studies that separated major from minor inconsistency, the level of major inconsistency ranged from 5% to 45% (median: 19%, interquartile range: 7% - 31%), which included discrepancies in specifying the study design or sample size, designating a primary outcome measure, presenting main results, and drawing a conclusion. A longer time interval between conference abstracts and the publication of full reports was found to be the only factor which was marginally or significantly associated with increased likelihood of reporting inconsistencies.
This scoping review revealed that abstracts are frequently inconsistent with full reports, and efforts are needed to improve the consistency of abstract reporting in the primary biomedical community.
Abstracts of primary research can provide concise information on a study’s purpose, methods, main results and conclusions. It is not uncommon that abstracts are used to aid decision-making, especially when full reports cannot be accessed [1, 2]. However, there is a risk that the reporting of abstracts is inconsistent with their corresponding full reports, which would then distract or mislead readers [3, 4]. Although the EQUATOR (Enhancing Quality and Transparency in Health Research) network has provided some guidelines to enhance the reporting of abstracts , adherence to them remains unsatisfactory [6,7,8,9]. Some studies assessing the inconsistency between abstracts and full reports have reported striking findings of abstract inaccuracy [10,11,12]. Nevertheless, there is a lack of summary showing the general mapping of the abstract reporting problem or providing overarching recommendations in primary studies for future research in the literature. Therefore, as part our series on the state of reporting of primary biomedical research , we used a scoping review to summarize the evidence from systematic reviews and surveys, in order to investigate the current state of inconsistent abstract reporting, and also to evaluate factors that are associated with improved reporting. This was done by comparing abstracts and their corresponding full reports.
We performed and reported our study based on the methodological guidance for the conduct of a scoping review from the Joanna Briggs Institute  and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline . Details on the methods can be found in our protocol . Surveys and systematic reviews were considered eligible if they compared abstracts within full reports with the reports themselves, or if they compared conference abstracts with subsequent full reports emanating from the same study. We did not distinguish between these comparisons in this scoping review.
Search strategy and study selection
Briefly, we searched EMBASE (Exerpta Medica Database), Web of Science, MEDLINE, and CINAHL (Cumulative Index to Nursing and Allied Health Literature) from January 1st 1996 to September 30th 2016 to retrieve relevant studies, using key descriptors for systematic reviews or surveys, abstracts, and reporting or inconsistency. All reference lists from the included surveys and reviews were also manually searched to assess their eligibility. All the searches were limited to the English language. Studies were excluded if: 1) they were not systematic reviews or surveys; 2) they did not have a study objective of comparing abstracts with full reports; 3) no data were reported on inconsistency between abstracts and full reports; 4) they were in duplicate (and then only one copy was retained); 5) they only published abstracts, letters, editorials or commentaries without full-text articles for further detailed information; or 6) they did not focus on primary studies.
All the screening of titles, abstracts and full-text articles was conducted by two reviewers (IN and YJ) in duplicate and independently. We used the Kappa statistic to quantify the level of agreement between the two reviewers . Discrepancies between the two reviewers were resolved by consensus, or, failing that, a third reviewer (GL) made a final decision.
The primary outcome was the level of inconsistency between abstracts and full reports, which was expressed as a percentage (lower percentage indicating better reporting) or categorized rating (such as major/minor difference, high/medium/low inconsistency), as reported by the authors. We also extracted details on inconsistency for the study-validity-related factors including research question or objective, population or sample size, intervention or exposure, comparator, outcome, study duration, study design, statistical analysis, result presentation, result interpretation, and conclusion or recommendation . Secondary outcomes were the factors associated with inconsistent abstract reporting.
Data collection was conducted independently by two reviewers (LA and IN) using a pilot-tested data extraction form. Data extracted were: 1) general characteristics of included systematic reviews or surveys (authors, journal, publication year, study design, field of study, data sources for abstracts and full reports, study search frame, numbers of included abstracts and full reports, study country of primary studies, study sample size in primary studies, and funding information for the systematic reviews or surveys); 2) definitions of inconsistency between abstracts and full reports and the main findings in the included studies; 3) information on inconsistency for the study-validity-related factors; 4) factors related to improved reporting between abstracts and full reports; and 5) authors’ conclusions or recommendations in the included studies. We also collected the terminologies that were used to describe the abstract reporting problem, and their frequencies.
Quality assessment of included systematic reviews
We assessed the study quality of included systematic reviews based on the AMSTAR (a measurement tool to assess systematic reviews) criteria . Some items, such as item 9 (“Were the methods used to combine the findings of studies appropriate?”) and item 10 (“Was the likelihood of publication bias assessed?”) were not applicable to the included systematic reviews, and these items were therefore excluded from our overall quality score evaluation. We calculated the scores by summing the number of items on AMSTAR that the included systematic reviews met. No study quality assessment was performed for the surveys because there were no validated evaluation tools available.
We used word clouds to show the frequencies of the terminologies employed to describe the problems identified in abstract reporting. The online program Wordle (www.wordle.net) was used to draw the word clouds, based on input of the terminologies and the numbers of included studies that used them to describe inconsistent abstract reporting. The relative size of the terminologies in the word clouds corresponded to the frequency of their use. We used medians and interquartile ranges to describe the level of inconsistency across studies. Evidence from the included systematic reviews or surveys was summarized qualitatively, but not quantitatively.
There were 9123 records retrieved from the electronic databases. After removing duplicates and having screened the titles and abstracts, a total of 84 studies remained for full-text article assessment (kappa = 0.85, 95% confidence interval [CI]: 0.79 - 0.91 for record screening). Of these, 16 studies met the eligibility criteria [2,3,4, 7, 10,11,12, 18,19,20,21,22,23,24,25,26] and one additional study  was identified from the reference lists, yielding17 studies that were included for the data collection and analyses (kappa = 0.65, 95% CI: 0.57 - 0.75 for data extraction). Figure 1 presents the flow diagram of the study selection process.
Table 1 summarizes the characteristics of the included studies, with three systematic reviews [3, 12, 19] and fourteen surveys [2, 4, 7, 10, 11, 18, 20,21,22,23,24,25,26,27]. Eleven studies compared conference abstracts with their subsequent full reports [2, 7, 19,20,21,22,23,24,25,26,27], while the others investigated inconsistency between the abstract section and the main text in the same publication [3, 4, 10,11,12, 18]. Three studies reported that their primary studies were mostly from North America or Europe [2, 19, 24]. The median sample size in the primary studies ranged from 5 to 452. Three studies reported that they received academic funding to support their surveys [4, 11, 21]. Study quality was evaluated for the three systematic reviews, in which their scores on AMSTAR were 8 (out of 9) , 7  and 6 , respectively. None of these three reviews provided information on conflict of interest for either the systematic reviews or each of the included primary studies [3, 12, 19]. Two studies did not consider the grey literature in their search strategies [3, 12]. Because there was no information available (such as protocol, ethics approval, registration) on the a priori research question and inclusion criteria before the conduct of the review, one study scored zero on the AMSTAR item 1 (“Was an 'a priori' design provided?”) .
The most frequently used terminology to describe the abstract reporting problem were “inconsistency” (n = 14, out of 17 the included studies, 82%), “deficiency” (n = 11, 65%), “accuracy” (n = 10, 59%), and “discrepancy” (n = 8, 47%). Other terminology included “omission”, “misreporting”, “discordance”, “poor”, “biased”, “inadequate”, “incomplete”, and “selective reporting”, each of which appeared in at most4 studies. Figure 2 shows the word cloud of the terminologies used in the included studies.
Table 2 shows definitions, main findings and authors’ conclusions of inconsistency between abstracts and full reports in the included studies. The level of inconsistency ranged from 4% to 78%, with a median of 39% (interquartile range: 14% - 54%). In the studies that differentiated major from minor inconsistencies [2, 19, 20, 27], the level of major inconsistency ranged from 5% to 45% (median: 19%, interquartile range: 7% - 31%), which originated from the specification of the study design (5%) or sample size (37%), designation of a primary outcome measure (from 14% to 28%), presentation of main results (19%), or drawing a conclusion (6%). All the included studies concluded that abstracts were frequently inconsistently reported, and that efforts were needed to improve abstract reporting in primary biomedical research (Table 2).
Table 3 shows the details on inconsistency for the study-validity-related factors between abstracts and full reports. Except for the research question or objective, intervention or exposure, study duration or design, and statistical analysis, inconsistencies were frequently reported in other factors of this type, with percentages of >10% in most cases. For instance, in the nine studies that assessed a total of 896 abstract-full-report pairs, conclusions in abstracts were found to be inconsistent with the full reports (ranging from 15% to 35%), or made stronger statements than in the full reports (17%). As presented in Table 4, three studies investigated factors related with inconsistent reporting between conference abstracts and full reports [2, 20, 21]. A longer time interval before publication of the full reports was found to be the only factor that was marginally or significantly related to an increased likelihood of reporting inconsistencies.
In this scoping review assessing inconsistency between abstracts and full reports, we summarized the evidence from systematic reviews and surveys to show the literature mapping for the inconsistent abstract reporting in primary biomedical research. Abstract reports were frequently different from their corresponding full reports, with a high level of inconsistency. The length of time between the appearance of conference abstracts and the publication of full reports was the only factor reported to be associated with inconsistent reporting.
Readers usually rely on an initial assessment of an abstract in deciding whether to access the full report, draw conclusions about the study, or even make their decisions, especially when a full report is not available [4, 28]. For instance, Bhandari et al. found that over 50% of the chapters in the latest editions of some most influential orthopedic textbooks referenced at least one conference abstract, and these abstracts would be frequently cited in lectures and rounds . Therefore, given their potential impact, all the summary information in abstracts should, at a minimum, be accurate and consistent with their full reports. However, abstracts are frequently prepared with the least care [1, 3].Our current review found a high level of inconsistency between abstracts and full reports, especially with respect to sample sizes, outcome measures, result presentation and interpretation, and conclusions or recommendations (Table 3). Unlike the included individual studies that evaluated a specific research area, or a group of journals or diseases, our review summarized all the available evidence from systematic reviews and surveys in various areas of the biomedical literature, and we consistently found severe problems in abstract reporting in the primary biomedical community. More efforts are warranted to reverse and prevent the inconsistency of abstract reporting.
There were two studies that also assessed the spin in the abstract reporting, in which the spin existed in the studies with overall non-significant results but with an overly-optimistic abstract that tried to claim significant results or strong recommendations [3, 4]. The spin may not always be relevant to our objective of identifying inconsistency between abstracts and full reports, because the spin could be the same in content and magnitude in both the abstract and the full report. However, when there is an attempt to incorrectly convince the audiences of a favorable finding or conclusion, the existence of spin in abstracts may pose a threat to distorting study findings and misleading readers of the biomedical literature, especially when readers do not go on to refer to the full reports for the study results in detail.
Many journals have adopted a policy of requiring structured abstracts, because they have been shown to be more informative, have greater readability, and be of better quality [29, 30]. However, one study has argued that the problem of inconsistent abstract reporting will not be mitigated by using structured abstracts . In contrast, if structured abstracts inappropriately emphasize their main points, the inaccurate information that they convey could have a stronger impact on the biomedical community. For instance, some structured abstracts used spin to over-emphasize favorable effects in subgroups of patients, for secondary outcomes or in deliberately modified populations, or they made over-optimistically strong conclusions and recommendations, which would further mislead the audience [3, 4]. Furthermore, word count limitations in structured abstracts can sometimes cause key information to be omitted . The effect of guideline checklists on improved consistent abstract reporting remains unknown, with sparse evidence available in the literature. The CONSORT (Consolidated Standards of Reporting Trials) guideline for abstracts, that was published to aid in improving structured abstract reporting for RCTs , might not prevent subtle inconsistencies, especially if editorial staff do not refer to full reports for painstaking scrutiny . Similarly, one trial provided instructions on ensuring data accuracy in abstracts to authors, but found that this was ineffective in actually improving abstract reporting .
Some included studies explored the interpretations of the inconsistencies in conference abstract reporting. For example, given that some conference abstracts were presented when studies were ongoing or at an early stage, sample sizes in full reports would probably be updated from the preliminary results described in abstracts [20, 21]. However, one study argued that the inconsistencies may be deliberate, because authors avoided providing details or explanations of the inconsistencies (such as differently handling patients who were lost to follow-up or withdrawal, or adding more exclusion criteria) to achieve more favorable results in full reports . Furthermore, in order to show significant findings in the full reports, authors may selectively report favorable findings, or deliberately change the way of defining primary outcomes, presenting and interpreting results or drawing conclusions [2, 19, 21, 23]. Three studies reported that having a longer time before the publication of full reports was associated with increased risk of inconsistent abstract reporting (Table 4) [2, 20, 21]. This might be partly explained if delayed publications had experienced several rejections from journals, and if authors then consciously or subconsciously modified their full reports to cater to the subsequent peer-review processes . Some delays in publishing were due to having an extended study duration. In long duration studies, large amounts of data may be collected, which may then yield different statistical analysis results from the preliminary results as presented in the conference abstracts [2, 33].
To reduce or prevent inconsistency between abstracts and full reports, we recommend that the authors, reviewers and editorial staff should carefully scrutinize the consistency and accuracy of abstract reporting during the submission and peer-review processes . Copyediting and proofreading should be performed strictly to avoid any confusion or inconsistency in abstracts after submissions are accepted. Journals may also consider more flexible word counts in structured abstracts to allow more details to be presented. Moreover, guidance and/or checklists are needed to facilitate authors, reviewers and editorial staff with their prompt assessment of inconsistency between abstracts and full reports. For conference abstracts, one might argue that conferences or meetings should require a publication-ready manuscript for their abstract submission . However, this expectation is probably unrealistic and its impact remains largely unknown. In contrast, we recommend that editorial staff and reviewers should refer to the previously presented conference abstracts during the peer-review process, and authors should provide explanations of any inconsistencies between those abstracts and the full reports.
Our scoping review has some limitations. We limited the search to articles in English, which would omit studies in other languages. As most included studies focused on RCTs and/or conference abstracts, the findings from non-randomized studies, basic science and/or comparisons between abstract sections and main texts in the same publications, remained largely unknown. Also, we could not assess the quality of included surveys because no such validated guidance was available. Similarly, lack of information on the factors associated with inconsistent reporting between the abstract section and main text in the same publications restricted our investigations and recommendations in this area.
In this scoping review of the state of abstract reporting in primary biomedical research, we found that abstracts were frequently inconsistent with full reports, based on evidence from systematic reviews and surveys in the literature. Efforts are needed to improve the consistency of abstract reporting in the primary biomedical community.
a measurement tool to assess systematic reviews
Consolidated Standards of Reporting Trials
Enhancing Quality and Transparency in Health Research
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
randomized controlled trial
Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, Michie S, Moher D, Wager E. Reducing waste from incomplete or unusable reports of biomedical research. Lancet (London, England). 2014;383(9913):267–76.
Bhandari M, Devereaux PJ, Guyatt GH, Cook DJ, Swiontkowski MF, Sprague S, Schemitsch EH. An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am. 2002;84-A(4):615–21.
Ochodo EA, de Haan MC, Reitsma JB, Hooft L, Bossuyt PM, Leeflang MM. Overinterpretation and misreporting of diagnostic accuracy studies: evidence of "spin". Radiology. 2013;267(2):581–8.
Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303(20):2058–64.
Simera I, Altman DG. Writing a research article that is “fit for purpose”: EQUATOR network and reporting guidelines. Evidence Based. Medicine. 2009;14(5):132–4.
Fleming PS, Buckley N, Seehra J, Polychronopoulou A, Pandis N. Reporting quality of abstracts of randomized controlled trials published in leading orthodontic journals from 2006 to 2011. American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics. 2012;142(4):451–8.
Hopewell S, Clarke M, Askie L. Reporting of trials presented in conference abstracts needs to be improved. J Clin Epidemiol. 2006;59(7):681–4.
Hopewell S, Boutron I, Altman DG, Ravaud P. Deficiencies in the publication and reporting of the results of systematic reviews presented at scientific medical conferences. J Clin Epidemiol. 2015;68(12):1488–95.
Seehra J, Fleming PS, Polychronopoulou A, Pandis N. Reporting completeness of abstracts of systematic reviews published in leading dental specialty journals. Eur J Oral Sci. 2013;121(2):57–62.
Pitkin RM, Branagan MA, Burmeister LF. Accuracy of data in abstracts of published research articles. JAMA. 1999;281(12):1110–1.
Ward LG, Kendrach MG, Price SO. Accuracy of abstracts for original research articles in pharmacy journals. Ann Pharmacother. 2004;38(7-8):1173–7.
Lehmen JA, Deering RM, Simpson AK, Carrier CS, Bono CM. Inconsistencies between abstracts and manuscripts in published studies about lumbar spine surgery. Spine. 2014;39(10):841–5.
Li G, Mbuagbaw L, Samaan Z, Jin Y, Nwosu I, Levine M, Adachi JD, Thabane L. The state of reporting of primary biomedical research: a scoping review protocol. BMJ Open 2017;7(3):e014749.
Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. International journal of evidence-based healthcare. 2015;13(3):141–6.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ (Clinical research ed). 2009;339:b2535.
Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37(5):360–3.
Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.
Harris AH, Standard S, Brunning JL, Casey SL, Goldberg JH, Oliver L, Ito K, Marshall JM. The accuracy of abstracts in psychology journals. The Journal of psychology. 2002;136(2):141–8.
Kottachchi D, Nguyen GC. Quality and publication success of abstracts of randomized clinical trials in inflammatory bowel disease presented at digestive disease week. Inflamm Bowel Dis. 2010;16(6):993–8.
Rosmarakis ES, Soteriades ES, Vergidis PI, Kasiakou SK, Falagas ME. From conference abstract to full paper: differences between data presented in conferences and journals. FASEB journal : official publication of the Federation of American Societies for Experimental Biology. 2005;19(7):673–80.
Snedeker KG, Campbell M, Totton SC, Guthrie A, Sargeant JM. Comparison of outcomes and other variables between conference abstracts and subsequent peer-reviewed papers involving pre-harvest or abattoir-level interventions against foodborne pathogens. Preventive veterinary medicine. 2010;97(2):67–76.
Davies MW, Dunster KR, East CE, Lingwood BE. Fate of abstracts published in the proceedings of the first annual Perinatal Society of Australia and new Zealand Congress in 1997. J Paediatr Child Health. 2002;38(5):501–6.
Klassen TP, Wiebe N, Russell K, Stevens K, Hartling L, Craig WR, Moher D. Abstracts of randomized controlled trials presented at the society for pediatric research meeting: an example of publication bias. Archives of pediatrics & adolescent medicine. 2002;156(5):474–9.
Preston CF, Bhandari M, Fulkerson E, Ginat D, Egol KA, Koval KJ. The consistency between scientific papers presented at the Orthopaedic trauma association and their subsequent full-text publication. J Orthop Trauma. 2006;20(2):129–33.
Toma M, McAlister FA, Bialy L, Adams D, Vandermeer B, Armstrong PW. Transition from meeting abstract to full-length journal article for randomized controlled trials. JAMA. 2006;295(11):1281–7.
Turpen RM, Fesperman SF, Smith WA, Vieweg J, Dahm P. Reporting quality and information consistency of randomized, controlled trials presented as abstracts at the American urological association annual meetings. J Urol. 2010;184(1):249–53.
Dyson DH, Sparling SC. Delay in final publication following abstract presentation: American College of Veterinary Anesthesiologists annual meeting. Journal of veterinary medical education. 2006;33(1):145–8.
Hopewell S, Clarke M, Moher D, Wager E, Middleton P, Altman DG, Schulz KF, Group C. CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS Med. 2008;5(1):e20.
Taddio A, Pain T, Fassos FF, Boon H, Ilersich AL, Einarson TR. Quality of nonstructured and structured abstracts of original research articles in the British medical journal, the Canadian Medical Association journal and the journal of the American Medical Association. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 1994;150(10):1611–5.
Sharma S, Harrison JE. Structured abstracts: do they improve the quality of information in abstracts? American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics. 2006;130(4):523–30.
Hopewell S, Clarke M, Moher D, Wager E, Middleton P, Altman DG, Schulz KF. CONSORT for reporting randomised trials in journal and conference abstracts. Lancet (London, England). 2008;371(9609):281–3.
Pitkin RM, Branagan MA. Can the accuracy of abstracts be improved by providing specific instructions? A randomized controlled trial. JAMA. 1998;280(3):267–9.
Li G, Taljaard M, Van den Heuvel ER, Levine MA, Cook DJ, Wells GA, Devereaux PJ, Thabane L. An introduction to multiplicity issues in clinical trials: the what, why, when and how. Int J Epidemiol. 2017;46(2):746–55.
Hopewell S, Eisinga A, Clarke M. Better reporting of randomized trials in biomedical journal and conference abstracts. J Inf Sci. 2008;34(2):162–73.
We acknowledge Dr. Stephen Walter for his professional editing of the manuscript.
This study received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Availability of data and materials
The data appeared in this study are already publicly available in the literature.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Li, G., Abbade, L.P.F., Nwosu, I. et al. A scoping review of comparisons between abstracts and full reports in primary biomedical research. BMC Med Res Methodol 17, 181 (2017). https://doi.org/10.1186/s12874-017-0459-5