- Research
- Open access
- Published:
Reporting quality of abstracts and inconsistencies with full text articles in pediatric orthopedic publications
Research Integrity and Peer Review volume 8, Article number: 11 (2023)
Abstract
Background
Abstracts should provide a brief yet comprehensive reporting of all components of a manuscript. Inaccurate reporting may mislead readers and impact citation practices. It was our goal to investigate the reporting quality of abstracts of interventional observational studies in three major pediatric orthopedic journals and to analyze any reporting inconsistencies between those abstracts and their corresponding full-text articles.
Methods
We selected a sample of 55 abstracts and their full-text articles published between 2018 and 2022. Included articles were primary therapeutic research investigating the results of treatments or interventions. Abstracts were scrutinized for reporting quality and inconsistencies with their full-text versions with a 22-itemized checklist. The reporting quality of titles was assessed by a 3-items categorical scale.
Results
In 48 (87%) of articles there were abstract reporting inaccuracies related to patient demographics. The study's follow-up and complications were not reported in 21 (38%) of abstracts each. Most common inconsistencies between the abstracts and full-text articles were related to reporting of inclusion or exclusion criteria in 39 (71%) and study correlations in 27 (49%) of articles. Reporting quality of the titles was insufficient in 33 (60%) of articles.
Conclusions
In our study we found low reporting quality of abstracts and noticeable inconsistencies with full-text articles, especially regarding inclusion or exclusion criteria and study correlations. While the current sample is likely not representative of overall pediatric orthopedic literature, we recommend that authors, reviewers, and editors ensure abstracts are reported accurately, ideally following the appropriate reporting guidelines, and that they double check that there are no inconsistencies between abstracts and full text articles. To capture essential study information, journals should also consider increasing abstract word limits.
Background
Academic writing is an invaluable tool for communicating research findings to the scientific community and the wider public. This is critical to the advancement of clinical practice and refinement of consensus guidelines for treatment of various diseases across medical disciplines. The title and abstract section of articles are intended to convey a brief yet comprehensive and systematic reporting of studies objectives, material and methods, results, and conclusions. Ideally, the title and abstract should provide a standalone summary aimed at informing authors about the appropriateness of the article content to their ongoing or upcoming research work, and informing clinicians about the recent trends in management of challenging disorders [1, 2].
Nevertheless, various systematic reviews have found inconsistencies between abstracts and published articles in orthopedic physical therapy [3], spine surgery [4], other surgical [5], and medical [6] disciplines. Additionally, studies found suboptimal reporting quality of abstracts of randomized control trials in relation to reporting guidelines recommendations [7]. Conference abstracts, i.e. preliminary unpublished research, were found to be more prone to such inconsistencies [6, 8]. Without adequate reporting, scientifically sound and innovative research could pass unnoticed, while wrongly reported research could mislead readers and patients [6, 9]. For example, misinterpretation of statistical significance in abstracts can result in overstated or unjustified conclusions. Additionally, studies have demonstrated inaccurate citation practices across various medical and surgical specialties, including different orthopaedic surgery subspecialties [10,11,12,13,14].
Inaccurate citation practices—can misrepresent or even contradict the conclusions or interpretations made in the article that is being cited [10, 15]. Inaccurate citation practices can also occur when authors cite abstracts, without reading the corresponding full-text articles [10, 15, 16], especially when there are inconsistencies between the two. This is particularly important for busy clinicians and for non-open access articles. In such situations the abstract becomes the most practical and occasionally the only available source of information [17].
Therapeutic/interventional studies that investigate surgical or nonsurgical treatment as pre/post observational case series are common occurrence in orthopedic surgery literature in general and pediatric orthopedic in specific [18]. The primary objective of this study was to investigate the reporting quality of abstracts of observational studies in three major pediatric orthopedic journals against an itemized checklist. The secondary objective was to analyze any reporting inconsistencies between the abstracts and their corresponding full-text articles.
Material and methods
We included original articles from the top three pediatric orthopedic society journals: 1) Journal of Pediatric Orthopedics, the official journal of the Pediatric Orthopaedic Society of North America; 2) Journal of Pediatric Orthopaedics part B, the official journal of International Federation of Paediatric Orthopaedic Societies; and 3) Journal of Children's Orthopaedics, the official journal of the European Paediatric Orthopaedic Society. To be included articles needed to be primary therapeutic research in pediatric orthopedics investigating the results of a treatment or an intervention, prospective or retrospective observational study. We excluded prognostic (natural history), diagnostic and economic studies, systematic reviews and meta-analysis, randomized control trials, case reports, editorials and other non-original research, and those whose full-text was irretrievable. The second author T.A.E searched the included journals’ websites for articles. We selected a convenience sample by including first eligible article per journal issue.
Screening and data analysis
To achieve the primary objective, we determined the abstract reporting quality using an itemized checklist (Additional file 1). The checklist was developed by us the authors and was based on (STROBE) guidelines for reporting of observational studies https://www.strobe-statement.org/. All items of the checklist had a dichotomous manner, i.e. item was present or absent. To achieve the secondary objective, we compared the information presented in each abstract with information presented in its corresponding full-text article. Thus, we retrieved all full-text articles and subjected them to free and in depth reading with the aim of; 1) identifying missing or inaccurately reported information in the abstract and 2) recording inconsistency/disagreement between information presented in the abstract and that presented in the full-text. The first author S.A.K performed analysis and revision of half the study sample, while the second author T.A.E performed the above screening, analysis, and revision of the whole sample, i.e. final 55 included articles. Disagreements between authors were resolved by joint meetings. We used descriptive statistics based on the total number of included studies.
Checklist items and definitions
Checklist consisted of 22 items, of which two asked about the title, 13 about quality of the abstracts, and 7 about inconsistencies between the abstracts and full texts. Definitions and items they refer to are presented below. We considered information when it was either explicitly or contextually implied.
Title
-
a)
Insufficient title (Question 1 of the checklist): we considered a title insufficient if one of the following items was missing; population, pathology, intervention, or follow-up period mentioned descriptively or numerically. This was irrespective of whether the title was in an indicative/informative or interrogative (question) format [1, 2].
-
b)
Misleading title (Question 2 of the checklist): we considered a title misleading when two or more of the above-mentioned items were missing or had discordant information [1, 2]. For example, the title indicates or implies long-term follow-up while in fact it is not, or title indicates cerebral palsy children while in fact study population comprises other neuromuscular disorders.
Abstract
-
a)
Reporting quality of abstract refers to the fulfilment of a certain reporting item scenario of the abstract per se and irrespective of the information present in its corresponding full-text article [1, 6, 7, 19, 20].
-
b)
Missing information: we identified missing as either essential item/information reported in the full-text article and not in the abstract or vice versa [3,4,5,6].
-
c)
Discrepant information: refers to discrepancies or disagreements between information reported in abstracts and that reported in its corresponding full-text article. Examples are; exaggerated or excessively generalized conclusions which are not substantiated by the results presented in the full-text article or discrepancies between sample size, gender distribution, P values in the abstract and in the full-text [3,4,5,6].
-
d)
Essential item of abstract: we identified essential information/items of the abstract as obligate reporting information the absence of which would compromise the reporting quality of the abstract, irrespective of the information being reported in the full-text article or not. Examples are; reporting at least one study objective, reporting study population characteristics, type of intervention/treatment, and so forth [1, 6, 7, 19, 20].
-
e)
Non-essential item of abstract: we identified non-essential information/items of the abstract as an occasionally obligate reporting information depending on the study context and settings, and the absence of which would not necessary compromise the reporting quality of the abstract. Vetting the full-text article can determine the applicability of reporting non-essential information to a particular abstract. Examples are; presence of a second or third study objective that was not reported in the abstract, yet reported in the full-text article or presence of a second or third outcome measure that was not reported in the abstract, yet reported in the full-text article, and so forth [1, 4, 7].
Definitions and virtual examples of what constitutes accurately reported items in abstracts are shown in (Table 1), while study’s raw data with explanatory comment is publicly made available (https://doi.org/10.5281/zenodo.7616147).
Results
We included a total of 55 articles (and their abstracts) published between 2018 and 2022. Twenty articles (36%) belonged to Journal of Pediatric Orthopedics, 18 (33%) to Journal of Pediatric Orthopaedics part B, and 17 (31%) to Journal of Children's Orthopaedics. Thirty seven abstracts (67%) were structured and 18 (33%) were non-structured. The included articles covered the following pediatric orthopedic themes; trauma, spine deformities, cerebral palsy, congenital/developmental, knee and foot deformities, adolescent hip disorders, osteoarticular infections, and sports injuries. The list of included articles is shown in (Additional file 1).
Quality or accuracy of reported items in titles and abstracts of included articles is shown in (Table 2) and inconsistencies between abstracts and full-text articles in (Table 3). In 48 (87%) of abstracts either age or gender or both were not reported, 21 (38%) of abstracts did not report the study’s follow-up period, and 21 (38%) did not report the study’s complications. Reporting inaccuracies were noted regarding study objectives and inclusion or exclusion criteria. However, there were inconsistencies between the abstract and full-text with respect to additional inclusion or exclusion criteria and study correlations that were reported in the full-text yet not in the abstract, in 39 (71%) and 27 (49%) of articles respectively. Study’s raw data with explanatory comments are provided (https://doi.org/10.5281/zenodo.7616147), (Fig. 1).
Discussion
In our study, which analyzed abstract reporting inaccuracies and inconsistencies with full-texts in 55 articles from three high-ranking journals in pediatric orthopedic subspecialties, we found abstract reporting inaccuracies and inconsistencies with full-texts. Specifically most common were abstract reporting inaccuracies related to patient demographics, study's follow-up, and complications. Whereas the most common inconsistencies between the abstracts and full-text articles were related to reporting of inclusion or exclusion criteria and study correlations. Our results are in alignment with results reported across diverse medical disciplines [4, 7, 21, 22].
Readers have been cautioned against depending on abstracts as sole source of information, and citing studies based solely on abstracts [10, 15, 23]. Although this practice violates the perquisites of research accuracy and integrity, it is difficult to estimate its true prevalence. It may even be more difficult to prevent such practices. Efforts to fix the former problem should go in-parallel with efforts to improve early career authors’ academic writing skills [9] and journal editorial and peer review scrutiny [24, 25]. Relevantly, receiving peer reviewer training or mentoring and adherence to journals’ review report checklist can aid early career authors in improving the reporting quality of submitted manuscripts [26]. Artificial intelligence may help editors and peer reviewers in detecting basic yet important reporting errors of submitted manuscripts or abstracts [27]. Nevertheless, vetting a manuscript’s scientific quality, completeness of the methodology, and validity of conclusions is more appropriately tasked with the peer reviewers themselves.
The title is an important yet under-researched element of any scholarly article. Rather than reading the abstract, busy readers may rely totally on screening article titles to identify a specific topic. There is no consensus on what constitutes an ideal title of a scholarly article. Broadly, an informative title should be chosen so as to reflect all main components of a study. However, the specific discipline, speciality, research type, title structure and format as indicative versus interrogative, and article dissemination/impact are factors that must be taken into account when formulating an optimal title [28]. For consensus and research purposes we adopted a specific definition of what constitutes a sufficient or standalone title suited for interventional pre/post studies. In that regard, we included follow-up period as one of the essential reporting item of the title because our included articles dealt with assessment of effects of therapy/treatment on outcomes. The current study did not show any misleading titles. Nevertheless, 60% of the titles were graded insufficient as per study definitions. This was overwhelmingly due failure to report the follow-up period neither descriptively nor numerically in the title.
This study provided a reasonable sample of pediatric orthopedic literature published in highly-sought journals in this subspecialty. In addition to assessing abstract reporting quality per se, we correlated between abstract and full-text reporting. This allowed us to uncover further anomalies of abstract reporting. The majority of articles had no author overlap (one or more authors in common). For articles that had author overlap, it was mostly not possible to verify the author's role in manuscript writing due to absence of author contribution statement in most of the included articles. For articles that provided such statement, one author played a fundamental role in writing of two articles/manuscripts. Therefore, we assume that author overlap did not influence our results fundamentally.
Study limitations
Our study had several limitations. We did not cover a representative sample of all orthopedic journals. We also did not perform a formal quantitative or qualitative statistical sub-analysis of the unreported or inaccurately reported items in the abstract or full-text which was beyond the study objectives. However, we recorded explanatory comments in the study’s raw data file for a deeper look. For instance, some study correlations that were not reported in the abstract yet were in the full—text proved to be highly supportive of the final conclusions in the abstract, while other instances of unreported/missing correlations in the abstract proved to be of secondary importance to the final conclusions. We did not correlate the abstract reporting quality with article or journal metrics, or characteristics of authors. The recording of some checklist items, as study implications, possessed a degree of subjectivity and may be linked to an author’s or assessor’s academic and clinical experience. This may have introduced recording bias. We did not implement a typical dual review or double screening in the search and screening stage of abstracts/titles. The fact that we selected a convenience sample and not an exhaustive one, mitigates this limitation.
Conclusions
In our study we found low reporting quality of abstracts and noticeable inconsistencies with full-text articles, especially regarding patient demographics, inclusion or exclusion criteria, and study correlations. Our study has shown that more work remains to improve the reporting of abstracts in pediatric orthopaedic literature. While the current sample is likely not representative of overall pediatric orthopedic literature, authors, reviewers, and editors should use relevant reporting guidelines, strive to report at least the items mentioned there, and double check that there are no inconsistencies between abstracts and full text articles. To capture essential study information, journals should also consider easing restrictions on abstract word counts.
Availability of data and materials
The raw data is available at Zenodo repository at: https://doi.org/10.5281/zenodo.7616147.
References
Cohen JF, Deeks JJ, Hooft L, et al. Preferred reporting items for journal and conference abstracts of systematic reviews and meta-analyses of diagnostic test accuracy studies (PRISMA-DTA for Abstracts): checklist, explanation, and elaboration. BMJ. 2021;372: n265.
Bahadoran Z, Mirmiran P, Kashfi K, et al. The principles of biomedical scientific writing: Title. Int J Endocrinol Metab. 2019;17(4): e98326.
Nascimento DP, Gonzalez GZ, Araujo AC, et al. Eight in Every 10 abstracts of low back pain systematic reviews presented spin and inconsistencies with the full text: An analysis of 66 systematic reviews. J Orthop Sports Phys Ther. 2020;50(1):17–23.
Lehmen JA, Deering RM, Simpson AK, et al. Inconsistencies between abstracts and manuscripts in published studies about lumbar spine surgery. Spine (Phila Pa 1976). 2014;39(10):841–5.
Assem Y, Adie S, Tang J, et al. The over-representation of significant p values in abstracts compared to corresponding full texts: A systematic review of surgical randomized trials. Contemp Clin Trials Commun. 2017;7:194–9.
Li G, Abbade LPF, Nwosu I, et al. A scoping review of comparisons between abstracts and full reports in primary biomedical research. BMC Med Res Methodol. 2017;17(1):181.
Chen Y, Li J, Ai C, et al. Assessment of the quality of reporting in abstracts of randomized controlled trials published in five leading Chinese medical journals. PLoS ONE. 2010;5(8): e11926.
Kleweno CP, Bryant WK, Jacir AM, et al. Discrepancies and rates of publication in orthopaedic sports medicine abstracts. Am J Sports Med. 2008;36(10):1875–9.
El-Sobky TA. An author’s guide to mastering academic writing skills: Discussion of a medical manuscript. J Musculoskelet Surg Res. 2021;5:227–34.
Pavlovic V, Weissgerber T, Stanisavljevic D, et al. How accurate are citations of frequently cited papers in biomedical literature? Clin Sci (Lond). 2021;135(5):671–81.
Bagga R, Cay P, Ricketts D, et al. Quotation errors related to the Proximal Fracture of the Humerus Evaluation by Randomization (ProFHER) study. Should Elb. 2020. https://doi.org/10.1177/1758573220950235.
Luo M, Li CC, Molina D 4th, et al. Accuracy of citation and quotation in foot and ankle surgery journals. Foot Ankle Int. 2013;34(7):949–55.
Davids JR, Weigl DM, Edmonds JP, et al. Reference accuracy in peer-reviewed pediatric orthopaedic literature. J Bone Joint Surg Am. 2010;92(5):1155–61.
Buijze GA, Weening AA, Poolman RW, et al. Predictors of the accuracy of quotation of references in peer-reviewed orthopaedic literature in relation to publications on the scaphoid. J Bone Joint Surg Br. 2012;94(2):276–80.
Rivkin A. Manuscript referencing errors and their impact on shaping current evidence. Am J Pharm Educ. 2020;84(7):ajpe7846.
Greenberg SA. How citation distortions create unfounded authority: analysis of a citation network. BMJ. 2009;339: b2680.
Pontis S, Blandford A, Greifeneder E, et al. Keeping up to date: An academic researcher’s information journey. J Assn Inf Sci Tec. 2017;68:22–35.
Souza KE, Chan JY, Campbell ST, et al. Research methodologic quality varies significantly by subspecialty: An analysis of AAOS meeting abstracts. J Clin Orthop Trauma. 2020;15:37–41 Erratum in: J Clin Orthop Trauma. 2021;21:101561.
Hopewell S, Clarke M, Moher D, et al. CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS Med. 2008;5(1): e20.
JerčićMartinić-Cezar I, Marušić A. Completeness of reporting in abstracts of randomized controlled trials in subscription and open access journals: cross-sectional study. Trials. 2019;20(1):669.
O’Donohoe TJ, Dhillon R, Bridson TL, et al. Reporting quality of systematic review abstracts published in leading neurosurgical journals: A research on research study. Neurosurgery. 2019;85(1):1–10.
Janackovic K, Puljak L. Reporting quality of randomized controlled trial abstracts in the seven highest-ranking anesthesiology journals. Trials. 2018;19(1):591.
The Office of Research Integrity [internet]. Relying on an abstract or a preliminary or version of a paper while citing the published version [cited 2022 June 20]. Available from: https://ori.hhs.gov/relying-abstract-or-preliminary-or-version-paper-while-citing-published-version.
Menon V, Varadharajan N, Praharaj SK, et al. Quality of peer review reports submitted to a specialty psychiatry journal. Asian J Psychiatr. 2021;58: 102599.
Superchi C, Hren D, Blanco D, et al. Development of ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research. BMJ Open. 2020;10(6): e035604.
RodrĂguez-Carrio J, Putrik P, Sepriano A, et al. Improving the peer review skills of young rheumatologists and researchers in rheumatology: the EMEUNET Peer Review Mentoring Program. RMD Open. 2018;4(1): e000619.
Schulz R, Barnett A, Bernard R, et al. Is the future of peer review automated? BMC Res Notes. 2022;15(1):203.
Hyland K, Zou HJ. Titles in research articles. J Engl Acad Purp. 2022;56: 101094.
Acknowledgements
None.
Funding
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Author information
Authors and Affiliations
Contributions
T.A.E conceived and designed the study. T.A.E. and S.A.K. contributed to data acquisition analysis and interpretation. T.A.E. wrote the manuscript draft. T.A.E. and S.A.K. revised the manuscript critically for important intellectual content and approved the version to be published. Both authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
T.A.E. is a reviewer for the Journal of Children's Orthopaedics and Journal of Pediatric Orthopaedics Part B. S.A.K. has nothing to declare.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1.
  Itemized checklist for assessment of reporting quality or accuracy of abstracts and inconsistencies with full-text articles. And List of final 55 pediatric orthopedic articles included in the analysis of this review.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Kamel, S.A., El-Sobky, T.A. Reporting quality of abstracts and inconsistencies with full text articles in pediatric orthopedic publications. Res Integr Peer Rev 8, 11 (2023). https://doi.org/10.1186/s41073-023-00135-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s41073-023-00135-3