Skip to main content

Research Integrity and Peer Review—past highlights and future directions

Abstract

In May 2016, we launched Research Integrity and Peer Review, an international, open access journal with fully open peer review (reviewers are identified on their reports and named reports are published alongside the article) to provide a home for research on research and publication ethics, research reporting, and research on peer review. As the journal enters its third year, we reflect on recent events and highlights for the journal and explore how the journal is faring in terms of gender and diversity in peer review. We also share the particular interests of our Editors-in-Chief regarding models of peer review, reporting quality, common research integrity issues that arise during the publishing process, and how people interact with the published literature. We continue to encourage further research into peer review, research and publication ethics and research reporting, as we believe that all new initiatives should be evidence-based. We also remain open to constructive discussions of the developments in the field that offer new solutions.

Introduction

These are exciting times for Research Integrity and Peer Review. Since launching in May 2016 [1], the journal has published a range of articles on diverse topics spanning broad themes within publication ethics, research reporting and peer review. Our four Editors-in-Chief have been on the road speaking at conferences and events including the World Conference on Research Integrity [2] and the International Congress on Peer Review and Scientific Publishing [3], a UK parliamentary inquiry into research integrity [4] and the Global Evidence Summit [5]. The past year has also been a year of change for the journal, as we have said goodbye (and thank you) to one of our founding editors, Iveta Simera, and hello (and welcome) to Joerg Meerpohl as the Editor-in-Chief of the journal’s research reporting section. Here, we share with you some of the most memorable moments from the past 18 months.

It's all about the research

An analysis of the articles published up to January 2018 (Table 1) reveals that the majority of articles fall within the research and publication ethics section, and perhaps this is to be expected given the section’s broad scope and the funding for research in this area. Published articles have covered issues that can occur during the phases of research and publication. These include how research is funded [6] and reviewed [7, 8], guidelines on research integrity [9], researchers’ perceptions on research misbehaviour [10] and their views on the publishing process [11]. Challenging topics have been tackled also, including how to handle appropriate disclosure of conflicts of interest [12], how to agree authorship contributions [13] and what is appropriate in terms of citation practices [14, 15] and text-recycling [16, 17]. Of course, potential concerns within publication ethics do not cease with publication and we have published articles that investigate the incidence of plagiarism [18], reasons for retractions [19] and the varied uses of expressions of concern [20].

Table 1 Article topics published within the areas of research and publication ethics, research reporting and peer review

Within the reporting section of the journal, initiatives that aim to increase the transparency and reproducibility of research have been published. These include research on standards of reporting [21] including how to correctly identify the most relevant reporting guidelines [22] and how to ensure they are up to date [23]. Of particular importance is the recognition that accurate reporting of sex and gender is key for research across multiple disciplines [24, 25], and the guidelines have been endorsed by EQUATOR [26]. Other initiatives which have been discussed include investigations of the extent of trial registration and the publication of clinical trials [27, 28] and the need for more collaborative efforts to facilitate and incentivise appropriate data-sharing [29, 30]. We can also learn about the factors that affect readers’ perception of research, and those which influence the media attention given to articles post-publication from research in these areas [31, 32].

In the peer review section, several articles published have focused on diverse aspects of the peer review process. These span a deeper dive into how grant review panels work [33] and the need for support for peer reviewers, particular in areas of training [34, 35] and mentoring [36]. Other topics include a study of reviewer recruitment in the field of ecology [37, 38] and research into the views of junior hospital doctors on their understanding of models of peer review [39]. It is great to see the journal is sharing experience of research into peer review from varied subject areas and connecting diverse communities, which was one of our aims in launching a multi-disciplinary journal.

Of course, the topics of the articles ultimately accepted for publication in the journal do not reflect the range of submissions we have seen. While we welcome all research within the areas of publication ethics, research reporting and peer review, we would particularly like to encourage submissions on the use of alternative metrics; research into different models of peer review; studies exploring the role of gender and diversity in peer review and how institutions work on supporting completeness, accuracy and transparency in research reporting and preventing and handling research misconduct.

Gender equality at Research Integrity and Peer Review

Given recent interest into gender and diversity in peer review [40] and inspired by investigations at other journals [41], we wanted to find out how the journal is faring in terms of gender and diversity (with respect to its editorial board, authors and peer reviewers). Of course, the journal is still growing, so the sample size is small, but we hope this will be a useful benchmark to monitor changes in future.

In most cases, it was possible to determine gender of individuals simply by name recognition, but in some cases, we had to check other sources for further information. The gender balance among our Editors-in-Chief is positively skewed in favour of females (3:1), but across the editorial board, the ratio is more even with 18 females (40%) and 27 males (60%). However, perhaps the gender composition of the board may reflect the composition of the research field as a whole? This may be the case given that of 33 articles published so far, 12 articles (36%) have female corresponding authors while 21 articles (63%) have male corresponding authors. In terms of peer reviewers, gender diversity is more balanced with 31 (45%) women and 38 (55%) men.

We were also able to investigate the geographic location of editorial board members, authors and peer reviewers who had agreed to review as one measure of ‘diversity’. The diversity among the Editors-in-Chief is biased in favour of Europe. However, the rest of the editorial board (comprising 45 individuals) span locations across the following continents: Europe (17, 38%), North America (13, 29%), South America (2, 4%), Africa (2, 4%), Asia (6, 13%) and Oceania (5, 11%), although there is a bias towards Europe and North America.

In terms of the location of corresponding authors, of 33 articles published up to January 2018, excluding conference proceedings [42, 43], we see a narrower range of diversity with the majority of authors located in North America (14, 42%), Europe (12, 36%) and Oceania (7, 21%).

In terms of 69 reviewers who supplied reviews for published articles, the diversity is somewhat broader with reviewers located across Europe (38, 55%), North America (25, 36%), Africa (1, 1%), Asia (1, 1%) and Oceania (4, 6%).

We are proud of the gender and diversity balance we are currently achieving across our editorial board and our reviewers, and we will continue to work to ensure appropriate balance in terms of gender and diversity in the future. While it is difficult to directly influence gender and diversity imbalances in submissions to the journal, we welcome submissions from across the world, especially Asia, Africa and South America.

Editors-in-Chief on the road

Our Editors-in-Chief attended key events in the area of research integrity [2, 4], peer review [3] and evidence-based policy [5] during 2017. The 5th World Conference on Research Integrity (WCRI), held in May 2017 in Amsterdam, focussed on transparency and accountability, themes which are very much interlinked at the journal through the open peer review model that the journal has adopted. Liz Wager (who works as an independent freelance consultant and trainer) played an active role in the conference programme. She championed what journals can do to improve transparency in research reporting [44] and also reviewed what countries can do to better support research integrity [45], a timely topic in the UK at the moment given the government inquiry into research integrity at which Liz also gave oral evidence [46]. Liz also led a discussion on how editors can cooperate and liaise with research institutions [47].

The WCRI was followed in September 2017 by the 8th International Congress on Peer Review and Scientific Publication, held in Chicago. The congress aims to encourage research into all aspects of peer review and scientific publication and establish the evidence-base on which researchers can improve the conduct, reporting and dissemination of research. The specific theme for 2017 was ‘enhancing the quality and credibility of science’ which very much corresponds with the scope of Research Integrity and Peer Review although the journal scope spans all research, including the humanities and social sciences. Stephanie Boughton, Maria Kowalczuk and Liz Wager were able to share their own recent research at the conference.

Stephanie presented an insight into the types of queries raised with BMC’s Research Integrity Group, showing that ‘ethics and consent’ and ‘data issues’ were the main themes, and two thirds of queries were raised by editors ahead of publication [48]. Maria compared journals operating on open, single-blind or double-blind peer review models, showing that while more invitations on average were sent to secure reviewers for the open model, the difference with other models was not insurmountable [49]. Liz’s analysis of PubPeer comments on articles in leading medical journals revealed that the frequency of comments requiring journal action was low, and although editors were unaware of the methodological issues raised in that way, they often followed up independently [50].

Also in September 2017, the global evidence-based health care community got together in Cape Town, South Africa, at the first ever Global Evidence Summit (GS), co-hosted by Cochrane, Guidelines International Network, Campbell Collaboration, Joanna Briggs Institute and the International Society for Evidence-Based Health Care (ISEHC). While the scope of the GES was broad focusing on evidence production as well as evidence use, e.g. in guidelines, reporting quality of trials and systematic reviews was discussed in various sessions such as ‘Reporting evidence synthesis’, ‘Improving conduct and reporting of evidence synthesis’, or ‘Tools to communicate and use evidence’. Joerg co-organized a special session on dissemination bias in qualitative research, and how this might impact on qualitative evidence syntheses [51], an area where further research is clearly needed.

Future directions on research integrity and peer review?

Looking back, it is encouraging to see the wealth of research being conducted in areas of research integrity, reporting and peer review, some of which we were able to publish in Research Integrity and Peer Review. This year, we look forward to hearing more about discussions and investigations that are being shared by the ASAPbio community into transparency, recognition and innovation in peer review in the life sciences ([52], the PEERE International Conference on Peer Review [53] and the PRINTEGER (Promoting Integrity as an Integral Dimension of Excellence in Research) conference [54].

Of course, some initiatives will help, others may fail, and that is fine as long as research in these areas is data-driven. In our role as co-Editors-in-Chief of Research Integrity and Peer Review we would love to hear about your research and how it could make a difference. Thank you to our authors and our reviewers, our Editorial Board and our readers for all their continued support. We look forward to working with you in 2018, and beyond.

References

  1. Boughton SL, Kowalczuk MK, Simera I, Wager E. A new forum for research on research integrity and peer review. Res Integ Peer Rev. 2016;1:5.

    Article  Google Scholar 

  2. 5th World Conference on Research Integrity http://www.wcri2017.org/. Accessed 24 Feb 2018.

  3. 8th International Congress on Peer Review and Scientific Publication https://peerreviewcongress.org/index.html Accessed Feb 2018. Accessed 24 Feb 2018.

  4. UK parliamentary inquiry into research integrity http://www.parliament.uk/business/committees/committees-a-z/commons-select/science-and-technology-committee/inquiries/parliament-2017/research-integrity-17-19/ Accessed 24 Feb 2018.

  5. Global Evidence Summit https://www.globalevidencesummit.org/ Accessed 24 Feb 2018.

  6. Barnett AG, Clarke P, Vaquette C, Graves N. Using democracy to award research funding: an observational study. Res Integ Peer Rev. 2017;2:16.

    Article  Google Scholar 

  7. Barnett AG, Campbell MJ, Shield C, Farrington A, Hall L, Page K, Gardner A, Mitchell BG, Graves N. The high costs of getting ethical and site-specific approvals for multi-centre research. Res Integ Peer Rev. 2016;1:16.

    Article  Google Scholar 

  8. Page SA, Nyeboer J. Improving the process of research ethics review. Res Integ Peer Rev. 2017;2:14.

    Article  Google Scholar 

  9. Räsänen L, Moore E. Critical evaluation of the guidelines of the Finnish Advisory Board on Research Integrity and of their application. Res Integ Peer Rev. 2016;1:15.

    Article  Google Scholar 

  10. Bouter LM, Tijdink J, Axelsen N, Martinson BC, Riet G. Ranking major and minor research misbehaviors: results from a survey among participants of four World Conferences on Research Integrity. Res Integ Peer Rev. 2016;1:17.

    Article  Google Scholar 

  11. Wallach JD, Egilman AC, Gopal AD, Swami N, Krumholz HM, Ross JS. Biomedical journal speed and efficiency: a cross-sectional pilot survey of author experiences. Res Integ Peer Rev. 2018;3:1.

    Article  Google Scholar 

  12. Dunn AG, Coiera E, Mandl KD, Bourgeois FT. Conflict of interest disclosure in biomedical research: a review of current practices, biases, and the role of public registries in improving transparency. Res Integ Peer Rev. 2016;1:1.

    Article  Google Scholar 

  13. Boyer S, Ikeda T, Lefort M-C, Malumbres-Olarte J, Schmidt JM. Percentage-based Author Contribution Index: a universal measure of author contribution to scientific articles. Res Integ Peer Rev. 2017;2:18.

    Article  Google Scholar 

  14. van der Vet PE, Nijveen H. Propagation of errors in citation networks: a study involving the entire citation network of a widely cited paper published in, and later retracted from, the journal Nature. Research Integrity and Peer Review. 2016;1:3.

    Article  Google Scholar 

  15. Duyx B, Urlings MJE, Swaen GMH, Bouter LM, Zeegers MP. Selective citation in the literature on swimming in chlorinated water and childhood asthma: a network analysis. Res Integ Peer Rev. 2017;2:17.

    Article  Google Scholar 

  16. Moskovitz C. Text recycling in health sciences research literature: a rhetorical perspective. Res Integ Peer Rev. 2017;2:1.

    Article  Google Scholar 

  17. Roig M. Comment on Cary Moskovitz’ “Text Recycling in Health Sciences Literature: A Rhetorical Perspective”. Res Integ Peer Rev. 2017;2:2.

    Article  Google Scholar 

  18. Higgins JR, Lin F-C, Evans JP. Plagiarism in submitted manuscripts: incidence, characteristics and optimization of screening-case study in a major specialty medical journal. Res Integ Peer Rev. 2016;1:13.

    Article  Google Scholar 

  19. Bozzo A, Bali K, Evaniew N, Ghert M. Retractions in cancer research: a systematic survey. Res Integ Peer Rev. 2017;2:5.

    Article  Google Scholar 

  20. Vaught M, Jordan DC, Bastian H. Concern noted: a descriptive study of editorial expressions of concern in PubMed and PubMed Central. Res Integ Peer Rev. 2017;2:10.

    Article  Google Scholar 

  21. Hamilton S, Bernstein AB, Blakey G, Fagan V, Farrow T, Jordan D, Seiler W, Shannon A, Gertel A. Developing the Clarity and Openness in Reporting: E3-based (CORE) Reference user manual for creation of clinical study reports in the era of clinical trial transparency. Res Integ Peer Rev. 2016;1:4.

    Article  Google Scholar 

  22. Shanahan DR, de Sousa IL, Marshall DM. Simple decision-tree tool to facilitate author identification of reporting guidelines during submission: a before–after study. Res Integ Peer Rev. 2017;2:20.

    Article  Google Scholar 

  23. Korevaar DA, Cohen JF, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, Moher D, de Vet HCW, Altman DG, Hooft L, Patrick M, PMM B. Updating standards for reporting diagnostic accuracy: the development of STARD 2015. Res Integ Peer Rev. 2016;1:–7.

  24. Heidari S, Babor TF, de Castro P, Tort S, Curno M. Sex and gender equity in research: rationale for the SAGER guidelines and recommended use. Res Integ Peer Rev. 2016;1:2.

    Article  Google Scholar 

  25. Welch V, Doull M, Yoganathan M, Jull J, Boscoe M, Coen SE, Marshall Z, Pardo Pardo J, Pederson A, Petkovic J, Puil L, Quinlan L, Shea B, Rader T, Runnels V, Tudiver S. Reporting of sex and gender in randomized controlled trials in Canada: a cross-sectional methods study. Res Integ Peer Rev. 2017;2:15.

    Article  Google Scholar 

  26. Equator network http://www.equator-network.org/. Accessed 24 Feb 2018.

  27. Gray R, Badnapurkar A, Hassanein E, Thomas D, Barguir L, Baker C, Jones M, Bressington D, Brown E, Topping A. Registration of randomized controlled trials in nursing journals. Res Integ Peer Rev. 2017;2:8.

    Article  Google Scholar 

  28. Asiimwe IG, Rumona D. Publication proportions for registered breast cancer trials: before and following the introduction of the ClinicalTrials.gov results database. Res Integ Peer Rev. 2016;1:–10.

  29. Hrynaszkiewicz I, Khodiyar V, Hufton AL, Sansone S-A. Publishing descriptions of non-public clinical datasets: proposed guidance for researchers, repositories, editors and funding organisations. Res Integ Peer Rev. 2016;1:6.

    Article  Google Scholar 

  30. Rowhani-Farid A, Allen M, Barnett AG. What incentives increase data sharing in health and medical research? A systematic review. Res Integ Peer Rev. 2017;2:4.

    Article  Google Scholar 

  31. Wager E, Altman DG, Simera I, Toma TP. Do declarative titles affect readers’ perceptions of research findings? A randomized trial. Res Integ Peer Rev. 2016;1:11.

    Article  Google Scholar 

  32. Haneef R, Ravaud P, Baron G, Ghosn L, Boutron I. Factors associated with online media attention to research: a cohort study of articles evaluating cancer treatments. Res Integ Peer Rev. 2017;2:9.

    Article  Google Scholar 

  33. Coveney J, Herbert DL, Hill K, Mow KE, Graves N, Barnett A. ‘Are you siding with a personality or the grant proposal?’: observations on how peer review panels function. Res Integ Peer Rev. 2017;2:19.

    Article  Google Scholar 

  34. Byrne JA. Improving the peer review of narrative literature reviews. Res Integ Peer Rev. 2016;1:12.

    Article  Google Scholar 

  35. Scarrow G, Angus D, Holmes BJ. Reviewer training to assess knowledge translation in funding applications is long overdue. Res Integ Peer Rev. 2017;2:13.

    Article  Google Scholar 

  36. Wong VSS, Strowd RE III, Aragón-García R, Moon YP, Ford B, Haut SR, Kass JS, London ZN, Mays MA, Milligan TA, Price RS, Reynolds PS, Selwa LM, Spencer DC, Elkind MSV. Mentored peer review of standardized manuscripts as a teaching tool for residents: a pilot randomized controlled multi-center study. Res Integ Peer Rev. 2017;2:6.

    Article  Google Scholar 

  37. Fox CW, Albert AYK, Vines TH. Recruitment of reviewers is becoming harder at some journals: a test of the influence of reviewer fatigue at six journals in ecology and evolution. Res Integ Peer Rev. 2017;2:3.

    Article  Google Scholar 

  38. Albert AYK, Gow JL, Cobra A, Vines TH. Is it becoming harder to secure reviewers for peer review? A test with data from five ecology journals. Res Integ Peer Rev. 2016;1:14.

    Article  Google Scholar 

  39. Patel J, Pierce M, Boughton SL, Baldeweg SE. Do peer review models affect clinicians’ trust in journals? A survey of junior doctors. Res Integ Peer Rev. 2017;2:11.

    Article  Google Scholar 

  40. Ross E. Gender bias distorts peer review across fields. Nature. 2018; https://doi.org/10.1038/nature.2017.21685.

  41. Our first year by numbers. Nat Ecol Evol. 2018;2:1–2. https://doi.org/10.1038/s41559-017-0440-z.

  42. O’Brien PS, Chan D, Leung F, Ko EJ, Kwak JS, et al. Proceedings of the 4th World Conference. Res Integ Peer Rev. 2016;(Suppl 1):9.

  43. Vasconcelos S, Watanabe E, Garcia LP, Duarte E, Cassimiro MC, et al. Proceedings from the IV Brazilian Meeting on Research Integrity, Science and Publication Ethics (IV BRISPE). Res Integ Peer Rev. 2017;2(Suppl 1):12.

    Article  Google Scholar 

  44. Wager E. WCRI presentation: What can journals do to improve research reporting http://www.wcri2017.org/images/documents/1.%20Monday%20May%2029,%202017/2.%20KC-07/E.%20Wager%20-%20What%20can%20journals%20do%20to%20improve%20research%20reporting.pdf Accessed Feb 2018.

  45. Wager E.WCRI presentation: Research integrity report card for the UK http://www.wcri2017.org/images/documents/3.%20Wednesday%20May%2031,%202017/1.%20Aula/E.%20Wager%20-%20Research%20integrity%20report%20card%20for%20the%20UK.pdf Accessed 24 Feb 2018.

  46. Wager E. Commons select committee, research integrity examined with academics. http://www.parliament.uk/business/committees/committees-a-z/commons-select/science-and-technology-committee/news-parliament-2017/research-integrity-evidence-17-19/ Accessed 24 Feb 2018.

  47. Wager E. WCRI presentation, Cooperation & Liaison between Universities & Editors http://www.wcri2017.org/images/documents/3.%20Wednesday%20May%2031,%202017/3.%20Auditorium/E.%20Wager%20-%20Coorperation%20&%20Liason%20between%20Universities%20&%20Editors%20%28CLUE%29.pdf Accessed 24 Feb 2018.

  48. Morawska M, Boughton SL. Types of research integrity issues encountered by a specialist research integrity group. https://peerreviewcongress.org/prc17-0228 Accessed 24 Feb 2018.

  49. Kowalczuk M, Michelle Samarasinghe M. Comparison of acceptance of peer reviewer invitations by peer review model: open, single-blind, and double-blind peer review. https://peerreviewcongress.org/prc17-0227 Accessed Feb 2018. Accessed 24 Feb 2018.

  50. Wager E, Veitch E. The role of PubPeer comments in alerting editors to serious problems with clinical research publications. https://peerreviewcongress.org/prc17-0144 Accessed Feb 2018. Accessed 24 Feb 2018.

  51. How can the impacts of dissemination bias in qualitative research be detected in the context of qualitative evidence syntheses? Identifying new approaches https://www.globalevidencesummit.org/how-can-impacts-dissemination-bias-qualitative-research-be-detected-context-qualitative-evidence. Accessed 24 Feb 2018.

  52. Transparency, Recognition, and Innovation in Peer Review in the Life Sciences (February 2018). http://asapbio.org/peer-review. Accessed 24 Feb 2018.

  53. PEERE International Conference on Peer Review (March 2018). http://www.peere.org/conference/. Accessed 24 Feb 2018.

  54. PRINTEGER Promoting Integrity as an Integral Dimension of Excellence in Research (February 2018). https://printeger.eu/. Accessed 24 Feb 2018.

Download references

Acknowledgements

We are grateful to Daniel Shanahan for steering the journal in its early days, to Jigisha Patel for previously chairing the Editorial Board and to all our Editorial Board members for their continued enthusiasm and support. We thank Marco Casola (Journal Development Editor, BMC) and Rhiannon Meaden (Associate Publisher, BMC) for their continued advice and help. We would also like to thank the EQUATOR Network and International Society for Managing and Technical Editors (ISMTE) for endorsing Research Integrity and Peer Review.

Funding

Not applicable.

Availability of data and materials

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

ECM wrote the first draft. SLB, MKK, JM and EW contributed to the review and revision of the text. All authors read and approved the final version.

Corresponding author

Correspondence to Elizabeth C. Moylan.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

SLB, MKK, JJM and EW are Editors-in-Chief and ECM is the Chair of the Editorial Board of Research Integrity and Peer Review. SLB and MKK are employed by Springer Nature, and ECM is employed by BMC (part of Springer Nature) and volunteers for COPE (Committee on Publication Ethics), EnTIRE (an EU proposal for Mapping the research ethics and research integrity framework) and MIROR (Methods in Research on Research).

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boughton, S.L., Kowalczuk, M.K., Meerpohl, J.J. et al. Research Integrity and Peer Review—past highlights and future directions. Res Integr Peer Rev 3, 3 (2018). https://doi.org/10.1186/s41073-018-0047-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41073-018-0047-1