The parallel session at WCRI in Montreal in 2013 identified 36 questions or challenges [2]. These were grouped under the headings: (1) When editors have concerns about research or publication integrity; (2) When institutions receive evidence or questions about research or publication integrity; (3) When institutions investigate suspected misconduct; (4) Role of funders and other bodies; (5) What can journals, institutions and funders do to prevent and detect misconduct and errors? The Workshop held in Heidelberg in 2016 was divided into 7 sessions on the themes of: institutions’ expectations of journals and (separately) journals’ expectations of institutions in cases of (i) suspected author or reviewer misconduct, (ii) proven misconduct, and (iii) other situations, with a final session focusing on other stakeholders (e.g. funders, journalists, lawyers, research integrity organizations). Following the meeting and discussion of a draft report, 14 key issues were identified and these were later organized into three themes. The issues are presented below, with a brief summary of the discussion and recommendations (except for Issues 3 and 8 where the group felt unable to make a universal recommendation because of the differences in laws and practices between countries).
Recommendations on best practice
(I) When journals suspect problems with submitted work
Issue 1: journals often have difficulty identifying somebody with responsibility for research integrity at an institution
In many areas outside North America, universities either do not have a research integrity officer or office (RIO), or the person or department with responsibility for research integrity (and their contact details) is not clearly identified on the institution’s website [5]. Identifying the right contact person is also difficult because various titles are used for this function.
If the corresponding author’s institution does not have a RIO, the authors may identify a suitable person at any of their institutions. If no such person can be identified at any of the institutions involved with the research, the authors should be asked to nominate a senior faculty member (e.g. dean or pro-vice chancellor with responsibility for research, or the chair of the research ethics committee or institutional review board) who was not directly involved with the research (and is not an author) who could be contacted if the journal has any concerns about research integrity.
Requiring researchers to provide contact details of a person with responsibility for handling research integrity allegations at their institution should not only enable journals to contact this person if concerns arise, but may raise awareness of RIOs among researchers and encourage institutions to make such an appointment and publish their contact details prominently on institutional websites. Details of the contact person for research integrity enquiries should not be published by the journal, but should be retained, should the need arise to contact them.
Recommendations
Institutions should have a research integrity officer (or office) and publish their contact details. National research integrity bodies (or other appropriate organizations) should keep a register of people with responsibility for handling research integrity allegations at their country’s institutions, to enable journal editors (and others) to contact them.
Where such lists are not available, journals should request corresponding authors to provide the name and email address (or telephone number) of their institution’s RIO (or of an individual with responsibility for handling research integrity allegations) when submitting the article.
Issue 2: what should journals do when reviewers say findings look ‘too good to be true’ in the absence of specific evidence?
If a peer reviewer raises a concern about the integrity of research, especially if s/he suggests that the results are ‘too good to be true’, the journal should ask the reviewer for more details (e.g. to explain why they gave this opinion) and may consider alerting the institution to concerns of fabrication or falsification if they consider they are well founded. Journals therefore need to determine whether to contact an institution and, if so, what information they should share, and whether this should be done before discussing concerns with the authors (see Issue 4).
Peer reviewer reports and comments to the editor should generally only be shared with authors’ institutions with the reviewers’ express permission. Similarly, the identity of the peer reviewer should not normally be revealed to the authors’ institutions in cases of suspected problems with submitted or published work (unless the reviewer has agreed to this or the journal publishes signed reviewer comments in an open peer review system).
It is helpful for journals to share suspicions about the integrity of reported research with institutions (as well as more specific concerns or clear evidence of problems) because institutions are able, and have a duty, to assess concerns about possible integrity breaches such as data fabrication or falsification by researchers. Another reason why journals should raise concerns about reported research is that the institution should have a more complete picture of the researcher’s behaviour than the journal (which usually has information only from one article), and such evidence may be important to trigger or inform an investigation. Sophisticated data fabrication or falsification may only become obvious when several publications are assessed, or when raw data or other forensic evidence are available [6]. Therefore, in such cases, while individual journals may have some suspicions, the full picture is usually available only to the institution. Furthermore, alerting an institution may prevent the research from being submitted to other journals (which would be unaware of the first journal’s concerns) before it has been properly assessed.
Recommendations
Journals should develop criteria for determining whether, and what type of, information should be passed on to institutions.
When journals share evidence relating to possible research integrity problems with institutions the identity of peer reviewers or other people raising concerns should not be revealed (unless this is already published or the individuals have given permission for this disclosure).
In addition to sharing any direct evidence of plagiarism, fabrication or falsification with institutions, journals should develop criteria for determining when reviewer or editor suspicions that work is ‘too good to be true’ or that something is ‘not right’ should be shared.
Issue 3: journals do not know whether an informal ‘off the record’ approach to an institution is possible
Since journals are typically not in a position (either legally or practically) to conduct detailed, formal investigations it is not always possible for an editor to obtain clear evidence or to judge whether an allegation is well-founded on the basis of submitted or published reports. While journals may request source data from authors, they do not have legal powers to obtain this, nor do they have access to laboratory notebooks or equipment logs, or the possibility to interview staff. Therefore, since journals do not normally have access to all the relevant information, their peer reviewers and editors may only be able to indicate they suspect that something is wrong.
Therefore, journals may sometimes want to contact institutions informally, to discuss their suspicions or concerns, or raise non-specific allegations about research integrity, without necessarily invoking a full investigation. Journals may also wish to know whether a researcher is currently being, or has recently been, investigated for suspected misconduct.
Journals need to understand that in some jurisdictions (for example, the United States) such an ‘off the record’ discussion is not always possible, as institutional research integrity officers and all those involved with investigations have to maintain the maximum confidentiality possible until the formal process has concluded and such conversations must be documented as part of the institutional record. Institutions risk being sued if they breach this confidentiality, e.g. by revealing that a researcher is under investigation. However, in other regions, the situation is different and it may be possible to discuss concerns informally and for universities to disclose whether an individual is currently under investigation.
Issue 4: should journals always contact authors about research integrity concerns?
In most cases, when journals have concerns about the integrity of submitted or published work, they should first raise them with the authors (usually starting with the corresponding author). This allows researchers to provide clarification, explanation or further information. Contacting authors is considered to reflect ‘due process’ or procedural fairness, and avoids wasting institutional and editorial time and resources over issues that arise from honest error and can be handled in a straightforward way by the journal. When approaching authors, journals are advised to describe concerns using neutral rather than accusatory language, for example highlighting the amount of text similarity rather than accusing an author of plagiarism. The presumption, at this stage, is that the authors are ‘innocent until proven otherwise’.
However, journals should be aware that in cases of suspected data fabrication or falsification, raising concerns with the authors first could enable researchers to destroy or alter evidence that might be important for an institutional investigation (http://retractionwatch.com/2014/07/22/accounting-professor-faked-data-for-two-studies-destroyed-evidence-university-report/, https://www.baruch.cuny.edu/rio/research_misconduct_examples.htm). Therefore, when journals have well-founded suspicions or evidence of falsification or fabrication they should consider informing the institution at the same time as, or before, they contact the author(s).
Such cases are likely to be rare, since the circumstances in which journals have access to raw data are currently limited (but may include Western blots and other images). This situation may change as publication of research data becomes more widespread (http://aims.fao.org/activity/blog/%E2%80%98-state-open-data%E2%80%99-figshare%E2%80%99s-report-global-trends-around-open-data).
If a journal discovers evidence of falsification (e.g. inappropriate manipulation of images) or major plagiarism (e.g. reports from text-matching software verified by an editor) the journal should retain the evidence and should offer to share it with the institution. (See, for example the EMBO Press classification for image aberrations [7].) However, care should be taken to avoid revealing the identity of peer reviewers, or other people raising concerns, to an institution against their wishes or without their permission. Ensuring the anonymity of internal whistleblowers (i.e. members of a research group or department who raise concerns about colleagues or collaborators) may be difficult since, even if their name is not revealed, the source may be obvious to the authors if only a few people know about certain details of the research.
Recommendation
Journals should develop criteria to determine when the authors’ institution(s) should be contacted immediately without (or at the same time as) alerting the author(s). This would occur only in exceptional cases when journals have evidence of substantive or significant falsification or fabrication of data.
Issue 5: if a journal suspects that an author or peer reviewer has failed to disclose a relevant competing interest, should they refer this to the institution?
Readers, authors or reviewers sometimes suggest that relevant competing interests have not been disclosed during the review process or in a publication. If such allegations or concerns cannot be resolved after discussion with the authors or reviewers concerned (e.g. by publishing a correction if information has been omitted from a publication, or seeking additional peer review), the journal may consider contacting an institution. However, institutional responses vary. Some institutions maintain lists of researchers’ current interests and have policies about disclosure of competing interests. In such cases, it is appropriate for journals to raise concerns with the institution and to ask them for relevant information. However, not all institutions register such information, and, if they do not, they may be unable to respond to the journal’s enquiries. While failure to disclose a relevant interest is not always categorised as research misconduct, it is generally recognised to be poor practice and usually requires action by the journal (which will depend on the severity of the case). The WCRI notes that ‘being transparent about and managing competing interests’ is one aspect of research integrity [4].
Recommendation
Institutions and funders should be responsive to journal requests for information to ensure that peer reviewers’ and authors’ competing interests are properly disclosed.
Issue 6: if a journal rejects an article about which either reviewers or editors have raised concerns about integrity, authors may simply submit it to another journal, perhaps after concealing problems more effectively
The COPE Code of Conduct for journal editors noted that ‘Editors should not simply reject papers that raise concerns about possible misconduct. They are ethically obliged to pursue alleged cases.’ [8] In other words, journals should seek explanations from authors even if they do not intend to accept their publication and should contact institutions, if required, regardless of publication status.
One suggestion made at the meeting was for each institution to maintain a repository of submitted manuscripts. Researchers affiliated to an institution would be expected to send a copy of all submissions to this repository. These would not be made public but the database could be used to check the history of a publication and document any changes made by authors (e.g. when submitting to a different journal after a rejection). Such a database of submitted manuscripts would be useful for institutional investigations and would permit assessment of all of a researcher’s work. To be workable this process would need to be straightforward and not excessively burdensome on researchers.
Recommendation
Journals should seek explanations from authors about research integrity concerns even if they do not intend to accept their publication and should inform institutions, if appropriate, regardless of publication status.
II. Investigating problems with research and its publication
Issue 7: institutional focus on strict definitions of research misconduct may hamper communication about broader issues of research integrity
Journals have a responsibility to correct or retract any publications that give misleading accounts of research methods, findings, analyses or authorship, regardless of whether this is determined to have been due (or related) to misconduct or to error. However, many institutions and research integrity bodies focus solely on determining narrowly defined misconduct and establishing the burden of proof for each particular case. Furthermore, definitions of misconduct vary between jurisdictions. For example, the US Office of Research Integrity (ORI) considers only cases of fabrication, falsification or plagiarism (FFP) in research funded by the US Public Health Service [9] while the Australian Code for the Responsible Conduct of Research takes a more inclusive approach [10].
Because of the possible serious consequences of a misconduct finding for individuals and institutions, and the importance of conducting rigorous and fair proceedings (and the costs associated with these), thresholds for launching a full inquiry or investigation may be high. This may give journals the impression that institutions are reluctant to cooperate or respond to their enquiries.
It would therefore be helpful if institutions had mechanisms for assessing the integrity of reported research in response to concerns raised by journals or others. The focus of such assessment should be solely on determining the integrity of the research itself, and its reporting (in other words, the likelihood that journal readers may be misled or confused), rather than on the behaviour or intentions of the researchers. Such assessments should permit institutions to respond more rapidly to journal enquiries and without concerns about breaching confidentiality related to institutional policies or employment processes. However, such assessments would not prevent further investigation through the institution’s established processes for handling misconduct allegations.
Recommendation
Institutions should develop mechanisms for assessing the integrity of research reports that are submitted to, or published by, academic journals, which could be used if concerns are raised; these mechanisms should be distinct from processes to determine whether misconduct has occurred.
Issue 8: institutions may feel legally bound to keep disciplinary hearings confidential and may therefore feel unable to communicate or share details of on-going investigations with journals
Journals have a responsibility to alert readers to published material that may be misleading (for whatever reason). Even when misleading research does not cause direct public harm, it may lead to the waste of other researchers’ time and resources. The need for journals to alert readers promptly to potentially misleading articles is especially great in applied research since decisions affecting individuals and public policies may be based on publications. Journals may therefore wish to know if an investigation has been started, and may wish to alert readers before an investigation (and appeal process) has concluded (e.g. by an expression of concern).
However, in many jurisdictions, research misconduct investigations and disciplinary hearings are considered confidential and institutions/employers may therefore feel unable to share details with journals. This approach may prevent journals from fulfilling their responsibilities to their readers, for example by publishing an expression of concern. Journals need to be aware that practices vary between countries. For example, the European Network of Research Integrity Offices (ENRIO) notes ‘when it comes to the level of transparency and confidentiality regarding investigations, traditions and legislative mandates vary across Europe’ [11].
Various solutions to this problem were discussed at the CLUE meeting. One suggestion was for journals to require authors to disclose any investigations or proceedings and thus waive the confidentiality accorded by law within their contract with the journal.
Another suggestion was that researchers’ employment contracts should specify that, in cases of suspected or proven misconduct, harm to research participants, or other circumstances affecting the validity of a research report, the employees’ usual right to confidentiality in disciplinary proceedings would be waived to allow the institution to communicate relevant details to the journal and other parties. The CLUE meeting participants recognised that such solutions might be hard for journals to enforce, or may require changes in employment legislation, and therefore put them forward for discussion rather than as recommendations.
Issue 9: journals and institutions may be asked about publications relating to research that took place many years ago
While investigation of historical research may pose more challenges than inquiries into more recent work, concerns should not be dismissed solely on the grounds that the research was done a long time ago. If plausible evidence of serious problems is raised, it should, ideally, be examined, regardless of when the problems occurred. However, since the difficulties of contacting authors and accessing original data increase over time, it is reasonable for journals and institutions to prioritise the investigation of recent over historical work.
Institutions should take responsibility for research performed under their auspices regardless of whether the researcher still works at that institution. Even if a researcher has moved to another institution, or has retired, the appropriate investigation should take place.
Investigations into the work of researchers who have died, are chronically incapacitated or have left research altogether, is especially difficult. However, in such cases, institutions should make their best efforts to establish whether the work has integrity, so that journals can determine whether readers should be alerted to concerns. Although probably a rare occurrence, this is another situation in which assessing the integrity of findings and reports needs to be separated from determining whether misconduct was committed by an individual researcher.
It is often impossible for an institution to investigate the integrity of a piece of research and its reporting without access to the source data. This can be problematic if data are not retained, as suspicions sometimes emerge several years after publication [6]. Also, if data are kept only by individual researchers, files may be lost unintentionally or deliberately destroyed or altered.
Permanent, public deposition of data is the ideal, because it allows immediate scrutiny by anybody interested, which may reveal errors or misconduct. However public posting of individuals’ personal or clinical data may not be possible due to the need for confidentiality (e.g. of medical records).
We encourage institutions and funders to review current data retention standards which may prevent effective investigation of historical data (e.g. we suggest that the 6-year period required for the retention of personal health data in the US under the Health Insurance Portability and Accountability Act (HIPAA) [12, 13] is too short). We also encourage debate on the risks and benefits of conventions in certain disciplines of destroying sensitive data, such as interview transcripts, to protect the confidentiality of research participants and to develop alternative systems (e.g. locked, secure deposition) to permit later investigation, if required.
Recommendations
Institutions should take responsibility for research performed under their auspices regardless of whether the researcher still works at that institution or how long ago the work was done.
Research institutions and major funders should ensure that essential research data are retained for at least 10 years (enabling the validation of published findings). Responsibility for data storage (e.g. for multicentre studies) should be defined in funding agreements.
Issue 10: concerns may be raised about research that involved several institutions
When research involves several institutions, one of the institutions usually takes a primary or coordinating role in relation to the funding. This primary institution should be the initial point of contact and take the lead in responding to concerns about research integrity. Ideally, research agreements should specify this and also set out responsibilities for data deposition and retention [14].
The International Committee of Medical Journal Editors (ICMJE) states that authors should be accountable for answering questions about research and identifying which author was responsible for each aspect if questions arise [15]. We suggest extending this guidance so that authors are also expected to identify where each component of a project was done, and therefore which institution should be responsible for investigating any concerns about it.
Recommendation
Research agreements should specify the primary institution that will coordinate the response to concerns about research integrity (following liaison with the other institutions, as required).
Issue 11: journals and institutions may be contacted by whistleblowers who conceal their identity, use pseudonyms or request anonymity
Allegations and concerns about the integrity of research and its reporting should be taken seriously by journals and institutions regardless of their source. Institutions should have policies about whistleblower protection and about the handling of cases from anonymous whistleblowers. Such allegations should be considered on their merits rather than being dismissed automatically. Therefore, an individual’s refusal to reveal their name, use of a pseudonym, or request to remain anonymous, should not prevent either a journal or an institution from taking allegations seriously. However, both journals and institutions need reassurance that an allegation is well-founded and is not simply a personal vendetta. Journals and institutions may therefore request further details or information from the correspondent and, if evidence of a problem is not forthcoming, it is reasonable for journals not to raise the concern with the university or for an institution to decide not to proceed with an inquiry or full investigation. However, this is a matter of judgement for both journals and universities (which should apply to both named and anonymous complainants), so we recommend a flexible approach, depending on the seriousness of the alleged problem or behaviour and the plausibility of the evidence provided. Journals should not feel compelled to respond to vexatious complaints.
Recommendation
Anonymous or pseudonymous allegations about research integrity made to journals or institutions should be judged on their merit and not dismissed automatically.
Issue 12: who should investigate if a peer reviewer is suspected of acting inappropriately?
Universities should recognize peer review as a legitimate part of research and academic activity and should encourage accountable and responsible behaviour from their researchers when they act as reviewers or editors [16]. However, even when peer review is viewed as part of general academic duties, the reviewer’s institution may not be equipped to investigate suspicions of reviewer misconduct since most of the relevant information will be held by the journal. In such situations, the journal may therefore have to initiate its own investigation, following the COPE flowchart about how to handle cases of suspected reviewer misconduct [17]. Since investigation of peer review manipulation requires access to journals’ editorial records, publishers should ensure these are retained for similar time periods as research records. Evidence of serious misconduct by researchers acting as peer reviewers (e.g. stealing ideas or material from the articles they were invited to review) should be shared with their institution. Therefore, journals should inform reviewers that their identity might be disclosed to their institution in cases of suspected misconduct and that possible serious misconduct will be addressed by the institution.
Recommendation
Journals should share evidence of misconduct by researchers acting as peer reviewers with their institution and follow the relevant COPE flowchart in such cases. Journals should inform reviewers that their identity might be disclosed to their institution in cases of suspected misconduct.
Journals and publishers should retain peer review records to enable the investigation of peer review manipulation or other inappropriate behaviour by authors or reviewers.
III. Issues arising after misconduct investigations
Issue 13: institutions sometimes do not share findings of misconduct investigations with journals that have published affected research and journals may be reluctant to publish informative retraction notices
Journals have a duty to avoid misleading their readers and therefore sometimes need to correct or retract published work that is incorrect or unreliable. Since problems can arise either inadvertently, from honest error, or from deliberate misconduct, the reason for a retraction should be clearly stated in the retraction notice including details of the affected findings and the type of problem detected [18]. This is important to ensure that honest researchers are not discouraged from alerting journals to problems with their work because of fears that a retraction will damage their career or be taken to imply that misconduct has occurred (when, in fact, such honesty and care for the research record should be praised [19]). Journals that have published affected work therefore need to receive details of misconduct investigations including clear information about all of the published articles (and submitted manuscripts) that are affected.
Being able to quote or cite an official report from an institution should facilitate the publication of clear and informative retractions (or corrections) since it reduces the journal’s risk of litigation. If a journal reports that University X has investigated the case and determined that a researcher has fabricated data this is a statement of fact and therefore unlikely to expose the journal to claims that it has published defamatory material.
Although, after misconduct has been found, institutions often require researchers to contact journals in which their work was published, we encourage institutions also to contact the journals directly. This direct communication between institution and journal allows relevant information to be shared and avoids situations in which researchers fail to contact affected journals, refuse to accept an investigation’s findings, or give a misleading account of the investigation to the journal. If an author tells a journal that the investigation was unfair or its finding was incorrect, this places the journal in a difficult position, but this problem may be avoided if the journal is allowed to see relevant sections or the full report of the investigation and can therefore verify whether it was properly conducted. We also recommend that institutions should be transparent about their processes for handling suspected misconduct or, at least be prepared to share information about such processes with journals, if requested.
Recommendations
Institutions should notify journals directly and release all relevant sections of reports of misconduct investigations (or a summary of their findings) to all journals that have published research that was the subject of the investigation. Names may be redacted to ensure privacy.
Institutions should allow journals to quote from misconduct investigation reports or cite them in retraction statements and related publications (e.g. explanatory editorials or commentaries).
Issue 14: journals sometimes fail to respond to requests for correction or retraction from institutions or authors
Communication with a journal should normally be addressed to the editor, but if the editor does not respond, the publisher should be contacted. If a journal is owned by an academic society, the leaders of that society may also be used as a point of contact, or to raise concerns about the behaviour of the editor.
Recommendation
Journals should respond to institutions about research integrity cases in a timely manner. The editor should be the first point of contact, and journals should publish their contact details. If a journal editor does not respond, the publisher or journal owner should be contacted.
