- Research
- Open access
- Published:
Responsible research practices could be more strongly endorsed by Australian university codes of research conduct
Research Integrity and Peer Review volumeĀ 8, ArticleĀ number:Ā 5 (2023)
Abstract
Background
This study aimed to investigate how strongly Australian university codes of research conduct endorse responsible research practices.
Methods
Codes of research conduct from 25 Australian universities active in health and medical research were obtained from public websites, and audited against 19 questions to assess how strongly they (1) defined research integrity, research quality, and research misconduct, (2) required research to be approved by an appropriate ethics committee, (3) endorsed 9 responsible research practices, and (4) discouraged 5 questionable research practices.
Results
Overall, a median of 10 (IQR 9 to 12) of 19 practices covered in the questions were mentioned, weakly endorsed, or strongly endorsed. Five to 8 of 9 responsible research practices were mentioned, weakly, or strongly endorsed, and 3 questionable research practices were discouraged. Results are stratified by Group of Eight (nā=ā8) and other (nā=ā17) universities. Specifically, (1) 6 (75%) Group of Eight and 11 (65%) other codes of research conduct defined research integrity, 4 (50%) and 8 (47%) defined research quality, and 7 (88%) and 16 (94%) defined research misconduct. (2) All codes required ethics approval for human and animal research. (3) All codes required conflicts of interest to be declared, but there was variability in how strongly other research practices were endorsed. The most commonly endorsed practices were ensuring researcher training in research integrity [8 (100%) and 16 (94%)] and making study data publicly available [6 (75%) and 12 (71%)]. The least commonly endorsed practices were making analysis code publicly available [0 (0%) and 0 (0%)] and registering analysis protocols [0 (0%) and 1 (6%)]. (4) Most codes discouraged fabricating data [5 (63%) and 15 (88%)], selectively deleting or modifying data [5 (63%) and 15 (88%)], and selective reporting of results [3 (38%) and 15 (88%)]. No codes discouraged p-hacking or hypothesising after results are known.
Conclusions
Responsible research practices could be more strongly endorsed by Australian university codes of research conduct. Our findings may not be generalisable to smaller universities, or those not active in health and medical research.
Introduction
High quality research that is rigorous, transparent and reproducible is needed for findings to be valid and trustworthy. Such research is more likely to result in practical and clinical translation of research outcomes and promotes public trust in science. However, the conduct and reporting of research is suboptimal in many disciplines. For example, the prevalence of questionable research practices (e.g.Ā fabricating data, and selectively deleting or modifying data) in many disciplines is high [1, 2]. Approximately 40% of clinical trials on exercise interventions for low back pain do not describe interventions in sufficient detail for independent replication [3]. Moreover, āspinā ā the misrepresentation and distortion of research findings ā is prevalent in biomedical science [4,5,6]. Overall, these examples suggest that the validity of scientific claims may, at times, be questionable.
Responsible practices in research conduct and reporting are needed to uphold research integrity and research quality. Responsible research practices often uphold principles of open science and research transparency, and include protocol registration, sharing of de-identified data and computer code, and transparent and rigorous reporting of research. However, there are few strong incentives for responsible research practices in research publishing, academic hiring and promotion, and grant funding [7,8,9]. Unsurprisingly, the lack of strong incentives may place researchers under pressure to compromise on research quality [10].
University codes of research conduct are documents that govern the behaviour of academic researchers. In Australia, codes of research conduct reflect the Australian Code for the Responsible Conduct of Research (the Australian Code), which outlines principles and responsibilities that both researchers and institutions are expected to follow when conducting research. It is not known to what extent university codes of research conduct endorse responsible research practices. Therefore, this study aimed to investigate how strongly codes of research conduct from research-intensive universities in Australia encourage responsible research practices, and discourage poor research practices.
Materials and methods
The study protocol and registration are available on the Open Science Framework [11]. Authorship order in this manuscript deviated from authorship order in the registered protocol to reflect author contributions in practice.
Sampling
We obtained a complete list of Australian universities from the Australian Government website Study Australia [12]. We selected 25 universities from this list active in health and medical research, as health and biomedical research receives substantial support from Australian funders [7].
The sample included all Group of Eight universities (nā=ā8), which are Australiaās leading research-intensive universities. Inherently, the Group of Eight is a company incorporated in 1999 that aims to influence the development and delivery of long-term, continual national higher education and research policy, and develop world-class international alliances and research partnerships. Consequently, the Group of Eight is highly influential.
Non-Group of Eight universities (nā=ā17) were sampled from universities reporting high research income awarded by the Australian National Health and Medical Research Council (NHMRC) in 2018, based on data from the Universities Australia Higher Education Research Data Collection [13]. High research income is a key measure to indicate that a university is research-active. Non-Group of Eight universities were purposefully sampled to obtain codes of research conduct from a range of research-active universities with high research income and sizeable public profile. We did not sample universities from which we could not obtain codes of research conduct from public websites (nā=ā2). Our final sample consisted of 8 Group of Eight and 17 non-Group of Eight universities.
Codes of research conduct and related documents were obtained from the public websites of the 25 included universities. The list of included universities is available in the Additional file 1.
Question development and pilot testing
The Australian Code outlines principles and responsibilities that researchers and institutions are expected to follow when conducting research. It includes more detailed guides on specific aspects of responsible research conduct. We interpreted how the 2018 Australian Code and its supporting guides could be translated into practice, and developed 19 questions and scoring criteria to assess if university codes of research conduct defined important aspects of research integrity, and how the codes advised or required researchers to implement responsible research practices. The questions were developed using questions from our previous audit of incentives for responsible research practices in grant funding schemes [7], and recommendations by large consensus papers [9, 14]. Questions assessed if codes of research conduct defined research integrity, research quality, and research misconduct (Q1-3, outcomes scored asĀ No: not mentioned, Default: the code defaults to the Australian Code, Specific: the code states its own definition), required research to be approved by an appropriate ethics committee (Q4-5, outcomes scored as No, Yes), endorsed (i.e.Ā advised or required) responsible research practices (Q6-14), and discouraged questionable research practices (Q15-19, outcomes in this and the preceding section scored asĀ No: not mentioned, Default: the code defaults to the Australian Code, Advised: implies strongly recommended but not mandated; a weak endorsement, Required: implies mandatory, likely with penalties if violated; a strong endorsement).
Questions were scored as āAdvisedā if codes of research conduct used words such as āadviseā, āsuggestā, āshouldā, āencourageā, and ārecommendā. Questions were scored as āRequiredā if codes of research conduct used words such as ārequireā, āmandateā, and āmustā. The word āshouldā was only used to score a question as ārequiredā if the code of research conduct used this word and stated penalties that researchers would face if they did not comply. The wording of the questions and scoring criteria are shown in Table 1.
Five university codes of research conduct were randomly sampled and used in a pilot study to assess the clarity of scoring instructions and agreement between investigators. Feedback from the pilot study was used to refine the wording of the questions and scores.
For the full audit, the text of the codes of research conduct and related documents was independently scored by 3 investigators (YKO, JD, KD). Scores that differed between investigators were discussed to reach agreement by consensus and refine the questions and scoring criteria. The refined questions and criteria were applied to the full sample (which included codes of research conduct used in the pilot study).
Document retrieval and data collection
University codes of research conduct and related documents referenced in hyperlinks within each code were retrieved by a single investigator (YKO) within 1Ā week (14ā18 Jan 2021). The following procedure was applied to retrieve documents:
-
The main website or document of the university code of research conduct was first retrieved using search terms such as āresearch conductā, ācode of research conductā, āresponsible researchā and āresearch codeā.
-
All documents referenced in hyperlinks from the main website or document of the code of research conduct were retrieved; we refer to this as Level 1 information.
-
Level 1 documents were screened. Only relevant documents referenced in hyperlinks in Level 1 documents were retrieved; we refer to this as Level 2 information.
When related documents could not be retrieved from a universityās public website, we contacted the university to request them. We were not able to retrieve the Research Ethics Manual referenced in the code of research conduct for one university. We contacted the universityās ethics board and they supplied us with an internal copy of the document.
Codes of research conduct and related documents were audited by a single investigator (YKO) and checked by another investigator (JD), using the questions and scoring criteria. Potential errors in scoring or lack of agreement were discussed with the investigator team and resolved by consensus. Decision rules to score questions are available in the Additional file 1.
A separate audit of the Australian Code and its supporting guides against our questions was also performed.
Analysis
Descriptive data are reported. All data processing and analysis were performed using Python (v3.9). Data and computer analysis code are available from the project repository [15].
Results
Codes of research conduct from 25 universities were audited (full list in Additional file 1). Based on HERDC data, the median (IQR) NHMRC research income obtained by these universities in 2018 was AUD $8.3 ($4.4 to $26.5) million.
The number of codes of research conduct that satisfied the criteria for each of the questions is shown in Fig.Ā 1. Results are stratified by Group of Eight (nā=ā8) and non-Group of Eight (nā=ā17) universities. Counts and percentages of scores for each university code of research conduct for each question are available in the Additional file 1.
Overall, a median of 10 (IQR 9 to 12) of 19 practices covered in the questions were mentioned (i.e.Ā the code of research conduct defaulted to the Australian Code), weakly endorsed, or strongly endorsed. Specifically, 7 (88%) Group of Eight and 16 (94%) non-Group of Eight codes of research conduct defined research misconduct, however only 6 (75%) and 11 (65%) defined research integrity, and 4 (50%) and 8 (47%) defined research quality. Twenty-four to 38% of codes of research conduct across Group of Eight and non-Group of Eight universities defined research integrity and research quality by defaulting to the Australian Code.
All codes of research conduct required ethics approval for human and animal research.
Overall, 5 to 8 of 9 responsible research practices were mentioned, weakly endorsed, or strongly endorsed, and 3 questionable research practices were discouraged. Specifically, all codes of research conduct required conflicts of interest to be declared, but there was substantial variation for other responsible research practices. The most commonly endorsed responsible research practices were ensuring researcher training in research integrity and other areas [8 (100%) and 16 (94%)], and making study data publicly available [6 (75%) and 12 (71%)]. The least commonly endorsed responsible research practices were making analysis code publicly available [0 (0%) and 0 (0%)], registering analysis protocols [0 (0%) and 1 (6%)], using reporting guidelines in reporting research [0 (0%) and 1 (6%)], and registering study protocols of study designs other than clinical trials [0 (0%) and 2 (12%)]. Proportionally, compared to Group of Eight universities, more non-Group of Eight universities endorsed that study protocols of designs other than clinical trials be registered, and findings be published on open access platforms.
Most codes of research conduct discouraged fabricating data [5 (63%) and 15 (88%)], selectively deleting or modifying data [5 (63%) and 15 (88%)], and selectively reporting results [3 (38%) and 15 (88%)]. Proportionally, compared to Group of Eight universities, more non-Group of Eight universities mandated against selectively deleting or modifying data and selective reporting of results. No codes of research conduct discouraged p-hacking or hypothesising after results are known (harking).
Results of the audit of the Australian Code and its supporting guides against our questions are available in the Additional file 1. Overall, in the Australian Code and its supporting guides, āresearch integrityā and āresearch misconductā were defined, but āresearch qualityā was not. The Australian Code required ethics approval for human and animal research. Eight of 9 responsible research practices were mentioned, weakly endorsed, or strongly endorsed, and 1 of 5 questionable research practices was discouraged.
Lists of all documents accessed and included for each university (including number, version, year of publication) are also available in the Additional file 1.
Discussion
Australian university codes of research conduct strongly endorse some aspects of research integrity (e.g.Ā requiring ethics approval, declaring conflicts of interest), as outlined in principle by the Australian Code for the Responsible Conduct of Research. However, other responsible research practices are weakly endorsed, or are not mentioned (e.g.Ā registering protocols, open code, discouraging p-hacking). In general, all codes of research conduct could endorse responsible research practices more strongly, and discourage questionable research practices more explicitly.
We sampled to include codes of research conduct from Australian universities active in health and medical research, based on research income awarded by the NHMRC. Thus, our findings would best reflect how strongly codes of research conduct from these universities encouraged responsible research practices, and discouraged questionable research practices. This is important because substantial research efforts in health and biomedical research are supported by the Australian government. In 2019ā2020, approximately AUD $202.5 billion was spent on health goods and services in Australia. This comprised 10.2% of overall economic activity [16], with large funds channeled through the NHMRC (e.g.Ā AUD $870 million in 2021ā2022) [17] and other funders. Researchers supported by these funders may be more likely to apply responsible research practices (or refrain from poor research practices) if their universityās code of research conduct explicitly endorse them.
Why do codes of research conduct not endorse responsible research practices more strongly, or not at all? First, substantial efforts to date have been directed towards preventing or minimising research misconduct and fraud, as these have broad, detrimental ramifications on public trust in science [18, 19]. Other poor research practices such as p-hacking or hypothesising after results are known may be regarded as less severe, and not prioritised. Compared to misconduct and fraud, upholding research integrity and quality through positive measures can seem less urgent, and more laborious to implement. Second, responsible research practices could be viewed by universities to fall under the remit of individual researchers, not institutions. Thus, universities may dismiss the need to endorse these practices more strongly. Third, universities might fail to adequately understand the value and need for responsible research practices, especially in context of the pressure to publish. Overall, these barriers may impede the broader implementation of responsible research practices in research conduct and reporting.
Codes of research conduct could endorse responsible research practices more strongly in several ways. Codes of research conduct could be regularly updated to mandate responsible research practices or hold researchers more accountable, especially when current wording simply defaults to the Australian Code, or only serves as a weak endorsement. Codes of research conduct could also be written to clearly attribute responsibility either to individuals or to institutions [20], or explicitly define good research ācultureā and how it can be developed. This could decrease reliance on research culture to drive the uptake of responsible research practices. More broadly, future research could investigate other strategies or themes to inform codes of research conduct and incentives for responsible research practices. Examples of these include the responsible evaluation of research and researchers, how open science and transparency impact responsible research practices, and how responsible mentoring, supervision and role modelling could be conducted [21]. Importantly, these efforts would allow codes of research conduct to implement policies that are evidence based [22].
Our findings should be interpreted in context of study limitations. First, we purposefully sampled codes of research conduct from universities active in health and medical research. This may mean our findings are not generalisable to smaller universities, universities that focus on other research areas, or universities that are not research-intensive. Second, document retrieval was difficult at times as some documents were not publicly accessible, or had multiple embedded hyperlinks. We contacted university research offices to request copies of documents that were not accessible. The lack of public access to these documents seems problematic: it may suggest universities do not highly prioritise transparency in the governance of research processes. Third, instructions on specific practices (e.g.Ā use reporting guidelines, discourage p-hacking and harking) may be present in internal university documents to relevant faculties, even though they were not stated in externally-facing documents to the University that could be accessed publicly. By auditing only the externally-facing documents, we would not have been able to capture internally-available information that is relevant to research conduct. Fourth, statements in some codes were ambiguous and difficult to interpret. In such instances, we endeavoured to make reasonable decision rules for interpretation, and we recorded the rationale for these decisions to ensure consistent assessment of subsequent codes of research conduct.
Conclusions
In summary, Australian university codes of research conduct could endorse responsible research practices more strongly, and discourage questionable research practices more explicitly. These measures could be implemented by clearly attributing responsibility for research rigour to individual researchers or to universities through the codes of research conduct. If the global scientific community collectively identifies and implements good research practices, in time, this may improve research integrity and research quality, and improve trustworthiness in science.
References
Gopalakrishna G, ter Riet G, Vink G, Stoop I, Wicherts JM, Bouter LM. Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in The Netherlands. PLoS One. 2022;17:e0263023.
Xie Y, Wang K, Kong Y. Prevalence of research misconduct and questionable research practices: A systematic review and meta-analysis. Sci Eng Ethics. 2021;27:41.
Davidson SRE, Kamper SJ, Haskins R, Robson E, Gleadhill C, da Silva PV, et al. Exercise interventions for low back pain are poorly reported: A systematic review. J Clin Epidemiol. 2021;139:279ā86.
Chiu K, Grundy Q, Bero L. āSpinā in published biomedical literature: a methodological systematic review. PLoS Biol. 2017;15:1ā6.
Diong J, Butler AA, Gandevia SC, HĆ©roux ME. Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice. PLOS One. 2018;13(8):e0202121.
Otte WM, Vinkers CH, Habets PC, van IJzendoorn DGP, Tijdink JK. Analysis of 567,758 randomized controlled trials published over 30 years reveals trends in phrases used to discuss results that do not reach statistical significance. PLOS Biology. 2022;20:e3001562.
Diong J, Kroeger CM, Reynolds KJ, Barnett A, Bero LA. Strengthening the incentives for responsible research practices in Australian health and medical research funding. Res Integr Peer Rev. 2021;2021(6):1ā7.
Lindner MD, Torralba KD, Khan NA. Scientific productivity: An exploratory study of metrics and incentives. Ouzounis CA, editor. PLOS One. 2018;13:e0195321.
Moher D, Naudet F, Cristea IA, Miedema F, Ioannidis JPA, Goodman SN. Assessing scientists for hiring, promotion, and tenure. PLoS Biol. 2018;16:e2004089.
NHMRC. 2019 Survey of research culture in Australian NHMRC-funded institutions. Survey findings report. 2020. Available from:Ā https://www.nhmrc.gov.au/research-policy/research-quality. [cited 2023 Mar 22].
Diong J, Bero L, Ong YK, Double K. Australian university codes of research conduct could endorse responsible research practices more strongly. 2020 [cited 2023 Feb 14]; Available from: https://osf.io/yfz8k/.
Australian Trade and Investment Commission. List of Australian universities. [cited 2023 Feb 14]; Available from: https://www.studyaustralia.gov.au/english/study/universities-higher-education/list-of-australian-universities/list-of-australian-universities.
Department of Education. Research income time series. Department of Education [Internet]. 2022 [cited 2023 Feb 14]; Available from: https://www.education.gov.au/research-block-grants/resources/research-income-time-series.
Rice DB, Raffoul H, Ioannidis JPA, Moher D. Academic criteria for promotion and tenure in biomedical sciences faculties: Cross sectional analysis of international sample of universities. BMJ. 2020;369:m2081.
Diong J. Responsible research practices could be more strongly endorsed by australian university codes of research conduct. 2023 [cited 2023 Feb 14]; Available from: https://github.com/joannadiong/Ong_et_al_2023_RIPR.
Australian Institute of Health and Welfare. Health expenditure Australia 2019ā20. AIHW, Australian Government [Internet]. 2022;AIHW, Australian Government:accessed 20 October 2022. Available from: https://www.aihw.gov.au/getmedia/f1284c51-e5b7-4059-a9e3-c6fe061fecdc/Health-expenditure-Australia-2019-20.pdf.aspx?inline=true.
Reeves T. Australian H&MR Research Facts. Research Australia [Internet]. 2016;accessed 16 January 2023. Available from: https://researchaustralia.org/australian-research-facts/.
Fang FC, Steen RG, Casadevall A. Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci. 2012;109:17028ā33.
Wager E. Coping with scientific misconduct. BMJ. 2011;343: d6586.
Valkenburg G, Dix G, Tijdink J, de Rijcke S. Making researchers responsible: Attributions of responsibility and ambiguous notions of culture in research codes of conduct. BMC Med Ethics. 2020;21:56.
Tijdink JK, Horbach SPJM, Nuijten MB, OāNeill G. Towards a research agenda for promoting responsible research practices. Journal of Empirical Research on Human Research Ethics. 2021;155626462110189.
Bouter L. What research institutions can do to foster research integrity. Sci Eng Ethics. 2020;26:2363ā9.
Acknowledgements
Not applicable.
Funding
This study is not supported.
Author information
Authors and Affiliations
Contributions
Yi Kai Ong: Investigation, Interpretation, Writing ā original draft, Writingāreview and editing. Kay Double: Conceptualisation, Methodology, Interpretation, Writing ā review and editing. Lisa Bero: Conceptualisation, Methodology, Interpretation, Writing ā review and editing. Joanna Diong: Conceptualisation, Methodology, Investigation, Formal analysis, Software, Visualisation, Interpretation, Data curation, Project administration, Supervision, Writing ā original draft, Writing ā review and editing. The authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Additional information
Publisherās Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Ong, Y.K., Double, K.L., Bero, L. et al. Responsible research practices could be more strongly endorsed by Australian university codes of research conduct. Res Integr Peer Rev 8, 5 (2023). https://doi.org/10.1186/s41073-023-00129-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s41073-023-00129-1