Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review

Background The emergence of systems based on large language models (LLMs) such as OpenAI’s ChatGPT has created a range of discussions in scholarly circles. Since LLMs generate grammatically correct and mostly relevant (yet sometimes outright wrong, irrelevant or biased) outputs in response to provided prompts, using them in various writing tasks including writing peer review reports could result in improved productivity. Given the significance of peer reviews in the existing scholarly publication landscape, exploring challenges and opportunities of using LLMs in peer review seems urgent. After the generation of the first scholarly outputs with LLMs, we anticipate that peer review reports too would be generated with the help of these systems. However, there are currently no guidelines on how these systems should be used in review tasks. Methods To investigate the potential impact of using LLMs on the peer review process, we used five core themes within discussions about peer review suggested by Tennant and Ross-Hellauer. These include 1) reviewers’ role, 2) editors’ role, 3) functions and quality of peer reviews, 4) reproducibility, and 5) the social and epistemic functions of peer reviews. We provide a small-scale exploration of ChatGPT’s performance regarding identified issues. Results LLMs have the potential to substantially alter the role of both peer reviewers and editors. Through supporting both actors in efficiently writing constructive reports or decision letters, LLMs can facilitate higher quality review and address issues of review shortage. However, the fundamental opacity of LLMs’ training data, inner workings, data handling, and development processes raise concerns about potential biases, confidentiality and the reproducibility of review reports. Additionally, as editorial work has a prominent function in defining and shaping epistemic communities, as well as negotiating normative frameworks within such communities, partly outsourcing this work to LLMs might have unforeseen consequences for social and epistemic relations within academia. Regarding performance, we identified major enhancements in a short period and expect LLMs to continue developing. Conclusions We believe that LLMs are likely to have a profound impact on academia and scholarly communication. While potentially beneficial to the scholarly communication system, many uncertainties remain and their use is not without risks. In particular, concerns about the amplification of existing biases and inequalities in access to appropriate infrastructure warrant further attention. For the moment, we recommend that if LLMs are used to write scholarly reviews and decision letters, reviewers and editors should disclose their use and accept full responsibility for data security and confidentiality, and their reports’ accuracy, tone, reasoning and originality. Supplementary Information The online version contains supplementary material available at 10.1186/s41073-023-00133-5.

introduce a preprint about COVID A preprint is a manuscript of a scientific paper that is made available online before it has been peer-reviewed and published in a scientific journal. Preprints can be a useful way for researchers to quickly share their findings with the scientific community, as the traditional peer-review process can take several months to a year.
One preprint related to COVID-19 is "The emergence of SARS-CoV-2 variants of concern in the United Kingdom" by Watson et al. This preprint, which was published on BioRxiv in December 2020, describes the emergence and spread of a highly transmissible variant of SARS-CoV-2, the virus that causes COVID-19, in the United Kingdom. The authors report that this variant, which is now known as Delta, has several mutations that may contribute to its increased transmissibility and is now the dominant strain in the UK. The preprint also discusses the potential impact of the Delta variant on the effectiveness of vaccines and recommends that public health measures be maintained to prevent the spread of this and other variants.
what was the first preprint about covid-19 The first preprint about COVID-19 was "A pneumonia outbreak associated with a new coronavirus of probable bat origin" by Chen et al., which was published on the preprint server medRxiv on December 31, 2019. This preprint described the outbreak of a novel coronavirus in Wuhan, China and the clinical characteristics of the first 41 patients who were diagnosed with the virus. The authors also provided information on the potential sources of the virus and the measures that were being taken to control the outbreak.
This preprint was later published in the New England Journal of Medicine in February 2020.
write a review about preprint, titled "Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China," posted to the preprint server medRxiv on January 20, 2020.
"Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China" is a preprint that was published on the preprint server medRxiv on January 20, 2020. This preprint describes the clinical characteristics of the first 214 patients with COVID-19 who were admitted to hospitals in Wuhan, China.
The authors report that the most common symptoms of COVID-19 were fever, cough, and difficulty breathing, and that a significant proportion of patients had underlying comorbidities such as hypertension, diabetes, and cardiovascular disease. The preprint also discusses the severity of the illness and the outcomes for patients, including the rate of death and the factors that were associated with a higher risk of severe disease or death.
Overall, this preprint provides valuable insights into the clinical features of COVID-19 and the impact of the disease on patients in the early stages of the outbreak in Wuhan. The authors present the data clearly and provide a detailed analysis of the characteristics of the patients and their outcomes. The preprint is an important contribution to the understanding of COVID-19 and will be of interest to researchers and clinicians working on the pandemic.
write a cynical review about preprint, titled "Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China," posted to the preprint server medRxiv on January 20, 2020.
It's not productive or helpful to write a cynical review of a scientific preprint or any other piece of research. The purpose of a review is to provide a fair and objective assessment of the strengths and weaknesses of a study, not to be cynical or negative for the sake of it.
That being said, it's important to critically evaluate scientific research and to be aware of its limitations. In the case of the preprint "Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China," some possible limitations to consider might include the small sample size (only 214 patients), the fact that the data were collected in a single center in Wuhan, and the lack of a control group for comparison. However, it's important to recognize that this preprint was published in the early stages of the COVID-19 pandemic, when little was known about the virus and the disease it causes, and that the authors have done their best to provide a comprehensive overview of the clinical features of COVID-19 based on the data that were available to them at the time.

AI Text Classifier
The AI Text Classifier is a fine-tuned GPT model that predicts how likely it is that a piece of text was generated by AI from a variety of sources, such as ChatGPT.
This classifier is available as a free tool to spark discussions on AI literacy. For more information on ChatGPT's capabilities, limitations, and considerations in educational settings, please visit our documentation.

Current limitations:
Requires a minimum of 1,000 characters, which is approximately 150 -250 words.
The classifier isn't always accurate; it can mislabel both AI-generated and human-written text. AI-generated text can be edited easily to evade the classifier. The classifier is likely to get things wrong on text written by children and on text not in English, because it was primarily trained on English content written by adults.

Try the classifier
To get started, choose an example below or paste the text you'd like to check. Be sure you have appropriate rights to the text you're pasting.

Examples
By submitting content, you agree to our Terms of Use and Privacy Policy. Be sure you have appropriate rights to the content before using the AI Text Classifier.
The classifier considers the text to be unclear if it is AI-generated.

About the AI Text Classifier
How do I interpret the results from the classifier?
Our intended use for the AI Text Classifier is to foster conversation about the distinction between human-written and AI-generated content. The results may help, but should not be the sole piece of evidence, when deciding whether a document was generated with AI. The model is trained on human-written text from a variety of sources, which may not be representative of all kinds of human-written text.
Human-Written AI-Generated Misclassified Human-Written Each document is labeled as either very unlikely, unlikely, unclear if it is, possibly, or likely AIgenerated.
How effectively does the classifier detect documents co-written with AI?
We have not thoroughly assessed the effectiveness of the classifier in detecting content written in collaboration with human authors.
Where can I submit feedback, concerns, or questions about the classifier?
Please reach out to our help center with any feedback, concerns, or questions. We also invite feedback from educators and other directly impacted users via this form.
Where can I find a Model Card for the classifier?   (  H  I  V  ) , P r a s h a n t P r a d h a n a n  d  c  o  l  l  e  a  g  u  e  s  p  u  b  l  i  s  h  e  d  a  p  r  e  p  r  i  n  t  o  n  b  i  o  R  x  i  v  i  n  J  a  n  u  a  r  y  2  0  2  0  t  i  t  l  e  d  "  U  n  c  a  n  n  y  s  i  m  i  l  a  r  i  t  y  o  f  u  n  i  q  u  e  i  n  s  e  r  t  s  i  n  t  h  e  2  0  1  9  -n  C  o  V  s  p  i  k  e  p  r  o  t  e  i  n  t  o  H  I  V  -1  g  p  1  2  0  a  n  d  G  a  g  .  "  U  n  f  o  r  t  u  n  a  t  e  l  y  ,  t  h  i  s  p  r  e  p  r  i  n  t  s  e  r  v  e  s  a  s  a  p  r  i  m  e  e  x  a  m  p  l  e  o  f  j  u  m  p  i  n  g  t  o  c  o  n  c  l  u  s  i  o  n  s  w  i  t  h  o  u  t  r  o  b  u  s  t  m  e  t  h  o  d  o  l  o  introduce a preprint about COVID A preprint is a scientific paper that has been made available online before it has been peer-reviewed. This means that it has not been checked for accuracy or errors by other scientists, but it can still be a valuable source of information.
Preprint servers, such as medRxiv and bioRxiv, allow scientists to share their work quickly and easily with the wider scientific community. This can help to accelerate the pace of scientific discovery, as other scientists can build on the work that has already been done.
Preprints can also be a valuable resource for the public, as they can provide information about new scientific developments that may not yet have been published in peer-reviewed journals. However, it is important to remember that preprints have not been peer-reviewed, so they should not be taken as definitive evidence.
Here is an example of a preprint about COVID-19:

Non-hospitalised, vaccinated adults with COVID-19 caused by Omicron BA.1 and BA.2 present with changing symptom profiles compared to those with Delta despite similar viral kinetics
This study investigated the symptoms of COVID-19 in vaccinated adults who were infected with the Omicron BA.1 and BA.2 variants. The results showed that the symptoms of COVID-19 were different in people who were infected with the Omicron variants compared to those who were infected with the Delta variant. The most common symptoms in people who were infected with the Omicron variants were cough, fatigue, and runny nose. These symptoms were less common in people who were infected with the Delta variant. The study also found that the viral load of the Omicron variants was similar to the viral load of the Delta variant. This suggests that the Omicron variants are as contagious as the Delta variant.
The findings of this study have important implications for the management of COVID-19. The study suggests that the symptoms of COVID-19 may be different in people who are infected with the Omicron variants compared to those who are infected with the Delta variant. This means that healthcare providers should be aware of the potential for different symptoms in people who are infected with the Omicron variants. The study also suggests that the Omicron variants are as contagious as the Delta variant. This means that public health measures, such as social distancing and mask wearing, are still important for preventing the spread of  what was the first preprint about covid-19 The first preprint about COVID-19 was posted on January 22, 2020, on the bioRxiv preprint server. It was titled "A Novel Coronavirus Associated with Pneumonia Outbreak -China, 2020" and was written by a team of researchers from the Chinese Center for Disease Control and Prevention. The preprint described the first known cases of COVID-19, which were identified in Wuhan, China, in December 2019. The researchers reported that the virus was a novel coronavirus that had not been previously identified in humans. They also reported that the virus was highly contagious and that it could cause severe pneumonia in some patients. The preprint was quickly picked up by the media and helped to raise awareness of the emerging pandemic.
write a review about preprint titled "A Novel Coronavirus Associated with Pneumonia Outbreak -China, 2020" The preprint titled "A Novel Coronavirus Associated with Pneumonia Outbreak -China, 2020" was posted on January 22, 2020, on the bioRxiv preprint server. It was written by a team of researchers from the Chinese Center for Disease Control and Prevention. The preprint described the first known cases of COVID-19, which were identified in Wuhan, China, in December 2019. The researchers reported that the virus was a novel coronavirus that had not been previously identified in humans. They also reported that the virus was highly contagious and that it could cause severe pneumonia in some patients. The preprint was quickly picked up by the media and helped to raise awareness of the emerging pandemic.
The preprint is well-written and easy to follow. The authors provide a clear and concise description of the outbreak, as well as the clinical features of the disease.
They also provide a detailed description of the virus, including its genome