Skip to main content

Table 1 Proposal for the future roadmap into peer review research

From: The limitations to our understanding of peer review

Topic

Topics recommended to be researched

Difficulty

Priority

 Role of editors in peer review

Justifications for editorial decisions

Medium

High

Factors that affect editorial quality, impartiality and their impact

Medium

High

How editors select reviewers

Medium

Medium

Impact of reviewer selection on relationships with editors and authors

Hard

Medium

Editorial competencies and motivations for decisions

Medium

High

Impact of decisions on epistemic diversity

Hard

High

Editorial conflicts of interest and relationships with other parties

Easy

High

Extent of editorial misconduct

Hard

High

Influence of reviewer recommendations on editorial decisions

Medium

Medium

Impact of editors’ careers on their scientific career

Medium

Low

 Role of reviewers in peer review

Factors that affect reviewer impartiality and their impact

Medium

High

Reviewer conflicts of interest and relationships with other parties

Easy

Medium

Reviewer competencies and motivations

Easy

High

Factors that affect inter-reviewer reliability

Medium

Medium

Extent of peer review misconduct

Medium

High

Expectations for reviewers

Easy

High

Impact of incentives for reviewers

Medium

Low

Conformation of reviewers to journal policies

Medium

Low

Extent to which anonymity is compromised

Hard

Medium

How do notions of expertise affect reviewer behaviour

Hard

Medium

Impact of reviewing on scientific careers of reviewers

Medium

Low

 Role of authors in peer review

Impact of author recommendations on reviews and reviewers

Medium

Medium

 Functionality and quality of peer review

What peer review actually is and does

Medium

High

How does peer review impact scientific discourse

Hard

High

Relationship between peer review and journal quality

Medium

Medium

Are there cases where peer review is redundant

Medium

Medium

Reproducibility of peer review

Hard

High

The development and impact of peer review standards

Medium

High

 Social and epistemic impacts of peer review

Homogeneity and centralisation of reviewer pools

Medium

High

Epistemic diversity of peer review

Hard

High

Impact of peer review on innovation or conservatism

Hard

High

Peer review as a vehicle for disseminating prestige

Hard

High

 Type of peer review

Factors influencing the choice of peer review type

Medium

High

Influence of peer review type on quality of review and potential misconduct

Medium

High

Do micro-publications impact reviewer engagement

Medium

Low

Is interactive peer review more effective

Medium

Medium

How have/will preprints impact peer review

Medium

Medium

Are overlay journals/services more effective

Medium

Medium

Which OPR services do researchers prefer

Easy

Medium

What measures can incentivise OPR

Medium

Medium

Researcher attitudes towards OPR

Easy

High

Researcher attitudes towards OPR for non-traditional outputs

Easy

Medium

The impact of OPR on participant diversity

Medium

High

The impact of blinding on biases and review quality

Medium

High

Impact of open review reports

Hard

High

Impact of review type on careers of reviewers

Medium

Medium

  1. The difficulty levels primarily refer to the relative ease of obtaining empirical data for study, should such data even exist. The priority levels relate to their perceived impact on the future of peer review. Both are subjective estimates of the authors