Table 3 Problems of open and anonymous peer review identified by the editors in chief of the 12 Social Sciences and Humanities journals

ANONYMIZED

Easy to abuse by editors

“I mean, obviously a position like mine can be abused in that way. Without question, it would be very easy to do so” (E1)

“ultimately, the decision is mine” (E4)

Difficult to credit reviewer

“one of the issues with blind peer review is, people never get credit for it. And increasingly, institutions are saying to faculty, tell us what you’re doing in your annual reviews. And we were getting requests to say, can you write a letter saying that I did the peer review” (E6)

Facilitates unwanted gatekeeping

“reviewers riding hobby horses about their own views” (E2)

“I’ve seen cases where it seems to me, even though you try to avoid it, that a reviewer has a disagreement or a grudge against something, just has a fixation on whatever. Has a indefensible opposition to something” (E8)

Lack of reviewer accountability

“reviewer is grinding an axe behind the veil of anonymity” (E8)

“the degree to which people have axes to grind or want to engage in some form of harassment or inappropriate behavior, a kind of single-sided process is probably not optimal at this point in time” (E3) *

Enables bad language use

“review is shameful or aggressive or unprofessional or unethical” (E5)

“the tone of the work has been more critical and constructive in a way that is not productive” (E4)

“newly minted academics are sometimes a little too severe ... and too square occasionally as well” (E11)

Difficult or impossible to carry out in practice

“Sometimes they say ‘I heard a paper at a conference two years ago and this looks like it’ and my response to that is: that’s fine” (E8)

“I’m increasingly impatient with the norms for anonymizing, which almost becomes a game for reviewers to then try and figure out” (E8)

Slows down communication

“Reviewer sends his/her comments to the editor who sends it over to the author who responds to the editor who decides whether s/he is able to evaluate or sends it back to the reviewer and then they send comments again. I don’t know if you could follow me” (E12)

        OPEN

Institutional discredit

“that academic articles are double blind peer reviewed, it’s sort of taken often blindly as a gold standard” (E11)

“how we can get the academy as a whole to value anything other than that kind of traditional double blind review?” (E3)

Does not protect reviewers

“the standard reason is that the reviewer whose identity is protected has a license to be more candid” (E2)

“I don’t know if it would be a very fair system to young researchers” (E10)

Does not protect authors

“[anonymization] protects the reviewer and the author from any personal issue that might arise” (E9)

“[only anonymity] will protect people from unfair biases by the reviewers” (E1)

Reviewers cannot review candidly

“It has issues with what you dare to do as a reviewer” (E1)

“to me the core is having it read by somebody who can be candid” (E8)

Facilitates biases

“the prestige of the author might blind the reviewers, or the fact that you’ve never heard of the person” (E8)

“it’s established as a research fact that e.g. women would have a greater chance of being published if they were going through a double blind peer review” (E1)

Hinders finding reviewers

“it would even reduce the willingness of reviewers to participate” (E12)

“it isn’t uncommon for me to go through maybe 12 declines before I find 2 reviewers. I’m not sure doing away with a blind review system’s the best” (E5)

Editorial challenges

“somebody who’s writing about queer studies may say ‘I think that a true peer is somebody who is queer’ but I will not ask somebody what their sexual orientation is” (E6)