Open Review and Open Science in The Social Sciences and Humanities: A "Double Blind" Gold Standard

Openings of research results, datasets, and scientic practices in general are currently being implemented across elds. Especially strongly data-driven areas like medicine are discussing publishing transparency too – in a context where open review formats now dominate. Social sciences and humanities (SSH), in turn, still rely on closed systems. In this study, we draw on 12 semi-structured interviews with chief editors of leading journals in SSH elds to better understand the transparencies of such review processes. We nd that, within SSH, ‘double blind’ peer review represents a gold standard that credible journals follow by default. However, the actual review processes of these journals are multi-stage and largely open with the authors’ names standardly visible to decision-making peers, with ‘double blind’ principles forming but part of it. We recommend journals to communicate the transparencies of their review in more detail, also and especially if they are ‘double blind’.

In comparison to the relatively recent emergence of organized open science approaches, the discussion and study of scienti c peer review (sometimes referred to as 'journalology') is old and widespread (e.g. Burnham 1990;Bornmann 2008). Already in the early 2000s, Gar eld (2004) found no less than 3.720 publications focused on the peer review process alone. Such a number of studies naturally covers a great range of subtopics, but the key question remains unsolved to date: how to verify scienti c quality in a way that is ethical, reliable, and possible to carry out in practice?
Whereas so-called open peer review processes have been a long-term solution in strongly data-driven elds like medicine, many SSH publications rely on the closed system of 'double blind' peer review. Despite these disciplinary tendencies, the debate about the Our study was set up to qualitatively chart the reasoning of SSH peer review processes and the transparencies behind them. With chief editors being expected to guarantee timely and smooth publication of journals -"they just want me to get the stuff on time, keep the trains running", as one interviewee put it -the policies they implement may be conservative and pragmatic, but what re ections and considerations take place under the surface? How transparently do SSH journals conceptualize and carry out their peer review processes? How do chief editors negotiate peer review policies, ethics, and pragmatics? And how do the chief editors situate their own powerful role in the process?
We answer these questions by means of 12 semi-structured interviews with chief editors of leading SSH academic journals. The article is divided in four sections. In the rst section we review relevant previous literature and establish a context. In the second section we introduce our qualitative method and data. The third section presents the results, and the fourth section is reserved for discussion.

Literature on Peer Review
We start by reviewing previous ndings related to peer review practices and their transparency elements in particular. Although the respective histories of different disciplines seem to have standardised various understandings of 'open peer review' and 'closed peer review' (blinded peer review), it is worth highlighting that both open and closed peer review practices have many forms. Ford's (2013) list of eight open peer review formats, based on 35 earlier studies, provides a useful illustration.
Signed review reveals the identity of the reviewer to the author. In disclosed review both the author and the reviewer are identi ed to each other. Editor-mediated review reveals the editor(s) as a decision-making or reviewing entity for the author. Transparentreview occurs in a public (typically online) environment with authors, reviewers, and editors all disclosed. Crowd-sourcedreview additionally allows community members to participate in the public review. Pre-publicationreview takes place during an extra preliminary review stage, as a work is evaluated in an open environment (e.g. preprint server) before entering the actual publication process. Postpublicationreview enables direct public commenting and criticism of a work after its publication on the journal platform. Synchronous review enables a work to be published in a dynamic format with continuous public review and editing. Ford's (2013) list is not exhaustive and lacks, for instance, one relatively common form of open review: a single blind review where the authors' identities are revealed to the reviewer, but not the other way around. Regardless, the above su ces to demonstrate that the notion of 'open review' is multifaceted and complex, and that there are several dimensions of openness that all potentially in uence the review process in diverse ways. These potential in uences have been studied respectively in detail too.
In a large-scale experiment carried out by Walsh and colleagues (2000) with the British Journal of Psychiatry, the scholars asked the peer reviewers of 408 manuscripts to sign their review reports and disclose their identities to the authors. A total of 76% agreed to sign, and their review reports were found to be of higher quality, more courteous, took longer to complete -and they were more likely to recommend publication. The scholars conclude: time commitment involved in reviewing might be too arduous for referees if the peer review process were opened up, especially when one considers the increased workload resulting from the loss of reviewers who refuse to sign their names. Although signing appears to make reviewers more likely to recommend publication and less likely to recommend rejection of papers, it is important to remember the role of the Editor in this process [moreover] junior reviewers may hinder their career prospects by criticising the work of powerful senior colleagues (Walsh et al. 2000).
While some studies have also reported less clear or no differences between signing and non-singing reviews (e.g. Godlee et al, 1998;van Rooyen et al, 1999), the current consensus points at the exact bene ts and de cits originally indicated by Walsh's group (see e.g. Shanahan & Olsen 2014; Moylan et al. 2014). These open review ndings have been studied further still, with various sub focuses such as reviewer bias. Manchikanti and colleagues (2015) cite evidence for at least six unique biases that have been found to corrupt the peer review process: con rmation bias (supporting existing beliefs), conservative bias (resistance against new methods/theories), bias against interdisciplinary research (applying single discipline criteria to multidisciplinary studies), publication bias (preference for positive results), bias of con icts of interest (judgment based on personal bene ts), and content-based bias (numerous biases produced by a subjective position -including 'ego bias' that makes reviewers favourable for work that cites the reviewer). The scholars entertain transparency via open peer review formats as an antidote to these issues; however, they also repeat the previously noted challenges that come along with open review: increased di culties related to recruiting reviewers and having honest criticism. Moreover, in the case of public open review, the review process can be complicated with commentators or critics who may not qualify as experts.
In addition to the above, a mutual problem that concerns both open and closed peer review formats is what Gorman (2007) has coined the "Oppenheim effect." This refers to a certain known-author syndrome according to which some editors and/or peer reviewers tend to provide established scholars with special treatment ("We both know that we are going to publish it anyway"). Recently, this theoretical concept has received strong empirical support, as Tomkins and associates (2017) had half a thousand papers in an annual computer science conference peer reviewed by two experts who were disclosed with the authors' names and two other experts who did not see the authors' names (i.e. four reviewers per paper). Peer reviewers were much more likely to accept papers from famous authors and top institutions, compared with their double blind counterparts. The same study also found a statistically signi cant bias against women authors -one more bias that has been known to handicap academia for decades (see Wennerås & Wold 1997).
In summary, the academic world -represented by 28,094 scienti c journals that publish some 2 million scienti c articles annually (Ware and Mabe 2015) -has come to employ multiple diverse review strategies in their publication processes in order to assure quality and ght biases. These peer review strategies have several transparency elements that surface at different stages of the publication process. Depending on the aspect, such transparencies (open or closed) resonate with various sought-after positives and to-be-avoided negatives. The general consensus is that fully closed peer practices are good for providing reviewers a safe anonymous space to criticize candidly, which also facilitates the speed of the review process. Fully open review processes, in turn, are generally good at motivating the reviewers and holding them accountable, thus increasing the overall quality of feedback and making biases as well as con icts of interest visible.

Social Sciences and Humanities
The previously cited literature on peer review transparency and open science has been published and discussed mainly in data-driven academic domains such as medicine and natural sciences. This makes sense, as many of the open science principles derive from issues of closed or limited datasets, and the peer review of such studies can be part of both the problem and the solution. Moreover, when biases like con icts of interest potentially have a strong direct clinical impact (e.g. via medicine testing and production), open peer review practices can be used to secure transparency and accountability. But what about elds such as those of the humanities, which do not necessarily rely on empirical samples?
The challenges in developing universally applicable procedures for open research and review appear to be at least partially rooted in conceptual differences such as the meaning of originality. For instance, Guetzkow and colleagues (2004) conducted a qualitative interview study with peer reviewers from diverse elds and found that whereas natural sciences conceive of originality very narrowly as the "production of new ndings and theories", for social sciences and humanities peer reviewers, originality could also be associated to novel elements in the approach, method, data, topic, or simply the fact that the research was linked to an understudied area. In addition, whereas topics such as research replication can, to some degree, address the humanities too (see Peels & Bouter 2018), it is evident that not all research designs can be based on one and the same reproducibility principle -and their peer review should not expect that either.
In the light of the above, there are good reasons to believe that social sciences and humanities maintain and evolve with a relatively different set of principles and strategies compared to, for instance, physical and life sciences. These differences, we expect, are re ected by the transparencies of the peer review processes, which journals in these elds/disciplines employ. Accordingly, this interview study was set up to answer the below three research questions. Since the research questions are explorative, we avoid exact hypotheses. However, before data collection, we did entertain and preregister some expected trends (https://osf.io/wjztp). Since the goal of this study was not to test or falsify these trends but to answer our noncon rmatory research questions, we do not discuss the expectations but rather aim at lling out the blanks of the SSH journal review processes in order to explore the issues of transparency that may lie within.

Method And Materials
Due to our research questions being explorative and qualitative, we chose semi-structured interviews with chief editors of leading SSH journals as a method. We use the term 'leading journals' here to indicate that our focus is on publications that are respected in their elds and recognized as quality academic platforms (e.g. excluding so-called 'predatory journals'). We did not choose the journals based on metrics such as impact factor, albeit many of the interviewed journals did rank very high according to these gures. We considered it important to include journals with not only different scholarly domains, but different pro les and sizes: the smallest journal from the most narrowly de ned eld publishes only around 15 articles per year, whereas the three biggest journals all publish more than ve times this volume. These boundaries give our anonymous materials an identity and save the reader from speculating with the relevance of interviewee responses.
The study followed the Finnish National Board on Research Integrity guidelines, according to which the local ethics committee should not be applied for a statement because the methods and themes do not represent risks that are higher than those that would occur in everyday interaction. Before reaching out for participants, however, we created an interview structure and work plan that were stored in the Open Science Framework on June 25, 2020 (https://osf.io/wjztp) The idea was that registering questions and general objectives of the research beforehand would build up both credibility as well as trust, and thus facilitate approaching busy chief editors who presumably receive dozens of emails and contact requests daily. That said, further details such as the complete question list and outcome expectations were not disclosed but embargoed, as we did not want to in uence the interviewees' responses.
The goal was to balance the 12 interviews so that diverse geographical regions, content orientations, and chief editor genders would be represented by the participating journals -with the caveat that each journal should be part of the SSH scienti c domain. Notably, all interviewed chief editors characterized their journal as multi-or interdisciplinary, even if their journal has a focus on or origin in a singular discipline. We created a list of journals that we would approach by personal email one by one so that when our invitation would be ignored or rejected, a new journal with similar representation was selected. An open call for participation was also distributed in social media. We had no trouble nding volunteers, as chief editors were generally interested in discussing the topic and willing to make time in their schedules within a short notice. Some journals were represented by multiple chief editors. We do not disclose further details about the editors or their journals in order to protect their privacy. Many of the interviewees explicitly wanted to remain anonymous.
The interviews were carried out by using Zoom remote communication software. Only the audio part of the video call was stored for analysis. A protected university cloud service was used for storing the data. The participants were informed of research details via email before the interview and in more detail at the beginning of the interview. Informed consent was collected in written PDF format.
Both researchers participated in the interviews equally. The average interview length was 70 minutes, the shortest one being 55 minutes and the longest one 90 minutes. The preregistered interview structure was followed systematically in each interview; however, if and when the interviewees opened new relevant themes that we could not foresee, follow-up questions were posed from outside the plan.
The interviews were transcribed into text format by using a General Data Protection Regulation compliant service (Konch.ai). The text was proofread with the help of a research assistant, as approximately 10% of the automated transcription was unreadable. The proofread text les were uploaded to a university-supported Atlas.ti system, which we used for software-based text analysis. The analysis was carried out by the rst author exploratively coding all text without a predesigned coding scheme (533 codes overall) and the second author reading the data. Coding reliability was tested by, rst, having an external assistant unitize 17% of the data (see Hodson 1999). Since the assistant did not have experience of the academic peer review, we did not pursue high numeric interrater reliability but negotiated a coding agreement (see Campbell et al 2013). After merging overlapping codes and coding negotiation, 510 codes remained.
We did not pursue themes or a manual for new coding rounds, but discrepancies regarding the codes were discussed and solved by the research team, i.e. reconciling differences via consensus (Syed & Nelson 2015). Because both researchers participated in the development of the interview structure and actual interviews, analytical agreement was reached in single one-day session. This produced 11 code families, which included descriptive information (e.g., how does the peer review process operate) and non-descriptive information (e.g., what do the editors personally think about the peer review process). These code families are: Authors: comments related to the authors who submit to the journal.
Decision: comments related to manuscript publication decisions.
Editing: comments related to the work of journal editing in general.
Editors: comments related to the chief editor's personal beliefs or role.
Interview: comments related to the interview in question.
Journal: comments related to the chief editor's journal.
Open science: comments related to open science in general.
Publisher: comments related to the publisher of the chief editor's journal.
Review: comments related to the review process in the chief editor's journal.
Reviewers: comments related to external peer reviewers of the chief editor's journal.
Science: other comments related to the scienti c world and its developments.
The second author then examined the data in the light of the families, not systematically re-coding but selecting relevant sections and units that related to our three research questions. Based on this we report the ndings in two overlapping parts: a. Descriptive: in this part we describe the actual journal peer review processes and the transparency issues related to it. This includes mainly the content of the families Decision, Editing, Journal, Review, and Reviewers.
b. Non-descriptive: in this part we move to the editors' personal views. This includes mainly the content of the families Authors, Editors, Open Science, Publisher, Science. The family Interview will not be discussed in this article.
Last, to address an issue that some readers will likely consider relevant: we do not numerically present the number of instances of speci c coded events. We acknowledge that a quantitative approach to similar qualitative data can be fruitful; however, in such case the target journals should have been selected with even narrower inclusion criteria for them to be clearly representative (of a selected small sub eld that could be well represented by 12 journals). Again, the present goals are not con rmatory, in which case qualitative analysis of the qualitative data, we assert, is more tting. Once more, due to the privacy concerns expressed by some interviewees, we do not share our data via an open repository. Parts of speci c interest can be made available by request to the corresponding author, however.

How do the editors employ open science and peer review in practice?
In this section, we outline the SSH review processes, as described by the interviewees. Although some experienced readers might consider this section not adding new knowledge to the eld, we consider it important to provide a detailed description in order to empirically illustrate what many believe these processes are-we were not able to nd previous studies that would have systematically analysed SSH review processes-moreover, this this baseline will also enable us to later pinpoint exceptions and re ect on the chief editors' personal views.
All interviewed journals employed multiple stages for reviewing their submissions. We start by summing up the general protocol that was similar in all journals: 1. Screening. When submission arrives at a journal, editorial body X (sometimes with Y) will screen the submission and decide if it will be reviewed. Negative decisions lead to desk rejection.
2. External review. Based on the screening and pre-review, X and/or Y look for appropriate external reviewers Z and invite them to review the submission.
3. Revisions. Based on the combined feedback of X, Y, and Z, the authors are asked to revise their submission and resubmit. Steps 2 and 3 are repeated until the submission is accepted or rejected.
4. Post-review. Accepted submissions are typically followed by copyediting and various debrie ng procedures related to the review process.

Screening
Sometimes the chief editors were responsible for all screening themselves, but often these decisions were informed by or entirely delegated to other parties such as assisting editors, associate editors, editorial boards, or the managing editor. The amount of personnel involved varied dramatically, from a sole editor to teams with more than ten people. The number of desk rejections at the end of this process varied even more, ranging from as low as 15% to over 85% of all submitted manuscripts. The average overall rejection rate lies at about 70%, with the lowest rejection rate at 33% and the highest at 95%.
Five unique reasons were identi ed as causing a desk reject: lack of t with journal scope, poor overall quality, ignoring relevant literature, narrowness or low impact, and instances where too many similar manuscripts were already in review or published. In one exceptional case, the chief editor noted almost all desk rejections to derive from the fact that the journal's name was similar to other journals with different pro les, which made authors constantly submit manuscripts that address out-of-scope topics.
The screening processes had combinations of closed and open elements. In the most common scenario, the chief editor and/or staff would screen the manuscript with the authors' names visible. In other words, the most common methods in the screening phase were disclosed or single-blind review where the reviewer saw the authors but not always the other way around, i.e. the authors might not know if they had been (desk) accepted or rejected by a chief editor, assistant editor, associate editor, editorial board, managing editor, or someone else. The interviewees' felt that double sided anonymity at this stage would be both impossible and impractical. One interviewee, for instance, was ready to let assistants make desk decisions, but in a way that would allow the chief editor to supervise the process in an open format: That has to be open. For example, you would never assign a reviewer who's in the same department as the author. So you need to know where the other is. You also want to avoid assigning a reviewer who was likely to have been the author's advisor. So that really can't be blind, or you'll end up just sending things to coauthors or work colleagues. I am much less concerned with the anonymity of authors than the anonymity of reviewers.
One journal practices a system that inverses the dynamic of the screening stage: a signi cant proportion of submissions are rst informally suggested to the chief editor, who then seeks input from experts in the eld to support authors in developing proposals. Only then proposals are submitted and subjected to a double blind peer review, which then has a very high acceptance rate. To this chief editor, nurturing promising submissions from the start strengthens the quality of the journal while guaranteeing a steady stream of high-quality publications -a desirable strategy because "we are in the business of publishing, not punishing." This feed-forward form of curation creates less need for critical feedback (and rejection) in the later review stages: "I like to think that we do a good job prior to the submission, so the author can con dently send an article and receive a positive, constructive feedback." The described process reminds one of the "registered report" article format (Chambers 2017), which was not explicitly used by any of the 12 SSH journals (not even those that were psychologically inclined).
In some journals, screening was supplemented with full editorial review. In these cases, one or more persons of the editorial staff read the entire manuscript and provided (signed or anonymous) decision feedback before moving to external review: Sometimes we get articles from [country] that are 1.5 page -then it's a desk rejection. But if it's a full article, the editorial manager will assign it to one of the other editors. And then it's the editor's task to go through the article.
Chief editors have a look at the rst round, we can then give a desk reject right away. But if it goes further, then two members of the editorial board read the text and assess whether it's good to go to review or not. We might not desk reject, but send 'OK, didn't go to review yet, but if you do these changes, we'll reconsider.' Summary: Screening is carried out by one or more editors. The full manuscript may be read or not, and feedback is optional. Since this phase is editorial, the main transparency issue is whether the authors' know who contributes to the desk decision, and based on what criteria.

External review
All editors participating in the study characterized the external review phase as crucial. Here we nd the strictest adherence to double blinded review practices. Virtually all interviewees supported the view that double blind external peer review was considered a standard by the academic community as a whole. With one exception, all journals employed strict double blind external peer review, with between two and four reviewers per submission for original submissions. Other formats, e.g. book reviews, are often treated differently.
The journals had several means for recruiting reviewers. This could be done by the chief editor or another staff member, or as a collective effort. Higher volume journals had reviewer databases with hundreds of potential experts; smaller journals would mainly recruit via the personal networks of the editorial staff. All editors agreed that academic quali cations were the primary selector, supported by a whole range of other criteria.
It's because they're considered experts. It's because sometimes we know them personally. It is because they are more committed, because they're on the board. And it is also because they are familiar with the journal, the direction of the journal, the expectations and the level of quality of the journal.
Altogether, nine unique criteria were considered relevant in choosing the reviewers: age ("the issue is how do you nd the really bright young people who are doing really thoughtful work"), biases ("we try to give papers to reviewers that are perhaps on different sides of a divide"), commitment ("because you know they are more committed"), diverse perspectives ("we certainly try to nd both reviewers from close to the paper's discipline and also from outside"), distance ("connection or lack of connection to the author"), expertise (this was mentioned by all interviewees in many ways), nationality ("you need someone who will pick up on the local nuances"), personal preferences ("I would be hesitant to send something to somebody who I didn't feel I knew"), and recommendations ("we also make very good use of the suggestions we receive"). Additionally, two journals were proud to have 'harsh' or 'super' reviewers who could be used for the most challenging tasks: I try my best to balance reviewers and also to sometimes avoid using harsh reviewers on material that I personally don't think is the strongest. So I might reserve my harshest reviewers for things that I myself nd to be of very high quality.
And then there are my sort of super reviewers. They are people whom I have just learned to trust. Who will rst of all do it if they say they'll do it, but also are good at sort of sifting through, reading and picking things up. These are often people who have been journal editors or just have a track record. And if I had a structure of associate editors, these are people who would be associate editors.
The role of external review was somewhat polarized among the journals. On one side, certain journals consider the external review to be advisory instead of decision-making. The editors of these journals de ne their roles rather as curatorial or akin to editors in book publishing. Beyond assuring high quality publications, they see their responsibility in stimulating innovative impulses to developing elds and helping authors bring their concepts to full fruition in a collaborative process.
we don't follow the peer review slavishly, but then again, the issue of not recognizing what the reviewer was saying has not really arisen -we end up reading every single piece submitted and everything, and then every piece that is published in the journals, we go over each one of us, more than once.
At the other extreme, however, chief editors were utterly clear about operating as nothing but 'mediators' between reviewers and the reviewed submissions. These journals pursued rst and foremost the assurance of scienti c quality, and in this picture, the external blind reviewers served as 'objective' measures.
If I'm in doubt as the editor, I will send it out to reviewers who can then make that decision for me. So the idea is that not one person, in cases of doubt, will make the decision. I don't make the decision, the reviewers do.
I tend to view myself as an umpire. I'm not quali ed to make these decisions. My job really is to try and ensure that the reviewers are appropriate and that the reviews are fair, to the extent that I can. And I'm sure there are mistakes.
It is worth noting that sometimes editorial review was merged with the external review. In these scenarios, internal editorial reports could be delivered based on either double or single blinded principles. None of the chief editors expressed concerns or policies regarding the disclosure of these internal processes, but we also did not inquire directly about them.
We have an editorial board. We have this board to kind of draw on in cases if we need a second opinion or third reviewer. But then they would also work as a blinded reviewer.
Sometimes it gets complicated and then the person I ask will be somebody on our editorial board who has been fairly helpful in the past, because you're asking them basically to follow through the entire editorial history of this. I'll send them the whole thing, like, 'here's the history, here are the reviews, what do you think.' Summary: For research articles, the general practice of disclosed external review was a rare outlier in our data. A consensus among the chief editors was that double blind review was the fairest and most objective means to carry out the process. Sometimes the process included internal review procedures, which could be non-blinded.

Revisions
With few exceptions, all research that had been published in the interviewed journals went through revision ("I don't think I've ever experienced a situation where a text would've been good to go"). The revision process was either conducted between authors and editorial bodies, or via additional external (blind) reviews after the rst revision. One editor explicitly stated that a manuscript which had not su ciently improved after a revision would not bene t from further revisions and was rejected. Other editors described an iterative process that could go through as many as eight revisions. A common method was to ask the same external (blind) reviewers to review the manuscript a second or more times.
Despite collectively following and agreeing on the previously discussed bene ts of closed external review, several editors voiced doubts regarding this phase in particular. These doubts derive from the nature of the revision process, namely, these interviewees felt that initial blind processes would bene t from increased transparency after the necessary 'gatekeeping' had been cleared out: [blind review] is not the gold standard that people often perceive it to be, and in fact, often what's far more useful is a sort of semicollaborative editorial process that follows after double blind peer review. That's where the improvements are really made. This is just a kind of initial gatekeeping, and sometimes it's useful and sometimes tokenistic.
In cases such as the above where chief editors expressed a personal liking for disclosing external reviews or revisions, they systematically cited institutional requirements that would not index their journal as a proper scienti c journal without 'blindness' involved. For instance: A completely open process, I think, is far more plausible. But then the issue is also that I've been on review committees where people have said, well, if this has not gone through a double blind review, it doesn't count as much. So I still think there's a huge hurdle to overcome in terms of how we can get the academy as a whole to value anything other than that kind of traditional double blind review.
Summary: All journals include a revision stage, which is internal, external, or a combination thereof. The transparencies in the revision process generally follow those of earlier stages, yet with increased open editorial input.

Post-review
In the post-review phase, most journals draw strongly on their editorial assistants for technical quality assurance such as checking the integrity and completeness of the citations.
In this phase, pragmatic differences, primarily relating to the ways in which the journals are nanced, emerge. The journals that are primarily or exclusively nanced by universities report dwindling subsidies, whereas journals with strong ties to associations or publishers appeared more stable. Relations to publishing houses were characterized unanimously as harmonious and unproblematic, except for some unease in cases of journals facing an upcoming periodic review of viability. One chief editor, addressing manuscript transformations into PDFs with DOIs, positively admits: I think the press handles all that, I've not been involved in any of that ... We've discussed occasionally whether [volume number should increase]. Other than that -cover design, occasional changes, that's always collaborative.
Only one journal employed technical means to assure quality control beyond peer review. They had recently started using software for an originality check, which would make sure that plagiation of all sorts could be detected before nal publishing ("now we run all papers through a system to see a possible relapse"). Transparency-wise, however, perhaps the most critical question at this phase was whether author or reviewer identities could be opened after a positive publication decision.
When I send an acceptance notice and say 'dear so-and-so we've accepted your article' I send that to the reviewers as well. It seems to me at that point I can include the name of the author. I mean, we've made the decision. But sort of automatically I take it out. But I keep thinking, why am I taking out the name of the author?
Finally, some chief editors perceived the articles that they publish in the larger continuum of scienti c evolution. Namely, the peer review of a publication is not something that takes place in or by the journal alone; rather, journal review is one evaluative event in an article's life, which continues post-publication as peers read and review it in academic forums: So if the paper is not good, it will be lost in history, it won't get citations. If it's really in uential but problematic, there will be some dialogue, there will be some criticism, there will be some contrasting results presented and so on.
Summary: The post-review phase consists mainly of technical tasks such as copyediting. Some interviewees considered a positive publication decision as a potential reason for disclosing otherwise closed identities. Public review of articles starts after publication.

What are the editors' personal opinions about open science and peer review?
In this section we move more explicitly toward the chief editors' subjective perspectives concerning the review process. After that, we discuss the results via our research questions.

Peer Review
A majority of the editors expressed, if not misgivings, then at least doubts about the universal viability of double blind peer review. The same chief editors likewise considered fully or partially disclosed open reviews problematic. For both sides, we identi ed seven unique but connected problems, which are presented in Table 1. In addition to these format-speci c problems, ve chief editors mentioned the general issue of external peer reviewers sometimes being too hasty and either providing little or no feedback to the authors. In these instances, the review reports were usually discarded and new reviewers were recruited.
Our space does not allow discussing all 14 identi ed problems respectively, but it is worth dovetailing some selected concerns. First, we highlight the notion listed as 'institutional discredit', which some chief editors considered a key obstruction that makes even thinking about moving to open peer review not worth the time. Despite the fact that several world-leading journals such as those in the Natureseries support open review formats, many interviewees recognized the double blind as a 'gold standard' that they could not move away from without sacri cing credibility. The pedigree of these standards appeared important particularly for new burgeoning journals: We did not want to be innovative or be radical in any way. We wanted to have a journal that would be regarded and identi ed as a very standard traditional scienti c scholarly journal, because we wanted to establish [our sub eld] as a typical, solid, and traditional.
Journals with a narrower regional or thematic focus likewise had a reason of their own for keeping the peer review process closed.
Drawing on smaller numbers of authors and reviewers, anonymization was perceived essential to reduce con icts of interest when virtually all experts are acquainted. One editor argued that, particularly in small, closely knit elds, peer-to-peer accountability based on disclosure of names might quickly devolve into "interpersonal as well as disciplinary con icts." The same editor mentioned a frequent need to revise the language of reviewers, because their observations seemed addressed to the editors rather than the authors, and formulated in a "language that would be shared among friends", i.e. not always respectful. "So of course, I remove that -it's unnecessary and insulting, and I rephrase it." In other words, the respectful tone of the reviewers that was characterized as central for their journal's review process is, at least in some cases, the result of an editorial process. Three other journals reported a similar policy, according to which a respectful tone was maintained by systematic review report editing.
For some chief editors, the author's identity was also relevant in the decision making process. Against the majority who considered anonymous double blind processes fair and objective, a minority felt that genuine fairness meant evaluating each submission in the context of the author's current career stage and background: We get senior scholars, well known people, and we get graduate students. And I think that work is going to be assessed in part in relationship to the identity of the author. And so I think it's important for me, who's going to be making some decisions about that, to know that. Now that then also means that I have to be conscious, as conscious as I can about my biases and so on, and I try to do that.
This was also the only one of our journals in which disclosed review practices were strongly present. At the other extreme, chief editors considered all transparency unethical and pursued complete anonymity at the screening phase too: We wanted to have a double blind review process, because that is, as far as one can tell, the most fair way of selecting what gets published. When I screen a manuscript, I also have them screened without any information about the authors. There might be a surname attached, but I normally will not look up who that person is before I screen it.
Considering that all but one of the 12 journals were running completely closed double blind peer review processes, a common problem was that sometimes peer reviewers wished to disclose their identities to the authors. When asked, three journals reported such instances to be somehow linked to the Peer Reviewers' Openness Initiative. The chief editors handled these requests in opposing ways: either allowing the reviewers to sign their reports, or denying disclosure. One chief editor felt that this transparency would challenge their values: I've pushed back on that, in the sense of we've stated very clearly that our journal is a double blind peer review journal. When they submit their work, that is the practice that we're going to function under. I believe that if there is a desire on the part of the reviewer to want to make his or her name known to the author, that's actually pushing on a value system that the author may not agree.
In line with the above, a majority of the editors described the conduct of their own journal as dependent on the context or conventions of their respective academic domain, suggesting that different elds needed different approaches to reviewing. For instance: Ultimately we all want to publish the best of possible articles. So if one way works for an editorial board, ne. If another way works for a different journal, ne. In the end, people will read the nal works published that contribute to scholarship. All the roads lead to Rome as far as I'm concerned.
One chief editor, representing the humanities, explicitly called out the entire eld lagging behind and lacking proper peer review to begin with. According to them, "there's too little blind review or even peer review in humanities ... we've been leaning on this curatorial model way too much, which also gives editors way too much power -that's something where we have a lot to learn from other elds." Another chief editor, speaking on behalf of communication studies, diagnosed open research practices as a reaction to bad research practices in other elds, and since "we have not encountered that type of di culty, we can go about having a real discussion about what are the upsides and downsides." Meanwhile, one interviewee felt that openness, as such, was not considered relevant within the SSH: authors seem to have very little interest in open science. I've also spent some time for an open data initiative and I'm surprised -the extent to which I don't see very many people actually interested. I just see a shrug of the shoulders. A kind of 'Eh, this doesn't really apply to us, why would I want to do this? It's just more work, it's more effort.' Almost every journal also supported publication formats that were not peer reviewed or peer reviewed differently. The problems with these formats, particularly for those in early career stages, were connected to the fact that scholars have to live up to quality criteria about their publication styles and venues, with double blind peer review as a universal criterium.
To create something that isn't going to provide people with a line they can put on their CV, under peer reviewed journal publication, that's a very tough thing to ask of people. And this is, to me, an incredible frustration because doing things like creating a podcast or blogging -or any number of things that we can think of that have happened over the last 15 years that would be valuable contributions to the scholarly conversation -are not going to count.
Several voices echoed the above reality contributing to their lack of motivation to create something new, or anything that would not be traditional peer review.

Reviewer Management
The chief editors widely agreed on some central issues, the most fundamental of them being an unanimous satisfaction with the work of their peer reviewers, describing this collective contribution in downright enthusiastic terms. Three journals estimated the prevalence of 'bad' review reports numerically, saying them to be but 1%-2% of all received reports. In order to maintain high quality in their review processes and make sure that the review system would work in the future as well, the interviewees disclosed systematic and nonsystematic means by which they keep track of both internal and external reviewers.
we're, of course, monitoring quality. We can tell reviewers that we don't nd the quality of their review high enough. We will quietly not use those reviewers if they over time display signs of lacking diligence. It's a very simple method and it also works well.
We actually have this internal system, and most of us remember to rate the reviewer. When you go out and look for a reviewer, if you look in our system, you would see if one of the other editors had rated the reviewer very low.
The concept of quality in the above and other cases was rarely a matter of content alone, but also reliability and speed. Reviewers who did not respect deadlines or were di cult to communicate with could likewise be classi ed bad quality, even though their feedback was appropriate.
Related to the transparency of the peer review process, the interviewees listed miscellaneous elements that they considered topical. For instance, one chief editor talked about marking submission, revision, and acceptance dates in the nal article as a feature that can remove doubts about the process, however, it may also turn against the journal: I see on some journals now the notation 'manuscript submitted on such and such a date, accepted such and such a date, published such and such a date'. I don't think that's a very revealing statistic or data point for a journal like ours where, in my opinion, so much of that timeline is outside my control. But it's part of greater transparency.
We should also add that none of the journals provided nancial compensation for their external peer reviewers who do all such work as free service. In general, nding external reviewers was one of the core challenges for journal editing, as multiple editors noted how it would not be unusual to ask up to 15 people to review before nding two who would agree. The trend was occasionally described as increasing ("there's a momentum building up"), in which case the chief editors felt unequipped to solve the problem due to lacking means for compensating the review work that they still needed to run the journal: "How do you reward reviewers? Because this whole gift economy depends on reviewers' unpaid labor." Systems like Publons were mentioned as possible solutions, with the caveat that they would not remove the original problem of volunteer work.
Some journals actively pursued editorial diversity, for instance, by carefully managing the board with ethnicity, gender, and regions in mind. A few expressed surprise that such diversity should even be considered. In the external review process, the de ning diversity concerns were about disciplinary or methodological domains, i.e. many chief editors felt that having 'both sides of the coin' would bene t the review, especially in polarized topics.

Publication Metrics
A further complicating matter in the picture were journal metrics, which for some served as a means to self-assess their own performance, yet at the other end of the spectrum, such numerical values were considered awed and irrelevant. A half of the chief editors would indicate interest in the statistics, usually provided by the publisher. Only those whose journals were up for review through publishers felt that clicks, subscriptions, and other metrics mattered in practice. When asked speci cally about the impact factor, replies ranged from moderate interest ("it's important for the publisher for sure -but it's also important for us") to complete de ance ("fuck the impact factor").
Although all chief editors, except one, professed to be aware of the impact factor among similar metrics, there were no attempts at in uencing journal policy from publishers or a liated academic associations. On the other hand, the editors often admitted being pleased about their journal's success, and since this success was typically validated by high journal rankings and peer recognition within the eld, some felt that careful self-re ection was needed when assessing potential 'high impact' manuscripts.
The thing goes back to the idea of rankings and stuff. So maybe I should publish more canonical stuff if I want to get higher. I don't want to think that way, but I know that. So how is that going to affect my practice?
Again, the question was conventionally tied to the reality of academic careers and work. Even if the editors did not consider the impact factor relevant, many of their submitters did. In this way, the metrics had a direct impact on the journal's pro le and prestige, and whenever such metrics were not disclosed, the editors could receive requests to make them transparent.
We occasionally get a request from an author for what our journal impact factor is. That typically comes up when an author is up for review, promotion or tenure. And I've written letters back to them saying it's not our job to participate in the tenure review.
To sum, the chief editors' personal viewpoints regarding the review process, its management, and related journal metrics were occasionally in con ict with the values they acknowledged or how their journal operated in practice. By and large, the chief editors were aware of this and often actively pursued solutions, which nonetheless were di cult to implement.

Discussion
RQ1: How do highly ranked SSH journals perceive open peer review processes and do these perceptions materialize in the actual processes they employ?
Our ndings show a mixed pool of perspectives that highlight pros and cons equally in both open and closed peer review practices. Whereas the review process in all journals consisted of multiple stages from screening and review proper to revisions and post-review, most of the concerns centered on the externally recruited reviewers in a context that was fully closed, i.e. double blind (one partial exception). To wit, the chief editors connected several bene ts and problems to review transparency, but applied transparent strategies mainly with regards to their own editorial work in which being able to identify the authors supposedly made the review more reliable and fair.
It would be incorrect to present a simple or single reason for why the journals relied on the double blind peer review processregardless of acknowledging numerous bene ts in the open peer review forms that have been in dominant use in other elds beyond the SSH. Whereas the majority of the chief editors considered the double blind approach a 'gold standard' and had never given a serious thought to any other review formats, sometimes such opinions were also accompanied by a strong belief that open peer review could never be carried out without losing institutional support and academic credibility. That said, we may also speculate with the possibility that the SSH domain, de ned by its humanistic and social roots, might just value the protective bene ts of closed review more than other scienti c elds and disciplines do.
RQ2: How do the chief editors position their journal's review and publication practice between policy, ethics, and pragmatism?
Given recent discussion of research ethics and calls for stricter fact checking in academic publications, questions of quality assurance were a key area highlighted in the interviews. There was a consensus that the existing (closed) structures and methods of their journals were tried and true, adequate means, given the various levels of oversight already in place.
The practices of peer review, reviewer selection, and editorial oversight were contextualized by editors as a part of an overarching system. In this system, journals play a small part for the articles they publish. Studies will have been vetted by university ethics boards and national or international grant givers. The journal then assesses through peer review the relevance and innovation of the research, while at the same time itself being subjected to oversight by a publisher or other funder, mostly based on quantitative success criteria like the impact factor, number of downloads etc. Emerging binding guidelines like Plan-S and best-practice recommendations by institutions such as the Committee on Publishing Ethics (COPE) complicate the 'system' still.
Within the above structures, especially the chief editors of our interdisciplinary journals leaning toward qualitative methods expressed satisfaction with the means and the degree of quality assurance done in their eld, and particularly their own publication. One journal had begun using a software tool to verify integrity and originality, yet the chief editor stressed that the results required cautious interpretation because of a high amount of false positive results. Ultimately though, such structures were deemed safeguards of good scienti c standards, and will not protect against hoaxes or conscious fraud. By and large, our study witnessed a view that editorial work and peer review serve not merely research publishing, but in the process elevate the quality of said research: "I think if we didn't believe that, we wouldn't do it -there's something valuable about vetting things." Before conducting the interviews (https://osf.io/wjztp), we expected smaller and younger publications to adhere more strongly to changing standards in publication practices and enforce open practices more vehemently than more established journals. We were wrong. As mentioned before, only one journal clearly practiced an open form of review, and this journal was not young, but an average age in our journal sample and does not adhere to other open science principles (also representing paywalled access).
RQ3: How do the chief editors situate their own powerful role in the peer review process and in what way is that role negotiating open science principles of the peer review process?
Few journals aimed at regulating editorial power. Several of our interviewees are founding members of their journal and perceive a strong identity between themselves and the journal. Only in two cases, interviewees reported xed chief editorial tenures, whereas multiple had been in their function for one or more decades. One of the former said: "this is the association's journal and certainly isn't mine -I just happen to be granted this opportunity to have a certain role at the time." The same editor would also highlight that for the duration of their tenure as a chief editor, they had the nal word in what gets published, potentially overruling external reviewers and associate editors, "so ultimately, the decision is mine." In similar ways, almost all editors expressed in detail their awareness of their power, sometimes with strategies for managing it.
Our ndings indicate that SSH editors are aware of their power and try to moderate it through (often simple) self-policies, such as "I publish things I disagree with but I think are really important." Journals with large editorial boards and shared editorial duties may have ne-tuned means for power management that our interviews were not able to chart; future research may shed more light on those.
We must stress, however, that several chief editors were currently in a process of introducing or considering new open science principles (like data sharing) to their journals, and if necessary at some future time, there seem to be no technical limitations for extending such rework to the peer review process too.

Conclusions
The pragmatics of organizing scienti c journal peer review are di cult to completely align with the professional ethics generally striven for. If nothing else, then the necessity of selecting quali ed reviewers that will not have an obvious con ict of interest is a step in the process practiced by all journals we spoke to, and one that can be assumed to be, in some form, practiced universally: "Their papers do go to referees, but then editors choose referees" (Macdonald & Kam 2007). Even though the issue itself is universal, our interviews showed that there are different strategies in place for addressing it.
Reviewer selection can be shared, distributed, and delegated, editors can act as collectives, and individuals can rotate out of their position of power periodically, all in order to safeguard the integrity of the editorial process through infrastructure. Other journals implement none of these procedures and instead entrust their review process to a seasoned editor-in-chief whose experience, reputation, and integrity assure quality in a very different -some might say more traditional -manner.
Furthermore, journals operate with bodies of reviewers ranging from a small circle of xed members to an ever-growing, systematically curated network, often correlating with the volume of submissions to be processed. One of the small journals in our study operates with a xed board of two dozen reviewing members, while the largest holds a database of 15.000 external reviewers. In all cases, editors were con dent that the constituency of their authors and readership is satis ed with this organization, often arguing with widely held practices of their eld. Based on our study, we cannot identify any method as better or worse for dealing with possible biases and other problems with reviewers or their selection (see Table 1).
With reference to fact that many chief editors considered blind review as something that they need, not least to meet institutional gold standards, our study yields evidence for the SSH double blind process typically having a) the majority of submissions accepted or rejected non-blindly via editorial desk decisions, b) accepted or rejected non-blindly by reviewing editorial staff in post-screening phases, and c) accepted or rejected non-blindly by a combination of editors, or the chief editor, with double blind reports as material to consult in their nal decision. In rare instances are the authors communicated about the one-sided transparencies of this largely open process -which is called 'double blind' because one or two of the four review phases includes selected closed reports. A key outcome of our study is a brief policy recommendation for all journals to communicate the transparency elements of their peer review processes stage by stage -also and especially if they are 'double blind'.
All our interviewees took pains to explain the intricacy of the process of selecting and editing submissions for the pro le of the journal. Although these pro les are found explicitly stated on the websites of all journals as a matter of course, the subtleties of the selection processes that inform the all-important early stages of screening and editorial review were di cult to articulate even for the seasoned editors we spoke to, and are accordingly not easily formalized in written, generalizable form. Some globally recognized multidisciplinary journals (e.g. Nature Human Behavior 2019) have already initiated efforts to clearly and comprehensively verbalize review transparency policies -however, to our knowledge, no journal has yet met the gold standards for communicating their transparencies of the actual multi-phase review process, the identi cation of which is hopefully easier after this article.

Declarations
Funding The work received funding from Academy of Finland (312397).
Con icts of interest/Competing interests. We have no con icts of interest regarding this manuscript.
Consent Written informed consent was collected from all participants.
Availability of data and material. The data will remain closed due to the requests of some interviewees. Parts of the data may be shared e.g. for peer review purposes.