Green Tea and Velociraptors

This is adapted from our recent paper in F1000 Research, entitled “A multi-disciplinary perspective on emergent and future innovations in peer review.” Due to its rather monstrous length, I’ll be posting chunks of the text here in sequence over the next few weeks to help disseminate it in more easily digestible bites. Enjoy! This section describes some of the traits and trends affecting modern peer review. Next week, the development of Open Peer Review!

1. Peer Review: An Introduction.
2. Peer Review: An Early History
3. Peer Review: The Modern Revolution
4. Peer Review: Recent Studies
5. Peer Review: Modern Role and Purpose
6. Peer Review: Criticisms of the Conventional System

The traits and trends affecting modern peer review

Over time, three principal forms of journal peer review have evolved: single blind, double blind, and open (Table 1). Of these, single blind, where reviewers are anonymous but authors are not, is the most widely-used in most disciplines because the process is considered to be more impartial, and comparably less onerous and less expensive to operate than the alternatives. Double blind peer review, where both authors and reviewers are reciprocally anonymous, requires considerable effort to remove all traces of the author’s identity from the manuscript under review (Blank, 1991). For a detailed comparison of double versus single blind review, Snodgrass (2007) provides an excellent summary. The advent of “open peer review” introduced substantial additional complexity into the discussion (Ross-Hellauer, 2017).

The recent diversification of peer review is intrinsically coupled with wider developments in scholarly publishing. When it comes to the gate-keeping function of peer review, innovation is noticeable in some digital-only, or “born open,” journals, such as PLOS ONE and PeerJ. These explicitly request referees to ignore any notion of novelty, significance, or impact, before it becomes accessible to the research community. Instead, reviewers are asked to focus on whether the research was conducted properly and that the conclusions are based on the presented results. This arguably more objective method has met some resistance, even receiving the somewhat derogatory term “peer review lite” from some corners of the scholarly publishing industry (Pinfield, 2016). Such a sentiment can be viewed as a hangover from the commercial age of non-digital publishing, and now seems superfluous and discordant with any modern Web-based model of scholarly communication. Indeed, when PLOS ONE started publishing in 2006, it initiated the phenomenon of open access “mega journals”, which had distinct publishing criteria to traditional journals (i.e., broad scope, large size, objective peer review), and which have since become incredibly successful ventures (Wakeling et al., 2016). Some even view the desire for emphasis on novelty in publishing to have counter-productive effects on scientific progress and the organization of scientific communities (Cohen, 2017), and journals based on the model of PLOS ONE represent a solution to this. The relative timing of peer review to publication is a further major innovation, with journals such as F1000 Research publishing prior to any formal peer review, with the process occurring continuously and articles updated iteratively. Some of the advantages and disadvantages of these different variations of peer review are explored in Table 2.

Tennant JP, Dugan JM, Graziotin D et al. A multi-disciplinary perspective on emergent and future innovations in peer review [version 3; referees: 2 approved]. F1000Research 2017, 6:1151 (doi: 10.12688/f1000research.12037.2)