Green Tea and Velociraptors

This is adapted from our recent paper in F1000 Research, entitled “A multi-disciplinary perspective on emergent and future innovations in peer review.” Due to its rather monstrous length, I’ll be posting chunks of the text here in sequence over the next few weeks to help disseminate it in more easily digestible bites. Enjoy! This section is on the recent history and modern development of peer review. 

1. Peer Review: An Introduction.
2. Peer Review: An Early History

1.1.3 The peer review revolution.

In the last several decades, and boosted by the emergence of Web-based technologies, there have been substantial innovative efforts to decouple peer review from the publishing process (Figure 2; Schmidt & Görögh (2017)), and the ever increasing volume of published research. Much of this experimentation has been based on earlier precedents, and in some cases a total reversal back to historical processes. Such decoupling attempts have typically been achieved by adopting peer review as an overlay process on top of formally published research articles, or by pursuing a “publish first, filter later” protocol, with peer review taking place after the initial publication of research results (BioMed Central, 2017; McKiernan et al., 2016; Moed, 2007). Here, the meaning of “publication” becomes “making public,” as in the legal and common use as opposed to the scholarly publishing sense where it also implies peer reviewed, a trait unique to research scholarship. In fields such as Physics, Mathematics, and Economics, it is common for authors to send their colleagues either paper or electronic copies of their manuscripts for pre-submission evaluation. Launched in 1991, arXiv (arxiv.org) formalized this process by creating a central network for whole communities to access such e-prints. Today, arXiv has more than one million e-prints from various research fields and receives more than 8,000 monthly submissions (arXiv, 2017). Here, e-prints or preprints are not formally peer reviewed prior to publication, but still undergo a certain degree of moderation by experts in order to filter out non-scientific content. This practice represents a significant shift, as public dissemination was decoupled from a formalised editorial peer review process. Such practice results in increased visibility and combined rates of citation for articles that are deposited both in repositories like arXiv and traditional journal venues (Davis & Fromerth, 2007; Moed, 2007).

The launch of Open Journal Systems (pkp.sfu.ca/ojs/; OJS) in 2001 offered a step towards bringing journals and peer review back to their community-led roots, by providing the technology to implement a range of potential peer review models within a low-cost open source platform. As of 2015, the OJS platform provided the technical infrastructure and editorial and peer review workflow management support to more than 10,000 journals (Public Knowledge Project, 2016). Its exceptionally low cost was perhaps responsible for around half of these journals appearing in the developing world (Edgar & Willinsky, 2010).

The past five to ten years have seen an accelerating wave of innovation in peer review, which we term “the revolution” phase (Figure 2; note that this is a non-exhaustive overview of the peer review landscape). Initiatives such as the San Francisco Declaration on Research Assessment (ascb.org/dora/; DORA), that called for systemic changes in the way that scientific research outputs are evaluated, and advances in Web-based technologies, are likely catalysts for such innovation. Born-digital journals, such as the PLOS series, introduced commenting on published papers, and Rapid Responses by BMJ has been highly successful in providing a platform for formalised comments (bmj.com/rapid-responses). Such initiatives spurred developments in cross-publisher annotation platforms like PubPeer (pubpeer.com/) and PaperHive (paperhive.org/). Some journals, such as F1000 Research (f1000research.com/) and The Winnower (thewinnower.com/), rely exclusively on a model where peer review is conducted after the manuscripts are made publicly available. Other services, such as Publons (publons.com/), enable reviewers to claim recognition for their activities as referees. Originally, Academic Karma (academickarma.org/) offered a similar service to Publons, but has since adapted its model to facilitate peer review of preprints. Platforms such as ScienceOpen (scienceopen.com/) provide a search engine combined with peer review across publishers on all documents, regardless of whether manuscripts have been previously reviewed. Each of these innovations has partial parallels to other social Web applications or platforms in terms of transparency, reputation, performance assessment, and community engagement. It remains to be seen whether these new models of evaluation will become more popular than traditional peer review, either singularly or in combination.

Reference

Tennant JP, Dugan JM, Graziotin D et al. A multi-disciplinary perspective on emergent and future innovations in peer review [version 2; referees: 2 approved]. F1000Research 2017, 6:1151 (doi: 10.12688/f1000research.12037.2)