Green Tea and Velociraptors

This is adapted from our recent paper in F1000 Research, entitled “A multi-disciplinary perspective on emergent and future innovations in peer review.” Due to its rather monstrous length, I’ll be posting chunks of the text here in sequence over the next few weeks to help disseminate it in more easily digestible bites. Enjoy! This section is the first half of the history of peer review. The modernisation of peer review comes tomorrow!

1. Peer Review: An Introduction.

1.1 The history and evolution of peer review

Any discussion on innovations in peer review must appreciate its historical context. By understanding the history of scholarly publishing and the interwoven evolution of peer review, we recognize that neither are static entities, but covary with each other. By learning from historical experiences, we can also become more aware of how to shape future directions of peer review evolution and gain insight to what the process should look like in an optimal world. The actual term “peer review” only appears in the scientific press in the 1960s. Even in the 1970s, it was often associated with grant review and not with evaluation and selection for publishing (Baldwin, 2017a). However, the history of evaluation and selection processes for publication clearly predates the 1970s.

1.1.1 The early history of peer review. The origins of a form of “peer review” for scholarly research articles are commonly associated with the formation of national academies in 17th century Europe, although some have found foreshadowing of the practice (Al-Rahawi, c900; Csiszar, 2016; Fyfe et al., 2017; Spier, 2002). We call this period the primordial time of peer review (Figure 1), but note that the term “peer review” was not formally used then. Biagioli (2002) described in detail the gradual differentiation of peer review from book censorship, and the role that state licensing and censorship systems played in 16th century Europe; a period when monographs were the primary mode of communication. Several years after the Royal Society of London (1660) was established, it created its own in-house journal, Philosophical Transactions. Around the same time, Denis de Sallo published the first issue of Journal des Sçavans, and both of these journals were first published in 1665 (Manten, 1980; Oldenburg, 1665; Zuckerman & Merton, 1971). With this origin, early forms of peer evaluation emerged as part of the social practices of gentlemanly learned societies (Kronick, 1990; Moxham & Fyfe, 2017; Spier, 2002). The development of these prototypical scientific journals gradually replaced the exchange of experimental reports and findings through correspondence, formalizing a process that had been essentially personal and informal until then. “Peer review”, during this time, was more of a civil, collegial discussion in the form of letters between authors and the publication editors (Baldwin, 2017b). Social pressures of generating new audiences for research, as well as new technological developments such as the steam-powered press, were also crucial (Shuttleworth & Charnley, 2016). From these early developments, the process of independent review of scientific reports by acknowledged experts, besides the editors themselves, gradually emerged (Csiszar, 2016). However, the review process was more similar to non-scholarly publishing, as the editors were the only ones to appraise manuscripts before printing (Burnham, 1990). The primary purpose of this process was to select information for publication to account for the limited distribution capacity, and remained the authoritative purpose of such evaluation for more than two centuries.

1.1.2 Adaptation through commercialisation. Peer review in forms that we would now recognize emerged in the early 19th century due to the increasing professionalism of science, and primarily through English scholarly societies. During the 19th century, there was a proliferation of scientific journals, and the diversity, quantity, and specialization of the material presented to journal editors increased. Peer evaluations evolved to become more about judgements of scientific integrity, but the intention of any such process was never for the purposes of gate-keeping (Csiszar, 2016). Research diversification made it necessary to seek assistance outside the immediate group of knowledgeable reviewers from the journals’ sponsoring societies (Burnham, 1990). Evaluation evolved to become a largely outsourced process, which still persists in modern scholarly publishing today. The current system of formal peer review, and use of the term itself, only emerged in the mid-20th century in a very piecemeal fashion (and in some disciplines, the late 20th century or early 21st; see Graf, 2014, for an example of a major philological journal which began systematic peer review in 2011). Nature, now considered a top journal, did not initiate any sort of peer review process until at least 1967, only becoming part of the formalised process in 1973 (nature.com/nature/history/timeline_1960s.html).

This editor-led process of peer review became increasingly mainstream and important in the post-World War II decades, and is what we term “traditional” or “conventional” peer review throughout this article. Such expansion was primarily due to the development of a modern academic prestige economy based on the perception of quality or excellence surrounding journal-based publications (Baldwin, 2017a; Fyfe et al., 2017). Peer review increasingly gained symbolic capital as a process of objective judgement and consensus. The term itself became formalised in research processes, borrowed from government bodies who employed it for aiding selective distribution of research funds (Csiszar, 2016). The increasing professionalism of academies enabled commercial publishers to use peer review as a way of legitimizing their journals (Baldwin, 2015; Fyfe et al., 2017), and capitalized on the traditional perception of peer review as voluntary duty by academics to provide these services. A consequence of this was that peer review became a more homogenized process that enabled private publishing companies to thrive, and eventually establish a dominant, oligopolistic marketplace position (Larivière et al., 2015). This represented a shift from peer review as a more synergistic activity among scholars, to commercial entities selling it as an added value service back to the same academic community who was performing it freely for them. The estimated cost of peer review is a minimum of £1.9bn per year (in 2008; Research Information Network (2008)), representing a substantial vested financial interest in maintaining the current process of peer review (Smith, 2010). Neither account for overhead costs in publisher management, or the redundancy of the reject-resubmit cycle authors enter due to the competition for the symbolic value of journal prestige (Jubb, 2016).

The result of this is that modern peer review has become enormously complicated. By allowing the process to become managed by a hyper-competitive industry, developments in scholarly publishing have become strongly coupled to the transforming nature of academic research institutes. These institutes have now evolved into internationally competitive businesses that strive for impact through journal publication. Often this is now mediated by commercial publishers through attempts to align their products with the academic ideal of research excellence (Moore et al., 2017). Such a consequence is plausibly related to, or even a consequence of, broader shifts towards a more competitive neoliberal academic culture (Raaper, 2016). Here, emphasis is largely placed on production and standing, value, or utility (Gupta, 2016), as opposed to the original primary focus of research on discovery and novelty.