19. Potential future models of Peer Review

This is adapted from our recent paper in F1000 Research, entitled “A multi-disciplinary perspective on emergent and future innovations in peer review.” Due to its rather monstrous length, I’ll be posting chunks of the text here in sequence over the next few weeks to help disseminate it in more easily digestible bites. Enjoy!

This section is the introduction to the next major part of the paper on potential different models of peer review. Next time, what would Reddit-style peer review look like? Previous parts in this series:

  1. An Introduction.
  2. An Early History
  3. The Modern Revolution
  4. Recent Studies
  5. Modern Role and Purpose
  6. Criticisms of the Conventional System
  7. Modern Trends and Traits
  8. Development of Open Peer Review
  9. Giving Credit to Referees
  10. Publishing Review Reports
  11. Anonymity Versus Identification
  12. Anonymity Versus Identification (II)
  13. Anonymity Versus Identification (III)
  14. Decoupling Peer Review from Publishing
  15. Preprints and Overlay Journals
  16. Two-stage peer review and Registered Reports
  17. Peer review by endorsement
  18. Limitations of decoupled Peer Review

As we have discussed in detail above, there has been considerable innovation in peer review in the last decade, which is leading to widespread critical examination of the process and scholarly publishing as a whole (e.g., (Kriegeskorte et al., 2012)). Much of this has been driven by the advent of Web 2.0 technologies and new social media platforms, and an overall shift towards a more open system of scholarly communication. Previous work in this arena has described features of a Reddit-like model, combined with additional personalized features of other social platforms, like Stack ExchangeNetflix, and Amazon (Yarkoni, 2012). Here, we develop upon this by considering additional traits of models such as WikipediaGitHub, and Blockchain, and discuss these in the context of the rapidly evolving socio-technological environment for the present system of peer review. In the following section, we discuss potential future peer review platforms and processes in the context of the following three major traits, which any future innovation would greatly benefit from consideration of:

  • 1. Quality control and moderation, possibly through openness and transparency;

  • 2. Certification via personalized reputation or performance metrics;

  • 3. Incentive structures to motivate and encourage engagement.

Potential pros and cons of the main features of the peer review models that are discussed.
Potential pros and cons of the main features of the peer review models that are discussed. Note that some of these are already employed, alone or in combination, by different research platforms.

While discussing a number of principles that should guide the implementation of novel platforms for evaluating scientific work, Yarkoni (2012) argued that many of the problems researchers face have already been successfully addressed by a range of non-research focused social Web applications. Therefore, developing next-generation platforms for scientific evaluations should focus on adapting the best currently used approaches for these rather than on innovating entirely new ones (Neylon & Wu, 2009Priem & Hemminger, 2010Yarkoni, 2012). One important element that will determine the success or failure of any such peer-to-peer reputation or evaluation system is a critical mass of researcher uptake. This has to be carefully balanced with the demands and uptakes of restricted scholarly communities, which have inherently different motivations and practices in peer review. A remaining issue is the aforementioned cultural inertia, which can lead to low adoption of anything innovative or disruptive to traditional workflows in research. This is a perfectly natural trait for communities, where ideas out-pace technological innovation, which in turn out-paces the development of social norms. Hence, rather than proposing an entirely new platform or model of peer review, our approach here is to consider the advantages and disadvantages of existing models and innovations in social services and technologies (Table 4). We then explore ways in which such traits can be adapted, combined, and applied to build a more effective and efficient peer review system, while potentially reducing friction to its uptake.

Tennant JP, Dugan JM, Graziotin D et al. A multi-disciplinary perspective on emergent and future innovations in peer review [version 3; referees: 2 approved]F1000Research 2017, 6:1151 (doi: 10.12688/f1000research.12037.3)

3 thoughts on “19. Potential future models of Peer Review

  1. The tradition of peer review has become ingrained in science over … Growth has also been driven by new operating models like open access, which has made publishing available to more potential authors than ever before.Thank you for the sharing, I was really looking for information like this. Please stay us share like this. Once again thank you for sharing.

Leave a Reply