Standardising Peer Review in Paleontology journals
Just finished a first-draft on a new paper, “Standardising Peer Review in Paleontology journals.” Full project guidelines are available here.
Feedback at this stage very much welcomed, as I have undoubtedly missed quite a bit! Here’s the introduction:
“However ill-defined it may be, the peer-review process is still the gold standard that will continue to drive scholarly publication.”
“While it is not a perfect process, traditional peer review remains the gold standard for evaluating and selecting quality scientific publications.”
That peer review operates as a ‘gold standard’ for scholarly research has become a common, institutionalised mantra. It is widely considered to be fundamental in maintaining the rigour and validity of scholarly research However, not only is peer review non-standardised, it also fails most basic tests of academic credibility and objectivity by being secretive, exclusive, and irreproducible. The process is often opaque, which can introduce bias into reporting standards for research and impact the overall quality of the published record. Despite being widely criticised, it remains almost ubiquitous as a critical part of scholarly communication systems. At the present, because we typically have almost no understanding of what peer review actually does, even generally, it can only be considered a ‘pseudo-standard’ at best. Training and support is generally lacking, and it is often the case that reviewers, through no fault of their own, are unaware of the critical questions to be asking with respect to research design, methods, reporting, and analysis.
In a modern knowledge production system, peer review is largely coupled to journals. Given that there are over 200 journals that publish palaeontology research Tennant and Lomax, 2019, this can make the process confusing for reviewers, irrespective of their relative expertise. As these journals are mostly owned by commercial entities in paleontology, they have a primary duty to increase financial value, and not to improve scientific quality or legitimacy. Because of this, management of the function of peer review is not in the hands of the research community, which means that our collective responsibility to ensure that it is performed to a high standard of integrity has been compromised. It is our duty as scholars to be critical of all elements of knowledge production, and if peer review is to be considered to be any sort of test of integrity or validation, then we must improve it.
The aim of this project is to formulate a clear set of guidelines explicitly for reviewers in palaeontology journals, or multi-disciplinary journals that include palaeontology submissions. Through this, peer review can be more transparent and objective, representing a valid form of peer review best practice and part of responsible research conduct. It can also help to improve the soundness and reporting standards for palaeontology research as a whole, increasing the field’s overall legitimacy. These guidelines were inspired by Parker et al. 2018, who created a similar checklist for the fields of ecology and evolution. Some of these points are adapted from the TTEE (Tools for Transparency in Ecology and Evolution) guidelines.
The purpose of this operational checklist serves one primary purpose: For an article to ‘pass’ peer review, articles must satisfy a specific quality threshold based on standardised guidelines. Thus, the quality of the peer review process is simultaneously ensured through an open and technical standardisation process. This should be of interest to all stakeholders engaged in the publishing process, including authors, editors, reviewers, and the publishers themselves, who all have a duty to uphold the integrity of the published research record. While it might initially increase the bureaucracy involved in publishing, ultimately it should save time and effort as it becomes more widely established as an embedded scholarly norm, with integrity a formative part of peer review culture.