The changing face of peer review
Article information
Abstract
It is a time of great innovation in peer review. Traditional models are being adapted and completely new ones introduced. Independent peer-review services are also starting to be offered by organizations outside the traditional journal ecosphere. In both new and established systems, the importance of increasing openness, transparency, and interaction between peer-review participants is being recognized, and these are being introduced to varying degrees. Concern with the ‘wastage’ of review effort in traditional peer review, where manuscripts often go from journal to journal, being reviewed afresh at each, before being accepted for publication, is also being addressed. Reviews are being transferred (‘cascaded’) and shared between some journals. The separation of the two basic functions of peer review—critical review and selection—as originally introduced by the journal PLOS ONE has been a major innovation, leading to the publication of sound work irrespective of its perceived novelty, interest, or importance. Post-publication review is also becoming more important and is another growth area. The concept of ‘portable’ reviews has been introduced, where authors can take reviews with them—either after they have obtained them from a peer-review provider in return for a fee or had their manuscript reviewed and declined at some journals—and include them with submissions to journals. The dynamics of publication are changing alongside, with journals able to ‘bid’ for papers that have been reviewed by independent organizations and make publishing offers to the authors. A number of innovations and ‘alternative’ peer-review models are described. They all, however, face many of the same issues as traditional peer review, and the same basic principles of good and ethical practice apply.
Introduction
There have been experiments in peer review in the past, but mostly at individual journals and publishers (e.g., see Nature, 2006 [1]), and none have achieved particularly widespread adoption. In the first decade of the 21st century this changed, and new models and initiatives began to appear that appealed more broadly. Some have already had a large impact on scholarly publishing. Others are still at a very early stage but have been greeted with considerable enthusiasm and support.
Concerns That Traditional Peer Review Is Wasteful
In the hierarchy of selective journals, manuscripts may end up being submitted to a number of journals before being accepted for publication, being reviewed afresh at each journal. Rejection on the basis of insufficient novelty or interest results in wasted time and effort by the authors and reviewers, with good sound work going from journal to journal, chasing publication. Concern at this wastage has led to the introduction of manuscript (and accompanying reviewer report) transfer (‘cascading’) between journals. This, as might be expected, works more successfully within stables of journals at a publisher or society (e.g., as at Nature Publishing Group, BioMed Central, and IOP Publishing) than between publishers (e.g., The Neuroscience Peer Review Consortium, http://nprc.incf.org/, where only a small percentage of transfers take place). However, some organizations are exploring other possibilities, for example the journal eLife, which allows authors to take their review reports with them to other journals [2].
Innovations in Peer Review
Separation of the functions of peer review
Amidst the changing scholarly landscape and growing unrest with some aspects of peer review, a number of innovations are being introduced. The field is wide open to new ideas and new players. Importantly, these ‘alternative’ peer-review models, as they are sometimes called, face the same issues as traditional peer review, and the same basic principles of good practice apply.
The first major innovation came with the launch of the open access journal PLOS ONE, and its importance shouldn’t be underestimated; it was a real ‘game changer’. For the first time, the two basic functions of peer review–critical review and selection–were separated. Publication would be based on the soundness of the research (methodology, results and reporting), not its novelty, importance or interest. That would be left for post-publication. Since its launch in December 2006, PLOS ONE has grown rapidly, and in 2012 published more than 23,000 articles (accounting for around 1.4% of the world’s scholarly literature), using more than 60,000 reviewers from 154 countries [3]. So successful has it been that journals with similar models have been launched by a number of other publishers (e.g., BMJ Open, Sage Open, AIP Advances). Such open access, repository-style journals with collections of sound work and relatively low rejection rates achieve two things. Firstly, they retain the submissions on which peer-review effort has been expended, and secondly, the article processing charges bring in revenues that can help support broader activities or other more-selective journals at that publisher. This move to publishing work on the basis of soundness brings with it the need, and great opportunities, for services that filter, evaluate, and draw attention to articles within these collections of work.
More transparent approaches
There is a move to more transparent approaches in peer review (making transparent the ‘black box’ of peer review [4]) with differing levels of openness. This has been welcomed and is regarded by many researchers as a positive move. Each journal/editor needs to decide the level of openness that is appropriate for them, as does each reviewer when deciding which invitations to review to accept. ‘Open’ peer review, which originally meant just the author knowing who the reviewer was, can now cover a number of things, e.g., the reviewers’ names being disclosed with the published article, reviewer reports being published (with or without names), the wider community being able to comment during review, and various combinations of these. Examples of journals with transparent processes are: the BMC series medical journals (http://www.biomedcentral.com/authors/bmcseries), the EMBO journals (http://www.embo.org/scientific-publications/transparent-process), BMJ Open (http://bmjopen.bmj.com/), eLife (http://www.elifesciences.org/), and PeerJ (https://peerj.com/).
Interaction between peer-review participants
Some journals, in an attempt to improve and enrich editorial decision-making, have started including an interactive stage during which reviewers and editors can discuss a manuscript before the decision is sent to the author. For example, reviewers at the four EMBO journals are given the opportunity to see each other’s reviews (‘cross-peer review’; http://www.nature.com/emboj/about/process. html#Cross_peer_review) before a decision is made, and to contact the editor if they have any concerns or further comments. At eLife, the editor and reviewers engage in a consultation phase before the editor compiles a summary decision letter [5]. The Frontiers journals include an interactive review stage during which authors and review editors collaborate online via a discussion forum until convergence is reached (http://www.frontiersin.org/about/reviewsystem).
New initiatives
In the short space of time since the end of 2011, a number of new initiatives providing peer-review services have been launched, some with models of review and publication very different to those in traditional journals. They have in common that they are trying to improve the scholarly review and publishing process from outside the traditional journal ecosphere, and are providing new tools and processes to do this. They are also changing the dynamics of interaction and publication.
Peerage of Science: http://www.peerageofscience.org
Peerage of Science is a ‘community of Peers’ and a ‘new service for scientific peer review and publishing for the Peers, by the Peers’. It offers services commercially to scientific publishers, funding institutions and other organizations that require peer review, but is free for scientists as authors. The initiative started in the areas of ecology, evolutionary biology and conservation biology, but is looking to expand across science. When a manuscript is submitted, any qualified, non-affiliated (i.e., without potentially conflicting associations) peer can engage as a reviewer, in a process managed automatically by software, following deadlines set by the authors, and overseen by a tracking editor. The reviewers also evaluate each other’s reviews (‘peer-review of peer-review’) and each review gets a quality index. The peer-review process is available concurrently to all participating journals, who can ‘bid’ for papers. Authors can choose to accept a publishing offer (and some have), or to export the peer reviews to a journal of their choice.
PeerJ: https://peerj.com
PeerJ is an open access publisher of scholarly research articles in the biological, medical and health sciences. There is a pre-print server (PeerJ PrePrints) alongside the main journal (PeerJ), which publishes peer-reviewed research articles on the basis of scientific and methodological soundness. There is a lifetime membership model, with a one-off subscription (starting, in May 2013, at USD 99 for the basic level, with USD 299 the most expensive), that allows authors then to publish free for life. All authors on a paper have to be paid-up members. All authors also have to review once a year, and reviewers who submit their reviews on time benefit from a ‘reviewer reward’ scheme which credits them with a limited-time (12 month) basic PeerJ membership.
F1000Research: http://f1000research.com
F1000Research is an open access publisher in the life sciences operating open peer review. All submitted articles are published immediately after a quick in-house check for inappropriateness (pseudoscience, language problems, unethical aspects). Peer review then occurs post-publication, with the peer-review status (and reviewers’ names and affiliations) visible with the article as part of the citation information: ‘approved’, ‘approved with reservations’, and ‘not approved’. Authors can make revisions in response to the reviewer comments, and all versions of the manuscript remain accessible and can be cited separately. Articles are published and stay on the site even if they receive only ‘not approved’ recommendations.
Rubriq: http://www.rubriq.com
Rubriq offers an independent peer-review service in the biological and medical fields, providing authors (for a fee) with reviews that are ‘portable’ and can be taken to journals with submissions. It also provides recommendations on matching manuscripts with the most appropriate journals. There is a standardized scorecard for reviewing, and the individual scores are used to calculate an overall score (the R-score) for a manuscript. The authors retain control over their scores and how they are used, and can choose to keep them confidential or to share them, inside or outside the Rubriq system. Journals in the Rubriq network have access to available manuscripts and the reports and scorecards, and can contact the authors through the Rubriq system to express interest and encourage submission to them.
These are all very young enterprises, with only a relatively small number of manuscripts and publications to date, so it is difficult to know which will survive. All, however, appear to have enthusiastic supporters, and Peerage of Science was the Winner of 2012 Association of Learned and Professional Society Publishers (ALPSP) Award for Publishing Innovation.
Post-publication Review
Peer review doesn’t stop at publication; work goes on being commented on and reassessed in the light of further work. Until relatively recently, there were only formal routes, such as letters to the editor and the submission of further manuscripts. The internet and social media have opened new channels, and responses can be rapid and extensive. The challenge is how to engage people. One successful example of post-publication review is the #arseniclife case, where, almost immediately after publication of a paper reporting a bacterium that could grow on arsenic instead of phosphorus and incorporate it into its DNA [6], experts took to Twitter and other social media to show there were flaws in the methodology and interpretation [7]. The online community knew there were problems with the paper long before anything about this appeared in the journal that published it.
Since the introduction of the Retraction Watch blog (http://retractionwatch.wordpress.com/) in August 2010, there has probably been greater scrutiny of published work, and increased awareness of retractions and other corrections of the literature and the importance of transparency in this process. Some initiatives, such as PubPeer (http://pubpeer.com/about), have been started because of frustration at the lack of post-publication peer discussion on journal websites. Commenters there have highlighted possible problems in published work, most notably with image manipulation and reuse in the paper by Tachibana et al. [8] just days after it was published (http://pubpeer.com/publications/23683578) [9].
Conclusion
Traditional models of peer review are being adapted, new models are appearing, and there is increasing emphasis on openness and transparency. Who organizes peer review is changing and moving outside the traditional journal sphere, with new initiatives and new players entering the arena. As more journals adopt the PLOS ONE model of publishing based on soundness and not novelty or impact, the demand for services that can filter and evaluate work after publication will increase. Could the learned societies and associations, with their pools of expertise and loyal members across all disciplines, have a role to play here? Post-publication commenting forums may well increase, possibly being organized by subject areas. There has never been a time of such great opportunities, and the possibilities for innovation are considerable.
Notes
No potential conflict of interest relevant to this article was reported.