Introduction

There have been experiments in peer review in the past, but mostly at individual journals and publishers (e.g., see Nature, 2006 [¹]), and none have achieved particularly widespread adoption. In the first decade of the 21st century this changed, and new models and initiatives began to appear that appealed more broadly. Some have already had a large impact on scholarly publishing. Others are still at a very early stage but have been greeted with considerable enthusiasm and support.

Concerns That Traditional Peer Review Is Wasteful

In the hierarchy of selective journals, manuscripts may end up being submitted to a number of journals before being accepted for publication, being reviewed afresh at each journal. Rejection on the basis of insufficient novelty or interest results in wasted time and effort by the authors and reviewers, with good sound work going from journal to journal, chasing publication. Concern at this wastage has led to the introduction of manuscript (and accompanying reviewer report) transfer (‘cascading’) between journals. This, as might be expected, works more successfully within stables of journals at a publisher or society (e.g., as at Nature Publishing Group, BioMed Central, and IOP Publishing) than between publishers (e.g., The Neuroscience Peer Review Consortium, http://nprc.incf.org/, where only a small percentage of transfers take place). However, some organizations are exploring other possibilities, for example the journal eLife, which allows authors to take their review reports with them to other journals [²].

Innovations in Peer Review

Separation of the functions of peer review

Amidst the changing scholarly landscape and growing unrest with some aspects of peer review, a number of innovations are being introduced. The field is wide open to new ideas and new players. Importantly, these ‘alternative’ peer-review models, as they are sometimes called, face the same issues as traditional peer review, and the same basic principles of good practice apply.

The first major innovation came with the launch of the open access journal PLOS ONE, and its importance shouldn’t be underestimated; it was a real ‘game changer’. For the first time, the two basic functions of peer review–critical review and selection–were separated. Publication would be based on the soundness of the research (methodology, results and reporting), not its novelty, importance or interest. That would be left for post-publication. Since its launch in December 2006, PLOS ONE has grown rapidly, and in 2012 published more than 23,000 articles (accounting for around 1.4% of the world’s scholarly literature), using more than 60,000 reviewers from 154 countries [³]. So successful has it been that journals with similar models have been launched by a number of other publishers (e.g., BMJ Open, Sage Open, AIP Advances). Such open access, repository-style journals with collections of sound work and relatively low rejection rates achieve two things. Firstly, they retain the submissions on which peer-review effort has been expended, and secondly, the article processing charges bring in revenues that can help support broader activities or other more-selective journals at that publisher. This move to publishing work on the basis of soundness brings with it the need, and great opportunities, for services that filter, evaluate, and draw attention to articles within these collections of work.

More transparent approaches

There is a move to more transparent approaches in peer review (making transparent the ‘black box’ of peer review [⁴]) with differing levels of openness. This has been welcomed and is regarded by many researchers as a positive move. Each journal/editor needs to decide the level of openness that is appropriate for them, as does each reviewer when deciding which invitations to review to accept. ‘Open’ peer review, which originally meant just the author knowing who the reviewer was, can now cover a number of things, e.g., the reviewers’ names being disclosed with the published article, reviewer reports being published (with or without names), the wider community being able to comment during review, and various combinations of these. Examples of journals with transparent processes are: the BMC series medical journals (http://www.biomedcentral.com/authors/bmcseries), the EMBO journals (http://www.embo.org/scientific-publications/transparent-process), BMJ Open (http://bmjopen.bmj.com/), eLife (http://www.elifesciences.org/), and PeerJ (https://peerj.com/).

Interaction between peer-review participants

Some journals, in an attempt to improve and enrich editorial decision-making, have started including an interactive stage during which reviewers and editors can discuss a manuscript before the decision is sent to the author. For example, reviewers at the four EMBO journals are given the opportunity to see each other’s reviews (‘cross-peer review’; http://www.nature.com/emboj/about/process. html#Cross_peer_review) before a decision is made, and to contact the editor if they have any concerns or further comments. At eLife, the editor and reviewers engage in a consultation phase before the editor compiles a summary decision letter [⁵]. The Frontiers journals include an interactive review stage during which authors and review editors collaborate online via a discussion forum until convergence is reached (http://www.frontiersin.org/about/reviewsystem).

New initiatives

In the short space of time since the end of 2011, a number of new initiatives providing peer-review services have been launched, some with models of review and publication very different to those in traditional journals. They have in common that they are trying to improve the scholarly review and publishing process from outside the traditional journal ecosphere, and are providing new tools and processes to do this. They are also changing the dynamics of interaction and publication.

Peerage of Science: http://www.peerageofscience.org

Peerage of Science is a ‘community of Peers’ and a ‘new service for scientific peer review and publishing for the Peers, by the Peers’. It offers services commercially to scientific publishers, funding institutions and other organizations that require peer review, but is free for scientists as authors. The initiative started in the areas of ecology, evolutionary biology and conservation biology, but is looking to expand across science. When a manuscript is submitted, any qualified, non-affiliated (i.e., without potentially conflicting associations) peer can engage as a reviewer, in a process managed automatically by software, following deadlines set by the authors, and overseen by a tracking editor. The reviewers also evaluate each other’s reviews (‘peer-review of peer-review’) and each review gets a quality index. The peer-review process is available concurrently to all participating journals, who can ‘bid’ for papers. Authors can choose to accept a publishing offer (and some have), or to export the peer reviews to a journal of their choice.

PeerJ: https://peerj.com

PeerJ is an open access publisher of scholarly research articles in the biological, medical and health sciences. There is a pre-print server (PeerJ PrePrints) alongside the main journal (PeerJ), which publishes peer-reviewed research articles on the basis of scientific and methodological soundness. There is a lifetime membership model, with a one-off subscription (starting, in May 2013, at USD 99 for the basic level, with USD 299 the most expensive), that allows authors then to publish free for life. All authors on a paper have to be paid-up members. All authors also have to review once a year, and reviewers who submit their reviews on time benefit from a ‘reviewer reward’ scheme which credits them with a limited-time (12 month) basic PeerJ membership.

F1000Research: http://f1000research.com

F1000Research is an open access publisher in the life sciences operating open peer review. All submitted articles are published immediately after a quick in-house check for inappropriateness (pseudoscience, language problems, unethical aspects). Peer review then occurs post-publication, with the peer-review status (and reviewers’ names and affiliations) visible with the article as part of the citation information: ‘approved’, ‘approved with reservations’, and ‘not approved’. Authors can make revisions in response to the reviewer comments, and all versions of the manuscript remain accessible and can be cited separately. Articles are published and stay on the site even if they receive only ‘not approved’ recommendations.

Rubriq: http://www.rubriq.com

Rubriq offers an independent peer-review service in the biological and medical fields, providing authors (for a fee) with reviews that are ‘portable’ and can be taken to journals with submissions. It also provides recommendations on matching manuscripts with the most appropriate journals. There is a standardized scorecard for reviewing, and the individual scores are used to calculate an overall score (the R-score) for a manuscript. The authors retain control over their scores and how they are used, and can choose to keep them confidential or to share them, inside or outside the Rubriq system. Journals in the Rubriq network have access to available manuscripts and the reports and scorecards, and can contact the authors through the Rubriq system to express interest and encourage submission to them.

These are all very young enterprises, with only a relatively small number of manuscripts and publications to date, so it is difficult to know which will survive. All, however, appear to have enthusiastic supporters, and Peerage of Science was the Winner of 2012 Association of Learned and Professional Society Publishers (ALPSP) Award for Publishing Innovation.

Post-publication Review

Peer review doesn’t stop at publication; work goes on being commented on and reassessed in the light of further work. Until relatively recently, there were only formal routes, such as letters to the editor and the submission of further manuscripts. The internet and social media have opened new channels, and responses can be rapid and extensive. The challenge is how to engage people. One successful example of post-publication review is the #arseniclife case, where, almost immediately after publication of a paper reporting a bacterium that could grow on arsenic instead of phosphorus and incorporate it into its DNA [⁶], experts took to Twitter and other social media to show there were flaws in the methodology and interpretation [⁷]. The online community knew there were problems with the paper long before anything about this appeared in the journal that published it.

Since the introduction of the Retraction Watch blog (http://retractionwatch.wordpress.com/) in August 2010, there has probably been greater scrutiny of published work, and increased awareness of retractions and other corrections of the literature and the importance of transparency in this process. Some initiatives, such as PubPeer (http://pubpeer.com/about), have been started because of frustration at the lack of post-publication peer discussion on journal websites. Commenters there have highlighted possible problems in published work, most notably with image manipulation and reuse in the paper by Tachibana et al. [⁸] just days after it was published (http://pubpeer.com/publications/23683578) [⁹].

Conclusion

Traditional models of peer review are being adapted, new models are appearing, and there is increasing emphasis on openness and transparency. Who organizes peer review is changing and moving outside the traditional journal sphere, with new initiatives and new players entering the arena. As more journals adopt the PLOS ONE model of publishing based on soundness and not novelty or impact, the demand for services that can filter and evaluate work after publication will increase. Could the learned societies and associations, with their pools of expertise and loyal members across all disciplines, have a role to play here? Post-publication commenting forums may well increase, possibly being organized by subject areas. There has never been a time of such great opportunities, and the possibilities for innovation are considerable.

Notes

No potential conflict of interest relevant to this article was reported.

References

• 1. Nature. Nature’s peer review debate [Internet]. Nature; 2006 [cited 2013 Jun 4]. Available from: http://www.nature.com/nature/peerreview/debate/.
• 2. Robertson M. Peer review: eLife goes portable [Internet]. BioMed Central blog; 2013 [cited 2013 Jun 5]. Available from: http://blogs.biomedcentral.com/bmcblog/2013/05/24/peer-review-elife-goes-portable/.
• 3. Somphanith S. Thanking our peer reviewers [Internet]. The PLOS ONE Community blog; 2013 [cited 2013 May 16]. Available from: http://blogs.plos.org/everyone/2013/01/07/thanking-our-peer-reviewers-4/.
• 4. Pulverer B. A transparent black box. EMBO J 2010;29:3891-92.http://dx.doi.org/10.1038/emboj.2010.307. Article
• 5. Schekman R, Watt F, Weigel D. Scientific publishing: the eLife approach to peer review. eLife 2013;2:e00799. http://dx.doi.org/10.7554/eLife.00799. PubMedPMC
• 6. Wolfe-Simon F, Blum JS, Kulp TR, et al. A bacterium that can grow by using arsenic instead of phosphorus. Science 2011;332:1163-6.http://dx.doi.org/10.1126/science.1197258. Article
• 7. Zimmer C. The discovery of arsenic-based Twitter: how #arseniclife changed science [Internet]. Slate; 2011 [cited 2013 May 17]. Available from: http://www.slate.com/articles/health_and_science/science/2011/05/the_discovery_of_arsenicbased_twitter.html.
• 8. Tachibana M, Amato P, Sparman M, et al. Human embryonic stem cells derived by somatic cell nuclear transfer. Cell 2013;153:1228-38.http://dx.doi.org/10.1016/j.cell.2013.05.006. Article
• 9. Oransky I. Cell reviewing allegations of image reuse in human embryonic stem cell cloning paper [Internet]. Retraction Watch blog; 2013 [cited 2013 Jun 4]. Available from: http://retractionwatch.wordpress.com/2013/05/22/cell-reviewing-allegations-of-image-reuse-in-human-embryonic-stem-cell-cloning-paper/.

Figure & Data

References

Citations

Citations to this article as recorded by  

• Experience with “Select Crowd Review” in Peer Review for The Thoracic and Cardiovascular Surgeon: 1-Year Experience
  Roman Gottardi, Peter Thomas Henning, Jessica Bogensberger, Markus K. Heinemann
  The Thoracic and Cardiovascular Surgeon.2023; 71(08): 609.     CrossRef
• Journal Peer Review and Editorial Evaluation: Cautious Innovator or Sleepy Giant?
  Serge P. J. M. Horbach, Willem Halffman
  Minerva.2020; 58(2): 139.     CrossRef
• The relationship of polarity of post-publication peer review to citation count
  Qianjin Zong, Lili Fan, Yafen Xie, Jingshi Huang
  Online Information Review.2020; 44(3): 583.     CrossRef
• Innovative Strategies for Peer Review
  Edward Barroga
  Journal of Korean Medical Science.2020;[Epub]     CrossRef
• Transparent Attribution of Contributions to Research: Aligning Guidelines to Real-Life Practices
  Valerie Matarese, Karen Shashok
  Publications.2019; 7(2): 24.     CrossRef
• The changing forms and expectations of peer review
  S. P. J. M. Horbach, W. ( Willem) Halffman
  Research Integrity and Peer Review.2018;[Epub]     CrossRef
• What is open peer review? A systematic review
  Tony Ross-Hellauer
  F1000Research.2017; 6: 588.     CrossRef
• What is open peer review? A systematic review
  Tony Ross-Hellauer
  F1000Research.2017; 6: 588.     CrossRef
• Peer‐review warning: system error, reviewers not found
  Lluís Brotons
  Frontiers in Ecology and the Environment.2015; 13(5): 241.     CrossRef
• Does ‘get visible or vanish’ herald the end of ‘publish or perish’?
  Joanne Doyle, Michael Cuthill
  Higher Education Research & Development.2015; 34(3): 671.     CrossRef
• Emerging trends in peer review—a survey
  Richard Walker, Pascal Rocha da Silva
  Frontiers in Neuroscience.2015;[Epub]     CrossRef
• Journal Metrics-Based Position ofDiabetes & Metabolism Journalafter the Change of Its Text Language to English
  Sun Huh
  Diabetes & Metabolism Journal.2014; 38(3): 187.     CrossRef
• How Far Has theInternational Neurourology JournalProgressed Since Its Transformation Into an English Language Journal?
  Sun Huh
  International Neurourology Journal.2014; 18(1): 3.     CrossRef
• Editing and publishing scholarly journals in the internet age
  Kihong Kim
  Science Editing.2014; 1(1): 2.     CrossRef