PS.8. COST PEERE session on peer review: research and training

Sunday 10th June: 10.30 – 11.55
Parallel session 8
Satellite, First floor

COST PEERE session on peer review: research and training
Chairs: Ana Marusic and Flaminio Squazzoni, Trans-Domain COST Action TD1306 New Frontiers of Peer Review
Content:

  • Large-scale exploration of peer review across research domains
    Flaminio Squazzoni, PEERE and University of Brescia, Italy
  • Motivations for peer reviewers to perform pre-publication review of manuscripts: a systematic review
    Mersiha Mahmić-Kaknjo, PEERE and University of Zenica School of Medicine, Bosnia and Herzegovina
  • Peer reviewer training as a means to boost a journal’s peer review capacity and quality
    Clarinda Cerejo, Editage Insights, India
  • Five Modes of “Sudden Death” – Experience with a Decision Option
    Markus K Heinemann et al., The Thoracic and Cardiovascular Surgeon, Germany
  • Innovations in peer review: Introducing VolunPeers
    Bahar Mehmani, RELX Group, Netherlands

Session overview
This session will use a trans-disciplinary approach to look into newest developments in peer review research and training, in order to explore the ways to improve efficiency, transparency and accountability of peer review.


Exploring peer review across domains, at a large scale

Flaminio Squazzoni, PEERE COST Action, University of Brescia, Italy

This paper presents a first analysis of peer review in a large sample of scholarly journals. It takes stock of the situation in different domains and suggests measures to improve our understanding of this important institution. Challenges and prospects of quantitative analysis that reconstructs the process are discussed.


Motivations for peer reviewers to perform pre-publication review of manuscripts: a systematic review

Mersiha Mahmić-Kaknjo, PEERE COST Action and University of Zenica, Bosnia and Herzegovina

The aim of the study was to identify studies and synthesize data on what motivates peer reviewers to perform peer reviews. Two authors systematically reviewed Medline, Scopus, Web of science, with no language or time limitations, following the search strategy developed by an experienced librarian. 3,585 titles were left after deduplication. Two authors chose 315 titles to be analyzed in full text. 14 manuscripts explored motivations: 4 agent based models (simulations), 3 theoretical papers, 4 qualitative qualitative studies of reviewers or editors (total 94 participants), 3 surveys (total 2,308 respondents, participation rates 62%, 63% and not listed). The most common incentives for performing peer review were: contribute to the community/scientific field, reciprocity, keep up to date on current research, improve manuscript quality, and acquire new skills and experience, career advancement. The most common disincentives were: lack of time, poor quality of manuscripts or journals, lack of formal recognition of performed work.

The study was funded by COST Action TD1306 New frontiers of peer review (PEERE) (http://www.peere.org/). The funder had no role in the design, execution, interpretation or writing up of the study.


Peer reviewer training as a means to boost a journal’s peer review capacity and quality
Clarinda Cerejo, Editage Insights, India

Journals and publishers often identify finding and retaining strong peer reviewers as a major pain point in the journal publishing workflow. While the demand for peer reviewers is increasing, owing to the growing volume of submissions journals receive overall and especially from emerging regions like China, Brazil, and the Middle East, the supply of peer reviewers continues to be restricted to traditional high-research-output and native-English-speaking countries, like the US and UK. This adds to the already-immense strain on the peer review system. As a possible solution, publishers have begun offering peer reviewer training for new and established peer reviewers. This session talks about one such publisher initiatives, focusing on the training curriculum, the medium of instruction, and the impact of such initiatives in helping to improve reviewing capacity and quality.


Five Modes of “Sudden Death” – Experience with a Decision Option

Markus K Heinemann, MD, PhD; Andreas Böning, MD, PhD; Graham Brumfield, The Thoracic and Cardiovascular Surgeon

Objective: The “Reject Without Review” (RWR) option was introduced into the manuscript management system of a cardiothoracic surgical journal to relieve the burden on the reviewers.
Design: Five recurrent patterns of insufficient quality could be identified. In a retrospective analysis the experience with this decision option was analyzed.
Results: Between 1/2014 and 8/2017 1273 original manuscripts underwent a First Decision. In 286 (22.5%) the “RWR” option was employed, accounting for 35.6% (286/802) of all primary rejections. The reasons were: 1. Submission of a Case Report (no longer a valid manuscript category, but to be submitted to a sister journal) n=109 (38.1%, 109/286); 2. Subject outside of Scope of journal n=78 (27.3%, 78/286); 3. Lack of Originality n=52 (18.2%, 52/286); 4. Faulty Science including plagiarism n=45 (15.7%, 45/286), 5. Incomprehensible English n=2 (0.7%, 2/286). The RWR manuscripts came from 23 countries, the top five being China (n=108), Turkey (n=74), Germany (n=15), Japan (n=14), Italy (n=12).
Conclusions: Over 20% of submissions could be decided upon without entering the review process, resulting in a respective reduction of workload for the reviewer pool. Apart from plain disregard of Instructions for Authors (submission of non-permitted Case Reports, research not covered by the journal), misconduct (including questionable ethical behavior) was also encountered. Lack of originality included unsolicited meta-analyses.


Innovations in peer review: Introducing VolunPeers

Bahar Mehmani, Reviewer Experience Lead, RELX Group, Netherlands

Elsevier Reviewer Recognition Platform has enabled reviewers to volunteer their services for particular journals since September 2015. Like many of its fellow Elsevier journals, the Journal of Molecular Biology (JMB) set up a module on the journal homepage which enables would-be reviewers to register via a simple form by identifying themselves using their Scopus or ORCID id, providing a motivation letter to editor, recording their areas of expertise and providing their contact details. The list Is provided to editors for further consideration.

“Volunpeers” is taking one step forward by grouping volunteers based on subject area. After controlling conflict of interest for a particular submission, editors send a group invitation to all of group members alerting them to new manuscripts matching their expertise. If they see something they like, the Volunpeers can respond to “book” the article for their review. This is done on a first-come-first-served basis. Once a Volunpeer has come forward, the handling editor sends the manuscript and instructions and awaits the review. Once a decision has been made on the paper in question, the editor provides concise and personalized feedback to the Volunpeer using Review Quality Instrument methodology and the Volunpeer is similarly invited to give feedback on the process from their perspective. As a result, the average number of days to submit review has been almost cut in half from 14 days to 7.7 days. Average quality rating by editor for Volunpeers is also high: 4.2/5 (for 38 volunpeers).