A step towards feasible payments for reviewers?

A new preprint posted to BioRxiv details a small trial at Company of Biologists journal, Biology Open, to test two different models of paying peer reviews for their contributions, as a means of improving reviewing speed and quality, and providing tangible reward to reviewers in exchange.

The pilot was held over a 6-month period from July to December 2024, including 20 manuscripts, and 80 pre-recruited reviewers who committed to providing a rapid response during the trial.

The trial comprised two payment models: a freelance model or a retainer model.

Retainer reviewers received £600 per quarter in return for reviewing every request sent, up to three manuscripts during that period. Freelance reviewers were paid £220 per manuscript on an ad-hoc invitation bases. Freelancers had one business day to accept or decline an invitation. All reviewers were required to submit their review within 4 working days.

Another condition of the trail was intended to maintain quality of reviewing. Each review was rated by the handling Editor on a three-point scale of 0, 50, or 100. A score of 100 indicates a helpful review; a score of 50 resulted in feedback and guidance for improving. A score of 0 meant the reviewer would not be paid nor be invited to review again under the initiative.

The results of this trail produced encouraging results which have prompted the journal to expand and continue with the trial into 2025 and beyond.

All review reports received a score of 100 (helpful, adequate). No reviews were rated as unhelpful. The mean average time for submission of reviews was 4.13 days, with the longest duration between submission and decision being 11 days.

This does seem an encouraging result, but comes at a cost.

The preprint reports the costs of this trail, which the authors estimate to be between £50,000-£60,000 (GBP). This total included legal fees for development of contracts (£5815), system development costs and staff costs (£28-38,000), in addition to the reviewer payments at the core of the trial which totaled £16,820.

Not all of these costs would be an ongoing expense, but still present a considerable obstacle for other journals to overcome should they wish to consider similar trials, and while this trial indicates some measures of success and feasibility, implementation at a larger scale remains to be tested.

Reference:
Daniel A. Gorelick, Alejandra Clark. Fast & Fair peer review: a pilot study demonstrating feasibility of rapid, high-quality peer review in a biology journal.
bioRxiv 2025.03.18.644032; doi: https://doi.org/10.1101/2025.03.18.644032