Ecology is not a dirty word

A fairly typical peer review process goes like this*. Author submits their paper to a journal. If it’s suitable to send to review, the handling editor sources a minimum of 2 or 3 relevant independent experts to review the paper. Very few papers are suitable for publication at first submission, so their review comments are returned to the author for consideration. Author revises the paper in response to the comments and resubmits the revised version. If the revisions are very minor and the response appropriate, the editor might make a decision immediately. Otherwise, this revised version is sent back to the original reviewers, who assess whether the authors have addressed the original comments appropriately and potentially pick up any new issues. The editor then makes the decision whether to accept (or reject) the paper or continue with further revisions.

This process can obviously take many months, but is fairly straightforward when it all goes smoothly.

One hiccup along the way can arise if one of the original reviewers isn’t available to review the revised version of a manuscript. This lengthens the process and means the editor has to cycle through another set of reviewer invitations (each one can take 1-2 weeks waiting for a response) to find a second reviewer for the revised version, or perhaps conduct the review themselves if they can’t find anyone. Prior to covid, I rarely experienced this as an editor, but have found this situation arising more frequently in the last year.

What are the pros and cons of having to bring on a new reviewer to assess the revision?

Given that most journals only seek a minimum of two or three reviews per paper, it can be a good thing to have a fresh pair of eyes look at the paper. But it can also delay the publication process unnecessarily, either through time taken to find another reviewer or because the new reviewer raises new or irrelevant issues that have to be responded to.

Continuity in the review process can be valuable – no paper is perfect, and new readers will always interpret things with different perspectives, so there are trade-offs involved in deciding whether endless revisions are necessary or appropriate.

For the new reviewer, it can be difficult to know how to tackle the review. Do you read the original version and response document first, or do you just read the clean revised version to avoid being influenced by previous comments? If you do read the whole revision, do you just focus on what’s been done to address the original comments, do you comment on the validity of previous review comments, or do you raise new issues?

For the author, it can be frustrating to go through the first revisions and then get additional comments from someone who didn’t see the first version and may not have read all the revision documents. The response document can become a messy headache of trying to explain why suggested changes are contradictory or unnecessary.

There is no right or wrong way here, just lots of things to consider in your decision-making. Given that this situation seems to be arising a lot more during this era of rapid change, here are a few points to consider from each perspective.

Editor:

• If an original reviewer is unavailable, I usually decide whether to invite a new reviewer based on the previous reviews. If the original submission was returned for very minor revisions and is now suitable for publication, I don’t bother sending it to another round of review. If the revisions were fairly minor, but reviewers had some specific concerns, I seek comments from the original reviewer that is available and do the second review myself. If it underwent major revisions or there were significant issues with the original submission, I value a fresh perspective and seek another reviewer.
• When I send the invitation email I mention that the paper is a revision. As a reviewer, I’ve accepted a few review invitations thinking it was an original submission, only to find otherwise. I do sometimes feel a bit annoyed that this wasn’t made clear to me in the review invitation, as it may have affected my decision to accept depending on my availability at the time (reviewing revisions can often be more time-consuming than original submissions).
• When making a decision with the new reviewer’s comments, I take into account that they may be overly critical or have missed some context (see below), and weigh up their recommendation with that of the original reviewer that was available.  

Reviewer:

• I always state in my review that I wasn’t one of the original reviewers. I do this because I know how frustrating it can be as an author, when you’ve spent weeks on comprehensive revisions, to get a whole lot of new comments that appear to overlook the context of the first revision. Sometimes I will suggest changes that contradict previous revisions, or I might miss some context in the previous revision, so I want the author to know where I’m coming from.
• To read the responses or not? This is a matter of personal choice, and also can depend on the paper. Usually I read the clean revised paper first to make up my own mind and identify any major issues that jump out at me. Then I go back to the response document and have a look at the main issues raised by previous reviewers to see if these have been addressed. I also look at how the author responded to the comments – this can tell a lot about how sloppy their revisions were or whether they were open to accepting constructive criticism. But overall, my comments largely focus on the new revised paper, not what was done previously.
• I try to focus only on major issues, rather than detailed specifics of grammar and style, or nitpick the methodology unnecessarily. The editor has already decided the paper is worthy of consideration, and after at least one round of thorough review, the author should have fixed most of the minor issues or they will do this prior to publication. So I focus on identifying any big methodological issues that the original reviewers may have missed and pointing out sections where more clarity is needed or key information is missing.
• What if the paper is fundamentally flawed and shouldn’t be published? I haven’t experienced this often, but occasionally you will receive a revised paper that should have been rejected at the first submission. Yet the editor and previous reviewers appear to have missed these major flaws. It’s important to say this – don’t feel that you have to recommend publication just because it’s gone through revisions and you’re ‘new’ to the scene. This is exactly why fresh eyes can be useful!
• Follow the same ‘How not to do peer review’ rules you would for any review.

Author:

• It’s frustrating getting a whole lot more comments to address, especially if you thought your previous revision was good enough to finally be published. I always try and remind myself that getting more reviewers than usual can be a good thing for improving the quality of my paper, no matter how frustrating the comments are!
• If the new reviewer requests unreasonable changes or changes that contradict previous revisions, don’t just follow the demands if it’s not right for your paper. Understand the editor has been with the paper from the beginning and knows the context of the previous revision, so they are unlikely to be sympathetic to a new reviewer demanding unreasonable or irrelevant changes at this stage. Justify why you are not making the changes, but make sure to be polite as the responses will likely go back to the reviewer! If the requested change is directly contradictory of a previous change and is largely a matter of preference, explain this in your response and say you are happy to follow the editor’s advice on what is appropriate for the journal.
• If the new reviewer is completely off base, or has fundamentally contradicted the assessment of the previous reviewers, email the editor and ask for advice. This should rarely happen and if the new reviewer is completely unreasonable, the editor should vet the comments or seek another reviewer before sending back to you. But sometimes these slip through the cracks. Email the editor and provide clear and non-personal reasons why you don’t think the reviewer’s comments are reasonable enough to address and see what they say.
• Follow the same standard rules you would when writing your original response document.

*This is written from my perspective as an ecologist working as editor, reviewer and author with ecology and general science journals. Other disciplines may have different norms and processes.

© Manu Saunders 2021