Since agreeing to take on the EiC job, I have been receiving the comments that author’s make on the post-submission survey that AGU sends out. One of the biggest comments in these anonymous survey results (AGU redacts the names before forwarding to me) and personal conversations about the journal is that the review process needs to be improved. The “shifting goalposts” phenomenon is a major complaint, as is concern about referees insisting on correction of a minor point. At the opposite end is the single-paragraph review, giving the referee’s general negative impression but only a few vague comments, with barely any actionable suggestions for improvement.
I want referees to thoroughly review a manuscript on the first pass. I want referees to comment on the introduction, noting whether the question is well-defined and well-supported by the historical context in this section. I want referees to comment on the methodology, scrutinizing the reasonableness of the approach for addressing the question posed. I also want reviewers to think about the reproducibility of the results from the description of the methodology in the paper. I want referees to examine the results and assess whether they uphold the conclusions. I want referees to carefully consider the discussion, contemplating the implications, broader impacts, caveats, and limitations of the findings with respect to previous results in this field. For all of these points, a thorough review not only points out what should be better, but make suggestions to the author on how to improve the manuscript and satisfy the referee.
Just as important as noting the negative aspects of a manuscript, a thorough review should also highlight the positive points of the study. Referees sometimes disagree, and it is then up to the editor to choose how to weigh these differing comments in deciding the fate of the manuscript. Including positive comments about a manuscript provides a counterbalance to the otherwise overwhelming list of negative comments and suggestions for improvement.
I have already seen several outstanding and thorough reviews in my short time as an editor for JGR-Space Physics. I hope to see many more.