How to Perform a Review

Invited to Peer Review. Now what?

Before accepting an invitation to perform a review, a potential referee should consider the following points:

  • the manuscript topic must match the reviewer’s area of expertise: even if the manuscript sounds fascinating, the potential referee should decline and/or suggest appropriate reviewers in case of insufficient expertise.
  • the review report should be provided within the proposed deadline: if extra time is needed to perform the review, a communication must be sent to the editor so he/she can alert the authors.
  • the review must be objective: in case of conflict of interest that can negatively or positively bias the review, the potential referee should decline the editor’s invitation.

Preliminary Assessment

After accepting the invitation, the referee should check the following points in order to get prepared to do the review:

  • System access.  Check if the manuscript and the other submission files are accessible and downloadable.
  • Journal guidelines. See if the journal has specific guidelines or special instructions for reviewers.
  • Review format. Make sure you know the Journal requirements about the review structure.

Please remember  that a reviewer have two main tasks:

  1. Determine if the authors’ statements are valid and consistent
  2. Help the journal editors make their decision. They need a reviewer for his/her specialized knowledge expertise, they do not need a copy editor.

Let’s do the review!

It may sound obvious but a referee should read the entire manuscript (including figures, captions, …) at least twice. Major and minor points should be carefully annotated during the analysis.

The goal is to figure out what the authors are claiming within their manuscript. The following points might be helpful to evaluate the work:

  • Individuate the main research question
  • Understand the methodological approach
  • Contextualize the manuscript into the published literature
  • Analyze the conclusions

Guide for Specific Sections

Abstract and introduction: the main research question and key results should be summarized in the abstract. The introduction is the overture: the authors should explain how the work is related to the published literature.

Figures and tables: figures and tables should be clear and readable, and support the results described in the main text. The captions should be complete and straightforward.

Methods

When you assess the methods used in the study, you are looking to determine whether the research is technically sound. Some questions you might consider, depending on the type of study, include:

  • What experiments or interventions were used?
  • Are the experiments or interventions appropriate for addressing the research question?
  • Are conditions adequate and the right controls in place?
  • Is there enough data to draw a conclusion?
  • Do the authors address any possible limitations of the research?
  • Was data collected and interpreted accurately?
  • Do the authors follow best practices for reporting?
  • Does the study conform to ethical guidelines?
  • Could another researcher reproduce the study with the same methods? In other words, have the authors provided enough information to validate the study?

Results, discussion, conclusions

  • Do the results support the conclusions?
  • Do the conclusions overreach?
  • Do the authors discuss any limitations of the study?
  • If the journal selects based on advance in the field does the study demonstrate this advance?

Statistical analysis

Is the statistical analysis adequate? If you do not have the expertise to consider the statistics, make sure you mention this in your report.

Data and supporting information

  • Do the data provide enough evidence for the authors’ conclusions?
  • Are the necessary data points provided?
  • Have the authors provided a sufficient amount of data and information for other researchers to recreate the analyses?

Other Things to Check:

  • Writing quality & clarity
  • Plagiarism
  • Reference list

How to Write a Peer Review

Think about structuring your review like an inverted pyramid. Put the most important information at the top, followed by details and examples in the center, and any additional points at the very bottom.

Here’s how your outline might look:

  1. Summary of the research and your overall impression

In your own words, summarize what the manuscript claims to report. This shows the editor how you interpreted the manuscript and will highlight any major differences in perspective between you and the other reviewers. Give an overview of the manuscript’s strengths and weaknesses. Think about this as your “take-home” message for the editors. End this section with your recommended course of action.

  1. Discussion of specific areas for improvement

It’s helpful to divide this section into two parts: one for major issues and one for minor issues. Within each section, you can talk about the biggest issues first or go systematically figure-by-figure or claim-by-claim. Number each item so that your points are easy to follow (this will also make it easier for the authors to respond to each point). Refer to specific lines, pages, sections, or figure and table numbers so the authors (and editors) know exactly what you’re talking about.

  1. Any other points

Confidential comments for the editors: use this space to mention concerns about the submission that you’d want the editors to consider before sharing your feedback with the authors, such as concerns about ethical guidelines or language quality. Any serious issues should be raised directly and immediately with the journal as well.

This section is also where you will disclose any potentially competing interests, and mention whether you’re willing to look at a revised version of the manuscript.

Giving feedback:

Giving feedback is hard. Giving effective feedback can be even more challenging. Remember that your ultimate goal is to discuss what the authors would need to do in order to qualify for publication. The point is not to nitpick every piece of the manuscript. Your focus should be on providing constructive and critical feedback that the authors can use to improve their study.

If you’ve ever had your own work reviewed, you already know that it’s not always easy to receive feedback. Follow the golden rule: Write the type of review you’d want to receive if you were the author.