How to Perform a Review

Invited to Peer Review. Now what?

Before accepting an invitation to perform a review, a potential referee should consider the following points:

  • the manuscript topic must match the reviewer’s area of expertise: even if the manuscript sounds fascinating, the potential referee should decline and/or suggest appropriate reviewers in case of insufficient expertise.
  • the review report should be provided within the proposed deadline: if extra time is needed to perform the review, a communication must be sent to the editor so he/she can alert the authors.
  • the review must be objective: in case of conflict of interest that can negatively or positively bias the review, the potential referee should decline the editor’s invitation.

Preliminary Assessment

Once you’ve accepted the invitation, get yourself prepared to do the review. Here are three quick things you can do that are helpful to check sooner rather than later:

  • System access. Make sure you can sign in to the peer review system and access the manuscript and other submission files.
  • Journal guidelines. See if the journal has specific guidelines for reviewers. Are there special instructions or other things to take into account? What are the criteria for publication? Keep these expectations and standards in mind from the very beginning.
  • Review structure and format. Make sure you know how your review will need to be formatted. Does the journal want you to respond to specific questions in a structured reviewer form? Will you need to recommend a specific decision action, e.g., major revision? You might also want to find out whether the journal has an open review option. Decide whether or not you will sign your name to your review.

Remember now you have two main jobs as a reviewer:

  1. Determine whether the authors’ claims are supported
  2. Help the journal editors make their decision

Journal editors make their decisions based on the journal’s publication criteria, read these before you start your review. Remember that the editors may not be experts in the field. They need you because of your specialized knowledge and technical expertise. They do not need you to be a copy editor.

Getting Started

The first step when reading is to figure out what the authors are trying to claim. It might be helpful to ask yourself these questions:

  • What is the study about? What is the main research question?
  • What is the approach? What did the authors do to address their research question?
  • What is the context? How does the study relate to published literature on this topic?
  • What are the conclusions? What are the authors’ main findings and what evidence do they provide for these conclusions?

Make sure you read the entire manuscript, including the figures. You should expect to read through the manuscript at least twice. It’s generally a good idea to read start to finish, but this is not always the case.

Here’s one way to read the manuscript that involves going a bit out of order:

  1. Read the abstract and introductionto get a sense of the overall context and approach (if the abstract and introduction do not do a good job summarizing the findings, you might need to read further to get this information).
  2. Look at the figures and tablescarefully in conjunction with the results.
  3. Read the conclusions.
  4. Then read the whole thingfrom beginning to end.

Take lots of notes as you go along to get a jump start on writing your report.

Tips for Specific Sections

Here’s a section-by-section guide of things to look for. You might want to take notes about what each section includes and identify the strengths and weaknesses.

Abstract and introduction

The introduction sets the stage. The authors should explain why the study matters and put the research in context.

  • Do the authors summarize the main research question and key findings?
  • Do the authors identify other literature on the topic and explain how the study relates to this previously published research?

Figures and tables

Make sure the manuscript text supports the data shown in the figures and tables. Do not just take the figures and tables at face value.

  • Are the figures and tables clear and readable? (Keep in mind that depending on the submission system you’re working in, you might have to click a link to view the high-resolution versions of the authors’ figures).
  • Are the figure and table captions complete and accurate?
  • Are the axes labeled correctly?
  • Is the presentation appropriate for the type of data being presented?
  • Do the figures and tables support the findings?

Methods

When you assess the methods used in the study, you are looking to determine whether the research is technically sound. Some questions you might consider, depending on the type of study, include:

  • What experiments or interventions were used?
  • Are the experiments or interventions appropriate for addressing the research question?
  • Are conditions adequate and the right controls in place?
  • Is there enough data to draw a conclusion?
  • Do the authors address any possible limitations of the research?
  • Was data collected and interpreted accurately?
  • Do the authors follow best practices for reporting?
  • Does the study conform to ethical guidelines?
  • Could another researcher reproduce the study with the same methods? In other words, have the authors provided enough information to validate the study?

Results, discussion, conclusions

  • Do the results support the conclusions?
  • Do the conclusions overreach?
  • Do the authors discuss any limitations of the study?
  • If the journal selects based on advance in the field does the study demonstrate this advance?

Statistical analysis

Is the statistical analysis adequate? If you do not have the expertise to consider the statistics, make sure you mention this in your report.

Data and supporting information

  • Do the data provide enough evidence for the authors’ conclusions?
  • Are the necessary data points provided?
  • Have the authors provided a sufficient amount of data and information for other researchers to recreate the analyses?

Other Things to Check:

  • Writing quality & clarity
  • Plagiarism
  • Reference list

How to Write a Peer Review

Think about structuring your review like an inverted pyramid. Put the most important information at the top, followed by details and examples in the center, and any additional points at the very bottom.

Here’s how your outline might look:

  1. Summary of the research and your overall impression

In your own words, summarize what the manuscript claims to report. This shows the editor how you interpreted the manuscript and will highlight any major differences in perspective between you and the other reviewers. Give an overview of the manuscript’s strengths and weaknesses. Think about this as your “take-home” message for the editors. End this section with your recommended course of action.

  1. Discussion of specific areas for improvement

It’s helpful to divide this section into two parts: one for major issues and one for minor issues. Within each section, you can talk about the biggest issues first or go systematically figure-by-figure or claim-by-claim. Number each item so that your points are easy to follow (this will also make it easier for the authors to respond to each point). Refer to specific lines, pages, sections, or figure and table numbers so the authors (and editors) know exactly what you’re talking about.

  1. Any other points

Confidential comments for the editors: use this space to mention concerns about the submission that you’d want the editors to consider before sharing your feedback with the authors, such as concerns about ethical guidelines or language quality. Any serious issues should be raised directly and immediately with the journal as well.

This section is also where you will disclose any potentially competing interests, and mention whether you’re willing to look at a revised version of the manuscript.

Giving feedback:

Giving feedback is hard. Giving effective feedback can be even more challenging. Remember that your ultimate goal is to discuss what the authors would need to do in order to qualify for publication. The point is not to nitpick every piece of the manuscript. Your focus should be on providing constructive and critical feedback that the authors can use to improve their study.

If you’ve ever had your own work reviewed, you already know that it’s not always easy to receive feedback. Follow the golden rule: Write the type of review you’d want to receive if you were the author.