Peer Review: A Practice That Sustains Science

Peer review is the process by which outside experts in a particular subject area assess the scientific rigor, validity, and other measures of quality to determine whether a scientific article meets the standards for publication,” according to Amy Drew, the APS director of publications. In the APS webinar The Basics of Peer Review, Drew was joined by Becca White, the APS peer review manager, alongside Robert L. Goldstone, editor in chief of Current Directions in Psychological Science, and Erin B. Tone, associate editor for Clinical Psychological Science, to share their experiences involving the process of peer review and tips for engaging in meaningful peer review.

Why Peer Review?

Peer review has two main purposes: (a) to filter out research of poor quality or content that does not fit within the scope of a particular journal, and (b) to improve the quality of journal articles. Reviewers not only help to select the content to be published but also are expected to provide meaningful comments that can improve the quality of the manuscripts they review. By incorporating reviewers’ comments and suggestions, authors can improve their manuscripts and even develop new research ideas.

Peer review “is one of the most important things we do as scholars,” Tone said. “Peer review sustains science,” Goldstone added, noting that reviewers should make their reviews as helpful as they can.

Context Is Key

Drew and White explained the peer-review process and what happens “behind the scenes.”

At APS, after an author submits a manuscript, the Editor in Chief determines whether or not it should be considered for possible publication and, if so, assigns the article to a Senior Editor, who assigns and coordinates with an appropriate Action Editor to determine whether the manuscript should be sent out for review. If so, the Action Editor seeks the advice of reviewers who are experts in the field or can otherwise contribute to improving the manuscript quality. These individuals send their reviews to the Action Editor, who then makes the decision to accept the manuscript, reject it, or ask the authors to revise it, taking into account the reviewers’ and editor’s comments before resubmitting it.

A written review usually follows a certain structure that helps the editor to ultimately answer the question: Should this particular article be published in this particular journal? Generally, the reviewers:

1. Summarize the manuscript;
2. Identify positive aspects of the manuscript;
3. Identify negative aspects of the manuscript, plus offer critiques and suggestions;
4. Recommend a decision (for some journals).

While writing a review, “context is key,” Drew explained. That is, reviewers should evaluate whether an article is appropriate for a certain journal, based on factors such as its submission guidelines and, for some journals, reviewer guidelines or additional information in the invitations sent to potential reviewers. “The length, level of analysis, and necessary expertise will vary based on what the editor is looking for in a review,” she added. She also underscored the importance of reviews following civility and ethics principles, including confidentiality, disclosure of conflicts of interest or competing interest, and a respectful tone.

In offering additional advice to peer reviewers, Tone suggested starting young and reviewing as often as you can, to benefit not only the field but also reviewers themselves. Also ensure that your recommendations and reviews are aligned—if your review only mentions positive aspects and then recommends a rejection, it is not helpful for the authors. “Be kind but helpful,” she said.

Goldstone emphasized the roles researchers have as reviewers:

• Gatekeeper: Make sure a manuscript is of sufficient quality and won’t mislead future research.
• Champion: Identify and promote good research.
• Coach: Provide ideas about how to improve a manuscript to increase its validity and impact, and to clarify the story it tells.

In their final remarks, the four speakers underscored the importance of peer review for the advancement of science. A good reviewer must have high standards, the speakers agreed, but also be understanding of the different standards and constraints imposed by experimental design and resource availability.

More tips:

• Graduate students can be reviewers.
• One of the best ways to start is by reviewing a manuscript with a mentor.
• You can list the journals you have reviewed for in your CV.
• If you keep being asked to review for the same journal, consider asking the editor to put you on the editorial board.
• Private communication to the editor is not common, except for some metacomments (e.g., “I’ve reviewed this before for a different journal).
• Journal staff are usually available for assistance.

View a recording of the Basics of Peer Review webinar and the full list of past and upcoming APS webinars here.