Reading the paper

• Read the paper fully, making notes about what is clear and what is not.
• Note technical issues, such as:
  • Are there experiments or analyses that should be done (or reframed)?
  • Do the authors describe new methods? If yes, how do these methods compare to existing methods?
• Note stylistic issues, such as:
  • are the figures and accompanying legends clear?
  • Spelling/grammar errors
  • Are acronyms used excessively and without justifiable reason?
  • Are there any missing or incorrect citations?

Composing the review

When formatting a first draft of a review, create three sections:

Summary

• Write 3-5 sentences on the major goals of the paper
• State the major successes of the paper.
  • “The major success of the paper is in developing a new model for side chain conformational heterogeneity”.
• State the major weakness(es) of the paper (if any - could also be the major confusion you have with the way it is framed).
  • “The major weakness of the paper is that the model proposed is not tested”.
  • These can refer to the major points, which are elaborated below
• Write 1-2 sentences on the place/impact of this paper in the field.

Major points

• Enumerate only major issues that deal with data quality and data interpretation and reflect on the ability of the paper to accomplish it’s major goals
• Each major issue should be its own paragraph or list of questions
  • this is your chance to ask both specific and open ended questions!
  • be specific and prescriptive, not passive agressive. Outline the experimental or textual revisions that are necessary to support the science.
    • DON’T: “it would be nice to include another technique.”
    • DO: “Your data do not fully support this specific (quote from paper) claim. Consider revising by either adding this specific control/technique described in this paper or altering the claim (paraphrase from paper) to allow for (reduced) alternative interpretation.”
    • REMEMBER: Most often, changes can be textual only! Do not suggest new experiments or controls just for the sake of suggesting something.

Minor Points

• List all minor technical questions you have
  • It is important that you distinguish between things you think are done incorrectly/incompletely and things that are not explained clearly enough for a reader to understand.
• List all minor stylistic issues
  • JF’s tend to focus a lot on clarity, but it’s great to have a reviewer who is eagle-eyed for all issues
• Note any parts of the paper you do not feel like you are able to assess
  • “A significant part of the paper relies on a sophisticated analysis of mass spectrometry. I cannot offer expert feedback on the technical merits of this part of the paper.”

After composing the review

• Reflect on the points you made and consider what is essential for supporting the arguments that are made in the paper
• Avoid the instinct to focus on the gatekeeping role of review and instead try to reframe all comments around what will help the science the most
• Re-read the review with an eye towards the reaction of the most junior author. Try to be empathetic to their point of view.
• Does your review follow the FAST (Focused, Appropriate, Specific, Transparent) principles to foster a positive preprint feedback culture