I wrote this for my lab a while ago. I’m hoping the course will generate feedback that will help to improve it!
Read the paper fully, making notes about what is clear and what is not.
Note technical issues:
are there experiments or analyses that need to be done (or reframed)
is the paper creating new methods
if so how does it compare to other methods
- are the intellectual leaps properly justified
Note any stylistic issues:
are the figures clear or do you need to parse the legend carefully
Format your first draft:
Write 2-3 sentences on the major goals of the paper
Assess whether the paper accomplishes those goals in separate paragraphs
in this section focus only on the major issues that deal with the interpretation of the data or the data quality
each major issue should be its own paragraph
if everything is in order, this can be relatively brief, rely on your notes on the technical issues!
Clearly state the major successes of the paper.
e.g. “The major success of the paper is in developing a new model for side chain conformational heterogeneity”.
Clearly state the major weakness(es) of the paper (if any). e.g. “The major weakness of the paper is that the model proposed is not tested”.
Write 1-2 sentences on the place/impact of this paper in the field.
Create a list of “Minor points”:
list all the minor technical questions that you have
especially important to make sure you distinguish between things you think are done incorrectly/incompletely and things that are just not explained clearly enough for you to understand them
list all the stylistic issues
I tend to focus a lot on clarity, but it’s great to have a reviewer who is eagle-eyed for all issues
Note any parts of the paper you do not feel like you are able to assess: “E.g. A significant part of the paper relies on a sophisticated analysis of mass spectrometry. I cannot offer expert feedback on the technical merits of this part of the paper.”
Nature family journals also ask these questions (which are good ones to keep in mind):
What are the major claims of the paper?
Are they novel and will they be of interest to others in the field?
If the conclusions are not original, it would be helpful if you could provide relevant references.
Is the work convincing, and if not, what further evidence would be required to strengthen the conclusions?
On a more subjective note, do you feel that the paper will influence thinking in the field? (this addresses points 1-4 of my guide)
This website is open-source and available on Github.