Peer Review in the Life Sciences

Spring 2021 Syllabus

Course Days/Hours: May 10, 12, 14, 17, 21, 24, 26, 28 from: 9-10AM

Location: Zoom!

Instructors: James Fraser

“At its best, peer review is a slow and careful evaluation of new research by appropriate experts. It involves multiple rounds of revision that removes errors, strengthens analyses, and noticeably improves manuscripts.

At its worst, it is merely window dressing that gives the unwarranted appearance of authority, a cursory process which confers no real value, enforces orthodoxy, and overlooks both obvious analytical problems and outright fraud entirely.

Regardless of how any individual paper is reviewed – and the experience is usually somewhere between the above extremes – the sad truth is peer review in its entirety is struggling…”

from The Lancet has made one of the biggest retractions in modern history. How could this happen?

Course Description:

By far, the most widely accepted means of communication is through publishing papers in scientific journals. The process of peer review plays an important role in refining the body of work prior to final publication. Yet, peer review is rarely taught to students in a formal setting, and is largely dependent on individual labs and mentors, leading to variable standards of peer review. Reflecting the evolution of technology, society and scientific culture, preprints have gained popularity in the life sciences in recent years, resulting in a shift in how progress in the life sciences is communicated, and raising questions of how we, as a scientific community, may work towards optimizing the peer review process in the life sciences. This class will use preprint servers (for example, BioRxiv) as a platform for formally teaching students how to peer review manuscripts in a critical and constructive way.

We have modeled this class based on a course at NYU organized by Gira Bhabha, Damian Ekiert, Liam Holt & Timothee Lionnet.

Racism and Bias in Peer review

We idealize peer review process as an unbiased assessment of science. But bias creeps in to all aspects of evaluation, especially if the evaluators are not willing to acknowledge their own potential biases. These biases accumulate and manifest in harming the careers of scientists from historically marginalized groups.

Course structure

Students will be paired to serve as “Co-Discussion Leaders” for one BioRxiv manuscript of their choosing. The co-discussion leaders will compose one joint written peer review, with a first draft due immediately to James prior to their presentation, reflecting their original thoughts on the manuscript. Please send your paper selections to James by May 3rd so everyone can start reading. For each class, everyone, not just the co-discussion leaders, should be prepared by having the paper carefully. We will start each day by calling on each participant in the class and asking them to share: 1 confusing thing about the paper and 1 cool thing about the paper.

After the round of sharing, the Discussion Leaders will present a joint talk similar in content and quality to a Tetrad or QBC Journal Club presentation, with an estimated 30-40 minute duration (noting that interruptions and discussion may take us closer to an hour).

After the group discussion, the co-discussion leaders will edit their review to reflect what emerged in the discussion. The deadline for this “final” review is June 1st 5PM. The review will be posted (either named or anonymously through James Fraser acting as an “editor”) as comments associated with the preprint on BioRxiv or a Zenodo record.

May 10

Optional event at 8AM PT:

May 12

Some good guidelines for reviewing

May 14

May 17

May 21

May 24

May 26

May 28