Peer Review in the Life Sciences

Spring 2023 Syllabus

Course Days/Hours: April 24, 25, May 1, 2, 3, 9, 10 from: 3:00 – 5:00 pm

Location: BH 212

Instructors: James Fraser, Christian Macdonald

“At its best, peer review is a slow and careful evaluation of new research by appropriate experts. It involves multiple rounds of revision that removes errors, strengthens analyses, and noticeably improves manuscripts.

At its worst, it is merely window dressing that gives the unwarranted appearance of authority, a cursory process which confers no real value, enforces orthodoxy, and overlooks both obvious analytical problems and outright fraud entirely.

Regardless of how any individual paper is reviewed – and the experience is usually somewhere between the above extremes – the sad truth is peer review in its entirety is struggling…”

from The Lancet has made one of the biggest retractions in modern history. How could this happen?

Course Description:

By far, the most widely accepted means of communication is through publishing papers in scientific journals. The process of peer review plays an important role in refining the body of work prior to final publication. Yet, peer review is rarely taught to students in a formal setting, and is largely dependent on individual labs and mentors, leading to variable standards of peer review. Reflecting the evolution of technology, society and scientific culture, preprints have gained popularity in the life sciences in recent years, resulting in a shift in how progress in the life sciences is communicated, and raising questions of how we, as a scientific community, may work towards optimizing the peer review process in the life sciences. This class will use preprint servers (for example, BioRxiv) as a platform for formally teaching students how to peer review manuscripts in a critical and constructive way.

We have modeled this class (and previous versions in 2020, 2021, and 2022) based on a course at NYU organized by Gira Bhabha, Damian Ekiert, Liam Holt & Timothee Lionnet.

Racism and Bias in Peer review

We idealize peer review process as an unbiased assessment of science. But bias creeps in to all aspects of evaluation, especially if the evaluators are not willing to acknowledge their own potential biases. These biases accumulate and manifest in harming the careers of scientists from historically marginalized groups.

Course structure

Students will be paired to serve as “Co-Discussion Leaders” for one BioRxiv manuscript of their choosing. The co-discussion leaders will compose one joint written peer review, with a first draft due immediately to Chris prior to their presentation, reflecting their original thoughts on the manuscript. Please send your paper selections to Chris by March 30th so everyone can start reading. For each class, everyone, not just the co-discussion leaders, should be prepared by having the paper carefully. We will start each day by calling on each participant in the class and asking them to share: 1 confusing thing about the paper and 1 cool thing about the paper.

After the round of sharing, the Discussion Leaders will present a joint talk similar in content and quality to a Tetrad or QBC Journal Club presentation, with an estimated 30-40 minute duration (noting that interruptions and discussion may take us closer to an hour).

After the group discussion, the co-discussion leaders will edit their review to reflect what emerged in the discussion. The deadline for this “final” review is May 15th 5PM. The review will be posted (either named or anonymously through James Fraser acting as an “editor”) as comments associated with the preprint on BioRxiv or a Zenodo record. Some commentary on how to name or be anonymous from PreReview

April 24

April 25

Some good guidelines for reviewing

Evaluate other reviews by this rubric

May 1

May 2 - JF at AVIDD meeting - CM to moderate!

May 3

May 9 - JF out of town - CM to moderate!

May 10

May 15