Calibrated Peer Review (CPR)

This is a marvelous system that can increase student learning while saving you up to 80% of the grading effort. This system was used by W. Prothero in UCSB's Oceanography class, with excellent results. The software that supports this pedagogy was originally written by UCLA chemists. The method was modified and implemented by W. Prothero. Currently, only the UCLA system is available to the public. 

Elements of a "CPR" assignment:

  1. The assignment description
  2. A scoring rubric that students will use to score papers
  3. Three "calibration" papers, scored by the instructor
  4. Other resources, instructions, etc. that students need to complete the assignment.
  5. For data intensive explorations, a list of links or resources to allow acquisition of data or data representations

To do the assignment, the student:

  1. Writes paper. Use EarthEd Online screen or write in Word and paste in text.
  2. Hands in paper before due date/time (click the "hand in button")
  3. Begins review of "calibration" papers 24hrs after the hand-in due date. (Students are prompted if their average item grade for a "heading" is greater than a preset value)
  4. Reviews 3 peer papers (randomly assigned and anonymous)
  5. Reviews own paper

The paper grade is computed from:

  1. Peer grades (3) weighted by each reviewer's rms variation from the calibration paper item scores
  2. Prof or TA grade if student requests it. This grade over-rides the peer grade.

A grade is also given for the quality of the student's reviews. It is based on:

  1. Difference between grade given peers and final grade each peer received
  2. Difference between final grade for own paper and score for own paper
  3. RMS deviation of calibration paper scores relative to "correct" scores
  4. All weights and point settings can be configured by the instructor.

The system encourages students to grade their peers carefully because they get points for the quality of their reviews. It also reduces the effect of students who do a sloppy job of grading. This will show up in errors in their calibration paper scoresheet item grades, thereby down-weighting the effect on their review on peers' grade.

<Link to Sample Writing and Calibration Papers>

<Link to UCLA CPR site>

Learning Resources: