Bias in peer evaluations

How to cite

Nicholls, Nicky. 2020. "Bias in peer evaluations." AEA RCT Registry. AEARCTR-0006697. https://doi.org/10.1257/rct.6697-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Nicky Nicholls*

* Primary investigator

Registry ID
AEARCTR-0006697
DOI
https://doi.org/10.1257/rct.6697-1.1
First registered
Last updated
Status
On going
Trial dates
2020-08-01 – 2021-12-01
Countries
Private
JEL codes
I23, J15, J16
Registry record
http://www.socialscienceregistry.org/trials/6697

Abstract

We study race and gender bias in peer evaluations of partners' relative contributions to a series of university assignments, where partners are randomly assigned.

Intervention

We randomly assign students in an undergraduate university course to partners to complete a series of paired assignments. Partners either share or do not share race and/or gender, and partners are different for each assignment, giving individual level variation in match/mismatch of own and partner race and gender. Following the assignment we ask students to report their own percentage contribution to an assignment (where the residual indicates the implied partner contribution). We compare the reported contributions for randomly assigned same and different race and gender partners to investigate possible race and/or gender bias that might arise in peer evaluations.

Experimental design

Students complete a series of assignments with different randomly assigned partners, where match of race and gender between the 2 students in the partnership varies both between individuals and within individuals (by assigning different partners for each assignment). We investigate whether student ratings of own contribution varies by race and gender, and by whether the reporters' race and/or gender is the same as or different from that of the partner. We control for the perceived difficulty of the assignment, as well as student grades, preference for working alone or in pairs and aspects of internet access (since the study was conducted in an unusual online semester in 2020 during COVID-19 lockdown).

Design details

Randomization method
Partners were assigned randomly using stata.
Randomization unit
individual
Clusters
350-400 individuals
Observations
350-400 individuals * 3 assignments = 1050-1200 observations
Arms
number of same and other race; and same and other gender pairs is determined by the random assignment
Primary outcomes
Student reports of own (versus partner) contribution to assignments

Cited by

No citation data available for this registration yet.

Keywords