Bias in Grant Peer Review: A Field Experiment

How to cite

Bento, Cindy Lopes; Katariya, L; Bento, C; Grimpe, C. 2025. "Bias in Grant Peer Review: A Field Experiment." AEA RCT Registry. AEARCTR-0016800. https://doi.org/10.1257/rct.16800-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Cindy Lopes Bento* , L Katariya , C Bento , C Grimpe

* Primary investigator

Registry ID
AEARCTR-0016800
DOI
https://doi.org/10.1257/rct.16800-2.0
First registered
Last updated
Status
In development
Trial dates
2025-04-23 – 2025-12-08
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/16800

Abstract

Growing empirical evidence reveals systematic risk aversion in science funding agencies. In response, alternatives to the traditional peer-review process have been sought. Previous research shows that evaluation formats can have a strong impact on the evaluation scores which determine the fundability of the idea. The traditional peer-review system relies on reviewers providing holistic scores across a set of loosely defined criteria. For instance, in the NIH peer-review process, reviewers first evaluate proposals along 5 dimensions and afterwards provide a final overall score which is used to decide the final funding decision. In the absence of any formal way to aggregate scores of different dimensions into the final score, the dimensions of risk and return get conflated. Using a field-experiment we will investigate whether changes to measurement methods can help to uncover risk aversion and other biases.

Intervention

New format of reviewer form.

Experimental design

The experiment empirically tests for risk aversion in the peer review process.

Design details

Randomization method
Sequential randomization: the time of entry in the experiment decides assignment to control vs. treatment. The last reviewer to enter is assigned to the treatment group, while the first three reviewers are assigned to the control group.
Randomization unit
Reviewer
Clusters
532 reviewer-proposal pairs
Observations
532 reviewer-proposal pairs
Arms
133 X 3 = 399 control, 133 X1 = 133 treatment
Primary outcomes
Our key outcome variables of interest are the scores assigned to a proposal by a reviewer.

Cited by

No citation data available for this registration yet.

Keywords