Expertise, Personal Experience, and Algorithm Aversion (new study 1)

How to cite

Gao, Yu; Wang, Cong; Chong. 2022. "Expertise, Personal Experience, and Algorithm Aversion (new study 1)." AEA RCT Registry. AEARCTR-0009542. https://doi.org/10.1257/rct.9542-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Yu Gao* , Cong Wang , Chong

* Primary investigator

Registry ID
AEARCTR-0009542
DOI
https://doi.org/10.1257/rct.9542-1.0
First registered
Last updated
Status
On going
Trial dates
2022-05-15 – 2022-06-15
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/9542

Abstract

Research shows that evidence-based algorithms perform better than humans in predicting the future. Yet people give less weight to AI advice than they should. By exogenously variating personal experience with AI predictions, we explore how personal experience impacts weight on algorithm advice and how the level of expertise moderates this relationship. Our results will help design algorithms that are better adopted by human decision-makers, and mitigate the biases that experts hold on algorithms.

Experimental design

In Stage 1, each subject will be randomly assigned to the expert or the laypeople group, where they will receive immediate feedback or delayed feedback regarding their performance in a prediction task, respectively. Then, subjects will receive advise from either an AI or a human in new prediction tasks.
In Stage 2, each subject will be exposed to a randomly drawn level of the advisor’s relative performance compared to herself (+24, +12. -12, -24). Then, subjects will receive advise from either an AI or a human in new prediction tasks.

Design details

Randomization method
Randomization will be done by the survey platform.
Randomization unit
individual
Clusters
8 treatments
Observations
234 subjects per treatment
Arms
234 * 2 * 2 * 2 = 1872 subjects in total. In theory, a WOA should be between 0 and 1. However, in our pilot study, we observe that about 5%-10% of our subjects report a WOA out of this range. We will oversample about 10% to account for this factor.
Primary outcomes
Main outcomes are weight on advice (WOA), the distance between optimal WOA and real WOA, and change in WOA after receiving the information about advisor’s incidental performance.
Secondary outcomes
confidence in self, confidence in the advisor, and their tolerance of the advisor

Cited by

No citation data available for this registration yet.

Keywords