Optimising Signals in Human-AI Interaction - Experiment 1

How to cite

Hoong, Ruru (Juan Ru); Dreyfuss, Bnaya. 2024. "Optimising Signals in Human-AI Interaction - Experiment 1." AEA RCT Registry. AEARCTR-0013716. https://doi.org/10.1257/rct.13716-6.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Ruru (Juan Ru) Hoong* , Bnaya Dreyfuss

* Primary investigator

Registry ID
AEARCTR-0013716
DOI
https://doi.org/10.1257/rct.13716-6.0
First registered
Last updated
Status
In development
Trial dates
2024-05-30 – 2024-12-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/13716

Abstract

Artificial intelligence (AI) has caught pace with — and in some contexts even surpassed — humans in the ability to make predictions from data, purporting to improve decision-making. However, in cases where humans are still responsible for the final decision, biases in probabilistic reasoning can render even informative AI predictions detrimental to decision-making outcomes. Using a randomised experiment with loan underwriters, we show that the provision of optimised AI signals can improve overall decision-making in spite of information loss.

Intervention

Participants see different types of AI signals to aid their decision-making.

Experimental design

We recruit 200 loan underwriters/officers/processors, and other participants to make loan application decisions.

Design details

Randomization method
Randomisation done through Otree programme
Randomization unit
Individual
Clusters
200 loan officers/processors/underwriters).
Observations
We will aim to recruit 200 loan officers/processors/underwriters, unless otherwise limited by our recruitment (i.e., 2 weeks go by without any new loan officers/processors/underwriters). We will filter out subjects in our analysis based on attention and time checks.
Arms
200 in each arm. Each subject will see every treatment arm in randomised order.
Primary outcomes
The main dependent variable we will look at is decision accuracy (i.e. whether the decision to approve/deny the loan was ex-post correct). Our main specification will use data from all rounds in the experiment. Though we will not be as well powered, we will later also perform a robustness check on just first round decisions.

Cited by

No citation data available for this registration yet.

Keywords