Belief reactions to sequential AI predictions

How to cite

Lambrecht, Marco; Herold, Theo; Wengström, Erik; GSE, Hanken School of Economics & Helsinki. 2025. "Belief reactions to sequential AI predictions." AEA RCT Registry. AEARCTR-0011057. https://doi.org/10.1257/rct.11057-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Marco Lambrecht* , Theo Herold , Erik Wengström , Hanken School of Economics & Helsinki GSE

* Primary investigator

Registry ID
AEARCTR-0011057
DOI
https://doi.org/10.1257/rct.11057-1.0
First registered
Last updated
Status
In development
Trial dates
2025-04-21 – 2025-05-04
Countries
Private
JEL codes
C91, D81, D83, D91
Registry record
http://www.socialscienceregistry.org/trials/11057

Abstract

Artificial intelligence (AI) has become integral in decision-making processes, with tools such as ChatGPT increasingly shaping outcomes. These AI systems are inanimate but operate based on human-coded algorithms. In this study, we explore how previous success and streaks influence beliefs over sequential AI predictions, comparing them to beliefs over sequences generated by humans and inanimate objects. We also examine how outcomes (the frequency of correct decisions) affect participants' willingness to switch to an alternative source of data generation. These insights may guide AI policy, such as efforts to address 'cognitive behavioral manipulation.'

Intervention

We conduct a two-stage experiment to investigate participants' beliefs over sequential AI predictions.

In the first stage, we compare belief reactions across three different treatments between subjects.

In the second stage, we compare across two treatments how participants react to information regarding their correct answers in the first stage.

Experimental design

Our design builds on a previous experiment and consists of two stages. In the previous experiment, we acquire binary outcomes from three different sources: (i) humans, (ii) an algorithm, and (iii) an inanimate pure chance object in the form of dice rolls.

In the first stage, we provide participants with outcomes from the previous experiment. Depending on treatment, outcomes will originate from one of the three sources of the previous experiment. Participants are tasked with predicting whether the ninth outcome in each sequence is correct or not.

In the second stage, participants receive information about their correct decisions made during the first stage and choose between a human- or algorithm-generated sequence of outcomes.

Design details

Randomization method
Treatment allocation is done by a computer within oTree. Similarly, any randomization during our experiment is done within our oTree code. Randomization for the preparatory stage (e.g. the random selection of stock market data) was implemented in Stata. Likewise, the generation of sequences from dice rolls was implemented in Stata.
Randomization unit
Individual
Clusters
900 total subjects
Observations
900 individuals
Arms
300 per first stage treatment, 450 per second stage treatment Given our 3x2 treatment design, we elicit data from 150 individuals in each of the 6 groups (150 in human+modal correct, 150 in human+modal incorrect, 150 in AI+modal correct, 150 in AI+modal incorrect, 150 in dice+modal correct, 150 in dice+modal incorrect)
Primary outcomes
Belief reaction ("decision" in pre-analysis plan)
Secondary outcomes
Choose AI-generated sequence ("choose_ai" in pre-analysis plan)

Cited by

No citation data available for this registration yet.

Keywords