Choosing Challenge: Humans and AI

How to cite

Kuruvila, Tara Mary; Song, Lena; Serra-Garcia, Marta. 2025. "Choosing Challenge: Humans and AI." AEA RCT Registry. AEARCTR-0017065. https://doi.org/10.1257/rct.17065-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Tara Mary Kuruvila* , Lena Song , Marta Serra-Garcia

* Primary investigator

Registry ID
AEARCTR-0017065
DOI
https://doi.org/10.1257/rct.17065-1.1
First registered
Last updated
Status
In development
Trial dates
2025-10-09 – 2026-12-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/17065

Abstract

This project empirically assesses how the availability of AI affects task performance and willingness to select into tasks of different difficulty levels with and without AI assistance, within a learning setting.

Experimental design

The study involves two phases. In Phase 1, participants complete tasks and are cross-randomized by question difficulty (easy vs. difficult) and access to an AI chatbot (“AI Advisor”). The AI Advisor allows participants to ask questions and upload materials.

In Phase 2, participants choose between tasks that vary in repetition and difficulty, both with and without AI access. We also vary incentives to measure WTA to complete the harder and longer tasks. For the choice question that is selected, the participant is then randomly assigned to receive their chosen option (75%) or the alternative option (25%).

Additionally, when choosing tasks for Phase 2, some participants will be randomized into an encouragement treatment, where the use of the AI Advisor is destigmatized.

The design measures how AI access and demographics such as gender influence performance, confidence, and willingness to engage in longer or more challenging work.

Design details

Randomization method
In-survey randomization on Qualtrics
Randomization unit
Individual
Clusters
100 MBA students, ca. 800-1100 lab participants, or as many as can be recruited in the fall and winter quarters.
Observations
100 MBA students, ca. 800-1100 lab participants, or as many as can be recruited in the fall and winter quarters.
Arms
In Phase 1, assignment to the easy vs. difficult condition and AI Advisor vs. No AI Advisor condition are with equal probability In Phase 2, participants choose their preferred task and this is implemented with 75% probability. Additionally, in the lab participant sample, participants are cross-randomized to be in the encouragement treatment vs. not.
Primary outcomes
The primary outcome variables in Phase 1 are: • Number of correct questions • Absolute and relative confidence (post-task) The primary outcome variables in Phase 2 are: • Choice of question (basic vs. more work, basic vs. more challenge) at different incentive levels • Number of correct questions • Absolute and relative confidence (pre- and post-task) • Willingness to accept to choose the harder and longer task

Cited by

No citation data available for this registration yet.

Keywords