AI Recommendations and Incentives Under Time Constraints: Evidence from an RCT
How to cite
Chupak, Maxwell. 2026. "AI Recommendations and Incentives Under Time Constraints: Evidence from an RCT." AEA RCT Registry. AEARCTR-0018623. https://doi.org/10.1257/rct.18623-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This paper studies how the size of the monetary penalty for inaccurate responses affects subjects' reliance on AI-generated recommendations. Subjects complete a series of CAPTCHA-style object-counting tasks, with and without access to a pre-generated recommendation from an AI system, permitting within-subject comparison of task accuracy and AI reliance across AI-available and AI-unavailable conditions; this contrast is implemented under one of three between-subjects penalty levels for incorrect responses. To help explain over- or under-reliance relative to the AI's true, undisclosed accuracy, we separately elicit subjects' subjective belief distribution over the AI's accuracy using an incentive-compatible quadratic scoring rule instrument (Matheson and Winkler, 1976), and their willingness to pay for AI access using a Becker-DeGroot-Marschak (1964) mechanism. Subjects' subjective beliefs about the AI's accuracy are compared to their own self-assessed and task-measured accuracy to provide further evidence on why subjects may over- or under-rely on AI recommendations (Moore and Healy, 2008). Subjects are recruited via Prolific and complete the experiment on Qualtrics.
Intervention
Subjects complete three modules, each containing 30 unique judgment tasks. Subjects are randomly assigned to one of three financial incentive conditions that differ in the monetary penalty for incorrect responses. AI recommendations are available in exactly one of the first two modules and unavailable in the other; this availability is varied within-subject and counterbalanced across subjects. Access to AI recommendations in the third module is instead determined by a Becker-DeGroot-Marschak (BDM) mechanism eliciting each subject's willingness to pay. In every module, the first task is one for which the associated AI answer is fully accurate, and the second task is one for which the associated AI answer is off by one; the relative order of these two tasks is randomized independently at each module load, regardless of whether AI recommendations are available in that module. All remaining tasks are presented in a fixed, predetermined order. When an AI recommendation is available, subjects may reveal it and, with an additional click, incorporate it into their response, so that viewing and adopting a recommendation are captured as distinct decisions.
Experimental design
Participants complete three modules of incentivized recognition tasks under time constraints. AI recommendations are available in one of the first two modules, with AI availability varied within subjects and module order counterbalanced across participants. Participants are randomly assigned to one of three financial incentive conditions that differ in the penalty for incorrect responses. In the third module, access to AI recommendations is determined through a Becker-DeGroot-Marschak (BDM) mechanism eliciting willingness to pay. Participants also complete an incentivized elicitation of their beliefs regarding the AI’s accuracy. Earnings are determined by one randomly selected module.
Design details
Cited by
No citation data available for this registration yet.