Measuring Economic Determinants of AI Use in Online Data Markets
How to cite
Kuruvila, Tara Mary; Naidu, Suresh; Song, Lena. 2026. "Measuring Economic Determinants of AI Use in Online Data Markets." AEA RCT Registry. AEARCTR-0018491. https://doi.org/10.1257/rct.18491-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This project assesses the extent to which bots and LLMs are used in the online market for survey responses, and how survey payments and performance-based bonuses affect adverse selection (low-quality or automated respondents entering the survey sample) and moral hazard (human workers using LLMs instead of providing authentic responses). We further test whether technical prevention tools and moral persuasion & penalties reduce LLM-generated responses.
Experimental design
We recruit approximately 3,000 participants aged 18–65 in the United States from Prolific, with comparison samples from other online survey platforms. Each participant completes a survey consisting of a screener and consent form, demographic questions, a battery of incentivized tasks, and a post-task section on self-reported AI use and attitudes. The task battery spans probability reasoning, beliefs and attitudes questions, open-ended summarization, and data labeling, reflecting the range of activities common in behavioral research and machine-learning data work. We cross-randomize four dimensions: base payment (high/low), bonus payment (none/low/high), prevention tools designed to discourage AI use (present/absent), and sanctions for AI use (none/moral persuasion and monetary penalty). Base payment is randomized at the study-listing level; the remaining dimensions are randomized at the individual level. To detect AI use, we combine behavioral measures (keystrokes, mouse movement, copy-paste activity, tab-switching, time on page), text-based AI detection on open-ended responses, and self-reported AI use, aggregated into pre-specified indices. After the study is complete, we invite participants to complete a follow-up survey to measure retention. For comparison, we also run this study on other platforms with design depending on budget and feasibility.
Design details
Cited by
No citation data available for this registration yet.