Detecting AI-Generated Text in Online Survey Responses
How to cite
Buschmann, Andy. 2026. "Detecting AI-Generated Text in Online Survey Responses." AEA RCT Registry. AEARCTR-0017629. https://doi.org/10.1257/rct.17629-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This study examines whether a syntax-based detection method based on character encoding patterns can identify AI-assisted text generation in open-ended survey responses. Specifically, it evaluates whether mixed or ASCII-only quotation mark usage, which arise from copy-pasting text generated in AI user interfaces such as ChatGPT, occurs in incentivized online survey responses and whether such patterns align with or diverge from respondents’ self-reported AI use.
Intervention
Participants are randomly assigned to one of three survey conditions that vary how artificial intelligence (AI) tools are described prior to a creative writing task. In one condition, participants are explicitly told they may use AI tools to assist with the task. In a second condition, participants are explicitly instructed not to use AI tools. In a control condition, AI tools are not mentioned prior to the task.
Experimental design
The study uses a between-subjects/split-ballot randomized survey experiment. Participants complete a creative writing task and related survey questions. The experimental manipulation consists of varying the instructional framing regarding AI tool use prior to the writing task. Outcomes are measured using both self-reports and post hoc analysis of written responses.
Design details
Cited by
No citation data available for this registration yet.