Evaluating User Bias in Content Preference: AI-Generated Texts Versus Popular Online Platforms
How to cite
Nesseler, Cornel; Paklina, Sofia; Parshakov, Petr; Naidenova, Iuliia; Matkin, Nikita. 2024. "Evaluating User Bias in Content Preference: AI-Generated Texts Versus Popular Online Platforms." AEA RCT Registry. AEARCTR-0014231. https://doi.org/10.1257/rct.14231-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This study explores user preferences between AI-generated explanations and those curated by popular platforms like Quora. We identify popular questions in five scientific fields and use responses from platforms and AI. Through a field experiment, participants were presented with texts from both sources and asked to indicate their preferred explanation. The findings have the potential to reveal trends in user bias. Our analysis provides insights into the evolving role of AI in content generation and its comparison with traditional human-curated knowledge platforms.
Experimental design
We recruit students at a higher education institute and participants from a popular survey platform.
All participants are first presented with an overview regarding the aim of the experiment, information regarding who to contact, and information regarding how the data will be stored. The experiment will gather no data that can reveal individuals.
We first ask demographic questions (age and gender) and the educational background. Participants are then presented with five questions from five different scientific fields (depending on the education they selected). Each question includes two responses. One response is generated through AI. The other is a human generated response from popular platforms. The participants have to state which response they prefer (the response options are binary: Text 1 or Text 2). We do not reveal if a response is AI or human generated.
Design details
Cited by
No citation data available for this registration yet.