Do People Trust ChatGPT to Write Accurate Statements?

How to cite

Buchanan, Joy; Hickman, William. 2023. "Do People Trust ChatGPT to Write Accurate Statements?." AEA RCT Registry. AEARCTR-0011546. https://doi.org/10.1257/rct.11546-1.2

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Joy Buchanan* , William Hickman

* Primary investigator

Registry ID
AEARCTR-0011546
DOI
https://doi.org/10.1257/rct.11546-1.2
First registered
Last updated
Status
Completed
Trial dates
2023-06-08 – 2023-08-31
Countries
United States of America ()
JEL codes
O33, C91, D8
Registry record
http://www.socialscienceregistry.org/trials/11546

Abstract

In this research paper, we aim to investigate the extent to which humans trust the output of LLMs to be factually accurate. While LLMs have showcased impressive capabilities in understanding and generating text, concerns have been raised regarding the potential for misinformation, biases, or inaccuracies in their responses. To explore this issue, we ask human subjects to rate the accuracy of statements written by humans and ChatGPT. We vary whether each statement was written by a human or by ChatGPT, and we also vary whether we inform participants about who/what wrote the statements.

Experimental design

We ask human subjects to rate the accuracy of statements written by humans and ChatGPT.

Design details

Randomization method
Each survey participant is randomly started into one of the information treatments by the computer.
Randomization unit
Individual
Clusters
450
Observations
450
Arms
150 will see AI-informed. 150 will see human-informed. There will be about 75 participants in each uninformed treatment.
Primary outcomes
The dependent variable is a choice by subjects to Trust, Fact-Check, or Not Trust

Papers from this trial

  1. Buchanan, Joy and William Hickman (2024) "Do people trust humans more than ChatGPT?" Journal of Behavioral and Experimental Economics, Volume 112, 102239. https://doi.org/10.1016/j.socec.2024.102239

Cited by

No citation data available for this registration yet.

Keywords