AI-Powered Promises: The Influence of ChatGPT on Trust and Trustworthiness

How to cite

Greevink, Ivo. 2023. "AI-Powered Promises: The Influence of ChatGPT on Trust and Trustworthiness." AEA RCT Registry. AEARCTR-0011511. https://doi.org/10.1257/rct.11511-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Ivo Greevink*

* Primary investigator

Registry ID
AEARCTR-0011511
DOI
https://doi.org/10.1257/rct.11511-1.0
First registered
Last updated
Status
In development
Trial dates
2023-06-14 – 2023-10-14
Countries
Private
JEL codes
D90, D91, D80, D84
Registry record
http://www.socialscienceregistry.org/trials/11511

Abstract

As Large Language Models (LLM) continue to improve at fast rate, it will not be long before they become an integral part of many of the digital tools we use today, including those focused on communication. In this project, we investigate how the availability of LLMs during written communication affects trust and trustworthiness between participant pairs in a trust game.

Experimental design

We will run a trust game with hidden action.

Two players will play as Trustee or Trustor.

The treatments differ in whether an artificial-intelligence-based language model can be used in order to communicate.

Design details

Randomization method
- Within each session, there is a randomized pairing before decision task 1 occurs. This is done by computer, namely via the method 'group_randomly' in Otree. This shuffles players such that they can end up paired with any random player. - After decision task one is done, all players will deterministically shift 'roles' in the group before decision task 2, from player A to player B and vice versa. They are then randomly repaired within their cluster (which is decided randomly from the start due to the random allocation into groups, and therefore clusters)
Randomization unit
Individual-level randomization into groups of two and clusters of four (or six at maximum).
Clusters
For the initial round, 160 participants in groups of two. For the analysis of the second round, ideally, 70-80 clusters of four (two groups). This depends on how many instances of clusters of 6 we will have.
Observations
320 student participants
Arms
160 participants in the GPT communication treatment, 160 participants in the Communication treatment
Primary outcomes
- "In", and "Roll" rates in each treatment arm (Communication, and GPT). - The frequency that messages contain promises (this will be rated independently by external coders).
Secondary outcomes
- Joint outcome ("IN", "ROLL") - first-order beliefs about trust (fraction of subjects playing “IN”), trustworthiness (fraction of subjects playing “ROLL”), and whether the message was generated by GPT. - Whether participants choose to rewrite the message generated by GPT - participants’ social value orientation - participants’ attitudes toward artificial intelligence - correlations between behaviors across the two periods - risk aversion - demographic characteristics

Cited by

No citation data available for this registration yet.

Keywords