AI-Powered Promises: The Influence of ChatGPT on Trust and Trustworthiness
How to cite
Greevink, Ivo.
2023. "AI-Powered Promises: The Influence of ChatGPT on Trust and Trustworthiness." AEA RCT Registry. AEARCTR-0011511.
https://doi.org/10.1257/rct.11511-1.0
Download:
BibTeX
·
RIS— for reference managers without a browser
translator. Zotero and most
others pick up this page's citation metadata directly.
As Large Language Models (LLM) continue to improve at fast rate, it will not be long before they become an integral part of many of the digital tools we use today, including those focused on communication. In this project, we investigate how the availability of LLMs during written communication affects trust and trustworthiness between participant pairs in a trust game.
Experimental design
We will run a trust game with hidden action.
Two players will play as Trustee or Trustor.
The treatments differ in whether an artificial-intelligence-based language model can be used in order to communicate.
Design details
Randomization method
- Within each session, there is a randomized pairing before decision task 1 occurs. This is done by computer, namely via the method 'group_randomly' in Otree. This shuffles players such that they can end up paired with any random player.
- After decision task one is done, all players will deterministically shift 'roles' in the group before decision task 2, from player A to player B and vice versa. They are then randomly repaired within their cluster (which is decided randomly from the start due to the random allocation into groups, and therefore clusters)
Randomization unit
Individual-level randomization into groups of two and clusters of four (or six at maximum).
Clusters
For the initial round, 160 participants in groups of two.
For the analysis of the second round, ideally, 70-80 clusters of four (two groups). This depends on how many instances of clusters of 6 we will have.
Observations
320 student participants
Arms
160 participants in the GPT communication treatment,
160 participants in the Communication treatment
Primary outcomes
- "In", and "Roll" rates in each treatment arm (Communication, and GPT).
- The frequency that messages contain promises (this will be rated independently by external coders).
Secondary outcomes
- Joint outcome ("IN", "ROLL")
- first-order beliefs about trust (fraction of subjects playing “IN”), trustworthiness (fraction of subjects playing “ROLL”), and whether the message was generated by GPT.
- Whether participants choose to rewrite the message generated by GPT
- participants’ social value orientation
- participants’ attitudes toward artificial intelligence
- correlations between behaviors across the two periods
- risk aversion
- demographic characteristics
Cited by
No citation data available for this registration yet.