Coordination and Leadership: the impact of Artificial Intelligence

How to cite

Paoli, Damiano; Bigoni, Maria. 2023. "Coordination and Leadership: the impact of Artificial Intelligence." AEA RCT Registry. AEARCTR-0012434. https://doi.org/10.1257/rct.12434-6.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Damiano Paoli* , Maria Bigoni

* Primary investigator

Registry ID
AEARCTR-0012434
DOI
https://doi.org/10.1257/rct.12434-6.1
First registered
Last updated
Status
Completed
Trial dates
2023-11-20 – 2024-12-20
Countries
Italy (Emilia Romagna)
JEL codes
C92; D83; D91
Registry record
http://www.socialscienceregistry.org/trials/12434

Abstract

We investigate the effect of Artificial Intelligence as a source of help for the leader of a group that is facing a coordination dilemma. In an on-line experiment, subjects play a one-shot minimum effort game with leadership. The group leader must send a short message to their teammates to enhance coordination. First, the leader writes the message, then they see ChatGPT's output for the same task and decide whether to send their own text or the one produced by the chatbot. Followers are informed whether the leader sent their own message or the one produced by ChatGPT, before making their decision. With this approach, we want to compare the differences in coordination levels and elicited beliefs between the groups with a human-written leader's message and those with an AI-generated one.

Intervention

Participants will play a one-shot Minimum Effort Game with a leader. The leader has to send a message to the other group members, trying to entice them to choose the highest effort level. The leader can send them his/her own human-written message, or the message generated by ChatGPT4 for the exact same task.

Experimental design

The study is an asynchronous online survey experiment in which participants will answer questions without any real-time interaction with the others. The study is conducted entirely on Qualtrics.

Participants will play a one-shot Minimum Effort Game with leadership. The leader of the group has to send a message to the other group members; the message can be written by himself/herself or generated by Artificial Intelligence. Followers will receive the message before choosing their effort level.

We will elicit leaders' and followers' beliefs about the effort chosen by others.

Design details

Randomization method
Randomization will be done at the recruitment level through ORSEE. We will identify a sample of eligible subjects and randomize them in the three different treatments.
Randomization unit
Randomization at the individual level (participants in the experiment).
Clusters
We will have 33 groups of 5 subjects that play the game in each treatment.
Observations
429 subjects. 64 subjects in the follow-up survey.
Arms
165 subjects control, 132 AI message, 132 Human message, 64 in the follow-up survey.
Primary outcomes
The first key outcome variable is the Individual Effort Level chosen by participants. Given that participants will be divided in groups of 5, the second key outcome variable will be the Minimum Effort in the group, defined as the lowest Individual Effort Level chosen among group members. This experiment aims to measure the Average Treatment Effect of receiving a human-written or AI-generated leader's message on the Individual Effort Level's choice. We will also elicit the beliefs of Leaders and Followers about other group members' choices, in order to understand the potential mechanism behind the effect that we expect to detect. Main hypotheses: 1. H0: Leadership with AI’s message does not worsen coordination with respect to leadership with human message. H1: Leadership with AI’s message leads to a lower Pareto-ranked equilibrium. 2. H0: Followers do not differentiate between AI and Human message. H1: Followers show algorithm aversion, choosing a lower effort level when the message is AI-generated, with respect to when it’s human-written. 3. H0: Subjects’ beliefs about others’ actions do not change based on whether the message is AI-generated or human-written. H1: Leaders and/or followers update their beliefs when the message is AI-generated, expecting lower effort levels chosen by their teammates than when the message is human-written.
Secondary outcomes
Given that the leader's choice between human and AI message is completely endogenous, we will observe leaders' preferences on it. We will check the quality of the text written by leaders and check if there is a significant difference between those who choose to keep their own text, and those who prefer ChatGPT's output. The quality of the texts will be assessed by a different sample of subjects, who are not informed about the purpose of the experiment and who do not know whether the messages have been written by a human participant or by ChatGPT. Moreover, we will ask leaders to what extent they think they have been convincing, how much they felt under pressure for the role they were given and how much they believe to be suited for a role of responsibility. Followers will also be asked about the leader's message persuasiveness. Finally, we will study whether the leaders' and followers' behavior are affected by gender, familiarity and trust in AI, risk aversion, education level and standard demographics.

Cited by

No citation data available for this registration yet.

Keywords