Do You Trust Your Own Voice? – An Experimental Approach to the Strategic Use of Audio Deepfakes

How to cite

Greif, Jannik. 2024. "Do You Trust Your Own Voice? – An Experimental Approach to the Strategic Use of Audio Deepfakes." AEA RCT Registry. AEARCTR-0014924. https://doi.org/10.1257/rct.14924-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Jannik Greif*

* Primary investigator

Registry ID
AEARCTR-0014924
DOI
https://doi.org/10.1257/rct.14924-1.0
First registered
Last updated
Status
On going
Trial dates
2024-10-23 – 2025-03-31
Countries
Private
JEL codes
C91, D83, D91
Registry record
http://www.socialscienceregistry.org/trials/14924

Abstract

The provision of personal information in online interactions - the social presence - reduces social distance and therefore increases trust. This includes profile pictures, personal descriptions, video or voice messages, and other forms of direct communication that enhance the perception of human interaction. Advances in artificial intelligence have enabled the creation of convincing and realistic deepfakes, which could potentially undermine the concept of social presence. Deepfakes are artificial but hyper-realistic media content (video, audio, image) and allow the creation of artificial profile pictures, audio or video messages, which can therefore be used to manipulate the social presence. This poses a severe challenge, as the quality and therefore believability of deepfakes increased to the point that people are often unable to reliably differentiate between authentic and artificial content. Audio deepfakes may alter vocal features such as gender, emotion, and perceived trustworthiness, which potentially affect others’ behavior.
This experimental study examines the impact of audio deepfakes on trust-based decision-making. It investigates whether manipulated vocal characteristics in an artificial audio message as pre-play communication affects behavior in a trust game - for both the sender and receiver. The findings may provide insights into how artificially generated media influences trust and decision-making in online interactions.

Intervention

Subjects participate in a laboratory-online experiment and are assigned to the roles Speaker (Trustee) or Listener (Trustor) and engage in a trust game. Listeners are assigned to one of the treatments, varying the type of message they receive: Control (no message), Text (transcribed message), Authentic (authentic voice message), Artificial-1 (artificial voice message, young woman), Artificial-2 (middle aged woman), Artificial-3 (young male), Artificial-4 (middle aged man).

Experimental design

The experiment consists of a trust game with one-sided pre-play communication and involves two roles: Speaker (Trustee) and Listener (Trustor). The study is split in three parts:
Part 1 (Laboratory, Speaker Role): Subjects assigned to the Speaker role record an authentic audio message and select an artificial audio message. They participate in trust game decisions using the strategy method and provide ratings for artificial voices. General trust, risk preferences, and demographic information are also elicited.
Part 2 (Online, Listener Role): Subjects assigned to the Listener role are recruited online and randomly assigned to one of seven treatments: Control (no message), Text (transcribed message), Authentic (authentic voice message), Artificial-1 (artificial voice message, young woman), Artificial-2 (middle aged woman), Artificial-3 (young male), Artificial-4 (middle aged man).. They make trust game decisions before and after receiving the message and classify the message as either authentic or artificial. General trust, risk preferences, and demographic information are also elicited.
Part 3 (Laboratory, Speaker Role): Speakers review aggregated trust game decision statistics based on message type and make an additional decision on which message type to use for a new trust game interaction, elicited using the strategy method.

Design details

Randomization method
Individual subjects in the role of Speaker are randomly recruited from the university’s subject pool. Due to known technical difficulties in reliably generating audio deepfakes, the planned number of subjects in the role of Speaker is doubled in order to ensure having sufficient technically sound audio deepfakes. Individual subjects in the role of Listener are recruited on Prolific and randomly assigned to the treatments.
Randomization unit
Individual subjects in the role of Speaker are randomly recruited from the university’s subject pool. Individual subjects in the role of Listener are recruited on Prolific and randomly assigned to the treatments.
Clusters
-
Observations
40 Speaker observations (5 audio messages + 1 text message each; 240 messages) 2800 Listener observations (10 observations for each message + control treatment)
Arms
Sample size of 80 Speaker (in order to get 40 technically sound artificial audio messages). Listeners and the respective treatment: - Control: 400 subjects - Text: 400 subjects - Authentic: 400 subjects - Artificial-1: 400 subjects - Artificial-2: 400 subjects - Artificial-3: 400 subjects - Artificial-4: 400 subjects
Primary outcomes
Trust game decisions (Amount transferred and amount transferred back) Perceived authenticity of audio messages Voice preference of artificial voices
Secondary outcomes
Expectations of trust game decisions Perceived trustworthiness ratings of artificial voices General trust and risk preferences Demographic factors

Cited by

No citation data available for this registration yet.

Keywords