Predicting compliance: Leveraging chat data for supervised classification in experimental research

How to cite

Hausladen, Carina Ines; Fochmann, Martin; Mohr, Peter. 2019. "Predicting compliance: Leveraging chat data for supervised classification in experimental research." AEA RCT Registry. AEARCTR-0005049. https://doi.org/10.1257/rct.5049-1.5

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Carina Ines Hausladen* , Martin Fochmann , Peter Mohr

* Primary investigator

Registry ID
AEARCTR-0005049
DOI
https://doi.org/10.1257/rct.5049-1.5
First registered
Last updated
Status
Completed
Trial dates
2019-11-26 – 2020-05-26
Countries
Germany (Berlin)
JEL codes
C55, C92, D83
Registry record
http://www.socialscienceregistry.org/trials/5049

Abstract

Behavioral and experimental economics have conventionally employed text data to facilitate the interpretation of decision-making processes. This paper introduces a novel methodology, leveraging text data for predictive analytics rather than mere explanation. We detail a supervised classification framework that interprets patterns in chat text to estimate the likelihood of associated numerical outcomes. Despite the unique advantages of experimental data in correlating textual and numerical information for predictive modeling, challenges such as limited sample sizes and potential data skewness persist. To address these, we propose a comprehensive methodological framework aimed at optimizing predictive modeling configurations, particularly in small experimental behavioral research datasets. We also present behavioral experimental data from a preregistered tax evasion game (n=324), demonstrating that chat behavior is not influenced by experimenter demand effects. This establishes chat text as an unbiased variable, enhancing its validity for prediction. Our findings further indicate that beliefs about others’ dishonesty, lying attitudes, and risk preferences significantly impact compliance decisions.

Intervention

For the out-of-sample performance of the pre-trained classifier, out-of-context chat-data is collected.
In this new online-experiment, participants work for a fictitious company in pairs of two.
In an online chat, they discuss which number of surplus hours they want to state.
Groups are controlled if reports do differ and/or if their group is one of the 30% percent of randomly controlled groups in each session.
If the inspections show that an individual's report was not truthful, (s)he needs to pay a fine.

Experimental design

Participants work for a fictive company in groups of two.
Both group members are informed about the surplus hours they worked.
They subsequently get the opportunity to chat about the amount of surplus hours they want to state.
The reports are controlled if the group members' reports differ and / or if the group is one of the 30 percent of randomly chosen groups to be controlled.
If a group is controlled and the number of stated surplus hours is not the same as the actually worked surplus hours, each group member as to pay a fine.

Design details

Randomization method
The experimental setting does not involve treatment. In order to minimize waiting-times in the online-experiment, participants are grouped by the time they log in to the experiment.
Randomization unit
In each session, 30 percent of the groups are randomly chosen to be controlled.
Clusters
100 groups
Observations
200 participants
Arms
This experiment does not involve treatments.
Primary outcomes
The reported amount of surplus hours by each participant. The group chat between two members of a group.

Materials

Papers from this trial

  1. Hausladen, Carina I., Martin Fochmann, and Peter Mohr. "Predicting compliance: Leveraging chat data for supervised classification in experimental research." Journal of Behavioral and Experimental Economics (2024): 102164. https://www.sciencedirect.com/science/article/pii/S2214804324000041

Cited by

No citation data available for this registration yet.

Keywords