Humans’ predictions based on text-based information

How to cite

CAI, XIQIAN. 2024. "Humans’ predictions based on text-based information." AEA RCT Registry. AEARCTR-0013622. https://doi.org/10.1257/rct.13622-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

XIQIAN CAI*

* Primary investigator

Registry ID
AEARCTR-0013622
DOI
https://doi.org/10.1257/rct.13622-1.0
First registered
Last updated
Status
In development
Trial dates
2024-05-22 – 2024-06-10
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/13622

Abstract

In this study, we investigate how humans make predictions based on text-based information. Our objective is to use statements of loan purposes from a crowdfunding website as the text-based information. We examine how prediction efficiency changes when we include additional information such as gender, loan amount, and loan interest rate. Additionally, we explore if and how their decisions are affected when we provide experts’ opinions alongside the text-based information.

The rationale is that humans may use text-based information to predict whether a loan borrower is trustworthy and whether a loan may default. We hypothesize that when the text-based information is limited, the prediction is not significantly different from random guessing. We also hypothesize that humans may exhibit some gender bias in their predictions and that their accuracy may improve when additional screening information, such as loan amount and loan interest rate, is provided. Finally, we hypothesize that humans’ predictions may be influenced by both experts’ opinions and opinions generated by large language models (LLMs).

Intervention

The experiment will be divided into six sections, each containing ten questions. In the first section, participants will receive only text-based information. In the second section, they will receive text-based information and gender information. In the third section, they will receive text-based information and loan interest rate information. In the fourth section, they will receive text-based information and loan amount information. In the fifth section, they will receive text-based information and ratings from financial experts. In the sixth section, they will receive text-based information and ratings from large language models (LLMs).

Experimental design

The experiment will be divided into six sections, each containing ten questions. In the first section, participants will receive only text-based information. In the second section, they will receive text-based information and gender information. In the third section, they will receive text-based information and loan interest rate information. In the fourth section, they will receive text-based information and loan amount information. In the fifth section, they will receive text-based information and ratings from financial experts. In the sixth section, they will receive text-based information and ratings from large language models (LLMs).

Design details

Randomization method
randomization done in platform by a computer
Randomization unit
individual
Clusters
1000 subjects
Observations
1000 subjects
Arms
1000 subjects
Primary outcomes
credibility rating for each statement on a scale from 0 to 100

Cited by

No citation data available for this registration yet.

Keywords