The impact of qualitative reviews in online markets: Empirical and experimental evidence on statistical discrimination

How to cite

Ioannidis, Konstantinos; Amano-Patiño, Noriko; Morris, James. 2025. "The impact of qualitative reviews in online markets: Empirical and experimental evidence on statistical discrimination." AEA RCT Registry. AEARCTR-0015282. https://doi.org/10.1257/rct.15282-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Konstantinos Ioannidis* , Noriko Amano-Patiño , James Morris

* Primary investigator

Registry ID
AEARCTR-0015282
DOI
https://doi.org/10.1257/rct.15282-2.0
First registered
Last updated
Status
In development
Trial dates
2026-03-30 – 2026-07-31
Countries
Private
JEL codes
D83, J15, L84
Registry record
http://www.socialscienceregistry.org/trials/15282

Abstract

We investigate the role of customer reviews and host demographics in statistical discrimination within the sharing economy (specifically in online rental markets). Using a controlled experiment in an Airbnb-like setting, we measure how a host's race, a host's gender, and customer reviews interact to affect accommodation demand. We create fictitious listings using scraped data from Airbnb and systematically vary host characteristics across three primary dimensions in a fully crossed 2x2x2 factorial design: Host Race (Black/White), Host Gender (Man/Woman), and a Review factor (High/Low). To isolate specific mechanisms of review-based discrimination, the exact nature of the Review factor varies across three between-participant treatments, manipulating either review quantity, positive informativeness, or negative informativeness. Our experimental design uses a forced-choice pairwise mechanism across three budget blocks (Low, Mid, High). To ensure perfect orthogonality and counterbalancing, the pairings are drawn from a comprehensive property map, with participants assigned to one of 56 block-randomised survey versions. This approach allows us to estimate the causal main effects of each attribute, as well as their interactions, to understand whether specific types of high-quality reviews can mitigate intersectional demographic penalties. Our findings will provide insights for platform design to reduce discrimination in the sharing economy.

Experimental design

General structure of the experiment

The experiment will run online on Prolific. After obtaining informed consent, participants will first report details about their most recent rental experience. The main task consists of 33 rounds divided into three budget blocks. In each round, participants are presented with two fictitious properties and must choose the one they would prefer to rent. To encourage truthful revelation of preferences and mitigate social desirability bias, participants are incentivised with bonus payments based on how closely their choices align with the modal choice (the option most frequently selected by other participants). The experiment concludes with a post-experimental survey comprising an Implicit Association Test to measure implicit biases and standard demographic questions.

Treatments

Our experimental design consists of three between-participant treatments. In all treatments, we use a within-subjects orthogonal design where participants evaluate 28 pairs of properties across three primary dimensions.
• Treatment 1 (Quantity): We vary the host race (minority/non-minority), host gender (man/woman), and review quantity (low/high, keeping the quality of reviews fixed).
• Treatment 2 (Positive Informativeness): We vary the host race, host gender, and the informativeness of reviews (low/high, keeping the number of reviews fixed) when all reviews are positive.
• Treatment 3 (Negative Informativeness): We vary the host race, host gender, and the informativeness of reviews (low/high, keeping the number of reviews fixed) when one of the reviews is negative.

Participants are randomly assigned to one of the three treatments. To prevent participants from seeing the exact same property twice while ensuring that all combinations of traits are tested against each other, the 28 experimental rounds are constructed using a predefined "property map". To achieve perfect counterbalancing, participants within each treatment are randomly assigned to one of 56 distinct participant types. This rotation ensures that property attributes are orthogonal to the round number and budget tier, minimising order effects.

Design details

Randomization method
Computerized randomization
Randomization unit
Individual randomisation
Clusters
1,200 individuals (400 per treatment arm)
Observations
Because participants make 28 valid experimental choices evaluated in pairs, the effective number of observations is 400 * 28 * 2 = 22,400 observations per treatment (67,200 total).
Arms
400 individuals per treatment
Primary outcomes
Whether target property was chosen

Cited by

No citation data available for this registration yet.

Keywords