The impact of qualitative reviews in online markets: Empirical and experimental evidence on statistical discrimination
How to cite
Ioannidis, Konstantinos; Amano-Patiño, Noriko; Morris, James. 2025. "The impact of qualitative reviews in online markets: Empirical and experimental evidence on statistical discrimination." AEA RCT Registry. AEARCTR-0015282. https://doi.org/10.1257/rct.15282-2.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
We investigate the role of customer reviews and host demographics in statistical discrimination within the sharing economy (specifically in online rental markets). Using a controlled experiment in an Airbnb-like setting, we measure how a host's race, a host's gender, and customer reviews interact to affect accommodation demand. We create fictitious listings using scraped data from Airbnb and systematically vary host characteristics across three primary dimensions in a fully crossed 2x2x2 factorial design: Host Race (Black/White), Host Gender (Man/Woman), and a Review factor (High/Low). To isolate specific mechanisms of review-based discrimination, the exact nature of the Review factor varies across three between-participant treatments, manipulating either review quantity, positive informativeness, or negative informativeness. Our experimental design uses a forced-choice pairwise mechanism across three budget blocks (Low, Mid, High). To ensure perfect orthogonality and counterbalancing, the pairings are drawn from a comprehensive property map, with participants assigned to one of 56 block-randomised survey versions. This approach allows us to estimate the causal main effects of each attribute, as well as their interactions, to understand whether specific types of high-quality reviews can mitigate intersectional demographic penalties. Our findings will provide insights for platform design to reduce discrimination in the sharing economy.
Experimental design
General structure of the experiment
The experiment will run online on Prolific. After obtaining informed consent, participants will first report details about their most recent rental experience. The main task consists of 33 rounds divided into three budget blocks. In each round, participants are presented with two fictitious properties and must choose the one they would prefer to rent. To encourage truthful revelation of preferences and mitigate social desirability bias, participants are incentivised with bonus payments based on how closely their choices align with the modal choice (the option most frequently selected by other participants). The experiment concludes with a post-experimental survey comprising an Implicit Association Test to measure implicit biases and standard demographic questions.
Treatments
Our experimental design consists of three between-participant treatments. In all treatments, we use a within-subjects orthogonal design where participants evaluate 28 pairs of properties across three primary dimensions.
• Treatment 1 (Quantity): We vary the host race (minority/non-minority), host gender (man/woman), and review quantity (low/high, keeping the quality of reviews fixed).
• Treatment 2 (Positive Informativeness): We vary the host race, host gender, and the informativeness of reviews (low/high, keeping the number of reviews fixed) when all reviews are positive.
• Treatment 3 (Negative Informativeness): We vary the host race, host gender, and the informativeness of reviews (low/high, keeping the number of reviews fixed) when one of the reviews is negative.
Participants are randomly assigned to one of the three treatments. To prevent participants from seeing the exact same property twice while ensuring that all combinations of traits are tested against each other, the 28 experimental rounds are constructed using a predefined "property map". To achieve perfect counterbalancing, participants within each treatment are randomly assigned to one of 56 distinct participant types. This rotation ensures that property attributes are orthogonal to the round number and budget tier, minimising order effects.
Design details
Cited by
No citation data available for this registration yet.