The Value of Recommender Systems

How to cite

Goncalves, Duarte; Aridor, Guy; Kong, Ruoyan; Konstan, Joseph; Kluver, Daniel. 2021. "The Value of Recommender Systems." AEA RCT Registry. AEARCTR-0007545. https://doi.org/10.1257/rct.7545-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Duarte Goncalves* , Guy Aridor , Ruoyan Kong , Joseph Konstan , Daniel Kluver

* Primary investigator

Registry ID
AEARCTR-0007545
DOI
https://doi.org/10.1257/rct.7545-2.0
First registered
Last updated
Status
Completed
Trial dates
2021-03-29 – 2021-10-31
Countries
United States of America ()
JEL codes
D82, D83, L15, L86, D12
Registry record
http://www.socialscienceregistry.org/trials/7545

Abstract

In this project we run a longitudinal field experiment in order to decompose the mechanisms that drive the influence recommendation systems have on consumption choices in terms of their informational and product discovery value.

Targeting a class of goods for which recommendations are ubiquitous -- movies --, we randomize the set of recommended movies on a movie-recommendation platform and compare outcomes to a control group of movies that holds fixed individuals' idiosyncratic preferences.
We test for whether recommendations meaningfully affect consumption patterns and beliefs about products.

Experimental design

We randomize the selection of recommended movies presented to users as recommended on an internet platform in order to study whether and how recommendations impact users' consumption patterns and their beliefs about both the recommended items and non-recommended items. Our study follows a within-subjects design. We have two treatment arms and a control group -- one set of movies that are excluded from both recommendation and belief elicitation (the control arm), one set of movies that we elicit beliefs about but exclude them from recommendation, and one set of movies that we both elicit beliefs for and include in platform recommendations. The sets of movies are fixed at enrollment and assigned to a treatment arm according to a matched pairs procedure using the predicted rating for the movies generated by the platform's recommendation algorithm. See PDF for more details.

Design details

Randomization method
The movies are assigned to the treatment arms when we target the participant for enrollment and the script is executed on the MovieLens servers. The movies we elicit beliefs about and recommend movies recommended are selected by an algorithm introducing computer-generated independent noise. See PDF for details.
Randomization unit
Our experiment is a within-subjects experiment where we randomize the set of movies participants are exposed to and recommended into two treatment arms and a control arm.
Clusters
Our experiment is a within-subjects experiment.
Observations
We target 4507 participants of the platform that match our eligibility conditions and ask them to participate in the experiment. We aim for 1100 participants to opt into the study with an average of 20 survey responses per participant.
Arms
Our experiment is within-subjects and for each participant we allocate 250 movies to each treatment arm.
Primary outcomes
Consumption choices (movie watching activity), elicited beliefs (expected rating and associated degree of uncertainty).

Cited by

No citation data available for this registration yet.

Keywords