Investigating Algorithmic Bias on Spotify
How to cite
Lumeau, Marianne; Moreau, François; Coavoux, Samuel; Aly-Tovar, Ramadan Jose. 2024. "Investigating Algorithmic Bias on Spotify." AEA RCT Registry. AEARCTR-0014495. https://doi.org/10.1257/rct.14495-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
In this “sock puppets” experiment, we test whether the recommendation algorithm of a music streaming platform is biased in favor of titles from large economies (here, USA) to the detriment of titles from a small economy (here, France). To do this, we created fictitious users that vary along four dimensions. Firstly, they vary according to their preference for small-economy titles (No, Small, Large, Total). In addition, the profiles vary according to their location (France and USA), their preference for recent or old titles and whether or not they consume titles recommended by the algorithm. Each day of the experiment, users will consume titles in the Get Recommendations API according to their profile (input data). After each consumption phase, the recommended titles are recorded (output data). We can then compare the difference between the proportion of titles of one nationality in a user's input data and the proportion of titles of the same nationality in the recommendation list (output data). We can test whether the result varies according to the preference for the small-economy music, but also to the user's location, the age of the titles and whether the user consumes recommended titles.
Intervention
Several countries have implemented broadcasting quotas for radio and TV, requiring stations to allocate a percentage of airtime to locally produced music. The proponents defend such strategy as a channel to promote and preserve local culture identities.
The rise of digital streaming platforms raised new concerns regarding the supply of cultural goods.
As streaming platforms rely on recommendation systems (editorial or algorithmic), partisans of govern- ment regulation are especially concerned with whether and to what extent these systems steer consumers, eventually having a negative impact on the consumption and production of local content.
For example, platforms could intentionally recommend certain content, putting them on a spotlight position such as playlists (Aguiar & Waldfogel, 2021) or other editorial recommendation systems.
Additionally, algorithms used to automatically recommend content to consumers could be biased, either intentionally or unintentionally, towards certain types of content, potentially undermining the representation of local content. For example, if a streaming service's algorithm predominantly recommends international hits over local music, it can reduce the visibility and reach of local artists. This could happen, for example, if streaming platforms intentionally favoured certain artists if such recommendations increased the platforms’ profitability (Bourreau & Gaudin, 2022). Recommendations could also be biased due to the lack of representativeness of data at a global level (Hesmondhalgh et al., 2023).
In this context, the aim is to study whether algorithmic recommendations introduce a bias against local titles and in favor of titles from large economy, such as USA.
To answer, we will run a “sock puppets” experiment on the Spotify's API Get Recommendations, based on the creation of fictitious users (bots). Each fictitious user is assigned to a profile, varying mainly according to their preference for small-economy titles (high, medium, zero). Here, the small economy is France.
Experimental design
We will run a “sock puppets” experiment on the Spotify's API Get Recommendations. This type of experiment is based on the creation of fictitious users.
Each fictitious user is assigned to a profile, varying according four dimensions. Firstly, they vary according to their preference for small-economy titles: No (0 French title and 5 US titles in input data), Small (2 French titles and 3 US titles in input data), Large (3 French titles and 2 US titles in input data), Total (5 French titles and 0 US title in input data).
In addition, the profiles vary according to their location (France versus USA), their preference for recent music (Yes, No) and whether they "consume" titles recommended by the algorithm (Yes, No). For location, we register each user in the API according to her assigned nationality and use proxy (VPN).
Each day of the experiment, users will "consume" 5 titles in the Get Recommendations API according to their profile. These titles are randomly chosen in a list of titles according to their profiles. These list have been created using the Top songs of Spotify and the SNEP (the dominant union in the French music industry). This procedure is used to check the success of input data. During the first seven days, the procedure is identical whether the fictitious user is assigned to the “consume recommendations” condition or not. However, from the eighth day onwards, the 5 titles of fictitious users assigned to the “consume recommendations” condition are drawn at random from the list of titles recommended to them the previous day (d-1). For the others, the experiment continues as before.
After each consumption phase, we will record the 20 recommended titles.
Design details
Cited by
No citation data available for this registration yet.