Ranking effects in recipe choices : a randomized controlled trial in a food app

How to cite

Pourieux, Matthieu; Lohéac, Youenn; Ackermann, Claire-Lise; Matson, Sheila. 2024. "Ranking effects in recipe choices : a randomized controlled trial in a food app." AEA RCT Registry. AEARCTR-0012705. https://doi.org/10.1257/rct.12705-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Matthieu Pourieux* , Youenn Lohéac , Claire-Lise Ackermann , Sheila Matson

* Primary investigator

Registry ID
AEARCTR-0012705
DOI
https://doi.org/10.1257/rct.12705-1.1
First registered
Last updated
Status
In development
Trial dates
2024-01-01 – 2024-02-29
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/12705

Abstract

Food consumption has two main impacts: (1) on the human health through nutrition, and (2) on the whole environment through the life cycle of products. From a policy perspective, favoring the consumption of healthy and environmental-friendly products is both a major public health issue and one important lever to address climate change. However, consumers only have limited information per se about the true qualities of a given food product, which may hamper their ability to decide on what they can safely eat or stop eating. A primary way of informing consumers about their food purchases is the use of labels, and especially 5-colors labels (from A to E) whose effectiveness has been scientifically documented. In Europe, and particularly in France, the NutriScore has been adopted as the main tool to inform consumers about the nutritional quality of products or meals. As for the environmental impact of goods, the Ecoscore is emerging as a potentially effective candidate.
There exists a large literature on the impact of nutritional labels, and in particular the NutriScore, on consumers' food quality intake as well as a growing literature focusing on the effects of environmental labelling (such as the EcoScore). However, studying the combination of both types of labels is relatively new. At the same time, that labels may enable to achieve the reduction in consumption of both unhealthy and environmentally-damaging products is debatable. Moreover, product information is only one dimension of the food decision, which is also heavily dependant on the broad choice architecture, and particularly on the way products are presented to consumers. In physical food purchases (online shop, supermarket, recipe book, etc.), products are organised by categories, price families, or other rankings that are decided by sellers. In web or mobile apps, which are increasingly used in daily food decisions, products are often ranked by popularity or by grades provided by past customers (e.g: restaurants). Selective ranking of products, which is a type of “nudge”, has indeed been documented as a powerful lever for behavioral change. If it was be possible to rank the products, meals or recipes by their NutriScore or their EcoScore (from A to E, from green to red), what would be the impact on the nutritional and environmental quality of food choices ?
To answer this question, we build a field experiment based on naturally-occuring food decisions using a mobile phone application which offers users the possibility to search for recipes that fit their needs and habits. We partner with the private firm that owns the application to exogenously manipulate the ranking of proposed recipes given users' inputs. Through randomization into treatments and observation of all users in a pre-intervention phase (with ranking by recipes' grades as baseline), we are able to assess the ceteris paribus effect of ranking recipes based on either the NutriScore or the EcoScore on both the nutritional and environmental quality of selected recipes.

Intervention

The experiment consists in modifying an existing food application that is maintained by a private-sector firm with which we partner. The application is explicitly designed to reduce food waste and offers its users access to a search engine of recipes that is tailored to the user’s food habits. First, users build a profile based on their food habits (diet, daily food supplies) and, second, they can search for recipes that contains an ingredient that they want to consume. The search engine then provides users with a list of recipes that fits the user’s profile. Upon clicking on a recipe, a user can access the details of the recipe (including, e.g., the ingredients, the steps of the recipe, estimations of the cooking time and budget, etc.). In the baseline version of the application, the recipes are ranked according to the grades given by users to the recipes. In addition to the grades, the search engine also displays the NutriScore and the EcoScore of recipes. With the agreement and support of our private partner, our intervention consists in modifying the ranking of recipes based on such scores: the first (resp. second) treatment ranks recipes based on the underlying values of the NutriScore (resp. EcoScore) from A to E. Our intervention is organized in two steps: first, a purely observational phase where the application is not modified, and, second, an intervention phase where the treatments are implemented (alongside a control condition).

Experimental design

The experiment is a Randomized Controlled Trial within a food application: all users of the food application will be followed over time and will be (uniform-)randomly assigned to one of three groups. One group serves as control and is not assigned to a treatment: the ranking of recipes in the search engine will still rely on recipes’ grades. The two remaining groups are assigned to a treatment that modifies the ranking of recipes: it will be based on the underlying value of either the NutriScore or the EcoScore. All rankings will appear in descending order: the recipe with the highest grade/value of NutriScore/value of EcoScore is always ranked first. Before the intervention phase where the treatments are implemented, all three groups will be observed in the no-modification environment (observational phase). Both phases last for a month: the observational phase started on 01/01/2024 and the intervention phase will begin on 01/02/2024. During the whole process the experimenters will not have access to the data nor to the application: it is only after all the data is collected that our private partner will give us access to the data (which is regularly saved on the partner’s servers). After the intervention, the application will resume to its pre-intervention status (i.e. recipes ranked by grades). Notably, upon registering a profile on the application, all users must agree on the principle that they may participate in scientific studies.

Design details

Randomization method
Computer-based randomization issued by the private-sector partner.
Randomization unit
Individual.
Clusters
Based on observations from past years, we expect from approximately 900 (2022) to 1200 (2023) active users in the application in both January and February 2024. Given that these numbers are compatible with our power analysis, and because the actual total number of users cannot be decided in advance, we decided to run the experiment on the the whole database rather than to randomize a power-compatible pre-defined number of users into our experiment.
Observations
Since the design is not clustered, the number of observations is identical to the number of clusters: from 900 to 1200.
Arms
About 300 to 400 for each treatment.
Primary outcomes
We have two primary outcomes which we construct as follows. First, we define a “selected” recipe as a recipe that is accessed by a user for more than 30 seconds. Second, for each “selected” recipe we extract the numerical values of both the NutriScore and EcoScore. Third, we compute the average value of each score per user per phase. Ultimately, we obtain two primary outcomes: the individual averages of both the NutriScore and the EcoScore.
Secondary outcomes
We also have two secondary outcomes we intend to use to run robustness checks on our main analyses: the individual averages of the ratio between the value of a score (NutriScore or EcoScore) of the selected recipe and the average score of the recipes that have been “viewed” by the user. “Viewed recipes” are defined as all the recipes above the lowest point in the list from the search engine that has been reached by the user.

Cited by

No citation data available for this registration yet.

Keywords