Evaluation of Interventions in Online Grocery Shopping for Sustainability and Health: An Adaptive Design Randomized Controlled Trial
How to cite
Rens, Thijs van; Oyebode, Oyinlola; Bentil, Helena. 2024. "Evaluation of Interventions in Online Grocery Shopping for Sustainability and Health: An Adaptive Design Randomized Controlled Trial." AEA RCT Registry. AEARCTR-0013247. https://doi.org/10.1257/rct.13247-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Effective interventions are needed to promote more sustainable and healthier food choices for both human and planetary health. This study will evaluate two interventions in two independent adaptive design randomised controlled trials (one 2-arm and one 3-arm trial) within the same study population (a factorial design is not powered). The interventions are (i) eco-labelling, which will provide participants with information on the environmental impact of their food purchases using a score ranging from A (most sustainable) to G (least sustainable); and (ii) price discounts on alternative products with a better sustainability profile (and equal or better nutritional profile) in place of specific products in their basket. To implement these interventions, we will use a browser extension on online shopping websites for one or multiple large UK supermarkets, accessed using the Google Chrome browser on a desktop or laptop computer. We will assess the effect of these interventions on the average eco-score of the basket (primary outcome) as well as its nutritional content (secondary outcome). Health outcomes of the intervention will be modelled using nutrition scores.
What is already known?
- The food system is responsible for more than one third of total global greenhouse emissions, while also contributing to land degradation and biodiversity loss. It will be necessary to reduce emissions from the food system to achieve agreed climate targets including a maximum global temperature increase of 1.5⁰C.
- Poor diets are responsible for 1 in 7 deaths in the UK. More environmentally sustainable diets are likely to be better for human health as well.
- Effective interventions are urgently needed to promote more sustainable and healthier food choices for both human and planetary health.
- Online grocery shopping is on an upward trend and understanding behaviour in this setting is therefore increasingly important. As well as elucidating effective approaches to shift online shopping towards healthy and sustainable purchases, studies of interventions in the online shopping environment also provide an opportunity to examine the behaviour of many consumers, which would be more difficult and more costly in physical supermarkets.
- There are no existing randomised controlled trials examining sustainability outcomes in real (not simulated) online grocery stores. Previous work has focused on healthier choices. Most of these studies found that economic interventions, such as price discounts, show the most promise, and a few found swaps to be effective as well.
- There are no existing randomised controlled trials evaluating the effect of price interventions in real online grocery stores. The setting of all previous studies on price interventions was either a physical supermarket or a simulated online supermarket. The very limited number of studies on swaps (2 studies) and information interventions (3 studies) in real online grocery stores provide mixed evidence, indicating that these interventions may prompt healthier purchasing in some circumstances for some people.
- Studying actual shopping behaviour in real online grocery stores is important because consumers often behave differently in simulated environments. For example, it has previously been noted that nutrition labelling appeared 17 times more effective in laboratory studies than real grocery settings.
- We have shown in a pilot study that it is feasible to run a field trial on an actual online supermarket website using a bespoke browser extension and have learned valuable lessons on how to implement such a trial.
What does this study add?
- This will be the first study to provide causal evidence on interventions to promote sustainable food purchases from a large sample of consumers in a real online grocery shopping environment.
- We will add to the evidence base on how choice architecture influences purchasing behaviours and provide evidence on two interventions: (i) eco-labelling, which will provide participants with information on the environmental impact of their prospective food purchases using a score ranging from A (most sustainable) to G (least sustainable); and (ii) price discounts on alternative products with a better sustainability profile (and equal or better nutritional profile) offered in place of specific products in participants’ baskets.
- Our study results will also allow us to calculate consumers’ willingness to pay for more sustainable groceries.
- As secondary outcomes, we will look at whether either intervention affects total purchases. We will also examine the impact on the nutritional composition of purchases and (modelled) health outcomes.
- This study will provide evidence about the potential for eco-labels, or for food subsidies that could alter the prices of sustainable food relative to less sustainable food, to change consumer behaviour to meet health and sustainability goals.
How will we determine this?
- A large-scale cross-sectional randomised controlled trial with real consumers doing their usual shopping over a two-month period on the website of a large UK supermarket. The trial will be implemented using an adaptive design in up to five waves for cost reasons.
- Members of the public were involved in the design of this study and associated study materials.
- Participants will be recruited from the UK Prolific panel and will be compensated for their participation in this study. The Prolific panel is not representative of the UK population but provides good coverage of different socio-economic and demographic groups. We will only recruit participants who are the primary grocery shoppers for their household and who report buying groceries online at least once a month.
- Participants will be randomly allocated into different arms at the start of the trial. The two interventions will be independently randomised within the same study population so that there are effectively two independent trials. One of these will be a 2-arm trial (eco-labels on or off) and the other one a 3-arm trial (price discount of £1 or £0.50 or no discount). The study is not powered to detect any interaction between the two interventions being tested.
- To implement the interventions, we will use a browser extension (plug-in) that manipulates the supermarket website when participants access it through the Google Chrome browser on a desktop or laptop computer.
- The plug-in will collect data on product selections, substitutions and purchases. Additionally, survey participants will be surveyed for information on demographics and attitudes towards food shopping and sustainability.
- We will assess the effect of these interventions on the average eco-score of the basket (primary outcome) as well as its nutritional content (secondary outcome). Health outcomes of the intervention will be modelled using nutrition scores.
- A separate process evaluation will consider what worked, for whom, and in what circumstances.
Intervention
We will evaluate the following two interventions:
1. Eco-labelling: Eco-labels provide consumers with information on the environmental impact of their food purchases, graded from A (lowest impact, most sustainable) to G (highest impact, least sustainable) based on an underlying numerical score ranging from 0 (most sustainable) to 500 (least sustainable). These environmental impact scores were calculated based on the ingredients in 1kg of each product, which are then linked to the life-cycle analysis (LCA) database Agribalyse. The scores take into account the environmental impact of a product in 16 categories, including land use, water scarcity, resource use, human health, wildlife damage, and climate change. More details on the methodology are provided in appendix A.
2. Price discounts on more sustainable alternative products: Price discounts will be offered through swaps at the first checkout screen. Participants will see a pop-up window suggesting a more sustainable alternative for a particular product in their basket, which is offered at a randomly varying price discount. The alternative products are selected to not cost £2.00 more than the original product, and price discounts will be offered at three levels: £1.00, £0.50 or no discount. The intention is that this will result in discounts being roughly equally spaced between 0 and 100% of the price difference between the original and the alternative products (explicitly offering percentage discounts is not feasible for technical reasons, because it would involve “live” scraping of the prices). We will use this information to calculate the willingness to pay for more sustainable products. As an example, a participant may be prompted to swap Greek yoghurt (500g) priced at £2.30 for dairy-free coconut yoghurt (600g) priced at £3.60, and the pop-up will notify participants that they can buy the coconut yoghurt for £2.60, £3.10 or the full price of £3.60, depending on the trial arm that the participant was assigned to. If the shopper accepts the swap, they will be reimbursed for the price discount. Each participant will get offered a swap for a (potentially discounted) alternative product on up to 3 products in their shopping basket. The products for which a swap is offered are randomly chosen from a list, composed by the researchers, of not-so-sustainable products with suitable alternatives. To avoid offering alternative products with an inferior macronutrient profile, alternative products on the list will be selected so that they are in the same or a better category for fat, sugar and salt content on the traffic-light label.
Experimental design
Setting, Recruitment and Study Design
This protocol has been informed by a pilot study, which was run in September and October 2023. Methods and results of the pilot are presented in Bentil et al. (2024).
Our setting is the online grocery shopping environment on the website of a large UK supermarket. We will use a browser extension (plug-in) to implement the interventions and collect purchase data. Underlying the plug-in is a database with information on the environmental impact of over 160 thousand products. The plug-in was developed by Sustained (https://sustained.com) and customised based on our specifications. It can be used to display eco-labels and/or offer price discounts on more sustainable alternative products, depending on the participant’s ID, which is randomly allocated to one of the arms of the trial. The plug-in also records information on products in the shopping basket, any swaps or clicks on the eco-labels, and products that are eventually purchased.
Participants will be recruited from the online research platform, Prolific (https://www.prolific.com/). To be eligible for the study, participants must:
- Be 18 years or older and located in the UK
- Be the primary grocery shopper of their household
- Frequently buy groceries online (at least once per month, self-reported)
- Usually shop at the supermarket(s) included in this study
- Usually use a laptop or desktop with the Google Chrome browser for online grocery shopping or be willing to do so for the duration of the study
- Consent to participate and be willing to download and install the plug-in and use it for the duration of the study
- Not have participated in the pilot study or previous wave(s) of the study.
These eligibility criteria are informed by the pilot study. We found that take-up is much lower among participants who were asked to shop at a different supermarket than where they usually shop, but participants who usually shop on a mobile device and therefore need to change their shopping behaviour to use the browser extension are no less likely to shop using the extension than those who usually use a desktop or laptop (Bentil et al., 2024).
The study will use a cross-sectional Randomised Controlled Trial (RCT) design. For participants in the intervention group, the intervention will be on from the start and there will not be a baseline period. The reason for this is that the results from our pilot study showed that inter-participant variation in the primary outcome was similar to intra-participant variation. Therefore, there is no efficiency gain from controlling for individual-specific fixed effects by comparing changes rather than levels of purchases. In a longitudinal design, some observations are lost, because some participants did not shop either in the baseline or in the intervention period, and our pilot results consistently showed a more precise estimation of the treatment effect in a cross-sectional design.
The trial will run over 8 weeks. If self-reported shopping frequency were accurate, our eligibility criteria would guarantee 2 shops per participant in a one-month trial. However, the pilot study showed that participants were shopping substantially less frequently than they said they would. On the other hand, we found that attrition did not increase over the one-month duration of the pilot, suggesting that a longer trial duration would be preferable. After the end of the trial period, participants are allowed and encouraged to keep using the plug-in and we will continue to collect their purchase data for use in future research. We will explicitly ask for consent for this post-intervention period of data collection at the start of the trial.
We will use an adaptive design for the RCT (Figure 3). An adaptive design, rather than the traditional fixed sample size RCT design, was chosen because uncertainty regarding recruitment rates and attrition makes determining the sample size challenging. The adaptive design will enable a reassessment of the sample size requirements and the potential to stop the trial early should interim analyses indicate success or futility, ensuring a more efficient allocation of resources and time (Pallmann et al., 2018).
The study will thus be implemented in phases or waves. We will approach Prolific panellists in four or five waves of 10,000 people, with the aim of recruiting around 700 participants in each wave, until we reach the sample size required for a 2-arm trial with a 2-month intervention period or until it becomes clear that it is futile to try and reach this sample size. As illustrated in Figure 4, we will use data from wave 1, anticipated to start in April 2024, to determine the number of people who shop during the trial (recruitment rate). Based on this information, we will recalculate the sample size requirements and continue with wave 2 only if it seems likely that after a maximum of five waves, we will successfully recruit a sufficiently large sample. This exercise is repeated after each wave. Between the second and third waves there is a two-months break to avoid the summer holidays, when shopping patterns may be atypical. We will use this break to consider whether we need to relax our eligibility criteria, add a fifth wave, add a second supermarket, increase compensation, or make other changes to the study design.
We have obtained ethical approval for our study from the University of Warwick’s Humanities & Social Sciences Research Ethics Committee (reference: HSSREC 123/23-24). We will obtain written informed consent from participants through the online survey, which will start with an informed-consent form, approved by the HSSREC.
Data Collection Procedures
Prior to the collection of data, participants will be informed about the study and will need to provide written consent to participate, see appendix B for the PIC forms. They will also be asked to answer survey questions on eligibility, demographics and other background information, see appendix C for questionnaires. Informed consent forms and surveys will be administered using Qualtrics. We will recruit potential participants via the Prolific (https://www.prolific.com/). Participants will be compensated for completing the online surveys at a rate of £12 per hour, and for keeping the browser plug-in enabled while they shop for groceries using the Chrome browser. Compensation for grocery shopping will be a fixed amount of £10 per month (plus any price discounts) for participants who shop at least twice in that month, as recorded by the plug-in. Participants will receive regular reminders via the Prolific platform to prompt them to complete their usual grocery shopping on Chrome with the plug-in enabled throughout the duration of the trial.
Screening/baseline survey: At the start of the intervention period, we will survey potentially eligible participants to collect background characteristics, including age, gender, ethnicity, household income, household size and composition, as well as attitudes towards food shopping and sustainability, see appendix C for the questionnaire. The plug-in will be distributed as part of the baseline survey using a customised download link for each participant that includes their participant ID. The plug-in will remember this ID number and pass it on with all purchase data that are generated, allowing us to match the purchase data to the information collected through the surveys. The personalised download links will allow only a single installation for each ID number, to avoid multiple households using the same participant ID. When there is an attempt to use a download link for the second time, the user will get a message to contact the researcher for a second link if appropriate.
Grocery purchases: The plug-in will collect data on grocery purchases at the point of checkout. These data are collected for initiated, updated, purchased (paid for), and cancelled checkouts. For each product, the plug-in will record a detailed description of the product and pack size, including a retailer-specific ID number, the purchase price, the quantity purchased, as well as information on the product including the environmental impact scores (eco-scores).
Endline survey: This survey will be administered at the end of the intervention period to gather quantitative and qualitative data for process evaluations, see appendix C for the survey questionnaire.
Design details
Cited by
No citation data available for this registration yet.