Measuring How Users Value Meta’s Family of Apps and Other Digital Goods

How to cite

Liaqat, Asad; Brynjolfsson, Erik; Collis, Avinash; Deisenroth, Daniel; Garro, Haritz; Kutzman, Daley; Lee, Jae Joon; Wernerfelt, Nils. 2022. "Measuring How Users Value Meta’s Family of Apps and Other Digital Goods." AEA RCT Registry. AEARCTR-0008990. https://doi.org/10.1257/rct.8990-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Asad Liaqat* , Erik Brynjolfsson , Avinash Collis , Daniel Deisenroth , Haritz Garro , Daley Kutzman , Jae Joon Lee , Nils Wernerfelt

* Primary investigator

Registry ID
AEARCTR-0008990
DOI
https://doi.org/10.1257/rct.8990-1.1
First registered
Last updated
Status
In development
Trial dates
2022-02-28 – 2023-02-28
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/8990

Abstract

We will administer incentivized, on-platform surveys in 15 countries using the double-bounded dichotomous choice (DBDC) method to elicit users’ valuation for Facebook. We will also elicit users’ hypothetical valuation of Instagram and Whatsapp using the single-bounded dichotomous choice (SBDC) method. In both DBDC and SBDC valuations, we will randomly vary offer values across users. Finally, we will also elicit users’ hypothetical valuations for other digital goods using the best-worst scaling (BWS) method where the options presented to users will vary randomly. We will run these surveys on the Facebook platform. We will estimate the median valuation of users for each app in each country.

Intervention

We will run the survey on the Facebook platform. Users will receive an invitation to participate in a survey. If users accept the terms and conditions, we will then elicit their valuations for Facebook using DBDC. They will first be asked if they would be willing to stop using Facebook for a month in exchange for an amount of money that will be randomized across users. Users who accept the first offer will be offered a smaller amount in the second round, and users who reject the first offer will be offered a higher amount (users will not be aware of this protocol to avoid strategic choices). For a small set of users, their responses will be taken into consideration and they will be offered the money they accepted in exchange for deactivating Facebook for a month. We will monitor their activation status and pay them the offered amount at the end of the 1 month period if they stay deactivated for the whole month.
We will then ask users hypothetical questions to separately elicit their valuations for Instagram and Whatsapp. These valuations will be elicited using the SBDC method, which uses a single question to users asking whether they would be willing to stop using the app in exchange for a random amount of money.
Finally, to contextualize these estimates, we will ask users hypothetical questions using BWS (best-worst scaling) to elicit their valuations for other digital goods such as Snapchat, Google Search, or YouTube. We will present users with a random subset of choices, where the combinations of choices are randomly chosen using a balanced-incomplete block design. Users will be asked to choose the best and worst options according to the BWS approach. Possible options could be: stop using Google Search for a month, earn $50 less for a month, and stop using Snapchat for a month.

Experimental design

We will randomize participants into a set of groups that will be offered different initial offer values. The group with the lowest offer values can be seen as a control group, while those with higher offer values can be seen as treatment groups. In each group, users will first be asked if they would be willing to stop using Facebook for a month in exchange for a random amount of money within a subset of monetary values.

For more efficient data collection, users will be offered a second monetary value: users who accept the first offer will be offered a smaller amount in the second round, and users who reject the first offer will be offered a higher amount.

By comparing take-up rates across groups that received different offer values, we will estimate the median valuation that users in a sample place on Facebook.

An analogous approach will be used to estimate the median valuation that users in a sample place on Instagram and WhatsApp, except that this valuation will be based on hypothetical questions asked using an SBDC approach.

Design details

Randomization method
Randomization done by a computer
Randomization unit
Individual
Clusters
N/A
Observations
We expect to have responses from around 70,000 users across 15 countries.
Arms
The total sample of 70,000 users will be evenly distributed across 9 initial offer values.
Primary outcomes
The main outcome of interest will be the value individuals across the 15 countries assign to Facebook. We will analyze how these valuations vary across subpopulations of respondents, such as country, age, gender, or education, etc. and how these valuations compare to users’ hypothetical valuations for other digital services such as WhatsApp, Instagram, Snapchat, or TikTok. We will calculate median valuations within groups of users using the methods described below.

Cited by

No citation data available for this registration yet.

Keywords