High-dimensional preference elicitation through Bayesian evaluation of binary bundle prompts - evidence from Bangladesh

How to cite

Schaffner, Nikolai; Rahman, Khandker Wahedur. 2022. "High-dimensional preference elicitation through Bayesian evaluation of binary bundle prompts - evidence from Bangladesh." AEA RCT Registry. AEARCTR-0009885. https://doi.org/10.1257/rct.9885-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Nikolai Schaffner* , Khandker Wahedur Rahman

* Primary investigator

Registry ID
AEARCTR-0009885
DOI
https://doi.org/10.1257/rct.9885-1.0
First registered
Last updated
Status
Completed
Trial dates
2022-03-16 – 2022-04-02
Countries
Bangladesh (Multiple)
Registry record
http://www.socialscienceregistry.org/trials/9885

Abstract

Preference elicitation is a key challenge within the analytical framework of microeconomics. In this intervention, we propose a novel empirical and Bayesian approach to measuring individual preferences in a developing-country context. We present participants with binary choices between bundles of goods - recorded choices are then aggregated into preference estimates using Gaussian Processes. In this proof-of-concept study, we aim to: i) demonstrate that this method is capable of delivering important preference metrics (such as cross elasticity estimates) in the field, and ii) illustrate the robustness of this method. Our analysis is based on novel survey data collected in collaboration with BRACU in Bangladesh.

Intervention

This Pre-Analysis Plan (PAP) concerns the analysis of survey data collected in
March 2022 as a product of a collaboration between Nikolai Schaffner (Univer-
sity of Oxford), Khandker Wahedur Rahman (University of Oxford and BIGD,
BRAC University), and Farhana Kabir (BIGD, BRAC University). Rahman
and Kabir have conducted a survey (subsequently “the frame survey”) among
a random sub-selection of households drawn from the microcredit recipient
database of BRAC. The frame survey was conducted as part of a separate
study (subsequently “original experiment”) aimed at improving the usage of
microcredit by female borrowers.
The collaboration that is the subject of this PAP represents a completely
distinct research project that only shares participants and access points with
the preceding survey. The data that is the subject of this PAP was collected as
part of an add-on module (subsequently “the preference module”) to the frame
survey described above.
The primary research question that motivates the preference module is:
“Can consumer preferences over high-dimensional good universes be measured
easily, quickly, and reliably in the context of a survey?” While being of highly
universal interest to economic research, this question is particularly interesting
(and challenging) in exactly the type of context that presents itself with the
respondent sample for the frame survey in this case. Namely, this includes the
absence of readily available market price signals and the presence of various
sources of mental stress on participants.
The relevance of this is rather clear: It is a routine challenge for organisations
involved in the distribution of aid/relief goods to compose bundles of goods most
likely to meet the demand (and therefore preferences) of recipients in need of
support.

Experimental design

After studying existing applied microeconomics literature on non-market-based
preference elicitation techniques, a key challenge appears to be to tackle the case
of measuring preferences over many different types of goods. Many approaches
that allow the expression of preferences over a large selection of goods simulta-
neously2 impose a significant attention cost and are therefore not feasible in a
high-stress environment typically faced by aid operations.
To solve this high-dimensionality challenge, this study makes use of two
elements:
• Binary bundle choice as a survey input device: The most basic building
block of the preference module is a presentation of two bundles of goods
at a time to the recipient, combined with the question of which of the two
(hypothetical) bundles the recipient would prefer.
• Gaussian Process (subsequently “GP”) estimation to support the trans-
formation of these binary preference input data points into a comprehen-
sive preference construct that enables deduction about relative preferences
across all queried goods.
One example of a question would therefore be:
• Which of the following bundles would you prefer if you could freely choose:
– Bundle 1: 5,000 Bangladeshi Taka (BDT) and 1 motorbike
– Bundle 2: 20 litres of petrol and 50 kilos of building-grade sand
The types of goods used in the bundles were drawn up from an informal
survey among BRAC loan practitioners, who were asked to name the goods
loan recipients would typically purchase with their funds.
Bundles and then binary comparisons of two bundles each were drawn ran-
domly: Comparisons always consist of at least three different goods, fixed at
at most 6 units per good - such that the summed monetary value of goods
in a bundle is always within 95 to 100 percent of a predefined budget. The
corresponding prices were collected in the following procedure: BIGD asked
enumerators dispersed all over the country to find out the current market prices
of different commodities in the areas where they were staying in for the purpose
of work. Once they sent the prices of these commodities from different areas,
the average price was calculated and used to estimate the bundle prices.
In order to highlight the effect of idiosyncratic bundle choice as well as to
detect potentially trivial questions, questions were pre-randomized as entire sets
of 30 questions each, called “question runs”.
The exact nature of bundles presented varied across two treatment arms -
within-subject: 15 questions of each run were designed as 2 goods + 1 good and cash, 15 goods of each run were designed as 2 goods + 2 goods, order of sections
is randomized; between-subject: one treatment answered questions drawn from
a universe of 25 goods, whereas another treatment answered questions drawn
from a universe of 50 goods, treatment assignment is randomized.
Finally, each run contains one redundant choice validation question - i.e. one
question that contains the same bundles shown previously but in reverse order.

Design details

Randomization method
Researchers randomly pre-assigned each unique participant ID to one of the four treatments using Stata.
Randomization unit
Treatment was (randomly) assigned at the level of individuals.
Clusters
50 branches of BRAC
Observations
Data was gathered through in-person surveys from 26 districts across 50 BRAC Microfinance (BRAC MF) Branch Offices. From each branch office, we randomly chose 2 Village Organizations (VO). So we surveyed 100 VOs in total. The number of observations is 500.
Arms
125 individuals were assigned to each treatment arm.
Primary outcomes
Two main sets of panel regressions shall be employed: • Determinants of best-fit demand bundles: LHS variable are the pseudo- demand vectors estimated to be best-fit. These are regressed on a broad selection of controls include demographics, household characteristics ( incl. treatment) and aim to indicate systematic predictors of demand. This analysis constitutes a key application of our preference elicitation method for any potential policymakers aiming to leverage it - and this regression is therefore an important part of our proof of concept. • Preference prediction quality: These regressions are analogous to those described above - however, the LHS variables contain the measures of preference elicitation quality described above (bundle estimate stability, rationality parameters etc.)

Cited by

No citation data available for this registration yet.

Keywords