High-dimensional preference elicitation through Bayesian evaluation of binary bundle prompts - evidence from Bangladesh
How to cite
Schaffner, Nikolai; Rahman, Khandker Wahedur. 2022. "High-dimensional preference elicitation through Bayesian evaluation of binary bundle prompts - evidence from Bangladesh." AEA RCT Registry. AEARCTR-0009885. https://doi.org/10.1257/rct.9885-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Preference elicitation is a key challenge within the analytical framework of microeconomics. In this intervention, we propose a novel empirical and Bayesian approach to measuring individual preferences in a developing-country context. We present participants with binary choices between bundles of goods - recorded choices are then aggregated into preference estimates using Gaussian Processes. In this proof-of-concept study, we aim to: i) demonstrate that this method is capable of delivering important preference metrics (such as cross elasticity estimates) in the field, and ii) illustrate the robustness of this method. Our analysis is based on novel survey data collected in collaboration with BRACU in Bangladesh.
Intervention
This Pre-Analysis Plan (PAP) concerns the analysis of survey data collected in
March 2022 as a product of a collaboration between Nikolai Schaffner (Univer-
sity of Oxford), Khandker Wahedur Rahman (University of Oxford and BIGD,
BRAC University), and Farhana Kabir (BIGD, BRAC University). Rahman
and Kabir have conducted a survey (subsequently “the frame survey”) among
a random sub-selection of households drawn from the microcredit recipient
database of BRAC. The frame survey was conducted as part of a separate
study (subsequently “original experiment”) aimed at improving the usage of
microcredit by female borrowers.
The collaboration that is the subject of this PAP represents a completely
distinct research project that only shares participants and access points with
the preceding survey. The data that is the subject of this PAP was collected as
part of an add-on module (subsequently “the preference module”) to the frame
survey described above.
The primary research question that motivates the preference module is:
“Can consumer preferences over high-dimensional good universes be measured
easily, quickly, and reliably in the context of a survey?” While being of highly
universal interest to economic research, this question is particularly interesting
(and challenging) in exactly the type of context that presents itself with the
respondent sample for the frame survey in this case. Namely, this includes the
absence of readily available market price signals and the presence of various
sources of mental stress on participants.
The relevance of this is rather clear: It is a routine challenge for organisations
involved in the distribution of aid/relief goods to compose bundles of goods most
likely to meet the demand (and therefore preferences) of recipients in need of
support.
Experimental design
After studying existing applied microeconomics literature on non-market-based
preference elicitation techniques, a key challenge appears to be to tackle the case
of measuring preferences over many different types of goods. Many approaches
that allow the expression of preferences over a large selection of goods simulta-
neously2 impose a significant attention cost and are therefore not feasible in a
high-stress environment typically faced by aid operations.
To solve this high-dimensionality challenge, this study makes use of two
elements:
• Binary bundle choice as a survey input device: The most basic building
block of the preference module is a presentation of two bundles of goods
at a time to the recipient, combined with the question of which of the two
(hypothetical) bundles the recipient would prefer.
• Gaussian Process (subsequently “GP”) estimation to support the trans-
formation of these binary preference input data points into a comprehen-
sive preference construct that enables deduction about relative preferences
across all queried goods.
One example of a question would therefore be:
• Which of the following bundles would you prefer if you could freely choose:
– Bundle 1: 5,000 Bangladeshi Taka (BDT) and 1 motorbike
– Bundle 2: 20 litres of petrol and 50 kilos of building-grade sand
The types of goods used in the bundles were drawn up from an informal
survey among BRAC loan practitioners, who were asked to name the goods
loan recipients would typically purchase with their funds.
Bundles and then binary comparisons of two bundles each were drawn ran-
domly: Comparisons always consist of at least three different goods, fixed at
at most 6 units per good - such that the summed monetary value of goods
in a bundle is always within 95 to 100 percent of a predefined budget. The
corresponding prices were collected in the following procedure: BIGD asked
enumerators dispersed all over the country to find out the current market prices
of different commodities in the areas where they were staying in for the purpose
of work. Once they sent the prices of these commodities from different areas,
the average price was calculated and used to estimate the bundle prices.
In order to highlight the effect of idiosyncratic bundle choice as well as to
detect potentially trivial questions, questions were pre-randomized as entire sets
of 30 questions each, called “question runs”.
The exact nature of bundles presented varied across two treatment arms -
within-subject: 15 questions of each run were designed as 2 goods + 1 good and cash, 15 goods of each run were designed as 2 goods + 2 goods, order of sections
is randomized; between-subject: one treatment answered questions drawn from
a universe of 25 goods, whereas another treatment answered questions drawn
from a universe of 50 goods, treatment assignment is randomized.
Finally, each run contains one redundant choice validation question - i.e. one
question that contains the same bundles shown previously but in reverse order.
Design details
Cited by
No citation data available for this registration yet.