Improving the Accuracy of Food Security Measurement: A Randomized Survey Experiment on Mode and Respondent Selection in Colombia
How to cite
Nastro, Espedito; Dietrich, Stephan.
2026. "Improving the Accuracy of Food Security Measurement: A Randomized Survey Experiment on Mode and Respondent Selection in Colombia." AEA RCT Registry. AEARCTR-0018176.
https://doi.org/10.1257/rct.18176-2.0
Download:
BibTeX
·
RIS— for reference managers without a browser
translator. Zotero and most
others pick up this page's citation metadata directly.
This study evaluates how survey mode and respondent selection protocols affect the accuracy of food security estimates. The experiment is embedded in a household survey implemented by the World Food Programme (WFP) in Colombia. Households are randomly assigned to one control group and three treatment groups: (1) a food diary benchmark group, (2) a face‑to‑face survey group, (3) a phone survey group without respondent selection protocol, and (4) a phone survey group with a respondent selection protocol designed to identify knowledgeable respondents involved in food‑related household decisions.
The research addresses three questions: (1) whether phone surveys generate less accurate food security estimates compared to face‑to‑face surveys when benchmarked against food diaries; (2) whether strategic and non‑strategic misreporting explain discrepancies across survey modes; and (3) whether implementing a respondent selection protocol improves the accuracy of phone survey estimates.
Food security outcomes are measured using standard food secuirty indicators, including the Food Consumption Score (FCS), the Food Insecurity Experience Scale (FIES), the Reduced Coping Strategy Index (rCSI), and the Livelihood Coping Strategy Index (LCSI). Strategic misreporting is measured by comparing self‑reported category in the national social protection registry (SISBEN) with verified administrative records. Non‑strategic misreporting is measured using attention, consistency checks and survey evaluation from the interviewed households embedded in the survey instrument.
The study contributes to the literature on survey methodology and food security measurement by identifying mechanisms driving measurement error across survey modes and by testing a protocol designed to improve the reliability of remote survey data.
Intervention
Four experimental arms are implemented: • Control (Food Diary Benchmark): Households record food consumption through a structured daily diary with remote enumerator follow-up and supervision. This group provides a benchmark estimate minimizing recall bias. • Treatment 1 - Face‑to‑Face Survey: Enumerators administer the full household questionnaire through in‑person interviews. • Treatment 2 - Phone Survey (Standard): Households receive the questionnaire via phone interview without any respondent selection protocol. This arm replicates the current remote survey modality commonly used by WFP. • Treatment 3 - Phone Survey with Respondent Selection Protocol: Phone interviews are conducted only if the respondent meets pre‑specified criteria indicating knowledge about food consumption, involvement in household food purchasing and food preparation decisions. If the respondent does not meet the criteria, another household member is identified.
Experimental design
The study is a randomized survey experiment conducted in Colombia. Households from selected departments previously interviewed in a nationwide WFP food security assessment serve as the sampling frame. Households are randomly assigned to four groups: one control and three treatments. The control group completes food consumption diaries, while the treatment groups receive surveys through different modes (face‑to‑face or phone). One phone survey arm includes a respondent selection protocol designed to ensure that the respondent is knowledgeable about household food consumption and purchasing decisions. The diary group provides a benchmark against which survey‑based estimates are compared to assess measurement accuracy.
Design details
Randomization method
Randomization occurs at the household level. Households are randomly assigned to one of the four experimental arms using statistical software prior to field implementation. Randomization ensures balance across groups.
Randomization unit
Households
Clusters
Total planned sample size is approximately 400 households, corresponding to approximately 100 households per experimental arm. Replacement households are pre‑randomized to address non‑response.
Observations
Total planned sample size is approximately 400 households, corresponding to approximately 100 households per experimental arm. Replacement households are pre‑randomized to address non‑response.
Arms
Total planned sample size is approximately 400 households, corresponding to approximately 100 households per experimental arm. Replacement households are pre‑randomized to address non‑response.
Primary outcomes
Food security is measured through the following indicators:
• Food Consumption Score (FCS)
• Food Insecurity Experience Scale (FIES)
• Reduced Coping Strategy Index (rCSI)
• Livelihood Coping Strategy Index (LCSI)
Primary outcomes measure differences in food security estimates across survey modes and relative to the diary benchmark.
The respondent selection protocol is designed to ensure that the individual answering the survey is knowledgeable about household food consumption and food-related decisions. At the beginning of the interview, enumerators ask the following screening questions to determine whether the respondent is involved in purchasing, preparing, or managing food for the household:
• In the last 7 days, how many days were you aware of the food consumed by all members of your household, both at home and away from home?
• In the last 7 days, how many days have you participated in food purchasing decisions for your household?
• In the last 7 days, how many days have you participated in food preparation for your household?
These screening questions are asked in all survey modes (face-to-face and phone). However, while responses to these questions do not affect participation in the face-to-face survey and in the phone survey without protocol, in the phone survey with protocol the responses determine eligibility to continue the interview. If the respondent does not meet the minimum eligibility criteria, the enumerator is prompted to identify and interview another adult knowledgeable household member. This procedure aims to reduce measurement error arising from interviewing respondents who have limited knowledge of household consumption.
Secondary outcomes
An additional research question of the study is to identify the mechanisms that may explain potential differences in food security estimates across survey modes. For this reason, the analysis also examines indicators of strategic and non-strategic misreporting. Strategic misreporting refers to the intentional misreporting of information when respondents perceive potential benefits from doing so, while non-strategic misreporting refers to reporting errors arising from recall limitations, inattention, or misunderstanding of questions. Investigating these mechanisms helps assess whether observed differences across survey modalities are driven by behavioral incentives or by cognitive and survey design factors affecting response accuracy.
Strategic misreporting indicator:
• discrepancy between self‑reported and verified SISBEN status.
Non‑strategic misreporting indicators:
• Comprehension and attention checks:
o the survey first introduces the World Food Programme (WFP) to respondents and later asks them to identify or explain what WFP is.
• Inconsistencies:
o The dairy group (milk, yogurt, cheese) in the Food Consumption Score module is repeated to check consistency in responses.
o The ‘reduce number of meals per day’ strategy of the reduced Coping Strategy Index (rCSI) is repeated to check consistency in responses.
• Respondents’ feedback:
o Survey experience on a scale of 0 to 10.
o Survey length feedback
o Enumerators’ feedback of respondents’ attention level during interviews
o Enumerators’ feedback of interruptions during interview.
Cited by
No citation data available for this registration yet.