The Effects of Phone-Based Surveys on Measurement Quality: When and Why Does Modality Matter?

How to cite

McKenna, Margaret; Chen, Lipeng; Baseler, Travis; Ginn, Thomas. 2022. "The Effects of Phone-Based Surveys on Measurement Quality: When and Why Does Modality Matter?." AEA RCT Registry. AEARCTR-0010053. https://doi.org/10.1257/rct.10053-3.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Margaret McKenna* , Lipeng Chen , Travis Baseler , Thomas Ginn

* Primary investigator

Registry ID
AEARCTR-0010053
DOI
https://doi.org/10.1257/rct.10053-3.0
First registered
Last updated
Status
On going
Trial dates
2019-11-22 – 2023-12-31
Countries
Private
JEL codes
C42, C81
Registry record
http://www.socialscienceregistry.org/trials/10053

Abstract

Applied research relies heavily on data quality. As more and more researchers collect their own data through surveys, how to ensure data quality is a central issue for survey data. In this project, we plan to study the impact of survey modality on data quality. Specifically, we introduce three treatments in a field experiment on a set of 900 micro-entrepreneurs in Uganda: 1) whether to conduct the survey in person or over the phone, 2) whether to fix enumerator-respondent pairings across survey rounds, and 3) whether to include a trust-building activity prior to the survey. We also cross-cut the three treatments, which can help us understand how these treatments interplay with each other. We will assess how measurement varies for 1) simple, objective questions, 2) complex, objective questions, 3) subjective, sensitive questions, and 4) subjective, non-sensitive questions. We also measure and bound experimenter demand effects (EDE) by telling respondents the results we expect in a donation allocation task. We plan to study how EDE vary with each of our treatments.

Experimental design

We randomize 1) whether to conduct the survey in person or over the phone, 2) whether to fix enumerator-respondent pairings across survey rounds, and 3) whether to include a trust-building activity prior to the survey for a survey conducted on 900 micro-entrepreneurs in Uganda. We cross-cut these three treatments. We will study how measurement differs across treatments.

We also randomize the respondents into one of the two groups which receive different priming scripts prior to a donation game to measure experimenter demand effects.

Design details

Randomization method
Randomization is done by computer.
Randomization unit
The unit of randomization is individual.
Clusters
900 individuals
Observations
900 individuals
Arms
There are 3 different treatments and we cross-cut them, so there are in total 8 treatment arms: 1) in-person survey, familiar enumerator, trust-building exercise at the beginning of the survey, 2) in-person survey, new enumerator, trust-building exercise at the beginning of the survey, 3) in-person survey, familiar enumerator, no trust-building exercise at the beginning of the survey, 4) in-person survey, new enumerator, no trust-building exercise at the beginning of the survey, 5) phone survey, familiar enumerator, trust-building exercise at the beginning of the survey, 6) phone survey, new enumerator, trust-building exercise at the beginning of the survey, 7) phone survey, familiar enumerator, no trust-building exercise at the beginning of the survey, 8) phone survey, new enumerator, no trust-building exercise at the beginning of the survey. Each treatment arm has around 115 individuals.
Primary outcomes
We are interested in assessing measurement differences in 1) simple, objective questions, such as basic household and business information, 2) complex, objective questions, such as business profit, 3) subjective, sensitive questions, such as social and political attitudes toward controversial topics, and 4) subjective, non-sensitive questions, such as subjective well-being. We are also interested in how experimenter demand effects, defined as the impact of priming on the donation task, vary with each of our treatments.

Cited by

No citation data available for this registration yet.

Keywords