Measuring the Unmeasured: Combining Technology and Survey Design to Filter Noise in Self-Reported Business Outcomes
How to cite
Anderson-Macdonald, Stephen; Zia, Bilal. 2014. "Measuring the Unmeasured: Combining Technology and Survey Design to Filter Noise in Self-Reported Business Outcomes." AEA RCT Registry. AEARCTR-0000251. https://doi.org/10.1257/rct.251-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Reducing noise and improving the measurement of business outcomes is critical to advancing research on private enterprise in low-income countries, particularly given the lack of record keeping (by entrepreneurs) and administrative data (by governments). To address many of the outstanding measurement challenges we have constructed an electronic survey tool that brings together the benefits of the latest surveying technologies with improvements in survey design to increase the precision of profit and sales estimates obtained through research on micro enterprises in low-income countries. We have identified four sources of noise, both in the literature and through our own research, which we will address through our new electronic surveying approach (e.g. respondent, business, surveyor and data management factors). By focusing on survey design and technology, our novel approach represents a first attempt at reconciling the recommendations of de Mel et al (2009) for increased attention to survey design, with those of Fafchamps et al (2012) for identifying ways to leverage an electronic survey tool beyond basic consistency checks.
Specifically, our new electronic survey tool is designed to improve the collection of data on three sections of a firm’s financials: (1) Money In; (2) Money Out; and (3) Money Left-Over. Through an iterative process, our electronic survey tool narrows in on a more precise estimate for firm sales, firm costs, and firm profits. Moreover, by default the electronic tool offers certain advantages (over a similar paper survey tool).
To test whether our new approach does a better job of filtering out noise and increasing measurement precision, we will conduct a randomized-controlled trial (RCT) to evaluate the effectiveness of a traditional paper survey tool compared to our new electronic survey tool. Our proposed study will include an independent sample of micro entrepreneurs from Ghana. The RCT will be carried out by the field research staff of Innovations for Poverty Action (IPA) in 2014. In Part 1, we will identify a sample of 600 micro entrepreneurs (50% female) operating their business in greater Accra. In Part 2, we will randomly assign half of the entrepreneurs into a treatment group (administered our new electronic survey tool) and half into a control group (administered a comparable paper survey tool). This field experiment will result in the construction of a novel dataset.
Experimental design
Our proposed study will include an independent sample of micro entrepreneurs from Ghana (n=600). The RCT will be carried out by the field research staff of Innovations for Poverty Action (IPA) in 2014. In Part 1, we will identify a sample of 600 micro entrepreneurs (50% female) operating their business in greater Accra. In Part 2, we will randomly assign half of the entrepreneurs into a treatment group (administered our new electronic survey tool) and half into a control group (administered a comparable paper survey tool).
Importantly, all members of the survey team will be trained for 3 weeks by IPA research managers. Each surveyor will implement the electronic tool (treatment group) or the paper tool (control group) to a participant on a randomized basis (supervised by IPA staff). We will control for individual surveyor fixed effects in our analysis. Thus, any differences between the two survey approaches (electronic versus paper) will be attributable to the survey design - and not to potential surveyor biases or training effects.
Design details
Cited by
No citation data available for this registration yet.