Agentic Feedback and Quality of Field Survey Data

How to cite

Pandey, Vivek; Elangbam, Kyamba Khanganba. 2023. "Agentic Feedback and Quality of Field Survey Data." AEA RCT Registry. AEARCTR-0011962. https://doi.org/10.1257/rct.11962-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Vivek Pandey* , Kyamba Khanganba Elangbam

* Primary investigator

Registry ID
AEARCTR-0011962
DOI
https://doi.org/10.1257/rct.11962-2.0
First registered
Last updated
Status
Completed
Trial dates
2023-08-05 – 2023-11-10
Countries
India (Gujarat)
Registry record
http://www.socialscienceregistry.org/trials/11962

Abstract

In the field of applied research, ensuring data quality has emerged as a significant concern, as evidenced by a substantial body of literature. Over the past two decades, there have been notable advancements in data collection procedures. This includes a transition from traditional paper-based data collection to tablet-based CAPI methods, as well as improvements in telephonic data collection during the COVID-19 pandemic. Despite several innovations on the technology side of the data collection process, field enumerator-level initiatives are quite few. Guidelines from most think-tanks and research labs focus on survey training and survey manuals. Some researchers recommend conducting tests at the end of the survey training, to enhance awareness of field enumerators.

The field experiment is mounted on a household survey that will be collected as part of a baseline survey of a school milk program in Gujarat, India. The state has historically experienced higher incidence of stunting and malnutrition among children and continues to do so. Therefore, the program seeks to provide liquid milk in government schools in order to improve child nutrition status. The experiment is therefore an attempt to generate causal evidence on how to improve the quality of anthropometric and other sensitive data at the individual and household level. In addition to standard household modules (including roster, assets, income, etc.), the study survey captures information on child anthropometry, child consumption, mother's well-being and her activity status.

The experiment consists of three treatment arms that randomly allocates 60 field enumerators using block-design across 120 villages: (i) enumerators in the control group villages receive a standard survey training with no tests (C), (ii) enumerators in the first treatment group solve three tests at the end of each day of the training and receive non-agentic feedbacks (T1), and (iii) enumerators in the second treatment group solve an identical tests as T1 but in addition they receive agentic feedbacks with an option for resubmission based on feedbacks (T2). The field experiment can examine the effect of frequent testing during survey trainings (T1 versus C) and variations in feedbacks on quality of survey data (T1 versus T2 and T2 versus C).

The experimental design facilitates the estimation of the test effect and agentic feedback on the quality of survey data. Quality of survey data is assessed from high-frequency checks that are completed on each day of the field work. The potential channel for generic testing includes awareness while agentic feedbacks may trigger mechanisms such as motivation, self-confidence, and self-responsibility.

Intervention

The experiment consists of three treatment arms that randomly allocates 60 field enumerators using block-design across 120 villages: (i) enumerators in the control group villages receive a standard survey training with no tests (C), (ii) enumerators in the first treatment group solve three tests at the end of each day of the training and receive non-agentic feedbacks (T1), and (iii) enumerators in the second treatment group solve an identical tests as T1 but in addition they receive agentic feedbacks with an option for resubmission based on feedbacks (T2). The field experiment can examine the effect of frequent testing during survey trainings (T1 versus C) and variations in feedbacks on quality of survey data (T1 versus T2 and T2 versus C).

Experimental design

The experimental design is a combination of cluster and block design ,i.e., we have utilized blocking on clusters. There are four blocks that combine high and low cognitive scores with high (H) and low (L) work experience in field work of enumerators. The size of the enumerator group is 60 and serve as clusters in the experiment. Blocking on these clusters will randomly allocate 15 enumerators in each of the four blocks (i.e., HH, HL, LH, and LL). Given that each block is comprised of three experimental arms (i.e., C, T1, T2), there will be 20 enumerators allocated to each experimental arm across the four blocks. The enumerators are allocated to 120 villages for the conduct of the field surveys in such a manner that each village is allocated to only one of the three treatment arms (C or T1 or T2).

Design details

Randomization method
Randomization done in office by a computer after the recruitment of field enumerators was completed
Randomization unit
Blocking on clusters where each strata are based on enumerator experience and cognitive score of the enumerators. Cognitive scores were measured at the time of recruitment process of enumerators.
Clusters
120 villages and 60 enumerators
Observations
The unit of data collection is at the household-level, i.e., 13 household surveys per village * 40 villages = 520 households per experimental arm, with a total of 1560 HHs across the three experimental arms. Therefore the number of observations is 1560 households. The anthropometric schedule will be canvassed to children in such households. Given that a number of outcomes will be reported at the child level, we expect to generate more than 3500 observations at the child-level (we assume 2.3 children per household).
Arms
40 villages and 20 enumerators (control-C), 40 villages and 20 enumerators (non-agentic feedback-T1), and 40 villages and 20 enumerators (agentic feedback-T2)
Primary outcomes
The primary outcomes include: (i) number of valid skips, (ii) missing values, (iii) number of flags, and (iv) consumption (calorie, macronutrients) and nutrition status (stunting, wasting, and malnutrition), and psychological well-being

Cited by

No citation data available for this registration yet.

Keywords