Do At-Scale Social Audits Improve Service Delivery of a Large Public Works Program?

How to cite

Schaner, Simone; Moore, Charity; Pande, Rohini. 2021. "Do At-Scale Social Audits Improve Service Delivery of a Large Public Works Program?." AEA RCT Registry. AEARCTR-0008640. https://doi.org/10.1257/rct.8640-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Simone Schaner* , Charity Moore , Rohini Pande

* Primary investigator

Registry ID
AEARCTR-0008640
DOI
https://doi.org/10.1257/rct.8640-1.0
First registered
Last updated
Status
On going
Trial dates
2019-04-01 – 2022-06-30
Countries
India (Bihar)
JEL codes
D73, H11, H53, J01, O10
Registry record
http://www.socialscienceregistry.org/trials/8640

Abstract

We study the recent rapid expansion of Bihar's social audits initiative as a measure to improve access and delivery of social protection under India's national workfare scheme. We undertake a causal evaluation utilizing administrative data from a randomized controlled trial across the state. Initial analysis using the administrative data finds that audits lead to a significant, sustained reduction in work as reported in administrative data, a result that could reflect some combination of reduced corruption --- a goal of audits, or declines in actual work --- an unintended consequence. Looking ahead, we will collect additional survey data needed to differentiate between these possibilities and provide further direction for program improvement.

Intervention

We study the impact of "social audits" of one of India's largest social protection programs, the Mahatma Gandhi National Rural Employment Guarantee Scheme (MGNREGS), in the Indian state of Bihar. MGNREGS guarantees at least one hundred days of unskilled manual labor to every rural household that demands work; the program is self-targeted and aims to provide income support to the rural population. Social audits of MGNREGS combine community-led monitoring with external oversight: an external team of mostly female auditors reviews the program's implementation, and the audit culminates in a public meeting where local residents discuss the results of the audits and hold local officials accountable. To identify the causal impact of social audits on MGNREGS program performance, we collaborated with the state agency responsible for implementing social audits to conduct a randomized controlled trial (RCT) that leveraged the state's ambitious audit expansion plan in the 2019-2020 fiscal year (FY).

Experimental design

To identify the causal impact of social audits on MGNREGS program performance, we collaborated with Bihar's Social Audits Society to implement an RCT that leveraged the state's ambitious audit expansion plan in FY 2019-2020.

1) First, the state set targets for the number of panchayats (the lowest level of administration in rural India, typically comprising 4-5 villages) it wanted to audit in each block.
2) Then, we randomly selected the panchayats to be audited in blocks where less than 100% of locations were targeted for an audit.
3) Thus, our randomization was stratified on the block, with different probabilities of treatment driven by the implementation agency’s audit target for each block.
4) The experimental sample includes 3,437 panchayats across 178 blocks. Of these, 2,178 panchayats were assigned to be audited (treatment), while the remaining 1,259 panchayats served as the control group.

The aim of our endline data collection is to assess whether administrative data-based reductions in work reflect reduced corruption or reductions in actual work. To understand whether workers may be discouraged from seeking work under the program due to audits, we will also incorporate a salience experiment into the endline. In this add-on survey experiment, we will inform a randomly selected subset of survey respondents in audit GPs of social audit findings. We will only conduct the audit salience experiment within our main RCT treatment GPs, randomizing at the individual level, and then collect data on interest in work and willingness to officially register work requests across all study locations.

Design details

Randomization method
Randomization was conducted in the office via a Stata program.
Randomization unit
Panchayat level
Clusters
Analysis from Administrative Data: 3,437 clusters (panchayats) Analysis from Survey Data: 274 clusters (panchayats)
Observations
Analysis from Administrative Data: 3,437 panchayats across 178 blocks. 2,178 panchayats in the treatment group (873 actually audited) and 1,259 panchayats in the control group. Analysis from Survey data: 5,480 respondents from 274 panchayats from 19 blocks
Arms
Analysis from Administrative Data: Of the 3,437 panchayats in the experimental sample, 2,178 panchayats were assigned to be audited (treatment), while the remaining 1,259 panchayats served as the control group. Analysis from Survey Data: Of the 274 panchayats in the endline survey sample, half will be in the treatment group, that is, they were assigned to be audited in 2019-20. The remaining half will be drawn from the control group.
Primary outcomes
Through this research, we aim to identify the causal effect of a policy innovation designed to improve accountability under MGNREGS, social audits, on program outcomes, including program expenditure, work provision, and program leakage. Our analysis will focus on the following primary outcomes: Access to MGNREGS entitlements */** Demand for work under MGNREGS */** Incidence of false work in official work records * Outcomes marked by a single asterisk (*) will be measured using survey data. Outcomes marked by a double asterisk (**) will be measured using administrative data.
Secondary outcomes
Our analysis will focus on the following secondary outcomes: Awareness of MGNREGS entitlements * Experience of misconduct in the program * Interest in participating in the program * Outcomes marked by a single asterisk (*) will be measured using survey data. Outcomes marked by a double asterisk (**) will be measured using administrative data.

Cited by

No citation data available for this registration yet.

Keywords