Information Design To Improve Service Delivery: Experimental Evidence from Guinea-Bissau

How to cite

Fracchia, Mattia. 2022. "Information Design To Improve Service Delivery: Experimental Evidence from Guinea-Bissau." AEA RCT Registry. AEARCTR-0009054. https://doi.org/10.1257/rct.9054-4.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Mattia Fracchia*

* Primary investigator

Registry ID
AEARCTR-0009054
DOI
https://doi.org/10.1257/rct.9054-4.1
First registered
Last updated
Status
Completed
Trial dates
2022-05-17 – 2022-08-31
Countries
Guinea-Bissau (Africa)
JEL codes
O12, D83, D91, I15
Registry record
http://www.socialscienceregistry.org/trials/9054

Abstract

Poor staff motivation can hinder the delivery of public services. We study how to design information to motivate front-line service providers in a field experiment with Community Health Workers in Guinea-Bissau. Study participants attend a one-day training workshop. We randomly assign half to a module conveying information about program effectiveness which blends scientific evidence with culturally relevant anecdotes. The primary measure of motivation is the performance in a real effort task. This involves phoning households to collect health information and deliver recommendations. We then investigate potential mechanisms via several follow-up surveys. Also, we track participants' performance in their role as CHWs in the months after the workshop. Finally, we follow up with households to measure downstream health outcomes.

Intervention

Study participants attend a one-day training workshop.

Pre-intervention
- Survey: elicitation of beliefs about health indicators and the program’s contribution to their evolution.
- Pre-test: questions on prenatal care covering topics from the program's training material.

Intervention - Experimental variation is at the Workshop level.
- Control Workshop: participants receive a placebo training module + refresher training on prenatal care.
- Treatment Workshop: participants receive a treatment module on program effectiveness + refresher training on prenatal care.

Post-intervention
- Survey: motivation and manipulations checks
- Survey: elicitation of beliefs about health indicators and the program’s contribution to their evolution.
- Post-test: questions on prenatal care covering topics from the program's training material.
- Real-effort Task: phoning households to test their knowledge of prenatal care and convey the recommendations covered during the refresher training.

Experimental design

We randomly assign selected participants to one of two alternative workshops: control and treatment. All CHWs attending a specific workshop will receive the same training. All workshops are held at the health area level and replicate the existing structure of the routine monthly meeting of the program.

The first activity of the workshop is a pre-test on prenatal care and a baseline questionnaire comprising a face-to-face survey and a self-administered one. The face-to-face survey elicits beliefs about health indicators at the local level, i.e., administrative region, and the program's contribution to their evolution. The self-administered asks about participants' engagement in the CHW program.

Participants then attend the training session. Depending on the treatment group, the training modules are the following:
• Control – receives a placebo module on mango, the seasonal fruit at the time of the workshop + refresher training on prenatal care.
• Treatment – participants receive a treatment module on program effectiveness taken from the official program evaluation presented for the first time to policymakers a few months before the workshop and still unknown to participants + refresher training on prenatal care.

During the training session, a research assistant observes and evaluates the attitude and participation of CHWs. After the conclusion of the training session, the trainer will prepare a debrief of the session, reporting his impressions and comments.

After the training, participants sit in a post-test on prenatal care and a follow-up questionnaire comprising a face-to-face survey and a self-administered one. The face-to-face survey has two modules. The first module replicates the baseline module eliciting beliefs about health indicators at the local level, i.e., administrative region, and the program's contribution to their evolution. The second collects measures of motivation and various manipulation checks. The self-administered survey collects essential socio-demographic characteristics and asks about participants' willingness to engage in the CHW program in the future.

Finally, participants perform a real-effort task with a clear and direct link to their role as CHWs. This involves contacting by phone a sample of their households. The target respondent is a pregnant woman or a woman between 18 and 49. During the call, CHWs test the respondent's knowledge of prenatal care and convey the recommendations covered during the refresher training,

We will follow up with CHWs over the phone in the month after the workshop. The phone survey will cover: i) extensive and intensive participation in the program and time allocation in other activities, and ii) motivation to work as a CHW. Moreover, when available, we will gather administrative data from the program for the months following the workshop. We will employ this data to measure: i) extensive and intensive participation in the CHW program and ii) individual performance.

We will also follow up with the contacted households to measure downstream outcomes in knowledge and behaviors. In the month after the workshop, we will contact by phone a sample of households contacted by CHWs during the workshop. The questionnaire will cover: i) satisfaction with the program, ii) knowledge of recommendations on prenatal care received during the phone call received during the workshop, and iii) health behaviors over the past week.

Design details

Randomization method
Randomization is done using Stata.
Randomization unit
The two regions of Farim and Oio have three hundred and seventy-nine villages with at least one CHW, for a total of six hundred and two CHWs. The universe of villages and CHWs are eligible to participate in the study. Assignment to Treatment and Control happens at the village level. We create pairs of villages stratifying on region, health area, distance from the health center, and number of CHWs in the village. We then randomly split each pair between Treatment and Control. We rank CHWs, sorting by sex, within each village in the sample. The first-ranked CHW within a village is selected to participate in the study. The second-ranked is selected as a substitute. Sixty-five percent of villages have only one CHW and no substitute.
Clusters
379 villages
Observations
We expect a total sample of three hundred and seventy-nine CHWs, one per village currently benefiting from the CHWs program.
Arms
Control: 190 Treatment: 189
Primary outcomes
All primary outcomes are collected during the Workshop. Survey measures at the CHW level - Performance in a post-test on prenatal care compared to pre-test level (before training module) - Beliefs updating about health indicators and the program’s contribution to their evolution - Motivation level after the training module - Stated willingness to participate in the program in the future at the extensive and intensive margin Training measures - Participation in the second part of the training, i.e., the refresher training on prenatal care Real-effort Task measures at the CHW level: - Quantity of attempts to contact households - Quality of communication with households evaluated by research assistants - Quantity and quality of reporting of the information gathered during phone calls
Secondary outcomes
CHW follow-up phone survey a month after the workshop - Self-reported extensive and intensive participation in the CHW program - Motivation level Program administrative data (if collected by the program) - Extensive and intensive participation in the CHW program - Performance in the program Household's level - Satisfaction with the CHW program - Knowledge of recommendations on prenatal care received during the phone call received during the workshop - Health behaviors over the past week

Cited by

No citation data available for this registration yet.

Keywords