Supporting healthier pregnancies and early child development one text at a time: Can personalized text messages, increased community participation and incentives to service providers help improve pregnancy and early childhood outcomes?

How to cite

Aguilar, Arturo; Gaitan, Pablo; Szteliga, Natalia; Gonzalez, Cristina Barnard; Silva, Monica Wills; Ita, Rodrigo Gonzalez de; Villar, Francisco del; Metland, Daphne; Ravenscroft, Luke; Kettle, Stewart; Vilar, Mireya; Vargas, Manett. 2016. "Supporting healthier pregnancies and early child development one text at a time: Can personalized text messages, increased community participation and incentives to service providers help improve pregnancy and early childhood outcomes?." AEA RCT Registry. AEARCTR-0001035. https://doi.org/10.1257/rct.1035-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Arturo Aguilar* , Pablo Gaitan , Natalia Szteliga , Cristina Barnard Gonzalez , Monica Wills Silva , Rodrigo Gonzalez de Ita , Francisco del Villar , Daphne Metland , Luke Ravenscroft , Stewart Kettle , Mireya Vilar , Manett Vargas

* Primary investigator

Registry ID
AEARCTR-0001035
DOI
https://doi.org/10.1257/rct.1035-2.0
First registered
Last updated
Status
On going
Trial dates
2015-12-07 – 2018-12-31
Countries
Private
JEL codes
I12, I15, I38, J13
Registry record
http://www.socialscienceregistry.org/trials/1035

Abstract

This project seeks to improve maternal health, birth outcomes and early child development, by empowering mothers through a SMS information system. The SMS information system is targeted at beneficiaries of Prospera, Mexico’s conditional cash transfer programme (formerly known as Progresa and Oportunidades). The SMS information system consists of appointment reminders, prompts to plan for birth and emergencies, information on potential concerns, and preventative health care advice. SMS are sent in a personalized manner using administrative information, medical records and the responses from the beneficiaries to SMS. The two-way system also allows beneficiaries to seek emergency care, report health concerns and change their regular appointments.
Three treatments variations will be evaluated with a randomized control trial (RCT) design. The first treatment arm will test the impact of the two-way information system. The second variation will test the SMS system with additional messages from local community members. Finally, the third group will test the SMS system with the additional component of enabling beneficiaries to provide feedback on the health services received. The feedback collected on the quality of the clinic services will be later used to provide incentives to clinic personnel.
Rigorous evidence about potential changes in habits, knowledge, health service demand, anthropometrics and developmental outcomes will be explored.

Intervention

The presented project (hereon Opening Prosperity) seeks to improve the delivery and impact of health services provided by Prospera. It was designed with the purpose of improving maternal health, birth outcomes and early child development of Prospera beneficiaries through timely and personalized SMS with healthcare information. Opening Prosperity consists of a two-way SMS information tool using RapidPro, an open source SMS tool designed by UNICEF. The messages were personalized and tailored to the specific circumstances of each participant using three sources of information: beneficiaries’ response to messages, Prospera administrative information, and clinical history information.

The design and content of messages to be sent during pregnancy, puerperium and early child stages was developed by the Behavioral Insights Team and BabyCenter. Que Funciona para el Desarrollo helped in the revision, translation and adaptation of messages to meet the platform requirements. The development of the messages also benefited from the expert advice from the Ministry of Health in the UK and in Mexico. Finally, the messages were revised by the Ministry of Health in Mexico to ensure compatibility with the information delivered to beneficiaries. The messages include a variety of topics relevant for pre- and post-natal care, such as: appointment reminders, prompts to plan for birth and emergencies, identification and what to do in case of potential concerns and alarm signs, preventative health care advice, lactation tips, vaccination reminders, among others. Additional flows of messages were also included to enable beneficiaries to seek emergency care, express health concerns, and change their checkup appointments through the platform. All these components are thought as complements to their regular clinic visits, which remain one of the mandatory conditionalities of Prospera.

Two rounds of training sessions with beneficiaries were scheduled at the clinic level. The SMS platform runs through a free number that was provided by the two major cell phone companies in Mexico. Non-Prospera pregnant women can also sign up to receive the messages. However, they are not invited to the training event and messages could have a regular SMS cost to them.

Beneficiaries in the first variation of the programme will only be receiving the SMS through the platform. In the second variation community members, which comprise Prospera vocales (beneficiaries elected at the community level to act as local programme representative) or health promoters, actively participate. This variation seeks to test whether the support of a local network can influence the behavior of pregnant women and encourage usage of the platform. Community members’ participation will happen in two main ways. First, community members attend the initial training session and sign mock contracts with the beneficiaries to obtain a commitment from them on either attending all their prenatal checkup visits, taking their pregnancy supplements (e.g. folic acid) or responding to all the messages. Second, community members are encouraged to send pre-designed messages with certain regularity to support the beneficiaries’ pregnancy.

Finally, the third variation enables the users to send feedback through SMS on the service provided by the clinic. By replying to SMS, beneficiaries are able to report compliance of basic protocols that should be followed during checkups and to evaluate the quality of the service provided in the clinic. To protect the beneficiaries, only a subset of the beneficiaries will receive messages asking them to evaluate the clinic, and anonymity will be guaranteed at all times. Based on the information gathered, clinics will receive information of the beneficiaries’ responses and how they compare to similar clinics in terms of geography and size. The best performing clinics will receive a set of incentives.

Experimental design

The unit of randomisation is the primary-level health clinic and the unit of analysis is the pregnant woman or her child. Each health clinic in the country is associated with a set of localities. Thus, the clinic conforms a cluster, with beneficiaries located in associated localities.

Clinics are randomly assigned to a control group or to one of three treatment arms. The control group means that Prospera beneficiaries are not offered any of the intervention components. Pregnant women and children that would have been eligible for the program will be tracked during the study.

The main component of the intervention, which consists of the delivery of personalized messages and the ability to contact health authorities to report emergencies and health concerns, is delivered to the three treatment arms. Thus, the comparison of the control group and treatment arm one will allow the research team to evaluate the impact of the main component in isolation. The comparison of the control group and the three treatment arms (grouped) allows to evaluate the impact of the project’s components.

Treatment arm two receives the second component of the project, which consists of the local network interaction, in addition to the main component. The comparison of treatment arms one and two will give evidence of the additional effect that results from the local network interaction. Since both groups receive the main component, this comparison will not convey the local network effect by itself since there is a potential complementarity between the main component and the local network interaction.

Treatment arm three receives the third component of the project, which consist of the possibility to evaluate and report compliance of health service received, in addition to the main component. The comparison of treatment arms one and three will give evidence of the additional evaluation and incentives component. Similarly as described above, the component will not be identified in isolation since there are possible complementarities with the main component.

The sample of clinics for the study contains 655 clinics located in five states: Chiapas, Guanajuato, Hidalgo, Puebla, and the State of Mexico. The five states contribute to the heterogeneity of the project. The following filters were implemented to determine the set of clinics where the study could be feasibly carried out: (i) hospitals, administrative buildings itinerant clinics and units administered by the municipality were excluded, (ii) clinics with less than five estimated Prospera pregnant beneficiaries are excluded, (iii) clinics in localities with no mobile signal available are excluded, and (iv) clinics in localities where less than 80% of females between 13 and 35 speak Spanish were excluded.

The set of clinics that remain after implementing these set of filters represent our population. We keep the clinics from the five selected states since a greater mass of clinics was found and the states represent very different contexts. Finally, to select the clinics for the study from these five states, a greedy algorithm was used with the objective of selecting clinics that maximize the distance between each other while keeping a representative sample with respect to the population of clinics. The reason for this is to ensure SUTVA and reduce the risk of contamination.

Once the clinics were selected, they were instructed to invite every pregnant Prospera beneficiary that had less than 32 weeks of gestational age. This restriction seek that the beneficiaries had exposure to the program at least 8 weeks.

Throughout the pregnancy a beneficiary will receive around 580 SMS. By the end of 2016 approximately 6550 beneficiaries will be part of the trial, half of them will be in treatment localities. We expect to have results from the impact evaluation by 2017.

Design details

Randomization method
The method for random treatment allocation was Coarsened Exact Matching (CEM). The purpose of this method is to maximize the likelihood of identifying a statistical significant effect through a randomized control trial. To do this, the method forms groups of similar clinics and randomises treatment status within the group. The following variables were selected to stratify clinics and form groups with the CEM: 1. A social lag index. This index is developed by the National Council of Social Policy Evaluation (CONEVAL) and updated every 5 years. Based on this index, CONEVAL classifies localities in five different categories: very low, low, medium, high and very high social lag. 2. Type of clinic. Clinics were segmented in blocks according to the type of locality where they are located [rural or urban] and the size of the clinic [small, medium or large]. The size is determined by the number of nuclei available. One nuclei is a set of two or three medical personnel attending in the clinic. Five blocks were defined: (i) small, rural clinics, (ii) medium and large rural clinics, (iii) small, urban clinics, (iv) medium, urban clinics, and (v) large, urban clinics. 3. Federal Mexican entity (State). Five states were selected in the sample: Chiapas, Estado de Mexico, Guanajuato, Hidalgo and Puebla. These states were selected based on their population, geographical, demographical and economic heterogeneity. Looking at different states creates differential ease of access to clinics, exposure to technology, traditions, cultures and languages. 4. Level of education. Using the Mexican census, we create two categories based on the percentage of female population above 15 years of age with incomplete secondary schooling. The following stages were followed to allocate each clinic to either of the three treatment groups or to the control: In the first stage, all the stratifying variables were used to form groups. Sets of six clinics were formed whenever a group had more than six clinics in it. If the number of clinics in the group is not a multiple of six, clinics were randomly selected to form sets of six clinics and those that were not allocated to a set of six moved on to the next stage. In the second stage, we drop level of education as a stratifying variable and repeat the procedure described above. The third and fourth stages repeat the previous steps by first dropping social lag and then state. This means that the third stage matched clinics based on state and type of clinic, and lastly, the forth state only matched clinics based on their type. Finally, the fifth stage collected the clinics that had not been allocated and randomly chose for each clinic their allocation. This method was executed with the 655 clinics sample, out of which 329 were allocated to the control group, 107 to treatment arm one, 111 to treatment arm two and 108 to treatment arm three.
Randomization unit
Clinic clusters are the randomization unit. A clinic cluster is defined by the population that is assigned to attend the clinic. Women in the study are Prospera beneficiaries and attendance to clinics is mandatory by design in the program. The assignment is generated geographically based on the distance between the clinic’s locality and the surrounding localities. In rural locations where a clinic is not available in each locality, the assignment often means that women have to attend a clinic located in a nearby locality. In urban locations where more than one clinic is available, beneficiaries are typically assigned to the clinic in their surrounding neighborhood. Our unit of observation for the analysis will be the pregnant woman and her child. All primary outcome measures will be measured at this level.
Clusters
655 clinics
Observations
6,550 beneficiaries (i.e. an average of 10 women per clinic)
Arms
Control group: 329 clinics Treatment arm one: 107 clinics Treatment arm two: 111 clinics Treatment arm three: 108 clinics
Primary outcomes
Our outcome measures will be extracted from the following sources: SINAC (Subsistema de Información sobre Nacimientos)- This is a country-wide birth registry that is administratively recorded by the Health Ministry and made available through the Dirección General de Información en Salud. The following measures will be used: 1. Weight of baby at birth (g) 2. Length of baby at birth (mm) 3. APGAR score (1-10) 4. Silverman score (1-10) 5. ICD-10 catalogued congenital diseases or complications This registry is updated every two months and contains unique identifiers that allow for identification of individual beneficiaries. To be able to match our Prospera beneficiaries to the registry we will need Prospera support since personal information is reserved for confidentiality protection. To evaluate potential indirect benefits, we will also use information from other births in the clinic. The information from SINAC will be collected at mid-2017 and again at the end of 2017. Prospera- We will request from Prospera information about compliance with their health conditionalities at an individual level. If this information cannot be made readily available, we will use public administrative information about the total bimonthly monetary amount awarded to the families of pregnant beneficiaries in treatment and control. This information is released by Prospera every two months and is available to the general public by law. Such monetary amount will be used to proxy the general compliance of families in Prospera Digital with the conditionalities set by Prospera. We understand this would be a noisy measure since there are other conditionalities (in addition to health check-up attendance). This information is expected to be collected at March-June 2017 and again November-December 2017. Primary-level Health Clinics- At the middle and end of the trial, medical records will be retrieved from all clinics in the trial for all beneficiaries in control and treatment arms. Such records will allow us to formulate the following outcome measures: 1. Total number of scheduled clinic visits 2. Attendance rate to scheduled clinic visits 3. Incidence of diseases during pregnancy and first two years of babies’ lives (by type of disease) indicating if they were attended at the clinic and what the outcome was 4. Percentage of women giving birth at clinics or hospitals 5. Percentage of women with full clinic record during pregnancy (we will revise specific characteristics being recorded) 6. Percentage of women with full battery of tests done during pregnancy (according to protocols) 7. Percentage of women with full (and incomplete) prenatal measurement follow-up (e.g. weight and height) 8. Percentage of women receiving pregnancy supplements 9. Incidence of specific health problems (malnutrition, anaemia, pre-eclampsia, gestational diabetes, post-natal sepsis, etc.) 10. Vaccination records 11. Delivery of lactation information 12. Percentage of babies that had the neonatal screening done 13. Incidence of premature babies 14. Maternal mortality rate 15. Infant mortality rate 16. Unplanned hospital or clinic visits This information is expected to be collected with a format that will be designed (and previously trialled) at March-June 2017 and again November-December 2017. INDICAS (Sistema de Indicadores de Calidad en Salud)- This is the national system of indicators used by the Ministry of Health to constantly monitor quality in the clinics. This information will be periodically gathered for the treatment and control clinics. We expect to gather it once every quarter. The outcome measures to be used include: 1. Proportion of times that protocols were followed during the check-ups 2. User self-reported satisfaction 3. Wait time to be attended RapidPro- This is the system that captures all the interactions that beneficiaries have with the platform that automatically sends messages. This information will be constantly (weekly) gathered and analysed mainly to follow-up the fidelity of implementation of the main treatment arm. The outcome measures are only gathered for the treatment groups and the following indicators are produced: 1. Intensity of participation by looking at the proportion of two-way messages that a beneficiary answer 2. Response rate of messages 3. Mistake rates for different kind of responses required 4. Proportion of beneficiaries not participating Post-participation survey- A survey will be implemented by the end of the intervention (circa June 2017) to gather additional outcome measures that are either imperfectly measured in the other sources or non-existent: 1. Knowledge of pregnancy and newborn care 2. Diet 3. Supplement consumption 4. Breastfeeding practices (duration, problems, knowledge) 5. Planning for emergencies and birth 6. Women empowerment indicators 7. Health service satisfaction 8. Malnutrition and obesity rates in both the mother and the newborn 9. Size and weight of infants (measured, not self-reported) 10. Incidence of gastrointestinal diseases in infants 11. Incidence of risk conditions during pregnancy: tobacco and alcohol consumption 12. Self-reported development milestones reached, e.g. crawling, walking, babbling, etc. (date and success) 13. Use and trust in the content of the messages 14. Main source of information about pregnancy and baby care topics 15. Tests with children to measure different aspects of their development, such as gross and fine motor skills, language development, neurological development, signs of risk, etc. Tests such as MSCA, MacArthur and EDI (all of them already used in Mexico will be favored)

Cited by

No citation data available for this registration yet.

Keywords