Promoting an evidence-based culture: an experiment with Latin-American policymakers

How to cite

CAF, Pilar; Ortega, Daniel; Gallego, Jorge. 2018. "Promoting an evidence-based culture: an experiment with Latin-American policymakers." AEA RCT Registry. AEARCTR-0003136. https://doi.org/10.1257/rct.3136-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Pilar CAF* , Daniel Ortega , Jorge Gallego

* Primary investigator

Registry ID
AEARCTR-0003136
DOI
https://doi.org/10.1257/rct.3136-1.0
First registered
Last updated
Status
On going
Trial dates
2017-03-22 – 2019-02-01
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/3136

Abstract

This study intends to identify the effects of a two day face-to-face training program, the Seminar of Impact Evaluation for Development (SEMIDE), that aims to promote impact evaluation (IE) as a tool for enhancing public management in Latin-America. This seminar was designed and implemented by the Impact Evaluation and Policy Learning Department of CAF - Development Bank of Latin-America - and targets Latin-American policymakers with the potential of promoting institutional changes within their institutions. The election of the participants consists of a competitive process that ends with a score for each candidate, followed by a randomized selection of the 35 places among the top 100 candidates. The seminar structure is divided into two sections for each day: 1) Presentations of impact evaluation cases from both academic experts and policymakers, and 2) Practical sessions where participants get the chance to land and apply IE knowledge. We intend to evaluate whether attending this seminar had a positive effect on participants in terms of the promotion and implementation of IEs inside their institutions, their perceptions about the utility of IE, long-lasting knowledge of IE concepts, and other variables. Two editions took place on 2017, in Peru (May 2017) and Argentina (November 2017). The first stage of this study focuses on the Peruvian edition, one year after it took place. We ultimately aim to conduct a multisite experiment that comprise findings on both editions, and potentially include others that are expected to occur during the next two years (most likely in Ecuador and Colombia).

Intervention

The intervention consisted in attending a two-day face-to-face seminar that discussed how Impact Evaluation (IE) can be a useful tool for policymakers to enhance public management in their institutions. The seminar structure was divided into two sections per day. First, both invited academics and policymakers delivered presentations that covered basic IE concepts and methods and/or personal cases where IE helped them to resolve a particular public policy issue and to improve decision making. The second section consisted of workshops where participants could learn how to identify key elements to assess the feasibility of a potential IE in their own projects, clarify their doubts regarding IE, and to work on an IE proposal which was presented to their peers.

Experimental design

For Peru's edition, each applicant got a score after the three-step application process for attending the seminar (online application assesment, video recording and interview). The top 100 were randomly assigned either to attend the seminar or to the control group (treatment=35, control=65). following a blocking strategy based on the score distribution. Three strata were defined: 1) From the first to the 33rd position, 2) From the 34th to the 90th position; and 3) From the 91st to the 100th position. The participants were then randomly selected within each block, based on the weight of each stratum in the final sample. For Argentina's edition, we conducted a simple randomization among the top 81 applicants, as their scores after the three-step application process were highly homogenous (for this edition, these three-steps were online application assesment, interview and score on an IE online course). Randomization design for the potential Ecuador and Colombia SEMIDEs will be detailed at a later stage of this multisite experiment.

Design details

Randomization method
Peru's edition: Blocking randomization Argentina's edition: simple randomization Both done by a computer.
Randomization unit
Applicant
Clusters
-Peru's edition: One cluster of 100 applicants -Argentina's edition: One cluster of 81 applicants
Observations
Peru's edition: 100 applicants Argentina's edition: 81 applicants
Arms
Peru's edition: -Treatment: 35 applicants -Control: 65 applicants Argentina's edition: -Treatment: 35 applicants -Control: 46 applicants
Primary outcomes
-IE knowledge score on survey test -Use of evidence generated by IE -Perception on the importance of IE -Knowledge and use of evidence websites -Engagement level on different stages of an IE process -Number of IEs implemented/supported

Cited by

No citation data available for this registration yet.

Keywords