Training Policymakers in Econometrics: Evidence from Two Experiments in Pakistan
How to cite
Mehmood, Sultan; Naseer, Shaheen; Chen, Daniel. 2022. "Training Policymakers in Econometrics: Evidence from Two Experiments in Pakistan." AEA RCT Registry. AEARCTR-0010583. https://doi.org/10.1257/rct.10583-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Over the last half century empirical economics has gone through a paradigm shift. The credibility revolution, with its careful attention to causality, has presented itself as a new paradigm for taking the "con out of econometrics”. We hope to study causal effects of a paradigm shift in the social sciences on practitioners–policymakers–using the training of the paradigm as its instrument. There seems to be consensus emerging in the literature that policymakers are highly averse to shifting their beliefs and engage in motivated reasoning to justify their initial policy choices. Sticking to priors and being inattentive to evidence may stymie the implementation of good policies that might otherwise spur economic development. How can policymakers be made more receptive to evidence? Will training them in concepts associated with the credibility revolution make them more likely to shift their beliefs? Will it induce them to change their policy choices? Can the econometrics training impact State capacity? Can it increase uptake of tax policy for which there is causal evidence? To answer these questions, we randomize policymakers in econometrics training and trace its impact on their attitudes, policy and the population.
Intervention
Experiment 1 with Junior Ministers. — The first experiment involves an intensive training where we aim to maximize the comprehension, retention, and utilization of the educational materials. Namely, we augmented the book receipt with lectures from the books’ authors, namely, Joshua Angrist and Daniel Siegel, along with competitive writing assignments. As part of the training program, deputy ministers were assigned to write two essays. The first essay was to summarize every chapter of their assigned book, while the second essay involved discussing how the materials would apply to their career. The junior ministers in each treatment group also participated in a zoom session to present, discuss the lessons and applications of their assigned book in a structured discussion.
Experiment 2 with Tax Officers. — The second experiment involved tax officers randomized into econometrics versus a placebo training. Identical to our first experiment, the officers receive writing assignments, present their learnings in class and engage in a structured discussion.
Difference Between Experiment 1 versus Follow-on Experiment 2. — There are two differences between the first and the second experiment. The first is that the second experiment also involves a cross-randomization of two signals. Half of tax officers randomly receive an email a one-page summary of results from a paper that provides experimental evidence that sending tax reminders increases tax collection, while the remaining half are randomized to receive the handout where a correlational study that tax reminders increase tax collection. Their prior and posterior beliefs on impact of tax reminders on tax collection are also solicited. The second difference between the two experiments is that in the experiment with customs officers, we have a natural policy outcome, on tax reminders sent and tax collection, linked from treated tax officers to their tax district jurisdictions.
Hypotheses.— We will test the following main hypotheses in the study:
H1: Econometrics training impact policy choices on deworming (Experiment 1)
H2: Econometrics training has no impact on policy choices on orphanage and school renovations (Experiment 1)
H3: Econometrics training has impact on sending tax reminder letters (Experiment 2)
H4: Econometrics training has impact on tax collection (Experiment 2)
We will test the following supplementary hypotheses:
H5: Econometrics training will increase more the adoption of sending tax reminder for those officers that received the signal study to send tax reminders with RCT evidence (Experiment 2)
H6: Econometrics training will not impact or decrease the adoption of sending tax reminder for those officers that received the signal study to send tax reminders with correlational evidence (Experiment 2).
Experimental design
In the first experiment we randomly assign 200 ministers into econometrics and placebo training unrelated to econometrics. In the first experiment, we also obtain demand for econometrics training prior to the experiment and control for this in all regressions.
In the second experiment, we randomly assign 300 tax officers into econometrics and placebo training unrelated to econometrics.
Design details
Cited by
No citation data available for this registration yet.