Are Teachers and Learning Software Complements or Substitutes? Evidence from a Randomized Experiment in El Salvador
How to cite
Buechel, Konstantin; Steffen, Daniel; Kuehnhanss, Christoph; Brunetti, Aymo; Jakob, Martina. 2018. "Are Teachers and Learning Software Complements or Substitutes? Evidence from a Randomized Experiment in El Salvador." AEA RCT Registry. AEARCTR-0002789. https://doi.org/10.1257/rct.2789-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
In this study we experimentally evaluate an education program in the Salvadorean district Morazan that expands schooling by an additional 180 minutes per week. The program comprises three different interventions that target primary school children in grades 3 to 6: (1) addtional math lessons taught by contract teachers, (2) additional math lessons taught by contract teachers using the software "Khan Academy", and (3) additional math lessons based on the software "Khan Academy" supervised by technical staff (but not teachers). This setup allows us to study the degree of complementarity/substitutability between computer-assisted learning software and pedagogically trained teachers. In particular, we appraise whether the (potential) impact on learning outcomes is primarily attributable to additional math classes or the use of computers, and further compare the cost-effectiveness of the three different versions of this school expansion program.
Intervention
This schooling intervention, which is implemented by the NGO Consciente in collaboration with the regional Ministry of Education (MINED), comprises three different versions of additional math classes that complement regular classes of 3 to 6 graders (i.e. primary school level). The additional math classes cover two lessons of 90 minutes per week and are implemented for a period of 24 weeks (mid April-- end of September 2018) during the school year 2018.
The first treatment arm offers additional math classes conducted by contract teachers. The contract teachers are hired by Consciente and teach refresher courses that repeat the math curricula of lower grades.
The second and the third treatment are additional math classes based on the computer-assisted learning software "Khan Academy". Students work with the Spanish Lite-Offline version of the software, which allows them to learn independently and -- more important -- at their own level and pace. The difference between these two latter treatments is that the additional computer classes are either conducted by a temporarily contracted supervisor or by a temporarily contracted math teacher. Supervisors provide technical support but will not assist with questions regarding math. Teachers, in contrast, are also allowed to explain mathematical concepts to students, and hence complement the instructions/explanations provided by the software.
Experimental design
Starting point are all primary schools in Morazan, i.e. about 300 schools. Our partner NGO faces oversubscription, meaning that it cannot reach all eligible beneficiaries due to limited financial resources.
Pre-selection: In a first step, we pre-select primary schools based on the following four criteria (ordered from most to least restrictive):
- (i) School size: Schools with integrated classes (across grades) or gaps in their grade structure (i.e. not at least one class per grade) are excluded.
- (ii) Security: Schools located in areas dominated by criminal gangs are excluded.
- (iii) Accessibility: Schools inaccessible by car are excluded.
- (iv) Electricity: Schools without electricity are excluded.
After this pre-selection 57 schools, 317 classes, and about 6300 students in grades three to six remain in our sample.
Randomization stage 1: Project resources do not allow to install computer rooms in all 57 pre-selected schools. Therefore, 29 of these 57 schools are randomly chosen to be part of the study. We stratify by school size (3 strata, at least 1 / 2 / 3 classes per grade), population density (4 strata, quartiles) and the existence of a computer room (2 strata, yes / no).
Randomization stage 2: 158 classes in the 29 remaining schools are randomly assigned to the different treatment arms, with each treatment/control group comprising 39 or 40 classes. We use rerandomization (see Morgan and Rubin, 2012) to make sure that treatments are balanced across schools and grades. The threshold criterion is defined as follows: 1) Each school has to receive all three treatment arms and a control class, 2) each treatment/control arm comprises at least 9 classes and at most 11 classes per grade, with a total of 39 or 40 classes.
Randomization within schools increases the risk of spillovers between control group and treatment groups, which may bias the estimated effect. Although no evidence for such spillovers is reported in the literature on related interventions, we take different measures (e.g. close monitoring, extensive briefing of staff) to actively prevent spillovers. Furthermore, we randomly match 40 classes from the 28 "pure control" schools that were excluded in the first randomization stage to the control classes in the "treatment" schools. The children in these pure control classes are also asked to complete the baseline and endline exams.
Design details
Cited by
No citation data available for this registration yet.