Evaluation of Conecta Ideas in Chile

How to cite

Romero, Mauricio; Gertler, Paul. 2022. "Evaluation of Conecta Ideas in Chile." AEA RCT Registry. AEARCTR-0009545. https://doi.org/10.1257/rct.9545-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Mauricio Romero* , Paul Gertler

* Primary investigator

Registry ID
AEARCTR-0009545
DOI
https://doi.org/10.1257/rct.9545-1.1
First registered
Last updated
Status
Completed
Trial dates
2022-04-28 – 2022-12-31
Countries
Chile (Region Metropolitana)
Registry record
http://www.socialscienceregistry.org/trials/9545

Abstract

We propose a large-scale evaluation of a learning platform designed in 2002 by AutoMind, a multidisciplinary team where the leadership is now at the University of Chile. The evaluation design will allow us to test the effect of Conecta Ideas vis-a-vis a control group. Moreover, the experimental design will feature three treatment groups (in addition to the control group). First, a group where students will earn purely symbolic incentives at the individual level. These incentives will be given for using the platform and for getting correct answers to exercises. Second, a group where these symbolic incentives translate to tangible prizes (e.g., books) for students. Third, a group where prizes are given as a function of the performance of the classroom. In addition to testing the effect of Conecta Ideas, this experimental design will allow us to test: a) individual vs group incentives, and b) tangible, but non-monetary incentives vs. purely symbolic incentives.

Intervention

Conecta Ideas (CI) sessions will be carried out in the computer labs of the schools. In a typical CI session, students work to solve the same set of 20 to 30 exercises assigned to them, which are aligned to the topics covered in regular math instruction that week. Students receive automatic feedback regarding whether their answers are correct. Lab coordinators, hired and supervised by the Conecta Ideas team, are
responsible for conducting the learning sessions at the computer lab in collaboration with regular classroom teachers. Lab coordinators will receive a day of training and ongoing supervision from the implementation team. While teachers will not receive formal training, the program promotes learning-bydoing. The software provides virtual stickers and prizes to students who do well in each CI session. It also keeps track of student’s progress (in the current session and across sessions) and offers each student personalized reports in the form of graphs to motive study effort.

Experimental design

The evaluation design will allow us to test the effect of Conecta Ideas vis-a-vis a control group, and at the same time, it will allow us to measure the effect of varying the implementation details of Conecta Ideas (in particular, by testing different incentive designs). We are able to study multiple questions by relying on standardized tests taken by student.

Design details

Randomization method
We conducted the randomization in Stata in a two-stage process. First, we randomized at the school level, stratifying by whether schools have one or more sections in Grade 4 and the quartile of an index that summarized the average IRT score and the standard deviation of scores within a school. The probability of being assigned to control was 0.4, while the probability of being assigned to each of the three treatments (plain vanilla Conecta Ideas, CI with individual incentives, and CI with group incentives) was 0.2. After schools were assigned to one of the four experimental groups, from each treatment school, we randomly selected a single section (in case there were multiple sections in Grade 4) for treatment.
Randomization unit
We randomized at the school level, stratifying by whether schools have one or more sections in Grade 4 and the quartile of an index that summarized the average IRT score and the standard deviation of scores within a school
Clusters
235 Schools
Observations
10,000 students
Arms
We have three treatment groups: plain vanilla Conecta Ideas (CI), Conecta Ideas with individual incentives (II), and Conecta Ideas with group incentives (GI). We assigned 50 schools to each of these groups, and we have 85 schools left that serve as controls.
Primary outcomes
Mathematics' test scores of student
Sponsors
Name: SUMMA Location: Santiago de Chile, Chile Url: https://www.summaedu.org/

Cited by

No citation data available for this registration yet.

Keywords