Identifying and Nurturing Math Talent at Scale: An Experiment with Art of Problem Solving Self-Paced Classes in Tamil Nadu, India

How to cite

Adajar, Phi; Duflo, Esther; Ellison, Glenn; Ellison, Sara; Kannan, Harini. 2024. "Identifying and Nurturing Math Talent at Scale: An Experiment with Art of Problem Solving Self-Paced Classes in Tamil Nadu, India." AEA RCT Registry. AEARCTR-0014090. https://doi.org/10.1257/rct.14090-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Phi Adajar* , Esther Duflo , Glenn Ellison , Sara Ellison , Harini Kannan

* Primary investigator

Registry ID
AEARCTR-0014090
DOI
https://doi.org/10.1257/rct.14090-2.0
First registered
Last updated
Status
In development
Trial dates
2024-08-12 – 2030-06-01
Countries
Private
JEL codes
D61, I24, I25, J24, O30
Registry record
http://www.socialscienceregistry.org/trials/14090

Abstract

In this project, partnering with the Art of Problem Solving (AoPS) and the Government of Tamil Nadu, we will test whether advanced online coursework and tutoring can identify and support high achievers who lack access to resources. Using administrative data and a screening mechanism, we will provide direct evidence of the distribution of high-potential students in Tamil Nadu. In an RCT with high-achieving students, we will test the effects of offering an advanced pre-algebra class and the additive benefits of support systems, such as tutoring. We aim to provide a model that can be extended to other Indian states or other countries if it proves successful.

Intervention

In our main treatment, students will receive one of four interventions.
a) Access to AoPS Pre-Algebra 1 course for six months, with various technological support, including WhatsApp groups and tablet + SIM access (n = 800)
b) Identical access as in (a), and weekly group tutoring sessions (n = 600)
c) Identical access as in (b), and weekly 1:1 tutoring sessions (n = 200)
d) Control group used as a baseline for comparison. (n = 800)

Treated students were also cross-randomized with two additional treatments.

1. 400 students in treatment groups were randomly assigned to receive support through school-provided computer labs. Working with school personnel, students were allowed after-school access to their school computer lab to work on AoPS.
2. Students were allocated to one of 3 "incentive" treatments
a) 533 students received no incentives.
b) 533 students were told that if they did not have adequate progress in the course, their tablet access would be revoked (though could be “earned back” if they kept up progress in the course). These incentives were enforced if a student had less than 80% of expected progess.
c) 533 students received the same information as group (b), but incentives were unenforced.

All treatments are randomized at the school level.

Students were selected for the program using one of several criterion, randomized at the school level. Selection will be based on four criteria:
a) (240 students) Performance on existing in-school tests
b) (240 students) Engagement with the internet-linked content and uploaded problems
c) (240 students) Performance on our virtual test
d) (240 students) Measures of student grit and persistence
The remaining 1440 students will be selected using combinations of the above criteria.
e) (240 students) an equal-weighted combination of (a) and (b)
f) (240 students) an equal-weighted combination of (b) and (c)
g) (480 students) an equal-weighted combination of (a) and (c)
h) (480 students) an equal-weighted combination of (a), (b), and (c)

Experimental design

Students will be selected for this intervention using a combination of test scores on government-administered public school examinations and a screening mechanism in which high-performing students will receive access to a sample of AoPS material.

After identifying students, they will be randomized to one of the four primary interventions, and potentially cross-randomized to additional interventions, as described above.

Design details

Randomization method
Randomization performed in Stata with a saved seed.
Randomization unit
Clustered at school level.
Clusters
1100 schools
Observations
2400 students
Arms
Main Treatment: 800 students in control, 800 with AoPS only, 600 with group tutoring, 200 with 1:1 tutoring. For subtreatments: Incentive: 533 students with no incentives, 533 students with enforced incentives, 533 students with unenforced incentives. (All belong to treatment groups in main design.) Comptuer Labs: 1200 untreated students, 400 treated students. (All belong to treatment groups in main design.)
Primary outcomes
1. Retention, grades, and engagement in AoPS class (only for the treatment groups) 2. Test scores on regular TN examinations (administrative data) 3. Test scores on a customized test created to measure skills in the standard math curriculum, the AoPS curriculum, and general mathematical reasoning, with a higher ceiling than standardized tests, administered at the end of the RCT. 4. Performance at the grade 10 board examinations (a high-stakes exam administered to all children in Tamil Nadu). 5. Admission into Tamil Nadu elite high schools during grades 9-12, and performance on any relevant entrance exams. For this to be a reliable outcome, we may encourage our entire sample to apply and take the entrance exams. 6. Performance on any state-wide or nation-wide exams, such as the NMMS. 7. Later-life outcomes, such as high school graduation, college matriculation, college selectiveness, and choice of a STEM major/career.
Secondary outcomes
1. Qualitative evidence on students’ interest in STEM, belief in math abilities, attitude to towards math, plans, comfort and use of technology, and other behavioral outcomes, collected at the end of 10th and 12th grade. 2. Parental beliefs about student abilities, confidence, and aspirations for their children. 3. Impacts on students' non-mathematics achievement, including Science, English, and Tamil.
Sponsors
Name: Agency Fund Url: https://www.agency.fund/ Name: Founder's Pledge Url: https://www.founderspledge.com/
Partners
Name: Art of Problem Solving Partner_type: private_company Url: https://artofproblemsolving.com/ Name: School Education Department, Government of Tamil Nadu Partner_type: government Url: https://tnschools.gov.in/welcome

Cited by

No citation data available for this registration yet.

Keywords