In-School Support to Teachers for Implementing Computer Assisted Learning Effectively: A Randomized Controlled Trial of Teacher Assistance Across Government Schools In India

How to cite

Oreopoulos, Philip. 2024. "In-School Support to Teachers for Implementing Computer Assisted Learning Effectively: A Randomized Controlled Trial of Teacher Assistance Across Government Schools In India." AEA RCT Registry. AEARCTR-0013921. https://doi.org/10.1257/rct.13921-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Philip Oreopoulos*

* Primary investigator

Registry ID
AEARCTR-0013921
DOI
https://doi.org/10.1257/rct.13921-2.0
First registered
Last updated
Status
In development
Trial dates
2024-08-01 – 2026-06-30
Countries
India ()
JEL codes
I2
Registry record
http://www.socialscienceregistry.org/trials/13921

Abstract

This proposal addresses the challenges of educational heterogeneity and the need for personalized instruction in India. This project provides personalized learning and high quality state aligned content in vernacular language to students through Computer Assisted Learning (CAL) by leveraging Khan Academy, a widely recognized online educational platform to address the foundational gaps and low academic levels among students in government schools in India.

The core strategy involves deploying 'khoaches' to selected government schools in Uttar Pradesh, who will facilitate the integration of Khan Academy into the curriculum, offer training to teachers and students, and ensure effective use of the platform. This intervention aims to improve math scores by allowing students to learn at their own pace, receive immediate feedback, and engage in mastery-based learning. The project will employ a randomized controlled trial to evaluate its impact, focusing on grades 6 to 8 mathematics, with an aim to provide credible evidence on the benefits of CAL when supported by trained personnel. The research will also examine how well teachers adopt the guided approach a year later, after completing their training.

Intervention

Many government schools in India have adopted some form of EdTech tools after encouragement from the State but only a small fraction of teachers and students are able to actually use these tools regularly and effectively due to multiple challenges - 1) Limited digital infrastructure, 2) Teacher shortage, 3) Insufficient digital knowhow across stakeholders, 4) Lack of teacher motivation and 5) Lack of sufficient time to practice with EdTech platform etc.

The project will assign Khoaches to intervention schools to maintain implementation fidelity. Khoaches will be hired on contract basis by Khan Academy and will undergo rigorous training to make sure they are fully equipped to handle the KA program in government schools and provide the necessary support and training to teachers and students. Khoaches will work collaboratively in consultation with the Social Welfare Department and schools to devise best strategies for ensuring adequate practice time (at least 30 minutes to 60 minutes of usage per week). The control schools will also have access to Khan Academy but the program will be driven by teachers themselves (current overall usage rate from this setup is low). Khoaches will visit the assigned school twice a week to supervise and conduct Khan Academy practice sessions. Khoaches will be responsible for - 1) Providing training to teachers and students; 2) Creating and helping teachers and students with their digital accounts (Rostering); 3) Supervising and ensuring Khan Academy sessions are conducted regularly, 4) Helping teachers assign the relevant content & track student performance and 5) Technical troubleshooting. Once teachers become comfortable with the process, they should be able to use and integrate KA with their curriculum with greater ease and effectiveness in subsequent years and the same should reflect in usage data.

Research Question
The purpose of this study is to evaluate the effectiveness of Khan Academy, a Computer Assisted Learning tool that offers personalized learning and high quality state aligned content in vernacular language to students, in improving Math test scores of government school students in India for grades 6 to 8 when supported by trained personnel (Khoaches).

Target Population
The study would be conducted in schools under Social Welfare department in Uttar Pradesh. There are a total of approximately 33,500 students in 105 schools across grades 6 to 12. The study would cover 75 schools out of these.

Experimental design

Research Design
The study would be carried out with government schools under the Social Welfare department in Uttar Pradesh for grades 6 to 8. There are a total of 105 schools under the department out of which 75 schools will be filtered on the basis of factors such as ‘Availability of Internet and Desktops/Tablets’ to ensure sufficient access to digital infrastructure. These 75 schools will be subsequently assigned to the ‘Treatment’ and ‘Control’ category randomly in 1 to 2 ratio leading to 25 Treatment schools and 50 Control schools. The schools in the Treatment category will receive the assistance of Khoaches for grades 6 to 8 from Jul’24 to Feb’25. These Khoaches will visit the schools twice a week as per a pre-determined schedule and supervise Khan Academy sessions. In Control schools, teachers themselves will be responsible for driving the Khan Academy program. This design will allow for credible causal impact evaluation.

Design details

Randomization method
Randomization will be done by office computer.
Randomization unit
School leve randomization, with clusters to incude two nearby treated schools and two nearby control schools.
Clusters
85 schools
Observations
~5000 students
Arms
29 treated schools, 56 control schools
Primary outcomes
The primary outcomes of interest will be student math achievement. The improvement in student’s math scores will be tracked using the Baseline and Endline assessments. Both Baseline and Endline assessments will be designed by the research team and data collection will be carried out by Khoaches. Assessments would cover grade appropriate skills and competencies up to that point.
Secondary outcomes
Secondary outcomes include teacher satisfaction, and impressions of their students’ learning progress. The study will also measure, quantitatively and qualitatively, student and teacher experiences with the program. We will also collect Khan Academy usage levels, during the year of implementation and in the following year. This will allow us to evaluate whether teachers maintain high dosage KA usage after the program ends in subsequent years.

Cited by

No citation data available for this registration yet.

Keywords