Teach for science: An impact evaluation of Science Education Initiative’s Fellows Program

How to cite

Mbiti, Isaac; Ganimian, Alejandro. 2018. "Teach for science: An impact evaluation of Science Education Initiative’s Fellows Program." AEA RCT Registry. AEARCTR-0002386. https://doi.org/10.1257/rct.2386-2.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Isaac Mbiti* , Alejandro Ganimian

* Primary investigator

Registry ID
AEARCTR-0002386
DOI
https://doi.org/10.1257/rct.2386-2.1
First registered
Last updated
Status
Completed
Trial dates
2017-06-01 – 2018-09-30
Countries
India (Pune, Maharashtra)
JEL codes
C93, I21, I24
Registry record
http://www.socialscienceregistry.org/trials/2386

Abstract

We propose to conduct a randomized evaluation of Science Education Initiative’s fellowship program in Pune, in the Indian state of Maharashtra, in 46 public and charter schools. Science Education Initiative seeks to improve STEM learning in public schools by recruiting highly motivated college students who are placed in schools to work with regular teachers as teaching assistants. The fellows work alongside teachers using scripted lesson plans to deliver potentially better instruction to students. The proposed RCT will examine the effectiveness of the SEI fellow program on student STEM learning outcomes, as well as gender achievement gaps in STEM. Detailed information on management, classroom practices, teacher surveys, and student surveys will provide insights into any potential mechanisms.

Intervention

The intervention is a program run by Science Education Initiative (SEI), a non-profit organization based in Pune, in the Indian state of Maharashtra.

The “SEI Fellowship Program” (SEI-FP) attempts to deliver high-quality science and math instruction in schools serving low-income students, by:
i. Developing high-quality content and pedagogical tools aligned with global standards;
ii. Selecting highly-motivated college graduates in Science, Technology, Engineering, and Math (STEM) to serve as “shadow” science and math teachers in schools serving low-income students;
iii. Providing intensive training to fellows on pedagogy, content, and classroom management; and
iv. Monitoring fellow performance and provides continuous feedback to improve classroom instruction.

In 2016, SEI placed 36 fellows across 52 classrooms, and 35 fellows in the 2015 academic year. In the current 2017-18 academic year, SEI has placed 68 fellows in 110 classrooms across 63 PMC schools, reaching approximately 4,000 students

Experimental design

We are undertaking a randomized evaluation of the SEI-FP in 46 public and Vidya Niketan (ViNi) schools under the Pune School Board (PSB). These schools were chosen in conjunction with SEI after considering a number of factors such as the language of instruction, the presence of other NGOs or teaching programs, and the remoteness of the school. Among our set of study schools, one SEI fellow per school was randomly assigned to either grade 5 or grade 6. This approach resulted in 23 grade 5 classrooms and 23 grade 6 classrooms in the treatment group and an equal number of control groups in both grades.

Design details

Randomization method
Randomization was done in office by a computer. This randomization was conditional on geographic and school timing constraints to minimize the chance that fellows were placed far from where they live or placed in schools that don’t match their availability.
Randomization unit
The randomization was done at the classroom level in each of the 46 schools.
Clusters
92 classrooms
Observations
There will be approximately 2,898 observations, comprising of: (i) an estimated 2,760 students (at an average of 30 students per classroom in 92 classrooms); (ii) 46 SEI fellows placed in the treatment classrooms; (iii) 92 public school teachers teaching in the treatment and control classrooms.
Arms
A treatment group of 46 classrooms (23 grade 5 classrooms and 23 grade 6 classrooms) will receive an SEI teaching fellow. The treatment classes will have about 1380 students in total (about 30 per class) A control group of 46 classrooms (23 grade 5 classrooms and 23 grade 6 classrooms) will not receive an SEI teaching fellow. The control classes will have about 1380 students in total (about 30 per class)
Primary outcomes
Our study will focus on the following outcomes: student achievement (test scores) and classroom instruction. Specifically, we collect the following data: i. Student assessments in math, science and language ii. An assessment of teacher subject matter expertise in science and math, based on both domestic (e.g. Pratham teaching capability test, Teacher’s Understanding of Subject Knowledge or TUSK) and international models (e.g. Context Knowledge for Teaching or CKT); iii. A survey of teachers, based on several international models (including the Teaching and Learning International Study or TALIS, the Program for International Student Assessment or PISA, the Program for International Reading Literacy Study or PIRLS and the Trends in International Mathematics and Science Study or TIMSS); iv. A survey of students (based on PISA and the MET project’s “Tripod” survey). Our focus will be not only to document the impact of the program on student achievement, but also to document changes in school management and classroom instruction practices. We will also conduct one-to-two rounds of unannounced visits to schools and conduct classroom observations to document the program’s impact on: (a) teachers’ presence in school and the classroom; (b) the share of classroom time devoted to instruction, classroom management, and other tasks drawing on the Stallings Observation Protocol, which has been previously used in India and other developing countries) (Stallings et al., 2014; Sankar & Linden, 2014; World Bank, 2016); and (c) different classroom teaching practices (drawing on the Classroom Assessment Scoring System or CLASS).

Cited by

No citation data available for this registration yet.

Keywords