System-Based Approaches to Improving Teaching Effectiveness
How to cite
Brown, Christina; Berry, Jim; Fitzpatrick, Anne; Lucas, Adrienne; Beg, Sabrin. 2025. "System-Based Approaches to Improving Teaching Effectiveness." AEA RCT Registry. AEARCTR-0014870. https://doi.org/10.1257/rct.14870-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Teaching quality is a key input into student learning but is typically poor in most low-income settings. One intervention that has been championed by the World Bank is the Teach tool, a classroom observation form that captures 30 elements of teaching practices that are each associated, but not yet rigorously causally linked, with student learning. Although this tool has been implemented in 30 countries to monitor and improve teaching quality together with observer feedback, whether it, in fact, does improve teaching quality is unknown. This study would be the first to test the effectiveness of this intervention as it is rolled out in 20,000 schools in Andhra Pradesh. We seek to address two main gaps in the evidence:
1. Does being observed using the Teach tool and receiving general feedback improve teacher effort and student outcomes?
2. How does improving in a given dimension of the rubric affect student learning?
Intervention
The Teach tool was developed to provide principals, observers, or other stakeholders a window into what teaching practices are happening in a classroom. It is a 30-item World Bank observation form that highlights teaching elements associated with better student outcomes, grouped into four broad categories: 1) Time on Learning, 2) Classroom Culture, 3) Instruction, and 4) Socioemotional Skills.
We will evaluate two different implementations of the Teach tool: the status quo of implementation and a variant that may improve its efficacy. In both implementations, all observers are trained to use the Teach tool through a cascade model – Learning for Equity, our implemntation partner, trains master trainers who train the observers. Once trained, observers are instructed to begin observations using the Teach tool. The observation is an in-depth undertaking, requiring each observer to sit and observe a class for 30 minutes and provide feedback into the Teach tool tablet app.
In the first treatment (T1), teachers will be observed, told their scores, and provided with a video on how to improve one of the teaching aspects that is deficient in the state. However, the content of this video may not reflect a particular teacher’s deficiency. In the second treatment (T2), teachers will receive the same T1, but additionally receive ongoing coaching and mentoring from the observer on how to improve in specific areas in which they are lacking. Control schools will not receive any changes. Observers were trained in June-July 2024 and began conducting classroom observations in September 2024.
We will randomize the videos that teacher receive that will allow us to separate the act of being observed (which should affect all competencies equally) from the effect of the randomly assigned targeted follow-up. In treatment 1, the video will be randomly selected from the entire video library. In treatment 2, the assignment will be randomly selected from the three lowest-scoring competencies of a given teacher.
Experimental design
We conduct a cluster-randomized controlled trial with two treatment groups and one control group. The randomization was conducted at the school level stratified by district.
In the first treatment (T1), teachers will be observed following the TEACH tool and told their scores. However, the content of this video may not reflect a particular teacher’s deficiency. In the second treatment (T2), teachers will receive the same T1, but additionally receive ongoing coaching and mentoring from the observer on how to improve in specific areas in which they are lacking. Control schools will not receive any changes.
As TEACH tool is also being implemented in all schools across the state (Andhra-Pradesh), we will further use state-wide data of students’ test scores to assess the effectiveness of TEACH tool compared to our control group.
School-level randomization:
Among the 20,000 schools in Andhra Pradesh that are due to receive the program, we first randomly selected 501 schools. Among these schools, we then randomly assigned 167 schools into each treatment arm. Following randomization, we discovered that several schools were closed. To increase statistical power, we then increased the sample size to add in additional 100 schools to the Teach arm. Therefore, the final sample is 601 schools.
Design details
Cited by
No citation data available for this registration yet.