Teacher observation, teacher effectiveness, and pupil attainment: An RCT in England’s secondary schools
How to cite
Burgess, Simon; Taylor, Eric; Rawal, Shenila. 2016. "Teacher observation, teacher effectiveness, and pupil attainment: An RCT in England’s secondary schools." AEA RCT Registry. AEARCTR-0001010. https://doi.org/10.1257/rct.1010-2.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
We study a program of informal, peer performance evaluation among classroom teachers who work in the same school. Maths and English teachers who teach these subjects in years 10 and 11 in “treatment” schools, all of which are secondary schools in areas of high deprivation in England, are asked to implement a program of peer observation for a period of two school years. The experiment is designed to estimate the effect of peer observation on teacher performance in the classroom, as measured by teachers’ contributions to student achievement and attainment, and by teachers’ observed teaching practices. The experiment is also designed to estimate different effects for teachers who are the “observer” and teachers who are the “observee”, and the influence of the frequency of observations. Observer teachers conduct an observation of their peer observee teacher and provide an evaluation; observers assign scores on specific skills using a structured rubric. However, the program is not a formal evaluation; there are no formal incentives or stakes attached to the scores.
Experimental design
Of the 82 schools in our study sample, 41 were randomly assigned to the treatment condition and 41 to a business as usual control. Random assignment occurred within blocks formed by two school-level measures: racial/ethnic composition and student test score growth attributable to the school (school value-added). As detailed below, additional randomization of teacher roles and the frequency of observations occurred within the 41 treatment schools.
In treatment schools, English and Maths teachers are asked to implement a program of peer observation and performance description for a duration of two school years. However, the program is not a formal evaluation. No explicit incentives or penalties are attached to teachers’ scores.
Within treatment schools, English and Maths teachers were randomly assigned to one of three role conditions: (i) teachers were assigned to the “observer” role with probability 1/3, (ii) to the “observee” role with probability 1/3, or (iii) to participate in both “observer” and “observee” roles with probability 1/3. Assignment to role was within school-by-subject blocks, where subject is either Maths or English.
Throughout the school year, “observers” periodically spend time watching “observees” teach in the observees’ classes. Each observation lasts 20-30 minutes. During each visit, observers are asked to pay particular attention to the observee’s performance in several specific teaching skills, and score those skills using an evaluation rubric and tablet computer program. Each skill is first scored as being “Ineffective (1-3)”, “Basic (4-6)”, “Effective (7-9)”or “Highly Effective (10-12)”.. The rubric provides a concrete description of what an observer should see happening in the classroom to warrant allocation to each one of these categories. After choosing one of these four categories, observers can choose a numeric score from within each of these categories creating a final 12 point score scale. Observers and observees were encouraged to meet and discuss the observation results however the form and nature of these feedback sessions were not prescribed. Note given the assignment of roles, some teachers will be the observer in one pair interaction, and the observee in a different interaction.
In treatment schools, English and Maths departments were randomly assigned to a “high frequency” observation condition or to a “low frequency” condition. In half of treatment schools the English department was assigned to “high frequency” and the Maths department to “low frequency.” In half of schools the department assignments were reversed. In the high-frequency condition, observees are required to be observed 12 times per year. In the low-frequency condition, the observees are to be observed 6 times per year.
These design features create three key “treatment effect” estimates of interest. First, the broad contrast of student achievement outcomes between the (i) peer observation treatment schools, and (ii) business-as-usual control schools. Second, the contrasts in teacher performance—as measured by student test scores—between (i) teachers in the observer role; (ii) teachers in the observee role; (iii) teachers in treatment schools with no role, but who might have gained through spillovers; and (iv) teachers in control schools with no role and no exposure to treatment. Third, the contrast in teacher performance between teachers who observed or were observed with (i) high frequency, or (ii) low frequency. Moreover, the design also permits estimates of the interactions between these three main contrasts.
Design details
Papers from this trial
- Teacher Peer Observation and Student Test Scores: Evidence from a Field Experiment in English Secondary Schools. Journal of Labor Economics Volume 39, Number 4 October 2021 https://doi.org/10.1086/712997
- Teachers’ use of class time and student achievement Economics of Education Review, 2023, vol. 94, issue C https://doi.org/10.1016/j.econedurev.2023.102405
Cited by
No citation data available for this registration yet.