Scaling Teacher Performance Pay Contracts in Rwanda

How to cite

Zeitlin, Andrew; Leaver, Clare; Ozier, Owen; Serneels, Pieter. 2023. "Scaling Teacher Performance Pay Contracts in Rwanda." AEA RCT Registry. AEARCTR-0011887. https://doi.org/10.1257/rct.11887-4.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Andrew Zeitlin* , Clare Leaver , Owen Ozier , Pieter Serneels

* Primary investigator

Registry ID
AEARCTR-0011887
DOI
https://doi.org/10.1257/rct.11887-4.0
First registered
Last updated
Status
Completed
Trial dates
2023-01-19 – 2025-09-01
Countries
Rwanda ()
Registry record
http://www.socialscienceregistry.org/trials/11887

Abstract

This study evaluates potential scale-up of a teacher performance pay system in Rwanda. Working under the mandate of a Ministry of Education task force on the incorporation of learning outcomes in teachers' "imihigo" contracts, the study A/B tests design alternatives and evaluates impacts relative to control. The contract design draws on evidence from the Supporting Teacher Achievement in Rwandan Schools (STARS) trial (Leaver et al., 2021), while working with the Sector Education Inspectorate to collect incentivized teacher input measures and while adapting the national Comprehensive Assessment as the basis for teacher learning metrics. Over the course of three years, the study will undertake an adaptive design, winnowing an initial set of five designs down to a single, leading design, while generating evidence about efficacy relative to status-quo contracts.

Intervention

The study evaluates five designs for scaling teacher incentive contracts. These are based on a core design "Model A", with each of four variations changing one dimension of the design at a time.
The base design, Model A, incentivizes teachers based on "Five Ps": Presence, Preparation, Pedagogy, Participation in CAMIS, and Pupil Learning. Presence and preparation are measured by Sector Education Inspectors (SEIs) during unannounced visits, with the latter referring to the completion of lesson plans. Pedagogy is measured by SEIs during an announced visit, using the Government of Rwanda's structured classroom observation procedure. Participation in CAMIS refers to the use of the Comprehensive Assessment Management Information System to record end-of-term marks for students on the Comprehensive Assessment, and is observed using the CAMIS database itself. Finally, pupil learning is defined as learning outcomes relative to others of similar baseline achievement levels (Barlevy and Neal, 2012) using the end-of-year Comprehensive Assessment in each year as the basis for learning assessment. Audits of 5 percent of pupils' recorded exam scores, using paper records in schools, will be used to enforce fidelity of the exam system. Teachers are ranked within districts on each metric, and the teacher's evaluation score is a weighted average of these components, with measures of teacher inputs comprising 50 percent of the total weight, and the pupil learning metric comprising the remaining 50 percent. Teacher payoffs are a linear function of their percentile rank on this composite metric, ranging from zero to five percent of salary.
The four variations modify this design as follows: Variation B1 adds head teachers' subjective assessment of teachers' activities to the set of "input" measures. Variation B2 provides termly feedback on the performance score, using end-of-term Comprehensive Assessment data to produce projected learning growth measures. Variation B3 replaces the linear payoff function with a three-tiered, stepwise function that provides no bonus to those below the 25th percentile, a bonus of 3 percent of salary to those between the 25th and 75th percentile, and a bonus of 5 percent to those in the top quartile. Variation B4 augments the audit scheme by retesting 5 percent of pupils.

Experimental design

Schools are included in the study sample on the basis of having lower- and upper-primary comprehensive examination data submitted for the 2021-2022 school year. There are 481 study-eligible schools, comprising 120 sectors in the 10 study districts.
Assignment is conducted at the "sector" level; there are 62 treated sectors, containing 248 treated schools, and 39 control sectors, containing 95 treatment-eligible control schools.

Design details

Randomization method
Active-treatment sectors were selected by simple randomization, stratified within districts. Among active-treatment sectors, 20,000 potential assignments of treatment were generated, stratifying assignment by district. Potential assignments were deemed "feasible" if they met criteria for balance in the shares of schools treated across treatment arms (given clusters have heterogeneous sizes). The implemented treatment assignment was selected by simple random selection from among feasible assignments.
Randomization unit
Assignment is conducted at the "sector" level, a sub-district geo-political unit.
Clusters
There are 120 study-eligible clusters ("sectors"). Of these, 101 sectors (343 schools) are selected for low-stakes measurement, though administrative-data outcomes will be observed in all clusters.
Observations
Low-stakes measurement will take place in 343 schools. All lower-primary teachers in these schools will be observed for presence, preparation, and pedagogy. A sample of 30 pupils per grade will be assessed in low-stakes measurements of learning outcomes.
Arms
Low-stakes measurement schools: 39 sectors (95 schools) Base design (version A): 14 sectors (54 schools) Variation B1: 12 sectors (48 schools) Variation B2: 13 sectors (48 schools) Variation B3: 12 sectors (57 schools) Variation B4: 11 sectors (41 schools)
Primary outcomes
The study has five primary outcomes, corresponding to the incentivized dimensions of teacher performance: - Pupil learning; - Teacher presence at the school; - Teacher pedagogy; - Teacher preparation; and - Teacher use of CAMIS for end-of-term and end-of year assessments.
Secondary outcomes
In addition to the primary outcomes above, we will estimate and test for impacts on the following secondary outcomes: - High-stakes analogs of all (low-stakes) primary outcomes (with the exception of CAMIS data completion); - Teacher retention; - Student drop-out; - Number of pupils unsupervised at time of surprise visit; - Teacher satisfaction; and - Head teacher activities
Sponsors
Name: Development Innovation Ventures, USAID Location: USA Url: https://www.usaid.gov/div Name: Fonds d'Innovation pour le Développement, Agence Française de Développement Location: France Url: https://fundinnovation.dev/en/
Partners
Name: Ministry of Education, Government of Rwanda Partner_type: government Url: https://www.mineduc.gov.rw/

Cited by

No citation data available for this registration yet.

Keywords