Using technology-enabled instruction to address cross-school, within-system heterogeneity in learning outcomes in Rwanda

How to cite

Rodriguez-Segura, Daniel; Turner, Elisabeth Jeannette; Sullivan, Tim. 2023. "Using technology-enabled instruction to address cross-school, within-system heterogeneity in learning outcomes in Rwanda." AEA RCT Registry. AEARCTR-0012245. https://doi.org/10.1257/rct.12245-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Daniel Rodriguez-Segura* , Elisabeth Jeannette Turner , Tim Sullivan

* Primary investigator

Registry ID
AEARCTR-0012245
DOI
https://doi.org/10.1257/rct.12245-1.1
First registered
Last updated
Status
Completed
Trial dates
2023-10-09 – 2024-07-30
Countries
Rwanda ()
JEL codes
I21 Analysis of Educational, I25 Education and Economic Development, I28 Government Policy
Registry record
http://www.socialscienceregistry.org/trials/12245

Abstract

Learning outcomes in low- and middle-income countries (LMICs) are consistently low (World Bank, 2018), yet curriculum expectations remain much higher than the benchmarks most pupils have met. This indicates a misalignment between pupil performance and policymaker expectations, and it is usually regarded as one of the contributing factors to the ongoing learning crisis (Pritchett and Beatty, 2015; Rodriguez-Segura and Mbiti, 2022). In addition to low average learning levels across many LMICs, researchers have also documented considerable heterogeneity in learning outcomes within classrooms (Muralidharan et al., 2019). This heterogeneity makes it difficult to deliver instruction that reaches most or all pupils in these classrooms, as teachers can deliver more targeted and effective instruction when classes are more homogenous (Duflo et al., 2011). In this sense, all else being equal, a broader baseline distribution of performance will result in instruction that only reaches a smaller number of students.
Given these pedagogical challenges posed by within-class heterogeneity, the literature has been appropriately concerned about the issue of within-class heterogeneity. Yet, on a macro-level, policymakers face a similar challenge with within-system heterogeneity when they perform activities like designing and implementing curricula, developing mastery benchmarks, and writing textbooks. They typically need to select a narrow level of learning to cater to in each grade, and it is therefore unlikely that they will effectively meet the needs of all pupils. Hence, collectively, the existing literature has not focused as much on within-system heterogeneity of foundational learning; instead, it often focuses on either overall low learning outcomes (World Bank, 2018) or discusses within-class heterogeneity (Muralidharan et al., 2019).
The present study aims to (1) quantify the extent to which within-system heterogeneity is present in a sample of 246 public schools across Rwanda and how this is mapped in conjunction with these pupils’ respective pedagogical needs toward mastering foundational literacy and numeracy, and (2) implement a randomized controlled trial where, in the treatment group, pupils’ pedagogical needs in each school*grade pair in Primary 4-6 are diagnosed at the start of the school year. Their teachers are then provided with customized teacher guides for the full year on literacy and numeracy. The control condition receives the same material across the board, regardless of their specific school*grade level, closer to the median level in each grade across the system.

Intervention

This study will consist of a three-part exercise: (1) At the beginning of the school year, data on foundational literacy and numeracy will be collected from a representative subsample of each school*grade pair in the study. (2) Each school*grade pair will then be randomized into either a treatment or control condition, stratifying by school size and cohort participating in the overall RwandaEQUIP program. (3) In the control condition, each class in the same grade will receive the same curriculum level across schools, similar to the approach used in most education systems worldwide, where the curriculum for a grade is standardized across all schools. However, this leveling was determined using system-wide learning data, which may result in it being closer to the median student's level in this system compared to other education systems globally. In contrast, the treatment condition will receive numeracy and literacy levels that are customized for the median student in their respective school*grade*subject. This intervention will last a full academic year, assessing whether a more tailored leveling of instruction on foundational learning skills in grades Primary 4-6 leads to higher learning outcomes.

Experimental design

The treatment assignment was stratified using school size and cohort entering into the RwandaEQUIP programme. The trial will be limited to Primary 4-6 classes. Leveling decisions in the treatment group are made based on the median pupil performance in each school*grade*subject. The control condition was assigned to approximately 20% of all school*grade pairs.

Design details

Randomization method
Randomization performed using Stata 18 by researchers.
Randomization unit
School*grade pairs.
Clusters
733 school*grade pairs, across 246 schools.
Observations
We aim to have administrative outcome data for at least 10 pupils in each school*grade pair, along with any additional data that is able to be collected via enumerators (budget pending).
Arms
147 school*grade pairs out of the 733 pairs in the sample are part of the control condition (20%)
Primary outcomes
A uniform numeracy assessment across treatment and control schools. Oral reading fluency and reading comprehension data to assess literacy.

Cited by

No citation data available for this registration yet.

Keywords