Integrating Educational Technology with Structured Pedagogy to Improve Learning Outcomes for Every Student

How to cite

Andersen, Erik; Kerwin, Jason; Graffy, Simon; Lambon-Quayefio, Monica. 2024. "Integrating Educational Technology with Structured Pedagogy to Improve Learning Outcomes for Every Student." AEA RCT Registry. AEARCTR-0014627. https://doi.org/10.1257/rct.14627-7.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Erik Andersen* , Jason Kerwin , Simon Graffy , Monica Lambon-Quayefio

* Primary investigator

Registry ID
AEARCTR-0014627
DOI
https://doi.org/10.1257/rct.14627-7.0
First registered
Last updated
Status
Completed
Trial dates
2024-09-16 – 2025-07-04
Countries
Ghana (Cape Coast)
Registry record
http://www.socialscienceregistry.org/trials/14627

Abstract

We are studying Inspiring Teacher's Tools for Foundational Learning Improvement (TFLI). TFLI uses best practices from early-grade reading instruction in a structured pedagogy format, in which teachers are given teacher guides with semi-scripted, high-quality lesson plans that tell them exactly how to teach their classes. These plans are linked to workbooks, training and support from Inspiring Teachers’ employees, who are expert teachers and teacher trainers. This basic approach has been proven to be highly effective in many other interventions and contexts across Africa. TFLI adds a digital layer to this structured pedagogy approach, with teachers accessing program material and training videos via the Smart Coach smartphone app. The app will also collect high-frequency data on student outcomes, which Inspiring Teachers can use to modify the intervention on the fly, capturing the best aspects of teaching at the right level (TaRL) interventions as well.
In this study we aim to use Inspiring Teachers’ TFLI approach to explore how support for assessment-informed targeted instruction can be built into structured pedagogy and how technology can be used to support that process to enable cost-effective and data driven teacher and school support. Using grade 1 students in primary schools in Cape Coast, Ghana, as our study context, we seek to determine how TFLI affects learning outcomes.

Intervention

The intervention to be implemented is “Tools for Foundational Learning Improvement” (TFLI). TFLI is a structured pedagogy program which uses a “science of reading” approach, combined with assessment-informed targeted instruction for early-grade reading. In addition to paper materials, training, and coaching, the program provides a companion app with video explainers and tools for running 1-1 literacy assessments, tracking student progress and managing coaching. This app (SmartCoach) adds a digital layer to the program, enabling high-frequency data collection on program delivery and student learning outcomes. This enables data-driven adaptive program management and is intended to cultivate a more purposeful learning centered approach to program delivery.

Experimental design

We will randomly assign a total of 80 schools to two study arms, with 40 schools in the control group and 40 schools in the treatment group. Schools in the treatment group will receive the structured pedagogy program over the 2024/2025 academic year. For this academic year, the focus is only on grade 1 students. There will be no baseline data. Within the intervention period, we will collect student lists for every school at the beginning of the 2024-25 school year. We will run a separate analysis studying whether the type of enumerator who collects the data affects measured test scores. Specifically, we have two groups of enumerators: 1) people with teaching certifications (which we call “teachers”, although they are not currently working as teachers) and 2) , and outside contractors hired directly by our organization and who are have no other not currently working in connection the education sector (“non-teachers”). Each group will have approximately 12 members. We will randomly assign an equal number of teachers and non-teachers to each school and randomize at the student level which enumerator collects each student’s data.

Design details

Randomization method
Randomization was done in the office by computer. Specifically, the randomization was run in R using the randomizr package.
Randomization unit
The treatment is randomized at the school level. The type of data collector (i.e. SISO or contractor) is randomized at the individual level.
Clusters
80 schools
Observations
80 schools (2400 pupils in total)
Arms
40 schools in treatment, 40 schools in control
Primary outcomes
English EGRA score (in SDs) - Score is the weighted average of the subtest scores, where the weights are the first principal component of the control-group data across all English EGRA components we tested in this wave of data collection, for every student in the relevant cohort (BS1 students in the 2024-25 school year). We will standardize each subtest score by the control-group mean and SD before running PCA. See the PAP for more details on how the scores are measured.
Secondary outcomes
Besides EGRA scores, we will also study the following outcomes from data collected during the school visits: a. Other questions from the student survey. Specifically, we plan to analyze how the intervention affects students’ perceived class ranks, career, and academic aspirations. We will also test if the intervention affects whether students practice reading and writing outside of school. b. Quality of teaching instruction, as measured via classroom observation. c. The distributional effects of the intervention using quantile regression, distributional regression, and K–S Tests. d. Effects on individual subtests of the EGRA (including distributional effects) We will also do three other secondary analyses. First, we will study the effect of the type of enumerator (SISO vs. outside contractor) on tests scores and estimated treatment effects. Second, we will validate the data collected by enumerators. This will be validated by audio recording the EGRA evaluations and re-scoring them using an AI tool. Third, the treatment effects of the intervention will also be expressed as Equivalent Years of Schooling (EYS). Since we have no baseline test scores, we will explore other ways of measuring typical progress on the EGRA each year. Finally, we will look at treatment effect heterogeneity analyses by student gender, teacher gender and teacher-student gender match. We may also do other exploratory analyses of the data; this list is not comprehensive.

Cited by

No citation data available for this registration yet.

Keywords