Decision-focused evaluation of an NGO-led TaRL model in Morocco

How to cite

Ricard, Claire; Plat, Estelle; Salas, Catalina; Waaziz, Oumaima; Zakraoui, Hamza. 2024. "Decision-focused evaluation of an NGO-led TaRL model in Morocco." AEA RCT Registry. AEARCTR-0013669. https://doi.org/10.1257/rct.13669-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Claire Ricard* , Estelle Plat , Catalina Salas , Oumaima Waaziz , Hamza Zakraoui

* Primary investigator

Registry ID
AEARCTR-0013669
DOI
https://doi.org/10.1257/rct.13669-1.0
First registered
Last updated
Status
On going
Trial dates
2023-10-23 – 2024-06-02
Countries
Private
JEL codes
I21 ; I38 ; J24 ; O12 ; O15
Registry record
http://www.socialscienceregistry.org/trials/13669

Abstract

To achieve the goal of ensuring that over 90% of students acquire basic academic skills by the end of primary school, the Government of Morocco is revisiting its remedial education policy. This study evaluates the impact of the Teaching at the Right-Level (TaRL) approach compared to the standard remedial education program implemented by an NGO, focusing on four key outcomes.

We are conducting a Randomized Control Trial (RCT) to assess the effects of TaRL on: (i) student performance on an extended ASER test, (ii) gender dynamics during remedial education sessions, and (iii) teachers' gender stereotypes towards students, measured through an Implicit Association Test (IAT). We will also evaluate if the increase in students' learning outcomes is proportionally higher than the increase in program costs. To the best of our knowledge, the gender-related impacts of TaRL have not yet been studied in the literature, making this an important area of investigation.

By leveraging administrative data and a difference-in-difference methodology, we will examine the spillover effects of TaRL on the academic performance of non-beneficiary students. Our access to extensive administrative data enables us to use schools without a remedial education program as a control group.

Intervention

Teaching at the Right-Level (TaRL) is a remedial education approach aimed at enhancing students' basic literacy and numeracy skills. This method involves grouping students by proficiency level and subject rather than by age, as is typically done in the national education system. TaRL incorporates a set of activities and a participatory pedagogical approach, with students attending remedial education sessions outside regular school hours.

The implementing foundation will administer Annual Status of Education Report (ASER) tests, adapted to the Moroccan context, to students in grades 3 to 6. Based on their performance in these tests, students will either receive TaRL support (beneficiaries) or no additional support (non-beneficiaries). ASER is a simple, standardized test that estimates children's educational status and basic learning levels.

The remedial education sessions will be organized as follows:

- 100 hours of TaRL activities (4 hours per week) over 25 weeks (approximately 6 months, or one full semester).
- Each week, there will be 1 session of French (2 hours), 1 session of Arabic (1 hour), and 1 session of Mathematics (1 hour).
- This schedule will provide students with 50 hours of French sessions, 25 hours of Arabic, and 25 hours of Mathematics.

Educators were trained in the TaRL methodology during the second semester of 2023. Concurrently, mentors were trained to supervise and support educators in implementing TaRL activities. The pilot phase began in October 2023 and will conclude, including national school holidays, around June 2024.

Experimental design

We selected a sample from a pool of 129 schools collaborating with the implementing partner. However, not all schools could participate in the study.

39 schools that implemented TaRL last year as part of a pre-test were excluded from the evaluation. Due to logistical constraints, the implementing partner chose not to implement its usual program in the provinces of Inezgane and Khmisset. Only one school in Sidi Slimane remained eligible for inclusion in the sample, as it was not part of the pre-test list submitted by the implementing partner.

To conduct impact analyses by province, a mix of schools implementing TaRL and the traditional remedial education program was necessary. Unfortunately, this arrangement was not feasible in Inezgane, Khmisset, and Sidi Slimane, leading to their exclusion from the sampling frame. Consequently, the final sample frame consisted of 89 schools.

From these 89 schools, a subset of 60 schools was selected for the impact evaluation. Each selected school was randomly allocated to either the TaRL group or the usual remediation group.

Schools were selected based on a stratified randomization procedure to ensure balance along outcomes of interest and representation of subgroups. Stratification was done by province, and within each school, students were randomly drawn in each stratum (gender and school grade) proportionately to the Sanady student population.

Due to implementation issues in the province of Taroudant, this province was excluded from our sample (8 schools). An additional school was removed because the implementing partner was unable to work there. The final sample comprises 51 schools with an average of 22 students per school. All teachers conducting remedial education in each school will take the Implicit Association Test, with an expected average of 3 teachers per school.

Design details

Randomization method
Randomization was conducted in office by a computer.
Randomization unit
Two-stage cluster random sampling: First stage is schools ; Second stage is students
Clusters
51 schools
Observations
1,140 pupils Approximately 153 teachers for the Implicit Association Tests
Arms
26 schools control, 25 TaRL schools
Primary outcomes
Scores at the extended ASER test we developed with our partners
Secondary outcomes
1) Difference in the implementation cost between the control and treatment interventions. 2) Difference in gender dynamics during remedial education sessions between TaRL and the control intervention. 3) Difference in teachers' implicit gender stereotypes towards students in treatment and control schools. 4) Difference in the trend of average grades given by teachers in treatment and control schools.

Cited by

No citation data available for this registration yet.

Keywords