The effects of a large-scale social and emotional learning program on educational outcomes

How to cite

Schildberg-Hörisch, Hannah; Breitkopf, Laura; Chowdhury, Shyamal; Kamhöfer, Daniel; Sutter, Matthias. 2024. "The effects of a large-scale social and emotional learning program on educational outcomes." AEA RCT Registry. AEARCTR-0012839. https://doi.org/10.1257/rct.12839-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Hannah Schildberg-Hörisch* , Laura Breitkopf , Shyamal Chowdhury , Daniel Kamhöfer , Matthias Sutter

* Primary investigator

Registry ID
AEARCTR-0012839
DOI
https://doi.org/10.1257/rct.12839-1.0
First registered
Last updated
Status
On going
Trial dates
2018-03-01 – 2025-12-31
Countries
Bangladesh (South Asia)
JEL codes
C93; D90; I21
Secondary IDs
DFG grants SCHI 1377/1-1 and SCHI 1377/1-2
Registry record
http://www.socialscienceregistry.org/trials/12839

Abstract

This pre-registration outlines the collection of new data and the formulation of additional hypotheses for our intervention study of the Lions Quest Skills for Growing (LQ SfG) program in elementary schools in Bangladesh. While the previous pre-registration (AEARCTR-0003129) and data collection aimed at describing our analysis of the short-term effects of the intervention on students’ socio-emotional development, this pre-registration considers effects of the intervention on teachers’ teaching style and their beliefs, attitudes, and behaviors with respect to teaching.

In the initial study, using an RCT in a sample of more than 3,000 children attending 135 schools, we assess the direct effectiveness of the LQ SfG program. LQ SfG is a school-based social and emotional learning (SEL) program that was taught in grades 2 to 5 of 69 randomly assigned elementary schools in Bangladesh during the academic year 2019 with specific lessons on, e.g., self-discipline, impulse-control, goal-setting, working cooperatively, empathy, perspective-taking, and self-confidence. The program is characterized by interactive and diverse teaching with example stories, open discussions, role plays, and group work.

Besides assessing LQ SfG’s effect on children’s socio-emotional skills such as self-control or prosociality, we also measure elementary school children’s educational performance by official school exam results in six subjects and by achievement test scores in Bengali and math.

Collected data on children’s skills and behaviors as well as first evidence from preliminary teacher surveys directly following the intervention suggest that teachers and an enhanced teaching style may play a role in explaining positive treatment effects on school exams and achievement test scores. The initial surveys covered only information on teachers’ socio demographics (age, gender, education, and experience), their absences, their attitudes towards teaching in a brief, rather ad-hoc manner and, for LQ teachers, their identification with the program and possible implementation problems they came across. To follow up on this, we will collect from our 135 sample schools and from teachers who taught at these schools during the intervention period a) refined matching data to link teachers to students (we currently only know whether students and teachers attended the same school in 2019 but not whether a specific teacher taught a specific student), b) measures of teaching style, beliefs, attitudes, behaviors, and personality, c) general information on schooling during the past Covid-19 pandemic years, and d) if feasible, pre-treatment and further post-treatment school exam results.

With these additional data, we plan to investigate the role of teachers and whether teaching style is related to both the intervention and children’s educational attainment. By improved knowledge of the matching of teachers to the students they taught, we can use individual-level teacher data instead of aggregates at school level which we are restricted to with our current data base.

The new data collection will start mid-January/beginning of February 2024.

Intervention

The intervention consists of conveying the content of the well-established Lions Quest Skills for Growing (LQ SfG) program during a designated school lesson in randomly assigned treatment schools. As a social and emotional learning (SEL) program, LQ SfG aims at helping children to make good decisions, among others, in terms of forward-looking behaviors such self-discipline, impulse-control, and goal-setting, or in terms of social awareness and relationship skills such as working cooperatively, empathy, and perspective-taking.

The program has a longstanding history and follows a curriculum that is implemented by the children’s regular teachers in the classroom environment. It is characterized by very interactive and diverse teaching with example stories, open discussions, role plays, and group work.

The full schedule comprises six units that each consist of several lessons: 1) building a positive learning community, 2) personal development, 3) social development, 4) health and prevention, 5) leadership and service, and 6) a reflection section on what has been learned.

In cooperation with the Lions Clubs International Foundation, learning materials got translated and adopted to the local context (for instance, pictures of children in the materials were changed to depict Bangladeshi rather than U.S. American or Indian children). In the course of our program adoption, school teachers got trained as LQ teachers in three-day workshops by international, qualified LQ trainers and received a textbook with detailed instructions. On top, refreshment trainings were organized half-way through the program. Children got a student journal in which they could summarize the topics and do homework. Parents were invited to a single, mid-program meeting with the LQ teachers, implementation staff, and local education authorities in their children’s school.

The program ran over the course of one school year following an official 28-week schedule (that omitted unit 5) between February and October 2019. Lessons took place on a weekly basis.

Each lesson lasts for about 30 minutes and is divided into four parts. First, the teacher presents an every-day situation and identifies together with the students why this situation is problematic (“discovery phase”). Second, students are encouraged to share similar experiences and the class discusses reasons and solutions for the problem (“connecting phase”). When learning how to make good decisions, for example, children are taught to act according to the “Think, Predict, Choose Model.” Being confronted with a decision, they are trained to follow a traffic-light approach and step back, calm down, reflect on their options and the consequences, and carefully work out what to do. They also discuss how to keep up their motivation for tedious tasks by not following immediate impulses but reminding themselves of why this goal is important to them or where they have been successful before. Third, students reenact the presented situations in role plays or solve tasks in pairs or small groups and employ the solutions and strategies they have talked about (“practicing phase”). Finally, teachers assign a homework related to the week's topic (“applying phase”). Students are, for instance, asked to discover and solve similar situations in their daily life and document their progress in the student journal.

Please see https://www.lions-quest.org/explore-our-selcurriculum/elementary-school-social-and-emotional-learning/ for more information.

Experimental design

We have a single treatment and a single control group. The 135 elementary schools in our sample are divided into 69 treatment and 66 control group schools by stratified randomization. In cooperation with the Lions Clubs International Foundation, we implemented the social and emotional learning (SEL) program Lions Quest Skills for Growing (PreK-5) in grades 2 to 5 of the 69 treatment schools. The remaining 66 schools in the control group did not receive any treatment.

Design details

Randomization method
Randomization was done by a computer using the statistic program Stata. In order to gain a balanced treatment-control group setting, we stratified by subdistrict (our schools are located in 11 subdistricts differing in their schooling authorities), by distance of the school to its respective subdistrict capital (as a proxy for schooling quality) and village literacy rate. No re-randomization was done, but the first draw taken.
Randomization unit
The treatment was randomized on school level.
Clusters
Treatment was assigned to 69 out of a total of 135 schools (hence, 66 schools in control group). These schools are serving 150 villages that we randomly selected from 11 subdistricts belonging to four districts of Bangladesh.
Observations
The RCT is based on 3,000 households from the 150 villages that are served by the 135 elementary schools in our sample: sampled by selecting at least 20 children from grades 2 to 5 of each school in 2019 (larger samples from schools serving multiple sample villages, e.g., about 40 children per school if the school serves two villages). We have a child sample that covers more than 3,000 students attending grades 2 to 5 in 2019 plus, if they have siblings, information on one of them (if they have siblings, the additionally sampled sibling is chosen randomly). Further, in general, two teachers per school from treatment and control schools got selected: constitutes a sample of 277 teachers who have been interviewed in a short survey directly following the intervention at the beginning of 2020. In treatment schools, these are Lions-Quest (LQ, the intervention program) teachers. In control schools, two teachers got randomly selected. We plan to follow up with this sample of teachers. If feasible, we extend the sample to other teachers who have taught at the 135 elementary schools during the intervention period.
Arms
69 schools in treatment group, 66 schools in control group
Primary outcomes
Measures of teachers’ teaching style, their beliefs, attitudes, and behaviors with respect to teaching
Secondary outcomes
Measures of teachers’ personality in terms of the Big 5 taxonomy, locus of control, as well as time and social preferences (A.); if feasible, school exam results of children (B.); assessments of intervention effects on teachers’ teaching and reports of how schooling was handled during the Covid-19 pandemic by schools’ headmasters (C.)

Cited by

No citation data available for this registration yet.

Keywords