Unlocking the Value of Social and Emotional Learning Interventions: Maximizing Benefits and Measuring the Opportunity Cost

How to cite

Llanes, Salome Aguilar; Bueno, Bernardo García Bulle. 2023. "Unlocking the Value of Social and Emotional Learning Interventions: Maximizing Benefits and Measuring the Opportunity Cost." AEA RCT Registry. AEARCTR-0012370. https://doi.org/10.1257/rct.12370-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Salome Aguilar Llanes* , Bernardo García Bulle Bueno

* Primary investigator

Registry ID
AEARCTR-0012370
DOI
https://doi.org/10.1257/rct.12370-1.0
First registered
Last updated
Status
On going
Trial dates
2023-10-23 – 2024-12-31
Countries
Private
JEL codes
I2 Education and Research Institutions
Registry record
http://www.socialscienceregistry.org/trials/12370

Abstract

In a previous RCT we documented gains of 0.08 SD in Math learning due to a Social and Emotional Learning intervention in an online tutoring program. We propose a follow-up RCT in the same setting with the aim to to quantify the opportunity cost of reallocating instructional time from other subjects, such as Math, to SEL. And also to uncover the driving mechanisms behind the observed learning effects of SEL interventions. The study hypothesizes three distinct channels influencing these effects: tutor-student relationships, tutors' emotional intelligence, and students' emotional intelligence.

Intervention

We propose to implement the RCT in Mexico through the NGO Jóvenes Ayudando a Niñas y Niños A.C. JANN is an organization that manages a free tutoring program that connects about K-12 students, with grades ranging from 1st to 9th, with university students that provide Math tutoring online. Tutors and students meet twice weekly for one hour from 12 to 24 weeks. We are concerned with measuring the impact of a social and emotional learning intervention on math learning. The intervention consists of 8 modules of social and emotional learning material based on RULER (Hoffmann, Brackett, Bailey Willner,2020).

In a previous RCT we documented learning gains in math associated with receiving the additional SEL curriculum.
This RCT follows up and aims to answer two policy relevant research questions.

What is the opportunity cost of devoting instruction time to SEL rather than other subjects, for example Math. The intervention comes at the cost of sacrificing instructional time that would have been devoted to math in the absence of the intervention. Do the benefits brought by SEL outweigh the costs?
What channel is driving the effects in learning derived from the SEL intervention? This can help understand the optimal delivery of SEL content to obtain the maximum benefits derived from SEL.We hypothesize that there may be three channels driving the effects observed, each of which has different policy implications for better implementation.

H1 SEL activities yield higher learning through the change they cause in the tutor-student relationship. Creating a climate of trust in the class, improving the relationship between the tutor and the students, and potentially increasing rapport. This is important because SEL would benefit most if taught by teachers who also teach other classes instead of specialized SEL teachers.

H2 SEL activities yield higher learning through the change they cause on the tutors' skills. The tutors who learned SEL improved their emotional intelligence. By improving these skills, they became more aware of their students’ needs, and thus, they became better teachers. In this case, SEL interventions could be delivered directly to teachers through professional development and would not need to reduce instruction time devoted to other subjects.
H3 SEL activities yield higher learning through the change they cause in the students. The students under the SEL condition, improved their emotional intelligence, and made the class easier to teach. This allowed the tutor to enjoy the class more and thus invest more effort. This would imply that teaching kids socioemotional skills can benefit them in Math learning regardless of who teaches SEL and that there are possibly spillover effects in other disciplines. In this case, the opportunity cost is more likely to be outweighed by the benefits observed in other classes.

Experimental design

A The first treatment will be assigning students to receive N SEL activities designed by pedagogist Cimenna Chao besides math.

B In this treatment tutors will be asked to teach Math and N History lessons to one group. Additionally the tutors of these groups will participate in N sessions designed to improve their emotional intelligence. The sessions will only be held in groups of tutors, and thus their students will never be exposed to the content, thus we can isolate the effect of the SEL activities on the tutors.

C In this treatment, each group of students will be assigned two tutors. One of the tutors will teach them Math and the N history activities. The other tutor will solely teach them the N SEL activities aforementioned. The reason for this treatment to exist is to isolate the effect SEL has on students.
D In this treatment, tutors will be asked to provide the students with Math and the N history lessons. This treatment was tested as-is on the past RCT and compared to A, where it yielded lower learning. We will compare A,B and C to D to test H1,H2 and H3.

E In this treatment, tutors will be asked to spend all instruction time teaching Math to their students with no additional activities, comparing treatments arms A,B,C will to E will allow us to benchmark the learning gains of SEL against a world where time devoted to SEL is used for instruction. Rather than the comparison with D where instruction time devoted to SEL does not substitute instruction time devoted to math. The comparison between D and E will allow us to identify the opportunity cost of SEL activities.

Due to time constraints some groups will receive only N=5 and most groups will receive N=8 either history or SEL classes. In the main specification we will control for this and we will do a subgroup analysis differentiating the groups that received partial treatment N=5 from the groups that received the full treatment.

Design details

Randomization method
We randomize using a computer.
Randomization unit
We are randomizing students to treatment arms once they are assigned to a treatment arm, they are randomly grouped in groups of 5 students of the same grade. On the other side, we randomize tutors to treatments and then, within each treatment, randomly assign groups to tutors. We will cluster standard errors at the tutor level for the main specification. Each tutor teaches 1 to 4 groups of 5 students.
Clusters
at least 1500 tutors depending on the ability of our partner organization to recruit participants.
Observations
15,000 students.
Arms
We will have 300 tutors per treatment arm.
Primary outcomes
Our main outcome is the academic learning of students measured by the Math and Spanish score on an evaluation designed by us, which is an adaptive exam. Students will answer a version of this evaluation at baseline and at endline. The second main outcome is the student-tutor attachment measured using the student version of the teacher–student relationship inventory (S-TSRI) (Ang, Ong, & Li 2020).
Partners
Name: Jóvenes Ayudando a Niñas y Niños A.C. Partner_type: ngo Url: https://jann.mx

Cited by

No citation data available for this registration yet.

Keywords