Supporting refugee integration using dialogue based school development programs: Evidence from a randomized targeted intervention in Swedish Municipalities

How to cite

Sjogren, Anna; Mühlrad, Hanna; Sibbmark, Kristina; Ahléèn, Malin Tallås. 2022. "Supporting refugee integration using dialogue based school development programs: Evidence from a randomized targeted intervention in Swedish Municipalities." AEA RCT Registry. AEARCTR-0010109. https://doi.org/10.1257/rct.10109-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Anna Sjogren* , Hanna Mühlrad , Kristina Sibbmark , Malin Tallås Ahléèn

* Primary investigator

Registry ID
AEARCTR-0010109
DOI
https://doi.org/10.1257/rct.10109-1.1
First registered
Last updated
Status
Completed
Trial dates
2016-03-01 – 2022-06-30
Countries
Sweden ()
JEL codes
I21 I22 I24 I28 H52 J45, F22
Secondary IDs
FORTE GRANT Diarienummer: 2021-00736
Registry record
http://www.socialscienceregistry.org/trials/10109

Abstract

We evaluate the effects of a dialogue based school support program targeted at Swedish municipalities which experienced a large influx of refugees during the 2015 refugee crisis. The support program was initiated by the Swedish government and administered by the Swedish National Education Agency (SNEA) in response to the crisis and the urgent need to support the accommodation of refugee children in schools. The program consisted of an initial 6-month stage during which a local team, in dialogue with consultants from the National Education Agency, identified local needs and agreed on a suitable package of support measures. During the second stage, with a duration of 18 months, the customized support package was implemented with financial and managerial support of the SNEA. Support packages typically involved teacher training in knowledge and language enhancing teaching strategies to support language development and learning of migrant students (Scaffolding language, Språk- och kunskapsutvecklande arbetssätt, SKUA), improved availability of tutoring in student mother tongue, managerial and administrative support in organizing refugee student reception and integration, including appointment of a local refugee reception coordinator and training of other personnel groups involved in the schools receiving refugee children. The support program was rolled out in seven waves during 2016-2019. Municipalities were first ranked according to need of support. In each wave the five most needy municipalities were guaranteed participation before randomization took place among subsequent pairs, i.e. 6-7, 8-9... each round, 5-12 pairs per round resulting in a total of N=63 in the treatment arm. Control municipalities, reentered the randomization procedure after 12 months. The support program thus affected some 112000 students, 10000 teachers, and 770 schools at the compulsory school level, excluding the guarantee municipalities.

The primary outcomes of the study are incumbent and asylum seeking students' test scores, compulsory school grades and qualification for upper secondary school. Other important outcomes are student and teacher mobility, school segregation, school resources, and effects on school markets, i.e. voucher school entry.

We will test a number of hypotheses:
1) Receiving targeted support improved schools ability to accommodate refugee students
2) Schooling outcomes of incumbent students were less negatively affected by refugee influx - or even positively affected - by municipal participation in the targeted support program
3) The effects of program participation are possibly larger for incumbent students with foreign background i) because of elements of the program also improves educational quality for this group ii) because they are more vulnerable to resource constraints.
4) It is possible that initial, short run effects effects are negative if program participation initially crowds out teaching capacity
5) Positive program effects may be mediated by enhanced reception and teaching capacity, increased school resources, more adequate special aid to children, reduced white flight and student mobility, less teacher turn over.

Intervention

The support program was initiated by the Swedish government and administered by the Swedish National Education Agency (SNEA) in response to the large influx of refugees in Sweden in 2015 and the urgent need to support accommodation of refugee children in schools. The SNEA design of the roll out was assisted by IFAU with the aim of facilitating effect evaluation.

Municipalities were ranked by the SNEA according to need of support in receiving and integrating refugees in their schools. Municipal rank, m_i, was determined by a regression based index increasing in refugee influx and declining in previous experience. In 7 rounds starting in the spring of 2016, pairwise randomization was used to select treatment and control municipalities. In each round, the 5 top-ranked, most needy municipalities, m_1-m_5, were guaranteed support, before randomization into treatment and control between m_6 and m_7, m_8 and m_9 etc took place. Control municipalities needed to wait at least one year before entering into the randomization process again. The initial plan was to select 15 pairs in each round. Given capacity constraints on the part of SNEA, a total of 63 pairs were selected in the seven rounds.

Treated municipalities were offered a customized support program. After an initial agreement of participation was signed, the content of the intervention was designed based on an analysis of local needs conducted by a local team assisted by consultants from the Swedish National Education Agency in the months following selection. A second agreement between the NEA and the municipality containing the locally adapted support program package and accompanying financial and other support was signed about 6 moths after agreement 1. This package was then implemented by the municipality with SNEA's assistance during 12-18 months.

Support packages typically involved 1) teacher training in knowledge and language enhancing teaching strategies to support language development and learning of migrant students (Scaffolding language, Språk- och kunskapsutvecklande arbetssätt, SKUA), 2) improved availability of tutoring in student mother tongue, 3) managerial and administrative support in organizing refugee student reception and integration, including appointment of a local refugee coordinator and 4) training of other personnel groups involved in the schools receiving refugee children. Many municipal packages also included strengthening of student health care services, improved feedback and quality enhancing routines, teacher training in intercultural teaching practices, improved quality of instruction in Swedish as a second language, mother tongue instruction availability and quality, parental introduction and support programs.

Experimental design

Identification
Identification of effects of the program relies on the randomized staggered roll out of the program, where the idea is to compare the outcomes of students (teachers, schools) in treated and control municipalities.

Estimation
It cannot be ruled out that treatment effects vary by wave, as the analysis and consulting capacity of the SNAE or quality of package components develop, nor that there are heterogenous treatment effects with respect to the municipality ranking in terms of need. Controls for wave and randomization pair, may be thus be necessary.

Furthermore, although randomization between adjacently ranked municipalities will likely achieve balance in predetermined characteristics between treatment and controls, this has to be assured in the data. It may be necessary to include controls for predetermined characteristics to assure balance and/or use a two-way fixed effects (TWFE) model.

Recent econometric developments in the analysis of TWFE staggered roll outs of policy has typically focused on non-random staggered roll outs (see e.g. Chaisemartin and D’HaultfŒuille 2020, Roth et al 2021). Given that the developments in the area are rapid, and methods applicable to random staggered roll outs will likely appear, we intend to follow the literature and to adopt our estimation methods to the state of the art, rather than state here which exact method is most adequate.

Randomization
Municipalities were ranked by the NEA according to need of support in receiving and integrating refugees in their schools. This ranking was was based on an index increasing in refugee influx and declining in previous experience. In 7 rounds starting in the spring of 2016, pairwise randomization was used to select treatment and control municipalities. In each round, the 5 most needy municipalities were guaranteed support, before randomization into treatment and control between 6 and 7, 8 and 9 etc took place. Control municipalities needed to wait at least one year before entering into the randomization process again. The initial plan was to select 15 pairs in each round. Given capacity constraints on the part of SNEA, between X and X were selected each wave, and a total of 63 pairs were selected in the seven rounds.

Treated municipalities received a customized support program. Through dialogue between the municipality and the National Education Agency in the months following selection, an analysis of local needs resulted in a locally adapted support program which was implemented by the municipality with NEAs assistance during 12-18 months.

References
Chaisemartin & D’HaultfŒuille 2020 "Two-Way Fixed Effects Estimators with
Heterogeneous Treatment Effects", American Economic Review 2020, 110(9): 2964–2996.
Jonathan Roth & Pedro H. C. Sant'Anna, 2021. "Efficient Estimation for Staggered Rollout Designs," Papers 2102.01291, arXiv.org, revised Aug 2022.

Design details

Randomization method
The pairwise randomization was done using a computer
Randomization unit
Randomization was done between municipal pairs.
Clusters
Randomization was done at the municipal level, but implemented in schools, affecting students, teachers, schools and families. All in all 63 municipalities were treated and 63 (58 unique) were control municipalities.
Observations
121 (63 treated, 63 controls of which 58 were unique since some controls entered the randomization again and became controls a second time) municipalities with some 190000 students in total, and 63000 in test taking years 3,6,9, about 16700 teachers and 1250 schools.
Arms
63 treated municipalities and 58 control municipalities. In the treated (control) municipalities there were in the year prior to the intervention 10000 (6700) teachers, 770 (480) schools, some 112000 (78000) students in total and 37500 (26000) students in test taking grades 3,6, and 9 at the compulsory school level.
Primary outcomes
Main student outcomes of interest: 1) Average test score across the three core subjects Swedish, Mathematics, and English taken in grades 3 (Swedish and Mathematics) 6 and 9 (Swedish, Mathematics, and English) and high school. These subject tests / grades will also be analysed separately. 2) qualifying for and completing high school. Measurement of test scores will be based on normalized scores within test year in the full population of test taking students, excluding asylum seekers and newly arrived immigrants. Analysis of heterogeneity and subgroups We will analyse effects across the distribution test scores The main analysis will focus on the effects on the incumbent student population, but we will if possible also analyse effect on the newly arrived refugee students. This split is motivated because it is not clear how well we will be able to follow, and thus determine the treatment status, of the newly arrived refugees since individual identifiers are unique to each register and background information is scarse. An attempt will be made to follow individual asylum seeking students, but its is unclear as yet if we will be able to match across registers and years for a sufficiently large fraction of the asylum seeking student population. We will conduct analyses by student gender and family background, i.e. parental education groups as defined by educational categories and percentiles in the distribution, by parental income groups, and by groups defined by an index of parental background based on predicted test scores. We will also consider outcomes by students’ migration background and how long they have been in the country. These analyses will be conducted using 1) interaction models and 2) split-sample regression models.
Secondary outcomes
In order to interpret effects on main outcomes: • In order to interpret effects on test scores we will analyse effects on test taking – to assess whether the fraction of test takers (or exempt children) is affected • In order to interpret effects on qualifying for and completing high school, we will assess any effects on grade inflation by studying effects on GPA in relation to test scores. Understanding possible mechanisms In order to understand and interpret effects on educational outcomes, we will study various channels through which families, teachers and schools may have adjusted differently to the influx of refugees as a result of the support program: This list is long. More detailed results on teachers, families and school markets may instead be presented in separate papers. • Receipt of special aid (home language instruction, extra instruction in home language, special instruction, adapted curriculum) • High School program choices • School mobility, i.e. students changing school for i) independent schools (profit/non-profit) ii) other municipal school Health and other social outcomes: • Health measures based on patient registers and medical drug prescriptions. Focus on psychiatric and behavioral conditions, indicators of risky behaviors and violence, substance abuse. Note that we do not have access to this data as yet. • Criminal behavior, Note that we do not have this data as yet. • Labor market • Earnings and employment School resources and organization: • School size/classroom size • Student - teacher ratios • Other personnel categories – assistants, librarians, nurses etc • Teacher qualifications, i.e. gender, educational qualification and experience within the teaching profession, as well as based on measures of predicted teacher salaries • Presence of head teachers • Fraction of children with special aid and provisions of home language instruction • Teacher turn-over and mobility - numbers and characteristics of teachers moving to and from school. Teacher flight to independent schools or other municipal schools • Headmaster turn over, mobility and qualifications School market: • Exit and entry of new schools, municipal and independent schools of different organizational form (i.e. for profit or non-profit) • Reorganiation of grade configurations Family environment • Parental employment and earnings • Fertility • Residential mobility

Cited by

No citation data available for this registration yet.

Keywords