Evaluation of a vocational school initiative to increase graduation

How to cite

Groes, Fane; Larsen, Anna Folke; Madsen, Edith; Hernandez, Marcos Vera; Stahlschmidt, Mikkel. 2022. "Evaluation of a vocational school initiative to increase graduation." AEA RCT Registry. AEARCTR-0009606. https://doi.org/10.1257/rct.9606-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Fane Groes* , Anna Folke Larsen , Edith Madsen , Marcos Vera Hernandez , Mikkel Stahlschmidt

* Primary investigator

Registry ID
AEARCTR-0009606
DOI
https://doi.org/10.1257/rct.9606-1.1
First registered
Last updated
Status
On going
Trial dates
2021-10-07 – 2026-06-30
Countries
Private
JEL codes
I21, I24, I 31
Registry record
http://www.socialscienceregistry.org/trials/9606

Abstract

This study examines the effect of an intervention at a 20 week’s basic course at Danish vocational schools. The activities in the intervention are designed to give the students real-world experiences in the industry and a professional network. The intervention is delivered by existing personnel at the participating schools who participate in an on-boarding course prior to starting the intervention. The intervention is implemented in nine different educations at four vocational schools with multiple locations from spring 2022 to spring 2024. The intervention is rolled out using a clustered stepped wedge design. The starting time of the intervention is randomized at the education-location level within stratas. The primary outcome of the intervention is graduation from the basic vocational course. Secondary outcomes are continued education and mental well-being.

Intervention

The intervention is called OS I BRANCHEN (Us in the Industry) and represents a new approach to the 20 week’s basic course at Danish vocational schools, called Grundforløb 2. Henceforth referred to with the common abbreviation GF2.

Background
The GF2 takes place at the vocational school as preparation for the main course. The main course consists of several apprenticeship and school modules. The intention of GF2 is to teach the students a range of vocational skills prior to their first apprenticeship.

The ROCKWOOL Foundation Interventions Unit has interviewed many GF2 students during the development of the intervention. Many students have chosen a vocational education because they want to be active in a workplace and use their skills to create value and make a difference for others. Quite a few have had bad experience with prior schooling and are eager to get out in the industry. Some students have limited experience with the industry and therefore need exposure to the industry to discover the possibilities and the everyday life in a company through practical experiences. Most students experience that the individual responsibility for their education and finding an apprenticeship is a heavy burden. The intervention is developed as a response to these student experiences. The intervention does not grant student apprenticeships, but elements of the intervention is meant to provide students with tools and experiences, which they can utilize in the process of getting an apprenticeship agreement with a company. Mostly, students carry the responsibility for getting an apprenticeship agreement with a workplace. However, a recent tri-party agreement on the vocational school area has shifted some of the responsibility of getting apprenticeship agreement away from the students towards the schools. The schools must then assist the students not able to get an apprenticeship agreement themselves. The tri-party agreement has not been fully implemented yet at the point of writing and therefore the effect of the responsibility shift is unknown, and there is uncertainty about the practical implementation of the tri-party agreement.

The intervention is being implemented and trialed at three vocational schools in Denmark. Zealand Business College (ZBC), NEXT Uddannelse København (NEXT) and Center for Erhvervsrettede Uddannelser Lolland-Falster (CELF). The intervention is being rolled out to the following nine educations or areas of education: Carpentry, Electrician, Bricklaying, Data & Communication, Car Mechanic, Plumbing, Social and Health Care (Social and health care assistants and social and health care helpers), Foods (Baker & confectioner, gastronome, waiter, nutrition assistant), and Business (Retail, trade, event coordinator).

The schools were recruited for the trial based on three parameters: They had a range of the educations which the intervention was designed for, the enrollment into these educations were large enough to utilize the network dynamics of the intervention, both in-between students and between the students and the companies, and lastly that graduation rates on the GF2 left room for improvement.

The intervention
The intervention rests on two principles:
- “The first day in school is the first day in the industry”
- “We succeed as a team”
The activities in the intervention are designed to give the students real-life experiences and a professional network. In all activities the students will try out the industry in a professional collaboration in a real world context. The intervention is delivered by existing personnel at the participating schools. The personnel participate in an on-boarding course prior to starting the intervention.

Experimental design

Identification
The identification of the treatment effect of the intervention on the outcomes will come from the stratified random staggered roll out of the intervention. See below for further details on the randomization.

Estimation
The econometrics literature that exploits the staggered roll out of a policy has expanded very significantly in the last four years (see for instance Borusyak et al 2022, Chaisemartin and D’HaultfŒuille 2018 and 2022, Freyaldenhoven et al 2021, Gardner 2021, Goodman-Bacon 2021, Roth et al 2022, Sun and Abraham 2020, Roth et al 2022). However, most of the literature has focused on non-random staggered rollout, in which identification of the treatment effects relies on the common trends assumption.

Recently, both Athey and Imbens (2022) and Roth and Sant’Anna (2022) have focused on the random staggered rollouts. Athey and Imbens (2022) assumes that the treatment effect is homogeneous with respect to the adoption date (invariance to history). Both papers assume a non-clustered design, in which the adoption date of a unit i is randomized, and the outcome for such unit i is observed during the entire time period. However, in our set up, we observe the outcomes of repeated cross sections (the students of a particular education-location in each semester, who will be different from semester to semester). Some of the ideas proposed by Athey and Imbens (2022) seem relatively straightforward to adapt to the repeated cross-section case. However, in order to use the methods proposed by Roth and Sant’Anna (2022), we would need to average the data at education-location-semester level, but this would lead to loss of power unless the intracluster correlation was one. Because the literature on staggered roll outs is evolving very quickly, we prefer not to select an estimation method at the moment of writing the pre-analysis plan.

Some secondary outcome variables are measured using a student survey. We anticipate the response rate to be different depending on whether the intervention is being implemented or not at the time of the survey. To deal with possible selection bias, we will use a Heckman Selection model in which the randomized reminders to answer the survey will be used for the exclusion restriction.

Design details

Randomization method
The randomization, or the order in which the education-locations can begin implementing the intervention, was done in-person at the ROCKWOOL Foundations premises. The randomization was stratified. Education locations were grouped into stratas according to the type of education that they provide (i.e. carpentry) as well as logistical constraints. The stratification allows to minimize the chance that all locations of a given education, say all locations which teaches Carpentry, started the intervention at the same time. When the number of education-locations was too low for a given education, we grouped several different educations in the same strata. Each strata had pre-defined sequences which determined the roll out scheme over a five semester period. The pre-defined sequences took into account logistical constraints such as not starting too many education-locations at the beginning of the trial. The education-locations where then randomly picked and applied to a sequence, which determined when the education-location location was going to implement the intervention. The randomization was done by randomly picking downward-facing cards, which identified the individual education-locations. The order of the draw determined the roll out scheme for the stepped wedge trial. The directors of education at the partnering schools participated in the draft to increase local support for the roll-out scheme among the implementation teams. ROCKWOOL personnel also attended the draft. The randomization was recorded and saved for documentation.
Randomization unit
The unit of randomization is an education-location, e.g. Carpentry in the city of Slagelse. Within some areas of education, different educations are taught together in GF2: All food educations are taught together, the same are all business educations and also the two social and health care educations. To avoid spillovers, we grouped these educations together that take lessons together during GF2. One school can have several education-locations for each education. In some cases, the same vocational teachers work at multiple locations, and here we grouped the education-locations together into one cluster to avoid spillovers.
Clusters
At the time of writing the pre-analysis plan, we have recruited 3 schools which provide 33 education-locations (clusters). We are planning to recruit an additional school into the study which will provide additional education-locations.
Observations
The number students participating in the trial, will depend on future enrolment into vocational schools and specifically the education-locations participating in the trial. Based on intake from historical administrative data, we estimate around 7600 students, which wil increase when we recruit an additional school.
Arms
Because of the staggered rollout design, all clusters end up receiving the intervention at some point.
Primary outcomes
Short-term: graduation of the GF2 program at 8 months after enrollment
Secondary outcomes
The secondary outcomes originate from two data sources: students’ answers to survey questions and data from the administrative record. Our first secondary outcome is the students’ mental well-being, which we measure by survey questions. The second and third set of secondary outcomes are students’ achievements and input factors, measured form the administrative records.
Sponsors
Name: ROCKWOOL Foundation Location: Ny Kongensgade 6, 1472 Copenhagen K, Denmark Url: https://www.rockwoolfonden.dk/
Partners
Name: NEXT Uddannelse Partner_type: government Url: https://nextkbh.dk/ Name: Center for Erhvervsrettede uddannelser Lolland-Falster Partner_type: government Url: https://www.celf.dk/ Name: Zealand Business College Partner_type: government Url: https://www.zbc.dk/ Name: Rambøll Management Consulting Partner_type: private_company Url: https://dk.ramboll.com/management-consulting

Cited by

No citation data available for this registration yet.

Keywords