How do pre-set deadlines and reminders affect graduation rates and performance in vocational colleges?

How to cite

Stryjan, Miri; Gershoni, Naomi. 2021. "How do pre-set deadlines and reminders affect graduation rates and performance in vocational colleges?." AEA RCT Registry. AEARCTR-0007146. https://doi.org/10.1257/rct.7146-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Miri Stryjan* , Naomi Gershoni

* Primary investigator

Registry ID
AEARCTR-0007146
DOI
https://doi.org/10.1257/rct.7146-1.0
First registered
Last updated
Status
On going
Trial dates
2018-04-01 – 2021-12-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/7146

Abstract

We evaluate an intervention taking place within vocational colleges in Israel, which establishes a framework with monitored deadlines and reminders for delivery of the proposal and submission of the final project required to graduate with a diploma. The goal is to study whether this way of adding structure and reducing uncertainty about the submission date can improve student performance, with particular focus on the completion of the requirements for a diploma. It will combine a randomized field experiment with collection of administrative and survey data on students in the vocational colleges. The research falls under the general headings of labor economics and economics of education, and also uses insights from behavioral economics.

The fundamental reason for low educational outcomes among students who are already enrolled in educational programs is the exertion of too little effort (see e.g. Oreopoulus et al., 2019; Clark et al., 2020). Several different yet interrelated reasons were offered for this shortcoming. First, low effort may be a results of flawed time perception leading to an underestimation of the time required to complete a task (Kahneman and Tversky, 1979). Second, procrastination which is a type of self-control problem may lead students to end-up exerting less effort than initially intended, even if their time estimations were accurate. Procrastination is often the product of present bias, which is the human tendency to focus on the here-and-now and value immediate rewards over long-term benefits (O'Donoghue and Rabin, 1999). Bisin and Hyndman (2020) document that present bias is prevalent among college students. Third, time management was pointed out as challenging for students, which again relates to problems with the perception of time (Francis‐Smythe and Robertson,1999). Previous studies show that setting clear, well-organized deadlines can increase timely submission of projects (Ariely and Wertenbroch, 2002). When looking at long-term goals, it was shown that setting such goals and then listing the intermediate steps required to achieve them can improve academic performance (Morisano et al., 2010). Inspired by these findings, in this experiment we implemented deadlines for the “final project” proposal and submission, accompanied by text message reminders.

Theoretically, diplomas and certificates are expected to decrease labor market frictions and alleviate adverse selection problems. Empirically, however, the causal effect of diplomas on labor market outcomes is difficult to identify, due to endogeneity problems (Gunderson and Oreopoulus, 2020). Provided that our intervention affects graduation rates and timing, a secondary aim of this project is to address the bigger question of the labor market returns to vocational education. Here, we plan to combine our data on students with administrative labor market data and utilize an Instrumental Variable approach to causally estimate the returns to vocational education.

Intervention

Context:
In Israel, high school graduates can earn a practical engineering diploma, which is a professional degree awarded by technical colleges. These studies are a combination of practical vocational training and broader theoretical studies. The Israeli National Institute for Technical Training (NITT), which operates within the Ministry of Labor, Social Affairs and Social Services, sets the criteria and regulates these studies, covering over 70 colleges and more than 30 fields of study. To graduate with a diploma, students are required to complete around 2,000 academic hours; to undertake standardized national certification exams; and to submit a final project which demonstrates the acquired knowledge and requires the application of practical skills. As in academic studies in Israel, the government regulates tuition and subsidizes the cost of training.
Israeli technical colleges, like many post-secondary educational settings, struggle with low graduation rates (or high drop-out rates). In 2016, it was estimated that only 49% of students enrolled in Israel’s technical colleges completed their course of study and only 63% of those who completed all courses were eventually eligible to receive a practical engineer’s diploma. In comparison, the drop-out rate among academic college students in Israel is only 8,7% (Feniger et al., 2016).
The timely submission and defense of the final project was pointed out as the main reason for failing to graduate (Ministry of Economy and Industry, 2017) and is therefore targeted in our intervention, which is focused on increasing the percentage of students who submitted and successfully defended their final project, thereby increasing the percentage of students who graduate with diplomas.

Intervention:
We partner with the NITT (National Institute for Technical Training) and use a randomized control methodology with randomization within matched pairs of college-by-major units, and ask how pre-announced deadlines and reminders affect completion and graduation rates, graduation timing, and for what type of students the intervention is more effective. Study participants included 4,900 students from 89 departments (major by college) in 33 technical colleges in Israel. Departments were randomly assigned to either the control group (43 departments or the experimental (treatment) group (46 departments). The experiment focused on four study majors: architecture, electrical engineering, civil engineering, and software programming.

Procedure and data collection:
The experimental intervention consists in recommending a deadline for making a project proposal and setting a specific deadline for the final submission (well in advance of the date) combined with sending text message (SMS) to students in treated departments, including motivational messages and reminders about upcoming deadlines. Control group departments use “business as usual” where no pre-defined deadline is set for proposal or project submission, but instead, defenses are scheduled on a rolling bases when at least 10 students are ready to submit and defend their project.
A short follow up survey will be administrated to students via SMS and email. The survey includes questions on the students' educational and labor market history, their attitudes and motivation for studying, and their labor market expectations. Students can choose to take the survey in either Hebrew or Arabic.

Experimental design

Starting in April 2018, the research team planned the intervention together with NITT. In January 2019, the deadline setup was announced and explained to the head of each department in the treatment group. The recommended deadline for proposal submission was April 2019. This deadline was communicated to department heads (who were asked to instruct the students accordingly), and backed-up by personal SMS reminders to the treatment group students. In August 2019 the treated units (departments) were assigned specific defense dates. SMS nudges and reminders were regularly sent to students in the treated units between April 2019 and the pre-scheduled defense date.

Treatment was assigned using a pair-wise randomization procedure (Bruhn and McKenzie, 2009) in which departments from the same field of study were matched based on previous rates of graduation with diploma, the ethnic/religious target population of the college and the number of students.

Design details

Randomization method
The randomization was conducted by the researchers on a computer.
Randomization unit
The treatment unit is the major-by-college combination, referred to here as “department”.
Clusters
89 major-by-college units, also referred to as departments
Observations
4,900 students from 89 clusters (major*college units) in 33 technical colleges.
Arms
43 departments; n = 2,215 students. Treatment group: 46 departments; n = 2,685 students.
Primary outcomes
Likelihood to complete final project, likelihood to graduate with a diploma, graduation timing.
Secondary outcomes
final project grade, exam grades, labor market outcomes.

Cited by

No citation data available for this registration yet.

Keywords