Labour market information and study choice: One-size-fits-all?

How to cite

Rodriguez, Sandra Perez; Fouarge, Didier; Künn-Nelen, Annemarie. 2023. "Labour market information and study choice: One-size-fits-all?." AEA RCT Registry. AEARCTR-0011785. https://doi.org/10.1257/rct.11785-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Sandra Perez Rodriguez* , Didier Fouarge , Annemarie Künn-Nelen

* Primary investigator

Registry ID
AEARCTR-0011785
DOI
https://doi.org/10.1257/rct.11785-1.0
First registered
Last updated
Status
In development
Trial dates
2023-03-01 – 2025-06-30
Countries
Private
JEL codes
J24, I23, I26
Registry record
http://www.socialscienceregistry.org/trials/11785

Abstract

Career guidance is essential for an informed study choice. Information on labour demand (e.g., wages) is particularly relevant in order to help youngsters make study choice that fit current and future demand. However, experiments that make such labour market information available to youngsters do not always have the expected effect, suggesting that there is no one-size-fits-all solution. Responsiveness to such information is likely to depend on personal characteristics (e.g., gender and migration background), time and risk preferences and skill level (e.g., numeracy, literacy, graph literacy or economic literacy), as well as on the way the information is presented. In collaboration with one of the largest career guidance platforms in the Netherlands, we conduct a field experiment for 15-year-old students to investigate what format of presenting wage information has the strongest effect on study choice. Our experiment aims at expanding the literature in two ways. First, we analyse what format of labour market information has the strongest effect on study choice. We distinguish between a pictograph (bar chart vs. icon arrays), a (neutral or colourful) ladder or a table in providing wage information for occupations. We hypothesize that the effects depend on gender, socio-economic status, time and risk preferences and migration background. Second, we analyse to what extent the potential heterogeneous effects are driven by differences in numeracy, literacy, graph literacy and economic literacy skills. This enables us to distinguish whether different individuals make intrinsically different choices, or whether different choices are a result of different skill levels. We hypothesize that at least part of the effect of making different choices is the result of different skill levels. We would like to investigate whether differences in skills can fully explain the expected differences in responsiveness towards labour market information. This insight would not only contribute to scientific literature, but would be also highly valuable for policy makers as it provides directions on how to reduce inequality of opportunities by improving the way career guidance is provided.

Intervention

In the Netherlands, secondary schools have the obligation to provide career guidance to their pupils. Often, they will do this by contracting a private company that specialized in career guidance. In collaboration with a career counseling platform called Qompas, one of the largest in the Netherlands, we propose a field experiment to answer the following research questions: What format of labour market information has the strongest effect on study choice? Does displaying labour market information using a pictograph (bar chart or icon arrays), a (neutral or colourful) risk ladder or a table have a different effect on study choice depending on the students’ numeracy, literacy, graph literacy and economic skills? Is there heterogeneity by gender, socio-economic status, time and risk preferences and migration background?

Our target population are 3rd year students at the general and academic high school tracks in the Netherlands, who use Qompas’ resources to help them choose their profile in secondary education as part of their school curriculum. We are building on the experiment conducted by de Koning (2022) in cooperation with Qompas; who showed that information has a positive effect on belief’s accuracy in the short-term, and occupational preferences. Nonetheless, there are several differences between that study and ours, as we would like to take into consideration students’ individual characteristics and to analyse their response to different formats of information.

Students take several tests that are already included in the platform regarding their competences, interests and high school profile preferences. Based on pupils’ answers, Qompas presents the occupation sectors that fit best each student, including a list of example occupations. We will ask the pupils to make a shortlist of 10 favourite occupations and rank them in terms of their preferences (ranking A). The average gross hourly wage corresponding with the shortlisted occupations will be displayed for the students in the treatment groups.
Each treatment group would see the corresponding information in a different format: table, (colourful or neutral) risk ladder or bar chart or icon arrays. Students that belong to the control group will not have access to wage information. Right after receiving the information, treatment group students shall rank the shortlisted occupations again (ranking B), which helps test the short run effect of the information intervention on preferences. We would compare the average wage corresponding to the preferred occupations among the pupils that received information in different formats and those that did not, given gender, socioeconomic background, migration background, numeracy, literacy, economic and graph literacy, time and risk preferences. On two later points in time during the experiment, after students have gone through other assignments, we ask all students again to rank a (reduced) list of occupations based on their preferences (rankings C and D) in order to study the mid run effect of the information treatment.

Reference
de Koning, B. K. (2022). Empirical studies on information, beliefs, and choices in education and work [Doctoral Thesis, Maastricht University]. Maastricht.

Experimental design

Students in the third year of the academic and the general track of high school in the Netherlands must choose a profile for the remaining of their secondary education. A large number of schools hire the services of career counselling companies, such as Qompas. Therefore, as part of their career guidance activities in their school curriculum, students already answer multiple tests regarding skills and preferences within the Qompas platform. These tests are presented in the following order: 1) “Profile choice test”, 2) “Competency test” and 3) “Interest questionnaire”. At a later stage, pupils are asked to fill in a Provisional profile choice and finally, their Definite profile choice. Within the platform, students receive information of what profiles and occupations would be a good match for them based on their responses to the test. The aim of our experiment would be to estimate the effect of providing labour market information on the gross hourly wage for these shortlisted occupations in different formats on students’ preferences for occupations and their study profile choice.

Prior to the beginning of the experiment, certain preparations have been done. First of all, having a direct match between the occupations that students will select and the labour market information is essential. Therefore, we substituted the occupational groups that are listed under the 11 categories of occupational sectors in the “Interest test” in Qompas with the ROA-BRC 2014 list of occupations, which is the adaptation to the Dutch context of the International Standard Classification of Occupations (ISCO). A graph literacy test has been incorporated into the platform, based on Okan et al. (2019). Furthermore, we have also included a shorter adaptation of the economic competences test from Oberrauch et al. (2023). Only schools whose headmaster has given consent will participate in the experiment. Moreover, students have the option to opt out of the experiment if they so wish.

Our intervention would take place after the “Interest questionnaire”. When pupils fill in the so-called “Interest questionnaire”, Qompas provides an overview on how the following 11 occupational sectors match them in a 5-point scale ranging from Very much (represented by ++) to Not at all (displayed as - -). A list of the occupations that are comprised within each sector is shown when clicking in the arrow on the right side of the category. All students will be asked to first select their preferred 10 occupations out of the top 3 of broad categories of occupations (beroepensectoren). Then we give them the option to choose additional existing occupations and exchange them for any of the already selected ones, so they keep a total of 10. Once they have their shortlist of 10 occupations, we would ask them to rank them (Ranking A). We have decided to have a shortlist of 10 occupations instead of 5 to reduce the risk of having too little variation between occupations (e.g. if a student chooses 5 occupations that are very similar in terms of wage, the intervention would have no effect).

Immediately after, students allocated to each treatment group will receive the average hourly wage corresponding to their shortlisted occupations in the format of table (T1), neutral risk ladder (T2), colourful risk ladder (T3), bar chart (T4) or icon arrays (T5), respectively. The average hourly wage will refer to the most up to date information available (currently 2021). Students in the control group schools will not receive information. Students in the treatment groups will be asked to rank these occupations according to their preferences immediately after seeing the information (Ranking B), in order to measure the short term effect of the intervention. Furthermore, all students (control and treatment groups) will be asked to rank their preferred occupations again when they fill in their provisional (Ranking C) and definite profile choice (Ranking D), which are completed, on average, 15 and 31 days after the interest questionnaire, respectively.

References:
Oberrauch, L., Kaiser, T., & Seeber, G. (2023). Measuring economic competence of youth with a short scale. Journal of Economic Psychology, 97, 102633.
Okan, Y., Janssen, E., Galesic, M., & Waters, E. A. (2019). Using the Short Graph Literacy Scale to Predict Precursors of Health Behavior Change. Medical Decision Making, 39(3), 183-195. https://doi.org/10.1177/0272989x19829728

Design details

Randomization method
We do a two-step randomisation: first we randomise at the school level whether schools get access to wage information or not. Then, within the schools that receive information, we randomise at the classroom level in which format the information is displayed (treatments 1 - 5). Randomisation method for the school level randomisation: Originally we received a dataset with 255 schools that were already in the system (i.e., schools that use the Qompas methods). We use the cvcrand command from STATA, which performs a covariate constrained randomisation of clusters in a two-arm parallel cluster randomised trial: cvcrand aantalleerlingen20212022 aantal_leerlingen pc4 cm em ng nt qualitylife qompas_intensity size sizeqompas qualitylifecat qompas_int_cat classrooms, ntotal_cluster(255) ntrt_cluster(192) categorical (size sizeqompas qualitylifecat qompas_int_cat classrooms) stratify(size sizeqompas qualitylifecat qompas_int_cat classrooms) seed(12345) * varlist are the baseline cluster-level variables to constrain on: - aantalleerlingen20212022: registered students per school in the platform - aantal_leerlingen: total number of students in the school - pc4: postcode (4 digit version) of the school - cm: number of students in the culture and society profile - em: number of students in the economics and society profile - ng: number of students in the nature and health profile - nt: number of students in the nature and technology profile - qualitylife: quality of life indicator at the 4 digit postcode (https://data.overheid.nl/en/dataset/leefbaarometer-meting-20201) - qompas_intensity: qompas intensity usage, determined by the share of tests completed at the school level - size: categorical variable that divides schools in quartiles based on the number of registered students - sizeqompas: categorical variable that divides schools in quartiles based on the number of students registered in the platform - qualitylifecat: categorical variable that divides schools in quartiles based on the quality of life indicator - qompas_int_cat: categorical variable that divides schools in quartiles based on the intensity of usage of the platform - classrooms: categorical variable that divides schools in quartiles based on the number of classrooms per school * ntotal_cluster are the total amount of clusters, in other words, 255 schools * ntrt_cluster refers to the amount of clusters that should be allocated to the treatment group, in this case, 192 schools (roughly 75%) * categorical specifies which variables are categorical (size, sizeqompas, qualitylifecat, qompas_int_cat classrooms) * stratify: we specify which categorical variables on which to stratify: size, sizeqompas, qualitylifecat, qompas_int_cat, classrooms. Each of them has 5 categories: 4 quartiles and missing values. They represent the number of students the school has, the number of students in Qompas platform, the quality of life indicator and the share of students that complete all tests in the platform * seed: we set the randomisation seed to 12345, to make sure every time that the randomisation is the same We check the summary statistics for the variables taken into consideration for the randomisation, by treatment or not. We also conducted a variance ratio test (sdtest) to check whether the variances of the two subgroups are equal or not, in order to take it into account in the t test below bys _allocation: summarize qompas_intensity, detail sdtest qompas_intensity, by(_allocation) // we fail to reject the null hypothesis that the variances are equal histogram qompas_intensity, by (_allocation) bys _allocation: summarize qualitylife, detail sdtest qualitylife, by(_allocation) // we fail to reject the null hypothesis that the variances are equal histogram qualitylife, by (_allocation) bys _allocation: summarize aantal_leerlingen, detail sdtest aantal_leerlingen, by(_allocation) // we fail to reject the null hypothesis that the variances are equal histogram aantal_leerlingen, by (_allocation) bys _allocation: summarize aantalklassen, detail sdtest aantalklassen, by(_allocation) // we fail to reject the null hypothesis that the variances are equal histogram aantalklassen, by (_allocation) bys _allocation: summarize aantalleerlingen20212022, detail sdtest aantalleerlingen20212022, by(_allocation) // we reject the null hypothesis that the variances are equal: adjust for this in the t test histogram aantalleerlingen20212022, by(_allocation) sdtest em, by(_allocation) // we reject the null hypothesis that the variances are equal: adjust for this in the t test sdtest cm, by(_allocation) // we reject the null hypothesis that the variances are equal: adjust for this in the t test sdtest nt, by(_allocation) // we fail to reject the null hypothesis that the variances are equal sdtest ng, by(_allocation) // we fail to reject the null hypothesis that the variances are equal bys _allocation: egen school_classrooms = total(aantalklassen) tab school_classrooms // check the total number of classrooms by treatment group: 367 classrooms in control group, 1102 in treatment group bys _allocation: summarize cm, detail // bys _allocation: summarize em, detail // bys _allocation: summarize ng, detail // bys _allocation: summarize nt, detail // We then run some balancing tests: tabstat qualitylife, by(_allocation) statistics(mean sd p25 p50 p75) ttest qualitylife, by (_allocation) // difference is not significant ttest qompas_intensity, by(_allocation) // difference is not significant ttest aantal_leerlingen, by(_allocation) // difference is not significant ttest aantalleerlingen20212022, by(_allocation) unequal // difference is not significant ttest aantalklassen, by (_allocation) // difference is not significant ttest em, by(_allocation) unequal // difference is not significant ttest cm, by(_allocation) unequal // difference is not significant ttest nt, by(_allocation) // difference is not significant ttest ng, by(_allocation) // difference is not significant Later in time, we received a list of new schools that have just hired Qompas' services for the first time and for which there is no historical data. Therefore, these schools get allocated using the command randtreat from STATA: randtreat, generate(_allocation) setseed(12345) unequal(1/4 3/4) misfits(global) Randomisation at the classroom level regarding the format in which information is displayed: It is done within Qompas' platform using the roll of a dice.
Randomization unit
We pursue a two-stage randomization. First, 25% of the schools will be randomly allocated into a control group that will not receive information, and the remaining 75% of the schools will receive information. Then, on the second stage, classrooms within the treatment schools are randomly assigned into the following five treatment groups: table, neutral risk ladder, colourful risk ladder, bar chart and icon arrays.
Clusters
Approximately 300 schools, with an average of 5.8 classes per school. Therefore, we expect approximately 1740 classes.
Observations
Between 25,000 and 30,000 pupils
Arms
We pursue a two-stage randomization. First, 25% of the schools will be randomly allocated into a control group that will not receive information, and the remaining 75% of the schools will receive information. Then, on the second stage, classrooms within the treatment schools are randomly assigned into the following five treatment groups: - Table: 1/3 of classrooms within an _allocation=1 school - Risk ladder no colour: 1/6 of classrooms within an _allocation=1 school - Risk ladder colour: 1/6 of classrooms within an _allocation=1 school - Bar chart: 1/6 of classrooms within an _allocation=1 school - Coin icon arrays: 1/6 of classrooms within an _allocation=1 school We received a list of schools that were already registered in Qompas' system, which were randomised as follows: - 192 schools in the extended treatment group that receives information - 63 schools in the control group that does not receive information There is an average of 5.8 classes per schools. Therefore, there are 368 classes in the control group and 1102 to be allocated into the different treatment groups. Then, we planned the following allocation: - 368 classes in the table treatment - 184 classes in the neutral ladder - 184 classes in the colourful ladder - 184 classes in the bar chart - 184 classes in the icon arrays Later in time, 49 new schools joined the platform, for which we have no information. We expect 12 schools (average of 70 classes) to join the control group and 37 to join the treatment group. Thus, on average we expect approximately 214.6 classes to be allocated among the 5 treatment groups: - 71 for the table - 35 for the neutral ladder - 36 for the colourful ladder - 35 for the bar chart - 36 for the icon arrays All in all, we thus expect the following number of classes per group: - 438 for the control group - 439 for the table - 219 for the neutral ladder - 220 for the colourful ladder - 219 for the bar chart - 220 for the icon arrays
Primary outcomes
- Ranking of preferred occupations. - Short term average wage differences of most preferred occupation(s). - Mid term changes in wage of most preferred occupation(s). - Heterogeneity analyses. Interaction terms between the treatment dummies and the following personal characteristics will be included: literacy, numeracy, graph literacy, economic skills, gender, migration background, socioeconomic status (proxied by a quality of life indicator at the postcode level), time and risk preferences.
Secondary outcomes
- Self reported high school profile choice. - Actual high school profile choices. - Enrolment in higher education programs.

Cited by

No citation data available for this registration yet.

Keywords