Testing new technologies for skill assessment and matching on expectations, upskilling and reallocation (pilot)

How to cite

Delfino, Alexia; Sadun, Raffaella. 2021. "Testing new technologies for skill assessment and matching on expectations, upskilling and reallocation (pilot)." AEA RCT Registry. AEARCTR-0008363. https://doi.org/10.1257/rct.8363-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Alexia Delfino* , Raffaella Sadun

* Primary investigator

Registry ID
AEARCTR-0008363
DOI
https://doi.org/10.1257/rct.8363-1.0
First registered
Last updated
Status
In development
Trial dates
2021-10-12 – 2022-03-01
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/8363

Abstract

We study the effects of an innovative tool for workers’ skills self-assessment and skills matching on individuals’ retraining choices, job search strategies and employment outcomes. Unemployed individuals who contact job centres to start their paperwork qualify as potential participants. Invited unemployed who agree to take part into the study will be randomised into receiving an invitation for our treatment, that is registering to an online platform that provides them with highly individualised information on their skills and skill-match within different occupations of their interest. The remaining individuals will be invited to register to a placebo website that only supplies aggregate information on a set of occupations. We expect those in the treatment group to increase their awareness on their current skill level, as well as align their expectations with the current labor market trends. Furthermore, we expect to find a positive effect on the likelihood that individuals retrain and search for occupations suited to their skills, even if in a sector/occupational category different from what they did in the past. This registration is for a pilot study, which we plan to then scale up. The pilot will happen in two job centres. In one of them, we will also randomise participants into having a phone conversation with a case worker, to measure the extent to which the actual interaction with caseworkers has an effect beyond that of the technology. By embedding our experiment within the organisational processes of the employment center, we will also test the effect of the technology on the job center organisational efficiency. On average, we expect treated individuals randomly selected for the call to achieve (weakly) better employment outcomes than control individuals selected for the call: during the phone conversation, case workers who can take advantage of the individualised information produced by the treatment platform on their users should provide more effective help and be a complement of the technology. In the other job centre, all the participants will have a conversation with a case worker.

Intervention

Our first treatment is an online platform that provides participants with highly individualised information on their skills and skill-match with different occupations. In one of the two job centres, individuals across treatment and control groups are also randomised into receiving a second intervention, which consists in having a phone conversation with their assigned case worker.

Experimental design

Our main hypotheses are that (1) providing unemployed individuals with highly individualised information on their skills and skill-match with different occupations should make them more likely to retrain and look for jobs that are more aligned to their skills, and that (2) providing the same information to job center case workers should allow them to be more effective in helping their users, thereby improving their employment outcomes.
This registration is for a pilot of the main study. The pilot will be conducted in two job centres. Unemployed individuals who contact the two job centres to start their paperwork will be sent an invitation to participate in our project. If they give their consent, they will be randomised into either receiving an invitation for the treatment website or the placebo website.
In one of the partner job centres, individuals across treatment and control groups will be further randomised into either receiving a phone call by their case worker to discuss their case or not. In the second job centre, all the individuals will be sent to have a meeting (online or in presence) with a case worker.
Baseline data on participants will be partly provided by admin records of each job centre and also collected via an online survey participants will fill in when they accept to participate in the project. A follow up survey will be sent to participants 2 to 3 weeks after their registration to the project. Depending on the number of case workers involved in the project, we will also send them a baseline survey before the beginning of the pilot. Each case worker will also complete an online survey after each call or meeting with the users. A follow up survey will be sent to all case workers.

Design details

Randomization method
Randomization into the treatment or placebo website is carried out on a rolling basis by the digital system that handles the project subscriptions, and it is stratified by gender, age, and job center. In the job center where calls are randomised, the additional randomisation into the call with the case worker is done by the researchers on a weekly basis using Stata, and stratifying for the same variables and for treatment status.
Randomization unit
Individual
Clusters
N/A
Observations
Employment center 1 (based on 2021 monthly data): - expected number of people invited in a month is 350 individuals - expected participation rate is 30%, hence 105 individuals will be randomised into treatment or control website Employment center 2 (based on 2021 monthly data): - expected number of people invited in a month is 320 individuals - expected participation rate is 30%, hence 96 individuals will be randomised into treatment or control website However, one of the purposes of the pilot is to gather information on the take-up rate (which we can only imprecisely guess right now), in order to do power analysis for the main study.
Arms
- Employment center 1: according to an expected flow of 105 participants in one month, we will assign 50% of them to the placebo platform and 50% of them to the treatment platform (thus approximately 52 in each treatment). Within each treatment, we will assign 50% to the call with a case worker and 50% to no call (thus approximately 25 in the cell call+treatment, 25 in the cell call+ placebo, 25 in the cell no call+treatment, 25 in the cell no call+ placebo). - Employment center 2: according to an expected flow of 96 participants in one month, we will assign 50% of them to the placebo platform and 50% of them to the treatment platform (thus approximately 48 in each treatment).
Primary outcomes
User level (i.e. the unemployed): job finding rate, job search breadth, job search intensity (time + applications), beliefs updating on occupability and skill fit in different occupations, confidence in own skills, willingness to change occupation/sector compared to past employment, actual reallocation across occupations, training and type of training Case worker level: job satisfaction, share of users remembered, perceived social impact and social worth Case worker-unemployed level: beliefs on user's employability, opinion on effectiveness of meeting, topics discussed
Secondary outcomes
Users: flexibiliy in commuting, aspirations in terms of salary/level of responsibility, overall life satisfaction, concerns on job finding Case workers: meetings take up

Cited by

No citation data available for this registration yet.

Keywords