Signaling skills online and labor market outcomes: evidence from an adaptive experiment

How to cite

Picard, Bertille; Hoffmann, Morgane; Marie, Charly; Bied, Guillaume. 2022. "Signaling skills online and labor market outcomes: evidence from an adaptive experiment." AEA RCT Registry. AEARCTR-0010085. https://doi.org/10.1257/rct.10085-3.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Bertille Picard* , Morgane Hoffmann , Charly Marie , Guillaume Bied

* Primary investigator

Registry ID
AEARCTR-0010085
DOI
https://doi.org/10.1257/rct.10085-3.1
First registered
Last updated
Status
In development
Trial dates
2023-01-09 – 2023-12-31
Countries
Private
JEL codes
J23, J24, C93, C80, C81, C90
Registry record
http://www.socialscienceregistry.org/trials/10085

Abstract

Digital job tools promise to reduce frictions on the labor market. In this randomized controlled trial, we study an intervention seeking to increase the usage of a public online platform in France. The intervention, implemented with an adaptive design, consists in sending mails to job seekers, providing information, help and motivation to register or update their profiles on the platform. We seek to discover which treatments maximize uptake of the tool among the several types of incentives considered. We then analyze the impact of the platform's use on labor market outcomes.

Intervention

We send emails to registered job seekers in order to encourage them to use their profile on the public employment service's platform, i.e to fill the profile and potentially publish it to become visible to recruiters. We test several emails to maximize take-up by either providing information or reducing the cost of the tool's use, in combination with behavioral levers. We also test two different sending hours.

- Informational treatments

(Info1) Information about the use by recruiters
In order to emphasize the potential gains of filling the profile, we highlight its use by other agents. We show information about the number of recruiters looking for candidates on the platform each month. We vary whether this information appears or not.

(Info2) Information about the gains in terms of service quality
Another information shared is related to the effect of providing information through the profile on the quality of the service given by the institution. Counselors rely on the profile to recommend job ads to job seekers and so do automatic recommender systems built by the institution. We highlight how a well-filled skill profile can enable Pôle emploi's counselors to better assist individuals in finding a job, a training program and more broadly communicate with them. We vary whether this information is present or not.

- Help provision treatments

(Help1) Including a list of steps
In order to reduce the cost of completion we write the various steps to follow in order to fully mobilize the tool directly in the body of the email. We vary whether this tutorial is proposed or not.

- Including an intensive help
Some job seekers may find it difficult to complete their profile independently. Some may have difficulties with the French language or with digital tools. Others may want to be accompanied in the process because of a lack of self-confidence. We propose an intensive help that can take 2 forms:
o (Help2workshop) Workshop: We propose job seekers to self-register to a half-day workshop designed to help them fill-in their profile at the PES.
o (Help2counselor) Counselor: We propose jobseekers to book an appointment with a counselor via a call-to-action button which redirects them to the appropriate webpage.
We vary whether the intensive help is proposed or not and whether it takes the form of a workshop or a counselor appointment.

- Scheduled time to send the email

(Schedule) Sending hours:
We vary whether the email is sent at 9am or at 3pm.

Possible combinations
We aim to test the interaction of some of these email contents. We selected a subset of interactions between email contents that we considered to have a chance of being relevant, leading to a total of 16 different emails summarized.

Since emails can be sent at either 9am or 3pm, this experiment includes a total of 32 treatment arms.

Experimental design

Each week, we follow these steps:
1. We sample 36 000 job seekers from the eligible population at week t.
2. We split them in two samples: 2/3 of the sample is allocated to a control group (who does not receive any incentive) and the other 1/3 is allocated to a treated group (who receives one of the incentives according to estimated individual probability of allocation).
3. For the treated group, an algorithm* determines allocation probabilities for each individual; treatments are then assigned accordingly.
4. Individuals in the treated group are sent a message shortly after the assignment.
5. We observe whether individuals in the treated group clicked on the pages of the online profile. This information is used by the algorithm to update the allocation probabilities.

After at most 10 weeks, we stop the adaptive assignment and keep some observations for the evaluation of the assignment policy learnt.

*Description of the algorithm: we use a honest generalized random forest to predict the impact of each treatment based on data collected the previous weeks. Assignments are generated using a Thompson sampling procedure based on the mean and variances of the indiviudal predictions from the forest (the first week, interventions are assigned at random).

Design details

Randomization method
The allocation to the control or treated groups is done by a computer with uniform probabilities (with mean 1/3). Within the treated group, the kind of intervention is chosen using a contextual bandits algorithm (i.e, randomization done by a machine learning algorithm with personalized sampling weights).
Randomization unit
The unit of the randomization is the job seeker.
Clusters
No cluster
Observations
~ 450,000 observations, one third in the treated group.
Arms
The treated group of size 150,000 is split between 32 treatment arms; the sample size for each arm is not known in advance (it is progressively adapted so that people are sent to the best treatments).
Primary outcomes
- Online profile related outcomes - Impact on the individual's usage of the platform - Labor market outcomes

Cited by

No citation data available for this registration yet.

Keywords