Friends Don't Lie: Allocating Jobs via Peer Information
How to cite
Walsh, Mark; Metzger, Jonas. 2022. "Friends Don't Lie: Allocating Jobs via Peer Information." AEA RCT Registry. AEARCTR-0009109. https://doi.org/10.1257/rct.9109-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
We often wish to target costly interventions (e.g. scholarships, financial aid, entrepreneurial loans) to the most suitable (e.g. hardworking, poor, trustworthy) individuals. However, suitability often remains unobserved and self-reports permit strategic misreporting. Peer prediction mechanisms, such as Robust Bayesian Truth Serum, aim to elicit suitability of an individual based on peer reports. However, these mechanisms are not robust to coordination among peers, which makes them unapplicable to digital screening problems. In this paper, we describe Algorithmic Truth Serum (ATS), the first peer prediction mechanism robust to coordination among peers. This mechanism would enable the automated, digital elicitation of peer information for the cost-effective targeting of interventions. To test this mechanism, we perform an experiment in which Indian job seekers digitally apply for a text transcription job. We elicit predictions from an applicant's peers and select the applicants with the highest predicted performance. To evaluate the performance of ATS, we compare eliciting peer predictions via ATS to not using incentives in an RCT. By improving screening, Algorithmic Truth Serum could aid in the growth of digital job market. Other possible applications include digital credit and targeting poverty programs.
Intervention
We test a novel information elicitation mechanism, Algorithmic Truth Serum (ATS), for the setting in which a large number of individuals, about whom we have only little verifiable information, apply for a desirable allocation (such as a loan, job, or scholarship). We test this mechanism on recommenders for digital job applicants. Our mechanism incentivizes recommenders to reveal additional information about the applicant, which allows it to make a more efficient allocation decision than would be possible based on the verifiable information alone. The mechanism consists of two machine learning models, both of which are estimated without access to any additional ‘ground-truth’ data: the first is a payment rule incentivizing truthful reports, which is estimated via a conditional moment restriction. The second is a decision rule which is robust to strategic misreports.
Experimental design
We will recruit digital job seekers to apply to a transcription task. In the application, the job seekers will choose a recommender, answer a few questions about themselves and take tests of their ability. We will send a recommendation survey to the chosen recommenders. In the treatment arm, the recommender will be incentivized to tell the truth via Algorithmic Truth Serum. In the control arm, the recommender will receive a fixed payment for completing the survey. Then, we will select the top 50% of job seekers in each arm based on the application and recommender surveys. The selected job seekers will then be able to access the transcription task and to collect their payment.
Design details
Cited by
No citation data available for this registration yet.