The Effects of Youth Letters of Recommendation

How to cite

Kessler, Judd; Heller, Sara. 2018. "The Effects of Youth Letters of Recommendation." AEA RCT Registry. AEARCTR-0002771. https://doi.org/10.1257/rct.2771-2.3

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Judd Kessler* , Sara Heller

* Primary investigator

Registry ID
AEARCTR-0002771
DOI
https://doi.org/10.1257/rct.2771-2.3
First registered
Last updated
Status
On going
Trial dates
2016-09-01 – 2023-09-01
Countries
Private
JEL codes
I25 Education and Economic Development; I38 Government Policy • Provision and Effects of Welfare Programs; J24 Human Capital • Skills • Occupational Choice • Labor Productivity; J08 Labor Economics Policies; C93 Field Experiments
Registry record
http://www.socialscienceregistry.org/trials/2771

Abstract

Summer Youth Employment Programs (SYEPs) are city-run programs that provide youth with paid work during the summer. These programs have been shown to significantly improve important youth outcomes including criminality, incarceration, and mortality. However, researchers have failed to find positive average effects on other important outcomes, such as future employment or college attendance. We aim to evaluate whether a minor program change, offering SYEP participants with personalized "letters of recommendation" (i.e., letters than can be shown to potential future employers, teachers, or guidance counselors), can improve outcomes for participating youth.

Intervention

This trial studies the impact of providing personalized letters of recommendation to youth participants of the New York City Summer Youth Employment Program (SYEP). SYEP provides youth with up to six weeks of employment at public and private organizations throughout New York City. We survey summer supervisors about youth performance. Each youth in the treatment group who is rated highly enough by their supervisor is sent a letter with personalized information from their supervisor about the youth's strengths. Youth in the control group are excluded from receiving a letter of recommendation through our program but are not prevented from requesting letters of recommendation from their employers directly.

Experimental design

The administrators of New York City's Summer Youth Employment Program (SYEP) select which summer employers (i.e., supervisors) will be part of the study. With the technical assistance of the research team, those supervisors will be sent a link to a survey instrument in which they will be shown the names of randomly selected youth who worked for them during the summer. After supervisors complete the survey, the research team will generate and send letters of recommendation to all youth who are rated highly enough by their supervisors to receive positive letters. We will subsequently link youth to existing administrative data on employment, and perhaps education and crime outcomes, in order to test whether the offer of a letter improves outcomes over a 3-year follow-up period. Data from a pilot of these measures conducted with 2016 SYEP participants will be combined with data from the full-scale trial of 2017 SYEP participants. Additionally, participants of the 2017 SYEP program will be invited to apply for a job, which will allow us to evaluate how the letter of recommendation affects the job application process directly.

Design details

Randomization method
Participants of the 2016 and 2017 NYC Summer Youth Employment Programs were randomly assigned to treatment or control groups. The research team conducted the lottery using Stata.
Randomization unit
The randomization was conducted at the individual youth level.
Clusters
No clustering, see planned number of observations response below.
Observations
70,000-80,000 individual youth who were SYEP participants, subject to data cleaning and de-duplication across study years. Additionally, in order to compare our study population to the broader SYEP applicant pool, we may collect data on the broader population of all youth who applied to SYEP in 2016 and 2017, even if they did not participate in SYEP. This broader group would be on the order of 200,000 individual youth. Additionally, we expect some non-compliance driven by factors such as whether supervisors received our survey recruitment emails and clicked on the survey invitation link. Whether a supervisor clicks on the survey invitation link should not differ between treatment and control youth on average, since survey recruitment emails give no indication of which youth were randomized to treatment or control. As such, we may separately analyze subsets where compliance is likely to be higher, but treatment is still random, such as youth who had a supervisor who clicked on a survey invitation link.
Arms
There is only one treatment arm and one control arm. The treatment and control arms each have 35,000-40,000 individual SYEP participants, subject to data cleaning and de-duplication across study years. Additionally, in order to compare our study population to the broader SYEP applicant pool, we may collect data on the broader population of all youth who applied to SYEP in 2016 and 2017, even if they did not participate in SYEP.
Primary outcomes
Employment and earnings in the formal sector
Secondary outcomes
Pending data availability: college enrollment and attainment; high school enrollment, performance, and attainment; crime outcomes, such as arrests and convictions; job application and letter use rates among a subset of study youth

Cited by

No citation data available for this registration yet.

Keywords