Supporting Evidence Building in Child Welfare Project – LifeSet Evaluation

How to cite

Pergamit, Michael; Courtney, Mark. 2021. "Supporting Evidence Building in Child Welfare Project – LifeSet Evaluation." AEA RCT Registry. AEARCTR-0008015. https://doi.org/10.1257/rct.8015-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Michael Pergamit* , Mark Courtney

* Primary investigator

Registry ID
AEARCTR-0008015
DOI
https://doi.org/10.1257/rct.8015-2.0
First registered
Last updated
Status
In development
Trial dates
2021-08-02 – 2026-12-31
Countries
Private
JEL codes
J13; I31; I38; I24; E24
Registry record
http://www.socialscienceregistry.org/trials/8015

Abstract

The evaluation of the LifeSet program will follow a sample of youth randomized to be offered the LifeSet program or, alternatively, services as usual. The LifeSet program focuses on supporting youth in successful transitions to independent living by providing supports and services in multiple domains of independent living. The program is structured around a core relationship between each youth and their assigned “LifeSet specialist” who works intensively with the youth throughout his or her time in the program.

This evaluation will study both program impacts as well as program implementation, and will draw on qualitative and quantitative data collection and analysis techniques. The goal of the evaluation is to determine whether LifeSet has significant impacts on key youth outcomes in New Jersey and will address the core evaluation question: does LifeSet have a causal impact on the well-being and successful transition to adulthood of youth in the target population? The primary research questions for the impact study are does LifeSet improve outcomes of target population youth in the domains of:

a. Connections to education and employment
b. Social connections
c. Housing stability
d. Youth well-being (including resilience and social-emotional competence)

The eligible population for LifeSet will include youth 17-21 years old who have an open New Jersey public child welfare agency (DCF) case, are in the custody or guardianship of DCF, are living in the community (i.e. not incarcerated, in residential treatment, or hospitalized), and do not have a disqualifying condition (e.g., serious violent criminal history, residence out of state, severe mental illness, or intellectual disability).

The target population for the LifeSet program includes many more individuals who are in need of, and are eligible for, the intervention than can be accommodated by the program’s limited slots. The evaluation will therefore allocate the limited program slots by lottery, which is a fair way to allocate scarce resources. Identification and randomization of target-age youth will be done by New Jersey DCF based on administrative records and information gathered from youths’ case workers. To prevent one LifeSet provider from randomly having all youth in their catchment area assigned to only one group, randomization will be stratified by provider catchment area using a 1T:1C ratio. We estimate that the program will be able to serve approximately 300 study-enrolled youth during a two-year intake period. With a 1:1 randomization ratio (1 treatment to 1 control, or 1T:1C), the evaluation design will provide us with a sample of 600 youth.

Intervention

The LifeSet program is operated by Youth Villages, a private nonprofit organization. Formerly known as Transitional Living, LifeSet provides at-risk youth and young adults leaving foster care, juvenile justice, and mental health systems with the intensive in-home support and guidance they need to make a successful transition to adulthood. The core of the program is a therapeutic case management relationship between participant youth and “LifeSet specialists,” who work with youth to develop and move towards youth-defined goals in multiple domains of independent living including education, housing, employment and financial security, health and safety, and social connections and support. Youth Villages is currently operating in seven states in the US—Georgia, Massachusetts, Mississippi, North Carolina, Oklahoma, Oregon, and Tennessee—and the organization trains and supports nonprofit partners to offer LifeSet in other communities in six additional states. The CWEST project is evaluating LifeSet in New Jersey, which will be delivered by four local providers in the state.

When a young person enters the LifeSet program, the LifeSet specialist works with the youth to develop an individualized plan based on the youth’s personal context and goals. After the initial assessment, the youth and the LifeSet specialist meet at least weekly for one-hour, face-to-face or virtual sessions. During these sessions, the LifeSet specialist provides clinically focused case management, support, and counseling aimed at helping the youth move towards his or her goals. These meetings occur in a community setting, either at the youth’s home, work, or at whatever other location is most comfortable and convenient for him or her. In addition to these regular, scheduled meetings, LifeSet assures that youth are able to get in touch with a LifeSet specialist 24 hours a day, 7 days a week. The maximum duration youth can receive LifeSet services in New Jersey is 12 months.

LifeSet specialists work with between eight and ten youth at any one time and have either a master’s degree in a relevant field, or a bachelor’s degree and relevant work experience. Team supervisors supervise four LifeSet staff, including conducting group supervision, individual and field supervision, and professional development, and have a master’s degree in a relevant field, or a bachelor’s degree with relevant work experience. Licensed Program Experts are responsible for ensuring model fidelity and ideally have LifeSet implementation experience at the specialist and supervisor level and hold a master’s degree and professional license in a social service field. Licensed Program Experts meet regularly with each LifeSet team to provide clinical supervision and guidance on the program model.

Experimental design

The LifeSet evaluation will employ experimental methods, i.e. a randomized controlled trial (RCT). The eligible population for LifeSet will include youth 17-21 who have an open DCF case, are in the custody or guardianship of DCF, are living in the community (i.e. not incarcerated, in residential treatment, or hospitalized), and do not have a disqualifying condition (e.g., serious violent criminal history, residence out of state, severe mental illness, or intellectual disability). The four local LifeSet providers serve the Southern, Northern, and Central Regions, covering 18 of New Jersey’s 21 counties.

Identification and randomization of target-age youth will be done by New Jersey DCF based on administrative records and information gathered from youths’ case workers. This process will be conducted by age cohort, starting with youth who are at least 19 years-old and moving to younger ages in successive monthly randomization rounds. Youth will be randomly assigned to LifeSet or services as usual prior to obtaining consent/assent for youths’ participation in evaluation data collection. New Jersey DCF will then refer youth in the treatment group to a LifeSet provider; youth in the control group will be referred to, or continue receiving, services as usual.

Randomization began in August 2021 using a 1:1 randomization ratio (1 treatment to 1 control, or 1T:1C). However, only 60 percent of treatment group youth had gone on to enroll in LifeSet by, much lower than the anticipated rate of 90 percent. To account for the decrease in excess demand, the randomization ratio was changed in March 2022 to a 2:1 randomization ratio (2 treatment to 1 control, or a 2T:1C). The anticipated sample size was increased from 600 to 661 to allow for as many eligible youth to be randomized into the treatment group as possible while preserving statistical power with unequally sized groups. A total of 340 youth were randomized at the 1T:1C ratio (170 in each group) and an expected 321 additional youth will be randomized at the 2T:1C ratio, for a final sample size of 661 (384 treatment, 277 control).

Design details

Randomization method
Identification of target-age youth will be done by New Jersey DCF based on administrative records and information gathered from youths’ case workers. New Jersey DCF will use its administrative data to identify youth who meet the age criteria for LifeSet and are not excluded due to other criteria identifiable in DCF administrative data. This list will be sorted first by region and then in descending order of youths’ age in days. The DCF Research Coordinator will conduct a secondary screening by contacting the caseworkers for youth deemed eligible based on administrative records to obtain caseworker confirmation that the youth does not meet any exclusion criteria. New Jersey DCF will send the list of eligible youth and the number of study slots needed per region to the Urban Institute project team who will perform the random assignment. New Jersey DCF will send this list via CSV file to the Urban Institute research team. Urban will conduct the random assignment in pairs per region using STATA. Urban will then send the list with group assignments added back to DCF. We note that the target population is not evenly distributed across the four providers’ catchment areas. To prevent one provider from randomly having all youth in their catchment area assigned to only one group, randomization will be stratified by provider catchment area using a 2T:1C ratio within each stratum.
Randomization unit
Individual
Clusters
NA
Observations
The study anticipates recruiting a total of 661 youth
Arms
The study estimates including 384 youth in the treatment group and 277 youth in the control group.
Primary outcomes
The primary research questions for the impact study are does LifeSet improve outcomes of target population youth in the domains of: a. Connections to education and employment b. Social connections c. Housing stability d. Youth well-being (including resilience and social-emotional competence)
Secondary outcomes
The secondary research questions for the impact study are does LifeSet improve outcomes of target population youth in the domains of: a. Mental health b. Contact with the criminal justice system c. Intimate partner violence d. Economic well-being
Sponsors
Name: US Department of Health and Human Services, Administration for Children and Families (ACF) Office of Planning, Research, and Evaluation (OPRE) Location: Washington, D.C. Url: https://www.acf.hhs.gov/opre

Cited by

No citation data available for this registration yet.

Keywords