Impact Evaluation of Online Program Delivery of Catholic Charities Fort Worth’s LIFT Program

How to cite

Kofoed, Michael; Turner, Patrick. 2025. "Impact Evaluation of Online Program Delivery of Catholic Charities Fort Worth’s LIFT Program." AEA RCT Registry. AEARCTR-0015336. https://doi.org/10.1257/rct.15336-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Michael Kofoed* , Patrick Turner

* Primary investigator

Registry ID
AEARCTR-0015336
DOI
https://doi.org/10.1257/rct.15336-1.0
First registered
Last updated
Status
In development
Trial dates
2025-02-04 – 2031-02-04
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/15336

Abstract

The Wilson Sheehan Lab for Economic Opportunities (LEO) has partnered with Catholic Charities Fort Worth (CCFW) to explore an experimental research study that will generate evidence of the impact of virtual case management within the LIFT program. CCFW is an innovative, independent nonprofit serving families across 28 North Texas counties. As part of their mission to provide lasting, empowering solutions to poverty, CCFW operates LIFT as a way of helping people achieve financial freedom. Through regular case management sessions with their program navigators, clients receive tailored emotional and economic support. CCFW’s program team aims to increase the number of clients served across diverse geographical areas and to understand whether providing their holistic case management services virtually will result in the same engagement and efficacy as their in-person services.
To evaluate this program, we will use a randomized control trial (RCT) to examine whether the in-person and virtual program settings produce the same impact on service engagement, goal attainment, financial knowledge, and trust and emotional resilience. Eligible clients are over 18, speak English or Spanish, and are willing and able to work. Approximately 1,300 clients will be randomly assigned to either in-person or virtual case management, with approximately 650 in each group. The study will utilize program data collected by CCFW and Texas state administrative data systems. Results from this RCT will be disseminated to policymakers and providers across the country to inform the expansion of programs designed to support families and how those services are delivered.

Intervention

The LIFT program combines individualized case management, resource connections, financial coaching, and strategic financial assistance with the goal of advancing an individual's long-term financial freedom, labor market success, and well-being. In one-on-one consultations, paired navigators focus on three areas contributing to self-sufficiency: financial resilience, resource stability and emotional resilience, the ability to plan and cope. A tenet of the LIFT intervention is that a client’s readiness for change is essential to her or his success, and to achieve financial independence, individuals must actively contribute to creating their goals, objectives, and service plans.
Since LIFT’s inception in 2016, the program has helped more than 500 clients in the Dallas-Fort Worth area. An earlier RCT conducted collaboratively by CCFW and LEO evaluated a similar case management program by CCFW called Padua. The results of the study show that Padua led to a significant improvement in labor market and housing stability for those in the Padua program. CCFW’s strong culture of evaluation extends into their Research & Analytics Department, which tracks program data suggesting that LIFT similarly increases earnings, employment, and overall well being. Combining the evidence from the Padua RCT and LIFT program data, it is likely that LIFT is an effective program, but it is vital to evaluate the effectiveness of various delivery methods (in-person and virtual) before expanding the program to the more rural areas of CCFW’s 28 counties. Therefore, all study participants will take part in the LIFT program and have access to the same resources, but randomization will determine if their case management sessions with their navigator occur in person or online.

Experimental design

The CCFW team and members of LEO's research team collaboratively designed an RCT to evaluate the impact of virtual program delivery on the outcomes of clients in the LIFT program. The following elements describe the experimental design, beginning with study enrollment:
Starting in February 2025, individuals will be referred or contact CCFW enquiring about services. They will be directed to complete an online application for the LIFT program. Based upon their online application responses, clients will be evaluated for their study eligibility. Eligible applicants to the LIFT program will undergo a study consent process during their program intake. Eligible clients who provide study consent will be randomized during an intake call with an Operation Specialist. Then, a Program Manager will assign them to a virtual or in-person navigator based upon their randomized group assignment. If an individual does not consent to be part of the study, the outreach specialist will refer him or her to Padua, CCFW's holistic case management services. Navigators will call randomized clients with an initial welcome call within 2 days of assignment to administer baseline surveys and to schedule the first assessment visit. This first assessment visit will initiate randomized activities as the virtual group will continue program delivery via calls while the in-person group will continue program delivery in-person with their navigator. The content and resources associated with the LIFT program itself will be consistent across the randomized groups, and only the delivery method differs.
As part of internal program evaluation for clients in LIFT, CCFW invites their clients to complete a set of surveys covering topics of emotional resilience, resource stability, trust in navigator, and financial knowledge. These surveys are collected as part of regular contact between navigators and clients, and these responses will be used in the construction of study outcomes. Longitudinal data will be accessed using a partnership The Ray Marshall Center at the University of Texas. This will include linking to data on earnings, employment, public benefits use, as well as other records. Credit reports and consumer reference data for study participants will be accessed via LEO’s existing partnership with Experian, as well as residential address data via a similar partnership with Infutor.

To verify that the outcomes of LIFT participants are similar to those of the evidence-based Padua program, applicants to CCFW’s Padua will undergo a consent process during the study’s time period in which willing participants will provide permission for release of identifiable data to researchers. This will merely add a descriptive element to the study and allow some comparison to aggregate outcomes of clients across both programs.
The study will enroll approximately 1,300 participants over four years. Eligible clients include those who are 18 and above, speak English or Spanish, are ready and able to work, and live or work within the 28-county service area. Anyone with a financial coaching need is eligible; there are no income limitations on the sample.
By comparing the outcomes of individuals who were randomly selected to receive virtual case management (“Treatment”) against individuals who were randomly selected to receive in-person case management (“Control”), LEO will be able to isolate the causal effect of the virtual program on outcomes of interest. Program effects will be estimated by an intent-to-treat design where outcomes of interest will be regressed on a vector of observed characteristics, and a treatment group dummy variable. We will also report treatment-on-treated effects using a two-stage least squares regression approach. Heterogeneous impacts can be obtained by estimating the baseline regression with additional controls for groups and group-by-treatment interaction effects.
Following the planned four years of enrollment, two-year results for the full study sample will be available in 2030.

Design details

Randomization method
Computer (STATA)
Randomization unit
Individual
Clusters
1,300 individuals
Observations
1,300 individuals
Arms
Approximately 650 individuals will be enrolled in the treatment group (virtual services), and 650 will be enrolled in the control group (in-person services) over the course of the study.
Primary outcomes
Program engagement, goal attainment index, credit score, employment
Secondary outcomes
Program engagement, debt, degree enrollment/completion, housing stability, benefits usage, self-reported trust in navigator, resource stability, emotional resilience, financial knowledge and stability

Cited by

No citation data available for this registration yet.

Keywords