Behavioral Interventions to Advance Self Sufficiency – Next Generation (BIAS-NG): Los Angeles County, Temporary Assistance for Needy Families
How to cite
Key, Clinton; Katz, Lawrence. 2022. "Behavioral Interventions to Advance Self Sufficiency – Next Generation (BIAS-NG): Los Angeles County, Temporary Assistance for Needy Families." AEA RCT Registry. AEARCTR-0010539. https://doi.org/10.1257/rct.10539-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
The Behavioral Interventions to Advance Self-Sufficiency-Next Generation (BIAS-NG) project, sponsored by the Office of Planning, Research, and Evaluation in the Administration for Children and Families (ACF), U.S. Department of Health and Human Services, uses principles of behavioral science to try to improve human services program design and outcomes. Behavioral science is an interdisciplinary field that incorporates psychology, economics, and other social sciences to provide insight into how people process information, make decisions, and take actions. BIAS-NG partners with state and local human services agencies to diagnose behaviorally-based barriers to program success, design interventions to address those barriers, and test the efficacy and cost efficiency of those behaviorally informed interventions relative to status quo service delivery.
For this study, BIAS-NG worked with the Los Angeles County Department of Public Social Service (DPSS). DPSS manages the Los Angeles County Temporary Assistance to Needy Families (TANF) program called California Work Opportunities and Responsibilities to Kids (CalWORKs). California’s TANF policies mandate that adult CalWORKs recipients participate in 20-35 hours per week of approved work activities, or welfare-to-work activities, as a condition of receiving benefits. If CalWORKs recipients are required to participate in welfare-to-work activities, they must also attend required meetings and report the hours they spent on these activities.
In Los Angeles County, the welfare-to-work activities are administered through the Greater Avenues for Independence (GAIN) program which is also overseen by DPSS. All adults in Los Angeles County who receive public assistance benefits through CalWORKs are required to participate in GAIN unless they meet criteria for an exemption, although those who are “exempt” may participate voluntarily. The GAIN program offers a variety of welfare-to-work activities that aim to support participants in finding sustainable work. One of these activities is Job Club, which is the focus of this BIAS-NG study. Job Club is a four-week program designed to support participants with building skills they need to find and obtain a job, such as developing a resume and interviewing techniques. Job Club participants then apply these skills to searching for a job during the program period.
When BIAS-NG and DPSS began working together, DPSS staff identified the challenge of low participant engagement in GAIN activities and set a goal of increasing engagement. Low engagement in GAIN activities is a challenge for both staff and participants. If a GAIN participant does not comply with the program requirements, GAIN program staff place the participant in a non-compliance status, initiating a process that may lead to the participant being sanctioned and having their benefits reduced. Having a participant enter non-compliance status is a very time-consuming process for both staff and participants. When a participant is placed in non-compliance status, the GAIN Service Worker (GSW) who manages the participant’s case must attempt to contact the participant multiple times, and if the participant does not respond to attempts to schedule a meeting, the GSW will make a visit to the participant's home. The GSW and participant must have a meeting to determine whether a participant had “good cause” for not complying with requirements before sanctions are issued. This whole process often takes two months to complete.
Additionally, participants can miss out on substantial benefits if sanctioned. In 2018, for one-person households, sanctioning reduced monthly benefits by as much as $355. For two- or three-person households, which often include children whose benefits are not reduced by sanctions, benefits were reduced by as much as $222 or $137, respectively. Of GAIN participants not exempted from the work requirement, 36 percent were in sanction status as of August 2018.
Working closely with DPSS, the research team identified key points in GAIN’s program processes (such as meetings, transitions, or activities) where participants did not meet GAIN program requirements or stopped participating in GAIN en masse. We refer to these points as “drop-off points.” The analysis of these drop-off points led the team to focus on attendance at Job Club. Of participants who were referred to Job Club in June 2017, only 38 percent started Job Club within 30 days of referral. Activity attendance is a problem across other GAIN activities as well.
The research team conducted interviews with GAIN staff and participants and used the information they shared to hypothesize several behavioral barriers that may be impacting Job Club attendance. These included participants not understanding GAIN program requirements, sanctions not being salient enough to achieve their purpose of motivating participants to comply with GAIN requirements, hassles associated with arranging child care and transportation, negative associations with GAIN programs, and having other life stressors that make attending Job Club a less pressing concern.
In response to these behavioral challenges, DPSS and the research team developed a text messaging intervention, further described in the Intervention section below. The goal of the texting intervention is to increase participation in Job Club, thereby reducing the incidence of sanctions due to failure to comply with GAIN requirements. In addition to initial attendance, the intervention was designed to increase the percentage of participants who complete Job Club successfully, meaning they find employment within the four weeks of Job Club and stop attending, or they complete all four weeks of Job Club. To capture these two goals, the primary outcomes will be attendance at Job Club in the first 30 days after referral, attendance at Job Club within the first 60 days after referral, and entrance into non-compliance status within 60 days of referral. The first two measures correspond with key drop-off points: 1) between the point someone is assigned to Job Club and the scheduled Job Club start date and 2) between the scheduled Job Club start date and completion.
For the impact evaluation, clients were randomly assigned to receive the behaviorally informed text messages (program group) or not (control group). The point of random assignment was when clients were referred, or assigned, to Job Club. The evaluation will compare outcomes of the program group to the control group for the full sample and several sub-groups. In addition to the impact study, BIAS-NG is conducting accompanying implementation and cost analyses to document how the intervention was delivered and at what cost.
Intervention
The BIAS-NG intervention aims to increase participation in Job Club, thereby reducing non-compliance with welfare-to-work requirements and reducing sanctioning, by using text message reminders with embedded behavioral messages. Such messages have been shown to improve attendance to healthcare appointments and appearances to court summonses. A study among TANF recipients in one Colorado county found that similar messages delivered by phone had significant positive effects on attendance to initial welfare-to-work meetings. Together, these findings suggest that behaviorally informed text messages are a promising tool for encouraging TANF recipients to attend welfare-to-work activities.
Experimental design
The impact evaluation uses a randomized controlled trial design. Study participants have an equal (50/50) chance of being assigned to the program or control groups.
The study sample includes Los Angeles County CalWORKs participants who are age 18 or older, speak English or Spanish, do not have a confidential case file or current domestic violence issues, are required to participate in welfare-to-work activities, and attend an in-person meeting with a GSW during which they are referred to Job Club. The sample includes participants who are both new and not new to GAIN. Study enrollment could occur at different points after enrolling in GAIN:
• During the initial appraisal meeting, after referral from CalWORKs.
• After completing an assigned activity and coming in to sign a new welfare-to-work plan.
• At a “cause determination” meeting, in which participants attend to correct an instance of non-compliance and sign a compliance plan.
The steps of the random assignment process were as follows:
• Step 1: During the meeting between the GSW and participant in which participants were referred to Job Club, the GSW explained to the participant that the county was interested in sending text messages to participants but was trying it out to determine whether it was helpful to participants. The GSW briefly explained the study, and if the participant was interested in receiving text messages, he or she signed an informed consent form agreeing to receive text messages if assigned to the program group. The GSW confirmed that GAIN had the right phone number to send the participant text messages. GSWs also emphasized that regardless of whether a participant received text messages, they were required to attend Job Club on their assigned date. Both control and program group participants received paper forms that included this information.
• Step 2: After the participant left the meeting, the GSW logged into the RA tool and entered the participant’s GAIN region, first name, and case ID.
• Step 3: The GSW was then prompted to enter in a few additional pieces of information including the participant’s phone number, confirmation that the assigned activity was Job Club, activity start date and time, activity end date and time, and activity location. The GSW then submitted the information to the RA tool, which randomly assigned the participant to the program or control group.
• Step 4: If the participant was assigned to the program group, the RA tool used the submitted information to populate text message templates and determine which messages to send each day. This report was automatically uploaded to LA County’s secure file transfer protocol each day. The IT Division’s system automatically downloaded the report and sent out the messages as texts.
• Step 5: A GSW co-located at each Job Club logged into the RA tool once a week to indicate if all the participants in the program group assigned to attend Job Club at the location where they worked were: (1) still attending Job Club, (2) had dropped out of Job Club or were no shows, or (3) had left Job Club because they found a job. This information was used to populate the appropriate text message templates sent after the participant’s scheduled Job Club start date. These messages were sent in the same way as the pre-Job Club messages. Note: this step was not needed at the start of the pilot, and was added beginning the week of August 12, 2019, when the complete intervention was first implemented.
• Ongoing: If the participant’s information changed (e.g., the start date of Job Club was delayed), the GSW logged into the RA tool and updated the information.
For two-parent cases, the second parent to be randomized was automatically assigned to the same research group as their partner to reduce potential contamination of information in the text messages sent to program group members being shared with control group members. For cases that were subject to randomization multiple times (e.g., a participant was assigned to a new GSW, who randomized the participant again during a cause determination meeting), the case retained its original research group assignment.
Design details
Cited by
No citation data available for this registration yet.