Learning from Administrative Data: NEWWS Evaluation

How to cite

Miller, Cynthia; Matsudiara, Jordan; Hoynes, Hilary. 2019. "Learning from Administrative Data: NEWWS Evaluation." AEA RCT Registry. AEARCTR-0004722. https://doi.org/10.1257/rct.4722-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Cynthia Miller* , Jordan Matsudiara , Hilary Hoynes

* Primary investigator

Registry ID
AEARCTR-0004722
DOI
https://doi.org/10.1257/rct.4722-1.0
First registered
Last updated
Status
On going
Trial dates
1991-06-01 – 2021-06-30
Countries
Private
JEL codes
I2, I3, J2, J3
Registry record
http://www.socialscienceregistry.org/trials/4722

Abstract

The Learning from Administrative Data (LAD) initiative seeks to extend the evidence of the long-term effects of various welfare-to-work programs evaluated in the 1990s, programs that included components such as earnings supplements, child care subsidies, work requirements, time limits on benefit receipt, and education services. Evaluations of these programs found that they typically increased mothers’ employment and earnings and childcare use. Some increased incomes and others did not, as increases in earnings were offset by reduced benefits. Questions remain about the long-term effects of these programs on children. Did they increase parents’ employment and/or family incomes enough to reduce poverty over the long-run or helped end the “cycle of poverty” by improving the outcomes of the children of welfare recipients?

The project will take advantage of multiple randomized controlled trials conducted by MDRC in the 1990s, including over 55,000 participants (mostly welfare recipients). It will merge data for the study participants with administrative data held by the U.S. Census’ Center for Administrative Records research and Applications (CARRA) to study the long run (over 20-year) effects of these policies on outcomes such as employment and earnings, fertility, marriage, and mortality, for adult participants and their children. This entry focuses on the analysis of the NEWWS evaluation, which tested the effects of 11 mandatory welfare-to-work programs that tested employment-focused versus education-focused approaches to increasing work and reducing benefit use.

Intervention

NEWWS was designed to directly and reliably test the effects of alternative approaches to helping welfare recipients find jobs and leave public assistance. The evaluation examined the effects of 11 mandatory welfare-to-work programs on welfare recipients and their children. The interventions operated in seven sites around the country and took different approaches to helping welfare recipients find jobs, advance in the labor market, and leave public assistance.

There were three approaches tested – two primary preemployment approaches (one that emphasized short-term job search assistance and encouraged people to find jobs quickly and one that emphasized longer-term skill-building activities, primarily basic education, before entering the labor market) — and a third approach that mixed elements of the other two. The programs were operated in seven sites across the country: Atlanta, Georgia; Grand Rapids, Michigan; Riverside, California; Columbus, Ohio; Detroit, Michigan; Oklahoma City, Oklahoma; and Portland, Oregon.

The study found that both the employment- and education-focused approaches substantially increased earnings during the five-year follow-up period, although the employment-focused approach produced larger increases in earnings and the effects emerged earlier in the follow-up period. Most of the programs increased earnings during the second and third years of the follow-up period, and the effects generally faded by the fourth and fifth years. Only two programs continued to produce statistically significant earnings impacts at the end of the fifth year. A subsequent analysis found no effects on work or earnings 10 to 15 years after study entry. The programs reduced public assistance receipt over the five-year period. These reductions offset the increases in earnings, however, with the result that there were no significant impacts on income (measured as earnings plus public assistance and tax credits minus payroll taxes).

See the following report for a summary of effects and a list of publications from the study: Hamilton, Gayle. 2002. Moving People from Welfare to Work: Lessons from the National Evaluation of Welfare-to Work Strategies. Washington, DC: U.S. Department of Health and Human Services. https://www.mdrc.org/sites/default/files/full_453.pdf

Experimental design

The evaluation of NEWWS used a random assignment research design, in which welfare recipients were randomly assigned to one of two or three research groups, depending on the site.
The analysis will focus on the full sample that was used in the main evaluation, including 41,715 adult welfare recipients and more than 81,000 children across all 11 sites.

Data sources to be used for the long-term analysis include the following: Employment and earnings from the IRS (if available) and from state UI agencies via the LEHD; federal tax credit (e.g. EITC) receipt, and marital status (based on tax filing status) from the IRS; Medicaid enrollment and utilization from CMS; Social Security disability program participation from SSA; Public housing program participation from HUD; Cash (TANF) and food assistance (SNAP) program participation from selected states; Fertility and death records from SSA; and California Franchise Tax Board data. The long-term analysis will also use data from the original evaluation, including a baseline survey, two- and five-year surveys, a child outcomes study survey, baseline reading and math tests, and a baseline private opinion survey.

Design details

Randomization method
Randomization conducted by enrollment staff via computer.
Randomization unit
individual
Clusters
41,175 adult participants, approximately 81,000 children
Observations
41,175 adult participants, approximately 81,000 children
Arms
Site T1 T2 Control 1 1495 1441 1497 2 1542 1557 1455 3 1596 3384 3342 4 2513 2570 2159 5 2226 n/a 2233 6 4309 n/a 4368 7 3529 n/a 499
Primary outcomes
The project will examine the long-term effects on key outcomes for study participants’ children, including employment and earnings, fertility, marriage, and mortality. Employment and earnings will be measured as average annual earning during the latest several-year age range available in the data. Outcomes related to fertility, marriage and mortality for the children will be measured as follows: had a birth before marriage, had a birth before age 20, married as of age 30 (pending outcome data availability), and mortality as of age 30 (pending outcome data availability). Effects will be estimated by program approach (employment-focused versus education focused) and for subgroups of children, defined by gender and age at exposure to the program.
Secondary outcomes
Secondary outcomes include: 1) employment, earnings and mortality for adult study participants; 2) participation rates in government assistance programs (Medicaid, subsidized housing, TANF, and SNAP for adult participants and their children; and 2) geographic mobility and neighborhood type for adult participants and their children. Effects on these and the primary outcomes will also be estimated for additional subgroups of participants, defined by parents’ employment history, welfare receipt history, education level, housing status, marital status, and residential locations. Effects will also be estimated for individual subgroups within the program approaches (employment versus education-focused). Effects will also be assessed at various points in the distribution of key outcomes, through quantile regression methods. Finally, depending on variation across the 11 programs in short-term effects on key mediators (such as parents’ employment and education and childcare use), the analysis will examine the potential mechanisms for any long-run impacts. To what extent do short-term impacts on these mediators help to explain variation in long-term impacts across sites and interventions?

Cited by

No citation data available for this registration yet.

Keywords