Pathways to Success for Adult Learners at Two Tennessee Community Colleges: Evidence from a Field Experiment

How to cite

Moreland, Amy; Deaton, Russ; Dell, Madison. 2023. "Pathways to Success for Adult Learners at Two Tennessee Community Colleges: Evidence from a Field Experiment." AEA RCT Registry. AEARCTR-0010784. https://doi.org/10.1257/rct.10784-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Amy Moreland* , Russ Deaton , Madison Dell

* Primary investigator

Registry ID
AEARCTR-0010784
DOI
https://doi.org/10.1257/rct.10784-1.0
First registered
Last updated
Status
In development
Trial dates
2023-01-20 – 2026-05-31
Countries
Private
JEL codes
C93, I23
Registry record
http://www.socialscienceregistry.org/trials/10784

Abstract

Over the past decade, adult students have constituted between 24% and 36% of fall enrollment in the Tennessee community college system. However, many of these students have struggled to persist and complete a credential. Three-year graduation rates for adult students have held steady at approximately 23% for the last four years. Adults who complete a credential stand to realize significant benefits to their employment prospects and earnings, but too few adults persist to completion to experience these benefits. To address the challenges adult students face on their path to a college credential, we will partner with two colleges (Jackson State Community College and Northeast State Community College) to redesign the experience of adults in their first year of college, including students who are new to college and students who are returning to college after time away. The intervention includes two components: redesigned general education courses and success coaches who are embedded within those courses. We will offer stipends to five faculty members per year at each partner college to incorporate adult-friendly teaching strategies into the general education courses they teach. Half of the eligible adult students who enroll in these redesigned courses will be randomly assigned to coaching. Coaches will have a caseload of no more than 150 students, which is substantially lower than the average caseload of 569 students per relational advisor at Tennessee community colleges. Coaches will meet with students two times in each semester (either in person or virtually) and will reach out at least weekly via text message and email, with the goals of connecting students with campus resources, helping them identify majors and careers that are most relevant to their goals, and supporting their academic progress. For students enrolled in the redesigned courses, coaches will work closely with instructors to monitor the progress of their assigned students and proactively intervene as students encounter difficulties. Because we will estimate the impact of each component of the intervention separately as well as the two components together, the experiment will have three treatment arms: (1) coaching only; (2) course redesign only; and (3) coaching and course redesign. Students are eligible to participate in the intervention if they meet the following criteria: enrolled as a degree- or certificate-seeking student at Jackson State or Northeast State; graduated from high school more than five years ago; either in their first-ever semester of college or in their first semester back in college after a break of at least one semester (excluding summer); and not part of any special populations that will be excluded from the sample (i.e., students who have access to enhanced advising through another program on campus and students in prison education programs). We will provide coaching for five cohorts (i.e., students beginning in each of the fall and spring semesters between spring 2023 and spring 2025), and students will receive coaching for a full academic year. Spring 2023 will serve as a pilot of the intervention, and data from this semester will not be included in the experimental evaluation. For each cohort, we will randomize students into the treatment and control conditions after stratifying by college (i.e., site), student type (i.e., first-time or returning student), and enrollment in a redesigned course section. Randomization occurs at the individual level. Across the two years, we will assign up to 600 students to the two coaches at Jackson State and up to 900 students to the three coaches at Northeast State to maintain a caseload of no more than 150 students per coach in a year. Based on enrollment data from the most recent academic year and expected adult enrollment in the redesigned courses, we estimate that there will be approximately 650 students in the treatment arms and 416 students in the control group at Jackson State. We expect that there will be approximately 950 students in the treatment arms and 768 students in the control group at Northeast State. Our primary outcomes of interest are course success rates, semester-to-semester persistence, year-to-year persistence, and two-year completion rates. We will treat all other outcomes and subgroup analyses as exploratory.

Experimental design

At the end of the registration period for each fall and spring semester between spring 2023 and spring 2025, Jackson State and Northeast State will send course enrollment data for all students registered for the semester. After restricting the sample to the students who meet our eligibility criteria, we will randomly assign students to the control group or to one of three treatment arms (coaching only, course redesign only, or coaching and course redesign). We will assign half of the eligible adult students enrolled in a redesigned course section to coaching. For students not enrolled in a redesigned course section, we will assign as many students to coaching as possible while still maintaining a maximum caseload of 150 students per coach. The remaining students who meet our eligibility criteria will be assigned to the control group. Spring 2023 will serve as a pilot of the intervention, and data from this semester will not be included in the experimental evaluation. Based on enrollment data from the most recent academic year, we anticipate having approximately 2,784 students in our eligible sample in academic years 2023-24 and 2024-25: 1,066 at Jackson State and 1,718 at Northeast State. Randomization will be stratified by college (i.e., site) and student type (i.e., first-time or returning student). From fall 2023 through spring 2025, randomization will also be stratified by whether a student is enrolled in a redesigned course section. We will account for this stratification in our analysis by including (cohort × site × student type × enrollment in redesigned courses) fixed effects in our models.

Design details

Randomization method
We will randomize students into one of three treatment arms or into the control group using random number generation in Stata. We will assign half of the students enrolled in a redesigned course section to coaching. For students not enrolled in a redesigned course section, we will assign as many students to coaching as possible while still maintaining a maximum caseload of 150 students per coach. The remaining students who meet our eligibility criteria will be assigned to the control group.
Randomization unit
For the spring 2023 pilot, randomization will occur at the individual level, after stratifying by college (i.e., site) and student type (i.e., first-time or returning student). For each semester from fall 2023 through spring 2025, randomization will also be stratified by whether a student is enrolled in a redesigned course section.
Clusters
N/A
Observations
Based on enrollment data from fall 2021 and spring 2022 (the most recent academic year), we anticipate having approximately 2,784 total observations in academic years 2023-24 and 2024-25: 1,066 at Jackson State and 1,718 at Northeast State. We expect the eligible sample to be larger in the fall semesters (n=1,792) than in the spring semesters (n=992).
Arms
The experiment will have three treatment arms: (1) coaching only; (2) course redesign only; and (3) coaching and course redesign. We conservatively assume that each of the 20 faculty members who redesign their general education courses will teach two redesigned course sections during the academic year (for a total of 40 redesigned course sections across two years). We also assume that there are, on average, 10 adult students in each of the redesigned course sections. Half of the students in the redesigned course sections will be assigned to coaching, which gives us 200 students in the course redesign only treatment arm and another 200 students in the coaching and course redesign treatment arm. To maintain a maximum caseload of 150 students for each of our five coaches, we can offer coaching to a total of 1,500 students across both years. Given our estimate that 200 students in the redesigned course sections will be assigned to coaching, we will assign an additional 1,300 students to coaching who are not enrolled in any redesigned course sections to maximize the number of students who have access to a coach. With 1,700 of our 2,784 total students assigned to one of the three treatment arms, there will be approximately 1,084 students in the control group.
Primary outcomes
To assess the impact of the intervention, we will focus on four primary outcomes of interest: 1. Course success rate: a binary indicator equal to 1 if a student received a grade of C or better in a course 2. Semester-to-semester persistence: a binary indicator equal to 1 if a student was enrolled at any Tennessee community college in the next fall or spring semester following their first semester of enrollment (or earned a credential in the intervening period) 3. Year-to-year persistence: a binary indicator equal to 1 if a student was enrolled at any Tennessee community college one year after their first semester of enrollment (or earned a credential in the intervening period) 4. Two-year completion rate: a binary indicator equal to 1 if a student completed a degree or credential within two years of their first semester of enrollment

Cited by

No citation data available for this registration yet.

Keywords