My Professor Cares: Experimental Evidence on the Role of Faculty Engagement
How to cite
Carrell, Scott; Kurlaender, MIchal. 2020. "My Professor Cares: Experimental Evidence on the Role of Faculty Engagement." AEA RCT Registry. AEARCTR-0005875. https://doi.org/10.1257/rct.5875-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Despite a growing body of literature that instructors “matter” in higher education, there is virtually no evidence about how their actions influence student outcomes. In this paper we provide experimental evidence on the impact of specific faculty behaviors aimed at increasing student success. We test the effect of professor feedback on student success in higher education classrooms though a "light-touch" randomized intervention. We present results from a small pilot in an introductory-level microeconomics course at a comprehensive research university, and the scale-up conducted in over 30 classrooms and nearly 4,000 students at a large broad-access university. The intervention consisted of several strategically-timed E-mails to students from the professor indicating keys to success in the class, the professor’s knowledge of the students’ current standing in the course, and a reminder of when the professor is available. Results from the pilot show that students in the treatment group scored higher on exams, homework assignments, and final course grade, compared to students in the control group. Results from the larger experiment at a broad access institution are more mixed—we find significant positive effects on student perceptions of the professor and course for all students. However, we only find positive achievement effects for our target population, first year students from underrepresented minority groups. Finally, we replicated the pilot to test the robustness of these results and again find positive effects on student achievement at the large comprehensive university. We conclude that in certain settings and with some students, targeted feedback from professors can lead to meaningful gains in achievement.
Intervention
We test the effect of increased and individualized professor feedback on student success. The “light-touch” intervention consisted of strategically-timed personalized e-mails to students from the professor indicating the professor’s knowledge of the students’ current standing in the course, keys to success in the class, and a reminder of when the professor is available.
Experimental design
The study was implemented in two separate waves during the spring of 2016 and fall of 2017 at a large representative, broad-access four-year institution. We randomly chose 30 large undergraduate courses (serving over 120 students) in each respective term and identified the instructor of record. Collaborating with the Campus Center for College and Career Readiness we recruited these professors by sending personalized letters signed by both the Provost and Dean of Undergraduate Studies. In total, 34 faculty members across 18 different course subjects participated in the study, with nearly 4,000 total students in the treatment and control groups.
In the spring of 2016 the treatment group comprised of a randomly selected one-half of the students in 14 large undergraduate classes. To assess whether spillovers may be biasing our estimates, in the Fall of 2017, we drew our treatment group in two different ways. In eight large classes (>120 students) we randomly selected one-third of students into treatment (in contrast to the half randomized in the first wave). Additionally, we randomly selected the entire class to receive treatment in the ten cases where the professor taught two sections of the identical course. Second, rather than providing two targeted emails as was done in the pilot (at a 10-week quarter system course), we chose to have three targeted emails in the scale-up (at 16-week semester system courses). During spring of 2016, the first email entailed an initial “welcome to my class” message containing strategies to succeed in the course. The second and third emails were targeted performance feedbacks at the midway point in the course and just before the final exam. In the fall of 2017, similar to the pilot, we asked professors to give students in the treatment targeted feedback based on the first “meaningful” assignment as well as midway through the course and just before the final exam.
Design details
Papers from this trial
- Scott E. Carrell & Michal Kurlaender AMERICAN ECONOMIC JOURNAL: ECONOMIC POLICY VOL. 15, NO. 4, NOVEMBER 2023 (pp. 113-41) https://www.aeaweb.org/articles?id=10.1257/pol.20210699&from=f
Cited by
No citation data available for this registration yet.