The Effects of Course Information on Usage

How to cite

Lindo, Jason; Tillinghast, Jonathan. 2019. "The Effects of Course Information on Usage." AEA RCT Registry. AEARCTR-0003654. https://doi.org/10.1257/rct.3654-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Jason Lindo* , Jonathan Tillinghast

* Primary investigator

Registry ID
AEARCTR-0003654
DOI
https://doi.org/10.1257/rct.3654-2.0
First registered
Last updated
Status
On going
Trial dates
2018-11-05 – 2019-12-31
Countries
Private
JEL codes
I21 I23 I26
Registry record
http://www.socialscienceregistry.org/trials/3654

Abstract

College students rely on a diverse set of information when making educational decisions. A highly uncharted area in the education literature is what information college students use to make course enrollment and field of study decisions. Our study aims to fill this gap by providing students course grade outcomes of previous courses taught at Texas A&M University in an easy to read graphical interface that allows comparison across classes. To study if students use previous course outcome information we implement a Randomized Control Trial (RCT). To measure the usage of our webpage, we introduce a brief survey asking for a student's Texas A&M email and major and measure the total visits to our webtool page. To examine the effects on academic outcomes we will also look at course tool and major selection in spring 2019 as outcomes.

Intervention

To study the impact access to previous grade information of courses we conduct a randomized controlled trail (RCT) by sending access to a random set of Texas A&M University (TAMU) student emails. The course tool utilizes previous grade information of courses taught at TAMU and displays it in an easy to read fashion using data visualization software. The course tool displays a previous course's average GPA on the y-axis with course number on the x-axis for a professor-course offered in the Spring of 2019. These data points are grouped by department type and allow the user to scroll through a graph window and view GPA's across departments. Hovering over a data point on the graph allows the user to see a courses grade distribution, professor name, course name, and number of q-drops. Furthermore, the tool includes several filters that allow the user to sort information based on GPA, department, core requirement, honors, regular, and whether it meets the International and Cultural Diversity requirement.

In Fall 2018, 5,095 emails were sent to undergraduate students notifying them they have been given access to our new course tool. A follow up email was sent to each student one week after the initial email. Clicking the link in the email directed a student to custom website that hosted the course tool. Upon arriving to the webpage, a student was prompted with a text box asking to enter their TAMU email as well as major (or undeclared if not applicable).

Experimental design

We use a complete list of student emails along with major and student unique identifying number to randomly select a 10% sample from each of four grade classifications (classification is based on student credit hour accumulation). A small subset of students (<.5%) were missing an email or student ID and were dropped before randomization. Our final sample size consists of 5,095 undergraduate students.

One week before the Spring 2019 signup period, an email was sent to a selected student advising them that they have been given access to our course tool with a brief description of the tool and link to access it (we also noted in the email that this was part of an IRB approved study). There were four different opening sign up times for students that were based on a classification given by TAMU (classification is a function of credit hour accumulation). Hence, each selected student within a classification group received an email one week before it's opening sign up period. The classification groups are defined as follows: group 1 consists of Undergraduate Non-degree and Freshman (0-29 hours), group 2 consists of Sophomores (30-59 hours), group 3 consists of Juniors (60-89 hours), and group 4 consists of Seniors (90+ hours) and Postbaccalaureate Undergraduates. As a reminder, one week after receiving the initial email a student received a follow up email with the same information as the first email.

Upon clicking the access link in an email, a student would be directed to our custom webpage where we hosted the course tool. Before accessing the tool, a student was asked to enter their TAMU email address and major (or undeclared if not applicable). A student had to complete these fields and click a submit button before each access of the course tool. We placed no limit on the time or number of times a student could access the course tool. As researchers, we can view the emails and majors entered along with a time stamp.

Design details

Randomization method
We randomly select a 10% sample within each of the four grade classification groups a student: Group 1 consists of Undergraduate Non-degree and Freshman (0-29 hours). Group 2 consists of Sophomores (30-59 hours). Group 3 consists of Juniors (60-89 hours). Group 4 consists of Seniors (90+ hours) and Postbaccalaureate Undergrads.
Randomization unit
We randomized students into treatment within each of the four university classification groups.
Clusters
no clusters
Observations
5,095 students in total were sent an email
Arms
The breakdown of 5,095 is as follows: Group 1: 1,102, Group 2: 1,134, Group 3: 1,142, Group 4: 1,717.
Primary outcomes
Course tool usage
Secondary outcomes
Course and major selection in spring 2019. Class registration completion.

Cited by

No citation data available for this registration yet.

Keywords