Time Pressure on Tests and Gender Gaps in STEM

How to cite

Livingston, Jeffrey; Gneezy, Uri; Sadoff, Sally; List, John. 2023. "Time Pressure on Tests and Gender Gaps in STEM." AEA RCT Registry. AEARCTR-0012104. https://doi.org/10.1257/rct.12104-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Jeffrey Livingston* , Uri Gneezy , Sally Sadoff , John List

* Primary investigator

Registry ID
AEARCTR-0012104
DOI
https://doi.org/10.1257/rct.12104-1.0
First registered
Last updated
Status
On going
Trial dates
2023-09-04 – 2024-05-31
Countries
Private
JEL codes
C93, I21
Registry record
http://www.socialscienceregistry.org/trials/12104

Abstract

This study will conduct an experiment investigating if time pressure on important standardized tests that are used in admissions decisions (the SAT math test and the GRE Quantitative Reasoning test) has differential impacts by gender. If so, tight time constraints may serve as an artificial barrier that prevents qualified women from advancing in STEM careers.

Student subjects take exams under two different time conditions: the normal time allowed on the exam, and 50 percent extended time. We then estimate whether easing time pressure has a bigger impact on the test scores of women than men, and see how easing time pressure affects how men and women rank on the test, particularly in the upper part of the distribution.

Intervention

All subjects take a full section of a GRE Quantitative Reasoning (GRE-Q) test. The test consists of 25 questions. There are two treatment conditions:

1. The test is taken with the normal time allowed for a similar section on the actual GRE-Q, which is 40 minutes.
2. The test is taken with extended time granted to students needing accomodations for a similar section on the actual GRE-Q, which is 60 minutes.

Experimental design

The design is between-subject. Students are randomized into one of the two treatment conditions described above by first blocking on gender, then randomizing into the treatment groups, and checking on the following:
1. how score on an 8 question GRE-Q pretest is balanced across the two treatments
2. how score on this pretest is balanced across the two treatments for women in the sample
3. how score on this pretest is balanced across the two treatments for men in the sample

We then rerandomize until the p-value of each of these differences is greater than 0.9.

Students are recruited by recruiting STEM major class sections to participate. The experiment is conducted in a class session in order to maximize attendance.

The exam is administered in Qualtrics. Students are emailed the link to the test that has the correct time allotted for the treatment condition to which they have been assigned. It is taken on their own laptops in their normal classroom. Each student is provided with a calculator and scratch paper for use in the exam. The Qualtrics instrument automatically ends the test when the allotted time is expired. The test page has a timer which shows the amount of time remaining that is visible in the upper left corner of the student's screen throughout the test.

Students were paid electronically when possible via services such as Venmo and Zelle. Students who have no electronic payment method available and cannot create an account are paid in cash. Before the test, students enter their account information. They are paid immediately after the conclusion of their session.

After the test, students are given a short survey that asks questions about the level of motivation and pressure they felt while taking the test.

Design details

Randomization method
Students are randomized into one of the two treatment conditions described above by first blocking on gender, then randomizing into the treatment groups, and checking on the following: 1. how score on an 8 question GRE-Q pretest is balanced across the two treatments 2. how score on this pretest is balanced across the two treatments for women in the sample 3. how score on this pretest is balanced across the two treatments for men in the sample We then rerandomize until the p-value of each of these differences is greater than 0.9.
Randomization unit
Students are randomized into treatment conditions at the individual level.
Clusters
n/a
Observations
128 in the initial wave for experiment 3 described in the attached preanalysis plan. We hope to generate 500 total observations.
Arms
250 per condition
Primary outcomes
raw and standardized (within sample) scores on the test score ranking on the test (within sample)
Secondary outcomes
Students are given a survey after taking the test that asks several questions about theeir motivation to do well on the test, pressure they felt when taking the test, and whether they felt they had enough time to complete the test. Each of these was answered on a 5-point Likert scale. We will explore how treatment (easing time pressure) affected each of these variables, and will control for them in some specifications of regressions with the primary outcomes listed above as the dependent variables.

Cited by

No citation data available for this registration yet.

Keywords