Role of Quality Signals in Student Evaluations of Teaching

How to cite

Arora, Puneet. 2025. "Role of Quality Signals in Student Evaluations of Teaching." AEA RCT Registry. AEARCTR-0016414. https://doi.org/10.1257/rct.16414-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Puneet Arora*

* Primary investigator

Registry ID
AEARCTR-0016414
DOI
https://doi.org/10.1257/rct.16414-1.1
First registered
Last updated
Status
Completed
Trial dates
2025-07-21 – 2025-12-31
Countries
India ()
Registry record
http://www.socialscienceregistry.org/trials/16414

Abstract

Student Evaluations of Teaching (SET) are one of the most widely used tools for assessing faculty performance in higher education. Despite their popularity, substantial research has documented concerns around their fairness, particularly highlighting that SET scores may reflect student biases rather than objective teaching quality. Gender-based disparities are of particular concern: female faculty members often receive lower ratings than their male counterparts even when teaching performance is equivalent. To counteract such bias, a commonly recommended strategy is for instructors, especially from underrepresented groups, to signal their professional quality through their academic qualifications, research accomplishments, and work experience. This approach is believed to offset the effects of stereotypes and provide students with a more complete view of the faculty member's merit.

However, one potential concern is whether such quality signaling has differential effects depending on the gender of the faculty member and the student. While signaling may help faculty from disadvantaged groups like women, the same strategy may backfire when used by faculty from traditionally advantaged groups such as men. In particular, there may be a perception that quality signals from male faculty constitute flaunting, which could adversely affect SET scores. Moreover, student gender may further moderate these effects, e.g. female students may be more sensitive to such dynamics than male students. This study conducts an experimental study to examine this interaction between faculty and student gender, and the impact of quality signaling, on SET scores in a higher education setting. I also examine the underlying mechanisms driving such interactions, if any.

Intervention

Information about faculty credentials/quality shared with some, while not shared with others.

Experimental design

The SETs will be conducted in the middle of the course (economics and other management courses, done only once with a cohort-section), and when students click on the survey, they'll be randomly assigned to one of two experimental conditions using Qualtrics’ built-in randomization algorithm:

• Information Treatment: Students in this group will first view a slide presenting detailed information about the faculty member’s academic and professional credentials. This may include the faculty's PhD institution, research publications, industry consultancy experience, teaching awards, and other notable achievements.

• No-Information Treatment Group: Students in this group will proceed directly to the evaluation without receiving any additional information about the faculty member.

Design details

Randomization method
Qualtrics in built randomization algorithm
Randomization unit
Student level
Clusters
500+
Observations
1000 (as against initial plan to do it in 2025 first year cohort, I am extending data collection to 2026 first year cohort as well, increasing the numbers from already collected 493 in 2025 to approximately 1000)
Arms
500 each
Primary outcomes
SETs
Secondary outcomes
Underlying mechanisms

Cited by

No citation data available for this registration yet.

Keywords