Understanding teacher perceptions of AI grading recommendations

How to cite

Goulas, Sofoklis; Megalokonomou, Rigissa. 2024. "Understanding teacher perceptions of AI grading recommendations." AEA RCT Registry. AEARCTR-0015004. https://doi.org/10.1257/rct.15004-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Sofoklis Goulas* , Rigissa Megalokonomou

* Primary investigator

Registry ID
AEARCTR-0015004
DOI
https://doi.org/10.1257/rct.15004-1.0
First registered
Last updated
Status
In development
Trial dates
2024-12-16 – 2025-12-16
Countries
Private
JEL codes
I21, C93
Registry record
http://www.socialscienceregistry.org/trials/15004

Abstract

The study involves a survey of teachers, administered online. The questionnaire will present different hypothetical scenarios of student answers to exam questions. Half of the participants will be randomly assigned a 'human first grader' questionnaire, while the other half will receive an 'artificial intelligence grading system' questionnaire. Participants will be asked to re-grade the hypothetical student answer. The study aims to (1) measure differences in teachers’ perceptions of grading recommendations when these are provided by a human first grader vs. an artificial intelligence grading system, and (2) identify the mechanisms that drive teachers’ perceptions of such recommendations.

Intervention

The study involves a survey of teachers, administered online. The questionnaire will present different hypothetical scenarios of student answers to exam questions. Half of the participants will be randomly assigned a 'human first grader' questionnaire, while the other half will receive an 'artificial intelligence grading system' questionnaire. Participants will be asked to re-grade the hypothetical student answer. The study aims to (1) measure differences in teachers’ perceptions of grading recommendations when these are provided by a human first grader vs. an artificial intelligence grading system, and (2) identify the mechanisms that drive teachers’ perceptions of such recommendations.

Experimental design

The study involves a survey of teachers, administered online. The questionnaire will present different hypothetical scenarios of student answers to exam questions. Half of the participants will be randomly assigned a 'human first grader' questionnaire, while the other half will receive an 'artificial intelligence grading system' questionnaire. Participants will be asked to re-grade the hypothetical student answer. The study aims to (1) measure differences in teachers’ perceptions of grading recommendations when these are provided by a human first grader vs. an artificial intelligence grading system, and (2) identify the mechanisms that drive teachers’ perceptions of such recommendations.

Design details

Randomization method
Randomization by computer
Randomization unit
Participant
Clusters
No clustering
Observations
600 participants
Arms
Two primary treatment arms, 300 participants in each arm.
Primary outcomes
The primary outcome will be the grade teachers provide in each hypothetical scenario. The goal is to measure how close teachers' grades are to those of the first grader when the first grader is a human colleague versus an artificial intelligence grading system.

Cited by

No citation data available for this registration yet.

Keywords