Teacher Biases and AI
How to cite
Goulas, Sofoklis; Tajali, Hedier; Megalokonomou, Rigissa. 2026. "Teacher Biases and AI." AEA RCT Registry. AEARCTR-0017679. https://doi.org/10.1257/rct.17679-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Artificial intelligence (AI) tools are increasingly used in education, including for providing feedback and supporting assessment. This randomized experiment studies how access to an AI chatbot affects teachers’ grading decisions and potential biases when marking real student work. Teachers recruited to an online Qualtrics study are each assigned one authentic student script consisting of a question prompt and handwritten responses. The experiment randomizes two dimensions: (i) the name attached to the student work, which serves as a signal of student characteristics, and (ii) whether teachers have access to an AI assistant while marking. One-third of teachers mark the script without AI support, one-third receive access to an untrained AI chatbot, and one-third receive access to a trained AI assistant that is randomly designed to provide either systematically “fair” or systematically “harsh” guidance.
The study has two primary objectives. First, it estimates the extent to which student name cues affect teachers’ evaluations and grades for identical work. Second, it tests whether AI support changes teachers’ perceptions, grading behavior, and potential bias, and whether trained AI guidance can reduce grading errors or attenuate name-based disparities relative to both no-AI grading and untrained AI access. Follow-up survey measures collect information on teachers’ perceptions of the script, confidence, decision-making process, and use of the AI tool, allowing exploration of mechanisms underlying any observed effects. The study contributes to evidence on how AI tools interact with human judgment and bias in high-stakes educational evaluation.
Experimental design
This study is an online randomized experiment implemented via Qualtrics with a sample of teachers. Each participant is asked to mark one authentic student script consisting of a prompt and handwritten responses. The experiment uses a randomized factorial design with two components. First, teachers are randomly assigned to receive an otherwise identical script labeled with different student names. Second, teachers are randomly assigned to one of three marking conditions: (i) marking without access to any AI tool, (ii) marking with access to an untrained AI chatbot, or (iii) marking with access to a trained AI assistant. In the trained-AI condition, participants are further randomly assigned to receive one of two versions of the assistant. After grading, teachers complete a short follow-up survey measuring perceptions of the student work and the marking process. The primary outcomes are teachers’ assigned grades and related perception measures, and the design allows estimation of how name cues and AI access affect grading decisions and potential disparities.
Design details
Cited by
No citation data available for this registration yet.