How Context Shapes Teachers’ Use of AI Grading
How to cite
Megalokonomou, Rigissa; Goulas, Sofoklis; Sotirakopoulos, Panagiotis. 2025. "How Context Shapes Teachers’ Use of AI Grading." AEA RCT Registry. AEARCTR-0016662. https://doi.org/10.1257/rct.16662-2.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Schools are beginning to deploy artificial-intelligence (AI) tools that suggest grades or feedback, yet little is known about how context and explanations shape teachers’ use of these recommendations. We study this with an anonymous, preregistered online survey experiment of K–12 teachers. Participants grade brief, realistic vignettes and are randomly assigned to (i) no AI suggestion (control), (ii) an AI grade recommendation, or (iii) the same recommendation accompanied by task/rubric context and a concise rationale (explainability). For a subset of vignettes we manipulate the accuracy of the AI suggestion, enabling tests of calibration (accepting correct and rejecting incorrect advice). Primary outcomes are AI uptake/override, accuracy-conditioned uptake, perceived trust, fairness/transparency, workload, and the between-teacher variance of assigned grades. We further explore heterogeneity by experience, subject area, and prior AI use. The design isolates whether contextual and explanatory cues help teachers discriminate between good and bad AI advice and whether such cues improve perceptions without increasing cognitive burden. Findings will inform the design and training of human-in-the-loop assessment systems that support—rather than substitute for—professional judgment.
Intervention
K–12 teachers complete a short, anonymous online survey (~9 minutes) with brief grading vignettes. For each vignette, teachers are randomly assigned to one of three conditions: (1) Control – no AI input; (2) AI recommendation – an on-screen suggested grade; (3) AI + context/explanation – the same suggestion plus brief task/rubric context and a one-sentence rationale. Teachers choose a grade and answer a few items on usefulness, fairness/transparency, and workload. No identifying information is collected, and no real student records are involved. Participation is voluntary.
Experimental design
Experimental Design (Public)
Anonymous online survey–experiment with K–12 teachers (~9 minutes). Each participant grades several short, realistic student-work vignettes. For each vignette, the platform randomly assigns one of three conditions: (i) Control (no AI input), (ii) AI recommendation (an on-screen suggested grade), or (iii) AI + context/explanation (the same suggestion plus brief task/rubric context and a one-sentence rationale). Order of vignettes is randomized.
Teachers choose a grade and answer a few brief questions on usefulness, fairness/transparency, and workload. Primary comparisons are across the three conditions (intent-to-treat). No identifying data are collected and no real student records are involved; participation is voluntary.
Design details
Cited by
No citation data available for this registration yet.
Keywords
- behavior
- education
- teachers
- grading
- assessment
- Artificial Intelligence
- Explainable AI
- decision support
- context
- framing
- calibration
- Trust
- adoption
- human-in-the-loop
- education technology
- survey experiment
- randomized controlled trial
- vignettes
- fairness
- Transparency
- reliability
- workload
- rubric
- feedback
- moderation
- heterogeneity
- preregistration