Transforming Economics Examinations: A Token-Based System for Enhanced Learning and Student Success

How to cite

Lagoudakis, Angelos; Elobeid, Amani; Kim, Beomyun. 2025. "Transforming Economics Examinations: A Token-Based System for Enhanced Learning and Student Success." AEA RCT Registry. AEARCTR-0016691. https://doi.org/10.1257/rct.16691-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Angelos Lagoudakis* , Amani Elobeid , Beomyun Kim

* Primary investigator

Registry ID
AEARCTR-0016691
DOI
https://doi.org/10.1257/rct.16691-1.0
First registered
Last updated
Status
In development
Trial dates
2025-09-22 – 2026-05-31
Countries
Private
JEL codes
A22, C90, I20
Registry record
http://www.socialscienceregistry.org/trials/16691

Abstract

This project introduces a token-based examination support system designed to transform
traditional assessment practices in higher education. Students will receive three types of
tokens that can be exchanged during exams for different levels of standardized hints, ranging
from basic concept reminders to detailed guidance. This innovative approach addresses multiple challenges in higher education by
enhancing student control to reduce test anxiety, offering flexible support options to
accommodate diverse learning needs, and promoting strategic decision-making to enhance
student learning and performance. The system is particularly beneficial in quantitative courses
where mathematical anxiety and varying learning speeds often impact performance. Unlike
traditional standardized accommodations, our approach enables students to customize their
support based on individual needs while maintaining academic rigor. Although initially
implemented in economics courses, the system's design allows for seamless adaptation across
STEM disciplines and any field requiring quantitative problem-solving skills.

Intervention

Every student in the class will have the chance to use tokens to access standardized forms of assistance, such as conceptual hints during one of the course’s exams.
If they participate, students will:
• Be asked to complete reflective journals and focus group interviews regarding using the token system during the exam.
• Permit us to use their data on Canvas, their token use data (e.g., when tokens were used, which exam questions they were applied to, and what type of hint or support was selected), and their responses to the reflective journals and focus groups. This combined dataset will allow us to assess token access and use’s performance and learning effects.
A reflective journal will be a Qualtrics survey implemented online and should take them 5-7 minutes. They may take them in the privacy of their home or another setting of their choosing. A focus group interview should take them 5-7 minutes and will be implemented in person on a prearranged date and time. We expect the participants to attend the focus groups and complete the reflective journals by the deadline, and we will give them about a week to do so in each case. These assignments will not count toward their Econ 1010 grade.

Experimental design

Our evaluation strategy integrates quantitative and qualitative methods to assess the token-
based examination support system. Traditional randomized crossover designs require
participants to receive identical treatments in different sequences with washout periods (to
eliminate prior effects) between treatments (Jones and Kenward, 2014). We employ a
"modified" randomized crossover design where tests on different modules effectively eliminate
carryover effects while maintaining experimental rigor. This adaptation is necessary in
educational settings where identical tests cannot be repeated. Using our teaching database, we
will calibrate test difficulties across modules.
Design Structure:
1. Midterm 1 (Module 1)
- Group A: With tokens
- Group B: Without tokens
2. Midterm 2 (Module 2)
- Group A: Without tokens
- Group B: With tokens
3. Final Exam (Module 1+2)
- All students: Without tokens
Our design identifies two causal effects using student-level observations: (1) direct
performance effect through treatment-control comparisons in each midterm, and (2) learning
effect by comparing final exam performance on specific modules between students who
previously used tokens for that module and those who did not. While order effects cannot be
identified in a single experiment, repeated implementations across courses would enable such
identification. Using standardized scores ensures fair evaluation while controlling for difficulty variations.

Design details

Randomization method
Randomization will be done in office by a computer
Randomization unit
Individual students
Clusters
280
Observations
280
Arms
140 per group. Since the experiment will be conducted in two semesters, we plan to have: 40 per group in Fall 2025 semester and 100 per group in the Spring 2026 semester.
Primary outcomes
(1) direct performance effect through treatment-control comparisons in each midterm (i.e., midterm exam score), and (2) learning effect by comparing the final exam performance on specific modules between students who previously used tokens for that module and those who did not.
Secondary outcomes
(1) Direct performance effect estimated using both exam-specific regressions and within-student fixed effects models to compare token and no-token conditions across midterms, controlling for midterm difficulty and student fixed effects. (2) Learning effect estimated on final exam module-specific scores using regression models with module fixed effects, assessing whether prior token exposure for a module predicts higher performance on that module in the final. (3) Additional analyses include treatment–covariate interactions (e.g., baseline GPA, diagnostic scores, demographic/background variables) to explore heterogeneous treatment effects. (4) Models incorporate pre-specified covariates (academic records, demographic/background data, standardized test scores, and other IRB-approved sources) and apply multiple testing adjustments, clustered standard errors, and robustness checks.

Cited by

No citation data available for this registration yet.

Keywords