AI Assistance and Student Problem Solving: The Role of Timing and Accuracy
How to cite
Mumcu, Ipek; Kubilay, Elif. 2026. "AI Assistance and Student Problem Solving: The Role of Timing and Accuracy." AEA RCT Registry. AEARCTR-0018411. https://doi.org/10.1257/rct.18411-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This study explores how access to AI-generated solutions influences problem-solving behavior and performance in 16–18-year-old students. Participants are randomly assigned to different conditions based on when they receive AI help and whether the AI solution is correct or incorrect, plus a no-AI control group. They complete an incentivized task with AI access and a second task without it to measure effects on independent performance.
Intervention
The intervention consists of providing participants with access to an AI-generated solution while completing a simple problem-solving task. The intervention varies along two dimensions: the timing of access to the AI-generated solution and the accuracy of the solution provided. Participants may receive access either immediately or delayed, and the AI-generated solution may be either correct or incorrect. Participants in the pure control group receive no AI assistance while solving the problem.
Experimental design
The experimental design follows a 2×2 between-subjects framework with an additional pure control condition. Participants are randomly assigned to conditions that vary along two dimensions: the timing of AI access (immediate versus delayed) and the accuracy of the AI-provided solution (correct versus incorrect), or to a control condition with no AI access. Participants are randomly assigned at the individual level with equal probability across all five conditions. Participants complete a simple, incentivized problem-solving task. In the immediate access condition, participants can request an AI-generated solution at any point from the start of the task. In the delayed access condition, the AI-generated solution is not immediately accessible and becomes available during the course of the task, with the point of access standardized across all participants in this condition. Within each timing condition, participants are randomly assigned to receive either a correct or an incorrect AI-generated solution upon request. Participants in the control condition are not given any access to AI assistance and are required to solve the task independently. This results in five groups defined by the combination of access timing, solution accuracy, and the absence of AI access.
Group 1: Pure control (no AI access)
Group 2: Immediate access – correct solution
Group 3: Immediate access – incorrect solution
Group 4: Delayed access – correct solution
Group 5: Delayed access – incorrect solution
All participants are first presented with a problem and instructed to solve it within a fixed time limit. After completing the first task, all participants proceed to a second problem. In this second stage, all participants are treated identically regardless of their initial assignment. They are given a fixed time limit to solve the problem and are not provided with any AI assistance. Both stages are incentivized.
Design details
Cited by
No citation data available for this registration yet.