Across-test human performance in mathematics

How to cite

Raux, Raphael; Dreyfuss, Bnaya. 2023. "Across-test human performance in mathematics." AEA RCT Registry. AEARCTR-0012253. https://doi.org/10.1257/rct.12253-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Raphael Raux* , Bnaya Dreyfuss

* Primary investigator

Registry ID
AEARCTR-0012253
DOI
https://doi.org/10.1257/rct.12253-1.1
First registered
Last updated
Status
In development
Trial dates
2023-10-06 – 2024-09-30
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/12253

Abstract

We study how human subjects perform on a series of math questions sampled from different standardized tests of various difficulty. The study will take place online and recruit subjects from the Prolific platform.

Intervention

We administer a math test online, where subjects attempt to solve 30 randomly-sample questions from 3 standardized tests of various difficulty levels (4th grade, 8th grade, and High school). These tests belong to the TIMSS family of tests, which are administered every four years in several countries. We are restricting our attention to multiple-choice questions, which represent the large majority of available questions and are more directly comparable between them. The size of the question pool will be around 420.

Experimental design

Subjects see a total of 30 questions. Questions are randomly sampled from the total pool of questions. For this stage there are no "treatments".

The goal of this exercise is to create an index of difficulty across all questions, using average success rates. We expect (tautologically) that a given subject' performance will be decreasing with question difficulty (computed with average performance). At the extremes of the ability distribution, a flatter slope can be expected given that performance is bounded between 0 and 1.

Design details

Randomization method
Randomization done through qualtrics survey flow.
Randomization unit
Individual
Clusters
We want to obtain a large enough number of responses for each item, to create a reliable difficulty index. We would need between 20 and 40 responses per question: for 420 questions this means between 280 and 560 subjects approximately.
Observations
Between 280 and 560 subjects
Arms
Between 280 and 560 subjects
Primary outcomes
Subject performance in the test.

Cited by

No citation data available for this registration yet.

Keywords