Machine Learning as a Tool to Detect and Validate Anomalous 2x2 Games

How to cite

Cloléry, Héloïse; Hollard, Guillaume; Prunet, Romain; Baron, Arthur; Smolarski, Hainui; Grosset, Alexandre; Fournier, Malo. 2026. "Machine Learning as a Tool to Detect and Validate Anomalous 2x2 Games." AEA RCT Registry. AEARCTR-0018316. https://doi.org/10.1257/rct.18316-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Héloïse Cloléry* , Guillaume Hollard , Romain Prunet , Arthur Baron , Hainui Smolarski , Alexandre Grosset , Malo Fournier

* Primary investigator

Registry ID
AEARCTR-0018316
DOI
https://doi.org/10.1257/rct.18316-1.0
First registered
Last updated
Status
In development
Trial dates
2026-04-19 – 2026-07-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/18316

Abstract

A central question in behavioral game theory is whether standard theoretical models can accurately predict human choices in simple strategic environments. Existing work suggests that some families of 2x2 games generate especially large discrepancies between theoretical predictions and observed behavior. These disparities motivate an approach in which machine learning is used not only to fit observed choices, but also to automatically generate 2x2 games specifically designed to expose the vulnerabilities of these theoretical frameworks.
The present study compares standard games and historical anomalies drawn from an existing database (\cite{Complexity2025}) with novel "anomaly" games generated by a machine learning procedure. The contribution is twofold. First, the project evaluates whether machine learning provides a more accurate predictive benchmark than standard models (Nash equilibrium, QRE, and level-k). Second, it tests whether ML-generated games reveal systematic blind spots of these standard models.
The experiment will involve participants recruited via Prolific and a pool of 120 games. These games are divided into three categories: 20 standard games, 20 historical "anomaly" games from the existing database, and 80 "anomaly" games generated by our machine-learning procedure. The games are randomly assigned to four fixed blocks of 30 games (5 standard, 5 database anomalies, and 20 ML anomalies). Each participant will play one block of 30 games. Within each block, game order will be randomized. Participants will make a strategic choice for each game and report perceived difficulty. They will also participate in a lottery game, a donation task, and a short IQ test. The main analysis will test whether database anomalies and ML-generated games produce larger discrepancies between theoretical predictions and observed behavior than standard baseline games.

Intervention

The current experiment constitutes an out-of-sample test of whether games selected by our ML procedure also generate larger prediction errors in a new participant sample.

Each participant will play a series of two-by-two games. To avoid cognitive fatigue, each participant will play a subset of 30 games in total. The games are taken from a pool of 120 two-by-two games divided into three categories: 20 benchmark games classified as standard in the original database, 20 historical anomaly games, and 80 anomaly games generated by our ML algorithm.

Experimental design

Each participant will play a series of two-by-two games. To avoid cognitive fatigue, each participant will play a subset of 30 games in total. The games are taken from a pool of 120 two-by-two games divided into three categories: 20 benchmark games classified as standard in the original database, 20 historical anomaly games, and 80 anomaly games generated by our ML algorithm.
To ensure uniform exposure, the 120 games are divided into 4 fixed blocks using a block-randomization design. Each block contains exactly 30 games (5 standard, 5 database anomalies, and 20 ML anomalies). Participants are randomly assigned to one of these blocks upon entry. Within each assigned block, the order of the 30 games is randomized to prevent sequence effects. Participants are randomly matched with each other, ensuring they face a different player at each game. Each player sees the game as the row player and, therefore, has to choose between playing "Up'' or "Down''. At the end of the survey, 4 games will be drawn at random to determine payment.
At the end of each game, participants are asked to report their perceived difficulty of the game.
After completing the 30 games and rating their perceived difficulty, participants complete a lottery-choice task, a donation task, and a short IQ test to elicit risk aversion, altruism, and cognitive ability, respectively. The lottery task and the donation task are incentivized with payments.

Design details

Randomization method
Randomization into block and randomization to match participants are done by a computer.
Randomization unit
Randomizations are at the individual level.
Clusters
200 participants.
Observations
We will have 24,000 observations at the player-game-model level (200x30x4)
Arms
Of all actions: 2/12 will concern standard games, 2/12 anomaly games from the original dataset, 8/12 newly generated anomaly games.
Primary outcomes
Our primary outcome is an accuracy variable equal to 1 whenever the action is correctly predicted by the model, and 0 otherwise. The unit of analysis is the subject–game–model observation.
Secondary outcomes
Games features, perceived games complexity, and response times on each game.

Cited by

No citation data available for this registration yet.

Keywords