Machine Learning as a Tool to Detect and Validate Anomalous 2x2 Games
How to cite
Cloléry, Héloïse; Hollard, Guillaume; Prunet, Romain; Baron, Arthur; Smolarski, Hainui; Grosset, Alexandre; Fournier, Malo. 2026. "Machine Learning as a Tool to Detect and Validate Anomalous 2x2 Games." AEA RCT Registry. AEARCTR-0018316. https://doi.org/10.1257/rct.18316-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
A central question in behavioral game theory is whether standard theoretical models can accurately predict human choices in simple strategic environments. Existing work suggests that some families of 2x2 games generate especially large discrepancies between theoretical predictions and observed behavior. These disparities motivate an approach in which machine learning is used not only to fit observed choices, but also to automatically generate 2x2 games specifically designed to expose the vulnerabilities of these theoretical frameworks.
The present study compares standard games and historical anomalies drawn from an existing database (\cite{Complexity2025}) with novel "anomaly" games generated by a machine learning procedure. The contribution is twofold. First, the project evaluates whether machine learning provides a more accurate predictive benchmark than standard models (Nash equilibrium, QRE, and level-k). Second, it tests whether ML-generated games reveal systematic blind spots of these standard models.
The experiment will involve participants recruited via Prolific and a pool of 120 games. These games are divided into three categories: 20 standard games, 20 historical "anomaly" games from the existing database, and 80 "anomaly" games generated by our machine-learning procedure. The games are randomly assigned to four fixed blocks of 30 games (5 standard, 5 database anomalies, and 20 ML anomalies). Each participant will play one block of 30 games. Within each block, game order will be randomized. Participants will make a strategic choice for each game and report perceived difficulty. They will also participate in a lottery game, a donation task, and a short IQ test. The main analysis will test whether database anomalies and ML-generated games produce larger discrepancies between theoretical predictions and observed behavior than standard baseline games.
Intervention
The current experiment constitutes an out-of-sample test of whether games selected by our ML procedure also generate larger prediction errors in a new participant sample.
Each participant will play a series of two-by-two games. To avoid cognitive fatigue, each participant will play a subset of 30 games in total. The games are taken from a pool of 120 two-by-two games divided into three categories: 20 benchmark games classified as standard in the original database, 20 historical anomaly games, and 80 anomaly games generated by our ML algorithm.
Experimental design
Each participant will play a series of two-by-two games. To avoid cognitive fatigue, each participant will play a subset of 30 games in total. The games are taken from a pool of 120 two-by-two games divided into three categories: 20 benchmark games classified as standard in the original database, 20 historical anomaly games, and 80 anomaly games generated by our ML algorithm.
To ensure uniform exposure, the 120 games are divided into 4 fixed blocks using a block-randomization design. Each block contains exactly 30 games (5 standard, 5 database anomalies, and 20 ML anomalies). Participants are randomly assigned to one of these blocks upon entry. Within each assigned block, the order of the 30 games is randomized to prevent sequence effects. Participants are randomly matched with each other, ensuring they face a different player at each game. Each player sees the game as the row player and, therefore, has to choose between playing "Up'' or "Down''. At the end of the survey, 4 games will be drawn at random to determine payment.
At the end of each game, participants are asked to report their perceived difficulty of the game.
After completing the 30 games and rating their perceived difficulty, participants complete a lottery-choice task, a donation task, and a short IQ test to elicit risk aversion, altruism, and cognitive ability, respectively. The lottery task and the donation task are incentivized with payments.
Design details
Cited by
No citation data available for this registration yet.