Counterfactual Reasoning and Algorithmic Descriptions

How to cite

Bo, Inacio. 2023. "Counterfactual Reasoning and Algorithmic Descriptions." AEA RCT Registry. AEARCTR-0011349. https://doi.org/10.1257/rct.11349-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Inacio Bo*

* Primary investigator

Registry ID
AEARCTR-0011349
DOI
https://doi.org/10.1257/rct.11349-1.0
First registered
Last updated
Status
In development
Trial dates
2023-05-03 – 2024-12-31
Countries
Private
JEL codes
C78, C73, D82, C91
Registry record
http://www.socialscienceregistry.org/trials/11349

Abstract

This project proposes a measure of complexity for mechanisms that is based on their algorithmic descriptions. We formulate the “mental” model followed by participants who are given an algorithmic description of a mechanism as an extensive game-form that reflects the steps of the algorithmic description. Players’ strategies in this extensive-form game reflect the information used from their (direct) strategies in the resulting mechanism in each step. In this framework, the standard direct mechanism can be interpreted as eliciting a strategy for said extensive game-form. We define a family of sequential mechanisms for a given algorithmic description as the set of nodes in which the continuation strategies are elicited. A mechanism A is deemed as less complex than B if during A’s execution, its strategies elicit the actions from a subset of the nodes of B. We test this theory in a controlled laboratory experiment, by studying whether subjects ability to best respond in direct and sequential versions of the Gale-Shapley DA and Boston mechanisms will improve when these are less complex with respect to our measure.

Experimental design

We conduct a matching experiment based on 6 experimental treatments that differ in matching rule and the mechanism used to implement them. They can be described in two phases.

Experiment Phase 1: Perfect information Gale-Shapley DA
We will have three different treatments: DA1, DA3 and DA18. There will be 60 subjects per treatment. All treatments will share the following features:
• The experiment will consist of 20 rounds.
• In each round, a new market consisting of 18 students and 18 schools is created. Each school has a unit capacity. The experimental subject will take the role of student S1, while S2,…,S18 are robot players.
• Students’ payoffs over schools and schools’ priorities over students were created such that under truth-telling, student S1 will be matched to his/her 1st, 3rd, 5th, and 7th highest-paying alternative in five out of the twenty markets each.
• There are four types of schools and four types of students (other than S1). Students/schools of the same type share the same payoff/priority ranking. Student S1 has a payoff ranking different from the other students.
• Subjects will be shown the entire payoff and priority table and will be told that all robot players will rank/choose schools in decreasing order of payoff value.
• At the beginning of each round, subjects will be asked to provide two schools that represent their prediction about which school they will end up matched to.
• Their payoff will consist of the average of two randomly drawn rounds for their assigned schools (which pay between CNY 10 and CNY 180) and two randomly drawn rounds for their predicted matchings (which pay CNY 10).

For DA18, we will ask subjects to submit a full ranking over all 18 schools in each round. The outcome will be the result of the Gale-Shapley DA mechanism with the ranking submitted by the subject as the preference for S1, and the preferences created for students S2,…,S18 as their submissions.

For DA1 and DA3, in each round the subject will act as student S1 in two different instances of the Iterative Deferred Acceptance Mechanism (Bó and Hakimov, 2022) in which every time the subject is asked to act, he/she will submit a choice of a single school (DA1) or a ranking of 3 schools among the ones that did not yet reject him/her (DA3). No feedback other than the fact that the subject was rejected from all previously ranked schools will be given.

Experiment phase 2: Imperfect Information Boston
We will have three different treatments: IIBOS1, IIBOS2 and IIBOS6. There will be 60 subjects per treatment. Each treatment will share the following features:
• The experiment will consist of 20 rounds.
• In each round, a new market consisting of 5 students (S2, S3,…,S6) and 6 schools is randomly created, using a model in which a student s’s utility from being matched to school c is the sum of two values:
o A common value for school s, drawn from the uniform distribution in the range [1,1000].
o An idiosyncratic value, drawn for each student/school pair, drawn from the uniform distribution in the range [0,1000].
• In each round the preference of student S1 will be chosen so that the ranking of his/her outcome under truth-telling is equally distributed between 1,2,3,4 and 5. That is, there will be four markets where under truth-telling student S1 will be ranked to his/her [1,2,3,4,5]th option.
• Each school has a unit capacity. The experimental subject will take the role of student S1, while S2, S3,…,S6 are robot players.
• Subjects will be shown the priority table, will be told the way in which the other students’ preferences were generated, and will be told that all robot players will rank/choose schools in decreasing order of payoff value.
• At the beginning of each round, subjects will be asked to provide two schools that represent their prediction about which school they will end up matched to.
• Their payoff will consist of the average of two randomly drawn rounds for their assigned schools (which pay between CNY 30 and CNY 180) and two randomly drawn rounds for their predicted matchings (which pay CNY 20).

For IIBOS6, in each round we will ask subjects to submit a full ranking over all 6 schools. The outcome will be the result of the Boston mechanism with the ranking submitted by the subject as the preference for S1, and the preferences created for students S2, S3,…,S6 as their submissions.
For IIBOS1 and IIBOS2, in each round the subjects will act as student S1 in two different instances of a sequential version of the Boston Mechanism, as follows:

IIBOS1
• In the first step, every student is asked to choose a school.
• Schools that receive one or more application permanently accepts the highest-ranked one with respect to its priority, and reject the other students.
• Students who were rejected are asked to choose a school among those that did not yet reject him/her, and the procedure continues until every student is accepted at some school.

IIBOS2
• In the first step, every student is asked to submit a ranking with two schools.
• The Boston Mechanism is executed with the submitted rankings.
• Students who are matched to some school are permanently matched to those schools.
• If there are students left unmatched, these are asked to submit a new ranking of two among those that did not yet reject him/her. The Boston mechanism is executed with the preferences submitted by these students in this second step, and where the schools that were matched to some student in the previous step have zero capacity. Students who are matched to some school are permanently matched to those schools.
• If there are students left unmatched, they are asked to submit a new ranking of two among those that did not yet reject him/her. The Boston mechanism is executed with the preferences submitted by these students in this third step, and where the schools that were matched to some student in the previous steps have zero capacity. Students who are matched to some schools are permanently matched to those schools.

Subjects’ Belief elicitation
Before each round in every treatment, the subjects will be asked for a prediction about which school they will end up matched in that round. For DA1, DA3 and DA18, they will have the opportunity of providing two schools. For the other treatments, only one school. If the outcome in that round is one of the schools predicted, the subject will earn an additional CNY 10 (resp. 20 in the Boston) in that round. (Notice that, as the payoffs for the main task, the average of two randomly drawn rounds will be used for the prediction payoff).

Design details

Randomization method
Otree based randomisation.
Randomization unit
Experimental subjects are randomly assigned to one experimental treatment.
Clusters
No clustering is necessary as each subject will be an independent observation.
Observations
20 rounds per subject, 60 subjects per treatment. 1200 (20 rounds x 60 subjects) observations per treatment. This results in 7200 observations (6 treatments) in total.
Arms
We are recruiting 60 Subjects per treatment who will make decisions over 20 experimental rounds each. We are conducting 6 treatments, resulting in a total of 360 subjects (6 treatments x 60 subjects).
Primary outcomes
The main outcomes of interest are the following: 1. In DA18, IIDA6 and IIBOS6, subjects rankings over schools submitted in each round. 2. In DA1, DA3, IIBOS1, and IIBOS2, the partial rankings over schools submitted every time the subject is asked for it, in each round. 3. The predicted assignments for each round.

Cited by

No citation data available for this registration yet.

Keywords