This study investigates how people extract information from data when they have no prior knowledge of the underlying relationship between explanatory variables and the outcome. Participants observe datasets consisting of colored lights (on or off) and a sound, and must figure out a rule (mapping from the lights' status to the sound) to predict when the sound occurs. The study evaluates theoretical predictions about how the statistical properties of a dataset influence rule extraction, with participants recruited online via Prolific from the United Kingdom.
Intervention
This experiment tests a cost–benefit analysis of rule extraction. Participants observe datasets consisting of two or three lights (explanatory variables) that vary in color and can be either on or off, paired with a sound (dependent variable). Participants receive no information about the underlying statistical relationships. We examine how the probability of extracting each rule (i.e. a mapping from the light configurations to the prediction of the sound) depends on the statistical properties of the dataset.
Experimental design
Each participant observes five datasets, each with a unique rule attaining the highest accuracy. Participants first read instructions and complete comprehension checks. They then observe each dataset in turn, taking notes as they wish, before predicting the sound. We collect 12 predictions per dataset in the main setting with 2 lights and 16 predictions for the setting with 3 lights. At the end of the experiment, we also elicit measures of self-confidence and the strategy participants used to solve the tasks.
Design details
Randomization method
Participants are recruited through Prolific in sessions of 50 subjects each. Each session includes five datasets, each associated with an optimal decision rule. The composition of datasets within a session is determined ex-ante to balance cognitive load across participants (e.g., ensuring variation in task difficulty within a session). Participants have no prior knowledge of which datasets they will encounter.
For each dataset, every participant is exposed to a single condition. The between-subjects design means that comparisons across conditions are conducted between participants within each dataset. Condition assignments are balanced to ensure equal sample sizes per condition within each dataset.
Randomization unit
The unit of randomization is the individual participant.
Clusters
0
Observations
We will recruit 250 participants in total. No observations will be excluded from the main analysis among participants who complete the experiment; however, participants who drop out before completion will be replaced by new subjects, and responses with completion times excessively fast will be excluded and similarly replaced. Robustness checks that we will report in the appendix will restrict the sample to: (1) participants whose maximum-likelihood strategy estimate achieves a posterior accuracy above 80%, and (2) participants who pass all comprehension checks. The 250 participants is the target before the robustness checks.
Participants will be recruited on Prolific from the United Kingdom and will be required to be native English speakers, aged 18–50, with at least a high school education (including higher education up to the doctoral level). To ensure data quality, only participants with an approval rate of at least 99% will be included.
Arms
Each of the three treatments with two lights includes 50 participants, with a total of 150. The two treatments with three lights likewise include 50 participants each, bringing the total to 250.
Primary outcomes
The main dependent variable will be the share of subjects extracting each rule, depending on its relative value and the number of explanatory variables (lights).
Secondary outcomes
We also measure participants' confidence and time spent learning.
Cited by
No citation data available for this registration yet.