Lab experiment to investigate individual preference for getting information generated by AI and the explanation of how the AI made that prediction. The experiment is done in a real-stakes decision where a Black-Box AI informs a decision to allocate actual US$10, 000-loans. I also investigate how and why people value explanations, and how behavioral factors might affect such valuations.
Intervention
Individuals make a decision to determine how a private lender allocate two $10000-loans (approve both or approve one). Participants are randomized into a neutral treatment where payoffs are not tied to actual loan repayment outcome, or a "lender-aligned" treatment where they have with direct stakes tied to loan repayment. A secondary experiment will elicit willingness-to-pay for explanations.
The actual date of the experiments will be early September 2025, it will be on Prolific. They will be run separately as 2 experiments.
Experimental design
I experimentally test how individual's preference for information regarding an AI is influenced by certain factors.
(Full design attached and randomization flow attached in this pre-registration as pdf/docs before the experiment launch)
Design details
Randomization method
Randomization done using Qualtrics' in-built function
Randomization unit
individual
Clusters
See below
Observations
2,500 for primary experiment; 1000-1600 for secondary (information for secondary attached in separate pre-reg, sample size and qualtrics for secondary posted there before secondary is run)
Arms
Evenly for non-lender-aligned and for lender-aligned for main arm. 1250 each.
3/4 will be in the arms where they choose whether or not to see the explanation: Same shares for each arm mentioned above plus a variant of the main arm where race-gender descriptors for borrowers are replaced by odd/even number phone no.
1/4 same as last one but subjects will consider whether to see the AI prediction/recommendation first
Secondary: experiment to elicit WTP for explanation in a task to guess whether a previous borrower actually repaid. uncorrelated with the previous:
1/4 each for complex model with private information, complex model with no private information, simple model with private information, simple model with no private information
then, 1/2 where contingent reasoning related to the use of private information is shown and illustrated in detail and 1/2 where no such hints are given (to see if people fail to contingent reason about the value of explanation in light of private information)
**** Please review attached full Qualtrics survey file for full primary experiment (as secondary experiment for the same paper is run in a later day, that will be registered separately and before that secondary experiment is run)
Primary outcomes
(1) Binary decision of whether the participant want to see an explanation of how the AI made the prediction of the default risk of the borrowers before making a loan allocation decision
(2) Binary decision of whether the participant want to see the predicted default risk of the borrowers before making a loan allocation decision
Secondary outcomes
(3) Whether the participant want to accept the AI recommendation and only approve one loan; or to override AI and just approve both (or if they made the decision before seeing explanation in arm where they choose to see recommendation or not, the first would be to approve one random borrower)
(4) How much, to reduce the second participant’s bonus at a personal cost of $0.01 per $1.00 reduction, up to the full $10 (3 scenarios -informed, not informed, chose not to be informed about explanation) - within subject
(5) What willingness to pay to see an explanation (BDM) in secondary experiment where the task is to predict whether a previous borrower defaulted (this is in secondary experiment, Qualtrics attached in different pre-reg and posted before that experiment is run)
Cited by
No citation data available for this registration yet.