This study examines how people develop trust in AI-driven resource allocation systems leveraging housing photos as a context. Using a hypothetical scholarship allocation scenario that uses AI to rank applicants based on photos of their house, we explore how the photos-based approach can help people understand both AI’s strengths and potential biases. We investigate how different types of AI errors affect trust across participants from varying socio-economic backgrounds. This work provides evidence-based insights for organizations considering AI adoption in high-stakes resource allocation decisions.
Experimental design
We plan to administer a survey where we show respondents various examples of how a hypothetical AI-based tool can rank housing imagery to assess socio-economic status and we ask respondents about their perceptions. After completing a baseline survey administered to all participants, respondents will be randomly assigned to one of four arms: one control arm and three treatment arms (see below). Within each arm, respondents will view two images along with ground truth rankings. The sequence in which these two images are presented will be randomized at the respondent level.
Design details
Randomization method
After completing a baseline survey administered to all participants, respondents will be randomly assigned to one of four arms: one control arm and three treatment arms (see below). Within each arm, respondents will view two images along with ground truth rankings as shown above. The sequence in which these two images are presented will be randomized at the respondent level.
Randomization unit
Unit of randomization: individual.
Clusters
Treatment is not clustered.
Observations
600 students
Arms
We will aim to allocate participants evenly across study arms. With one control arm and three treatment arms, each arm will comprise 25% of the overall sample.
Primary outcomes
1. Based on this additional information, how likely are you to recommend that University U expands the piloting of this AI tool for scholarship decisions?
2. The AI ranks are accurate (5-point agreement scale)
3. I am confident in the AI tool’s rankings (5-point agreement scale)
4. The AI tool may seriously misjudge some applicants (5-point agreement scale).
5. The AI tool can ensure wealthy applicants with low reported income are excluded (5-point agreement scale).
6. The AI tool can ensure all poor applicants are included (5-point agreement scale).
7. Using the AI tool is fair (5-point agreement scale).
Secondary outcomes
1. Consistency: AI applies the same criteria to every application (5-point importance scale).
2. Scalability: AI can assess many applicants at low cost (5-point importance scale).
3. Visual cues: AI can extract useful information from photos not reflected in application forms (5-point importance scale).
4. Irrelevant cues: AI may be misled by lighting or photo quality (5-point importance scale).
5. Use of neighborhood information: Only the applicant's home should be considered, and not the surroundings (5-point importance scale).
6. Privacy: Using publicly available Street View photos may give AI excessive access to personal data (5-point importance scale).
7. Would you be willing to have your own home evaluated by this AI tool as part of a financial-aid process?
Cited by
No citation data available for this registration yet.