This preregistered experiment investigates how two organizational factors -- trust in an AI forecasting tool and accountability for AI‑generated errors -- shape individuals’ reliance on algorithmic advice. Participants complete a forecasting task where they predict product demand using two numerical features, with or without the aid of an AI tool. We independently manipulate: (1) Trust, via information about the AI’s development, training data, and reliability; and (2) Accountability, via information about who is responsible for errors generated by the AI tool (the participant vs. an external Analytics Team responsible for monitoring and correcting the AI). In a subsequent experiment, we further decompose the accountability construct into three sub-dimensions -- liability (who bears blame), auditability (whether performance is monitored), and contestability (whether users can challenge AI outputs) -- to identify which specific governance feature drives reliance. Primary outcomes include weight on advice (WoA) and forecast accuracy. This preregistration specifies the hypotheses, sample size, exclusion criteria, and analysis plan in advance of data collection.
Experimental design
We independently manipulate: (1) Trust, via information about the AI’s development, training data, and reliability; and (2) Accountability, via information about who is responsible for errors generated by the AI tool (the participant vs. an external Analytics Team responsible for monitoring and correcting the AI). In a subsequent experiment, we further decompose the accountability construct into three sub-dimensions -- liability (who bears blame), auditability (whether performance is monitored), and contestability (whether users can challenge AI outputs) -- to identify which specific governance feature drives reliance.
Design details
Randomization method
Data is collected through Prolific, which randomizes the treatments over stratified groups (e.g., by gender) of individuals.
Randomization unit
Individual participant.
Clusters
This trial is not clustered. The intervention is implemented at the individual level.
Observations
Experiment 1
Planned total: approximately 400 participants (approximately 80 per arm across 5 arms)
Per-arm target: 75-100
Experiment 2
Planned total: approximately 400 participants (approximately 80 per arm across 5 arms)
Per-arm target: 75-100
Arms
Experiment 1
Planned total: approximately 400 participants (approximately 80 per arm across 5 arms)
Per-arm target: 75-100
Experiment 2
Planned total: approximately 400 participants (approximately 80 per arm across 5 arms)
Per-arm target: 75-100
Primary outcomes
Primary dependent variable:
Weight on Advice (WoA):
Defined only in conditions with AI advice (T1–T4). For each forecasted product:
WoAi=FinalForecasti−InitialForecastiAIForecasti−InitialForecasti\text{WoA}_i = \frac{\text{FinalForecast}_i - \text{InitialForecast}_i}{\text{AIForecast}_i - \text{InitialForecast}_i}WoAi=AIForecasti−InitialForecastiFinalForecasti−InitialForecasti
For cases where the AI forecast equals the initial forecast (denominator = 0), we will mark WoA as missing for that observation. We will also consider bounding WoA to [0,1] or excluding extreme values (see Analysis Plan).
We will use either:
The average WoA per participant across all products, or
A mixed-model specification with trial-level WoA and random effects for participants.
The preregistered primary specification will use average WoA per participant as the main DV.
Secondary dependent variables:
Forecast error: absolute percentage error (APE)
APEi=100×∣FinalForecasti−TrueDemandiTrueDemandi∣\text{APE}_i = 100 \times \left|\frac{\text{FinalForecast}_i - \text{TrueDemand}_i}{\text{TrueDemand}_i}\right|APEi=100×TrueDemandiFinalForecasti−TrueDemandi
and participant-level average APE.
Bonus attainment: indicator = 1 if participant’s average APE corresponds to ≥ 95% “accuracy” threshold according to the bonus rule; 0 otherwise.
Initial forecast error (to check whether AI improves accuracy relative to T0).
Cited by
No citation data available for this registration yet.