Public Attitudes Towards AI in Children's Social Care: An Online Conjoint Experiment with British Adults
How to cite
Gibbons, Daniel. 2025. "Public Attitudes Towards AI in Children's Social Care: An Online Conjoint Experiment with British Adults." AEA RCT Registry. AEARCTR-0015042. https://doi.org/10.1257/rct.15042-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This trial aims to investigate the trade-off between procedural safeguards and accuracy in how citizens perceive AI systems in the context of children's social care. The relevant procedural safeguards examined are transparency regarding the overall system, transparency in the decision-making process for specific children, and the role of the system in decision-making. The measures of accuracy studied are whether the system reduces false negatives (i.e., how many fewer at-risk children had their cases incorrectly closed) and false positives (i.e., how many fewer safe children had their cases incorrectly progressed). Trade-offs will be studied in terms of the willingness to "pay" for procedural safeguards in relation to each accuracy measure, e.g., how much reduction in false positives will respondents be willing to give up to ensure transparency in the overall AI system? To achieve this, statistical models known as mixed logit models will be employed, which measure the degree to which individuals vary in their willingness to "pay" for accuracy without needing to observe the factors that determine that variation, while also allowing for consideration of whether preferences for the different safeguards are correlated. The differences in willingness to pay associated with trust, both in general institutions (such as local councils and the UK's legal system) and in social workers, will be studied, with trust in social workers broken down into competency, motivation, and resources to undertake their work. As a more exploratory analysis, differences in willingness to pay in terms of age and education level will also be explored.
Intervention
Conjoint experiment, so a table of two AI systems using the following.
Instructions:
Children's services often receive reports, for example, from schoolteachers, about a child who might be at risk of abuse or neglect. The local council wants to use a computer program that learns from past cases to help social workers. This program helps predict if the child might be harmed if no action is taken.
Now, imagine you are a council worker tasked with choosing which program to pick. On the following pages, please indicate which of the two programs shown you would personally prefer to see used by the council.
You will see seven pairs of programs, please choose one from each pair.
Background:
Before the introduction of the program, social workers made correct decisions about 80 children out of every 100 children referred to them.
They made two kinds of mistakes:
"False negatives": 10 children from every 100 referred children were at risk of harm, but were missed by social workers, and their cases were closed.
"False positives": 10 children from every 100 referred children were not at risk of harm, but were incorrectly progressed to further investigation.
Below are descriptions of two programs that could help social workers with screening referrals. Please read the descriptions and indicate which of the two programs shown you would personally prefer to see used by the council.
Attributes:
- Use of the AI system, "How the program is used by social workers in decisions made about children" (levels: The program makes a preliminary decision that the social worker must override to change; The program only makes a recommendation to the social worker)
- Measuring overall transparency, "Information about the program is:" (levels: Easy to understand and publicly available; Hard to understand without expert advice, but publicly available; Not publicly available)
- Measuring transparency in individual decisions "For each specific child, information about how the program made a prediction is:" (levels:
- How many fewer at-risk children had their cases incorrectly closed (levels: 0, 2, 4, 6, 8, 10) Note for context: background includes information that for social workers alone "10 children from every 100 referred children were at risk of harm, but were missed by social workers, and their cases were closed".
- How many fewer safe children had their cases incorrectly progressed (levels: 0, 2, 4, 6, 8, 10) Note for context: background includes information that for social workers alone "10 children from every 100 referred children were not at risk of harm, but were incorrectly progressed to further investigation".
Experimental design
This will be an online conjoint experiment, consisting of six tasks with two randomly assigned options (with a seventh task as the first task, flipped to test for consistency), randomly assigned separately across five sets of attribute levels described above under "Interventions". The order of the attribute levels will be randomised across participants, but the order will be consistent across their set of six tasks.
Design details
Cited by
No citation data available for this registration yet.