Human assessment of chatbot misunderstandings
How to cite
Raux, Raphael. 2024. "Human assessment of chatbot misunderstandings." AEA RCT Registry. AEARCTR-0014039. https://doi.org/10.1257/rct.14039-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
We will study the reaction that potential users of an AI chatbot have when exposed to various misunderstandings it made in the context of parenting conversations. We will first look at acquire a measure of "how reasonable", from a human perspective, such misunderstandings are deemed to be. Then, we will study how future user engagement with the chatbot is affected by such reasonableness, by varying the types of conversations potential user are exposed to.
Intervention
The intervention involves two steps, with two separate surveys. First, an "initial labeling" which has a first sample of subjects rate the quality of randomly drawn real user-AI conversations, and establish a measure of the "reasonableness" of AI misunderstandings. The conversations (defined as 1 user query and 1 AI answer) are arranged by "pairs", where in each pair the user queries have the same "intent" (i.e., ask for the same thing) and the AI misunderstands the query in both cases. These misunderstandings vary in how "reasonable" they are, from a human perspective: we will explicitly look to obtain conversation pairs for which there is a large "reasonableness gap" in AI answers. Among the candidate conversation pairs we will rate, we will focus on the subset for which the average gap is largest, and the variance of reasonableness is lowest. Among those, we select a subset of pairs which do not present significant differences in "usefulness" between the 2 parts of the pair. Usefulness is elicited using a highly similar to the one used for reasonableness. We establish a final list of 5 pairs of low-quality conversations (along with 3 high-quality ones), that we use in the second step: the present both a large delta in reasonableness and a small and insignificant delta in usefulness (based on median and averages, rated by current or expecting parents).
The second step will have another subject sample observe conversations among the shortlist we establish in the first step. We will experimentally vary the "reasonableness" of misunderstanding: one arm will see "high-reasonableness" misunderstandings, while the other will see "low-reasonableness" misunderstandings. Subjects will then be given the opportunity to engage with the AI chatbot via a link, and we will measure engagement as the main outcome. To increase the baseline willingness to engage with the chatbot, we will include in both treatments a number of high-quality conversations (where the AI understood the question properly and provided a useful answer), which we will hold constant across treatments. For each conversation, we will elicit beliefs in chatbot performance (incentivized) and trust in the chatbot, which will also be main outcome variables.
Experimental design
The main treatment varies the reasonableness of conversations. All subjects will see a few low-quality conversations (not useful answers) between users and AI, chosen among the suitable pairs. Subjects in the "high" treatment will see the conversation that was rated as more reasonable (relatively speaking), while subjects in the "low" treatment will see the one rated as less reasonable. This is the only treatment difference. A sub-arm may be included where we vary the relative share of high-quality (useful answers) and low-quality (not so useful answers) conversations: the corresponding prediction is that the likelihood of user engagement with AI increases with the number of high-quality conversations seen.
Initial labeling: We start by screening for parents of young children and currently expecting parents, in order for our sample to be similar to the chatbot's user base. After a consent form, subjects see experimental instructions, which describe the task: they will see a number of real conversations in which answers were deemed unhelpful because the AI misunderstood the question. They will have to assess how reasonable the various misunderstandings are from a human perspective. We will make clear and explicit that we are not asking about how useful/helpful the answers are (they are all unhelpful) but rather how reasonable is the misunderstanding. We then run a similar design to elicit "usefulness" of answers and select only the pairs that are rated as equally-useful (not very useful in our case).
Experiment: We start by screening for parents of young children and currently expecting parents, in order for our sample to be similar to the chatbot's user base. After a consent form, subjects see experimental instructions, which describe the task: they will see a number of real user-AI conversations and will have the opportunity to engage with the AI at the end. As explained above, the main treatment will vary the reasonableness of the AI answers in the low-quality conversations. Subjects draw 2 low-quality conversations among a final list of 5.
Sub-arms may vary the relative share of high- and low-quality conversations. The main outcomes are elicited after the conversations: incentivized beliefs in performance and trust, as well as requesting a link to the chatbot or to parenting articles.
The survey will end with usual demographic questions.
Design details
Cited by
No citation data available for this registration yet.