BallotBot: Can Chatbots Strengthen Direct Democracy?
How to cite
Opocher, Giacomo; Ash, Elliott; Galletta, Sergio. 2024. "BallotBot: Can Chatbots Strengthen Direct Democracy?." AEA RCT Registry. AEARCTR-0014172. https://doi.org/10.1257/rct.14172-1.2
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
In this study, we investigate the impact of new information technologies on American voters' political knowledge through a two-wave online survey experiment. We developed BallotBot, an interactive chatbot powered by state-of-the-art large language models (LLMs) trained to provide official information about statewide ballot measures. A random sample of California voters is assigned to either use BallotBot or the digital version of the official voter guide to answer questions about upcoming ballot initiatives. We want to compare participants' short- and medium-term political knowledge, as well as the perceived and objective costs of acquiring that knowledge, across the two experimental groups. Additionally, we collect data on voting intentions, turnout behavior, and the reasoning behind participants' vote choices to assess how these factors differ between the groups.
Experimental design
The experiment takes the form of a survey where participants receive an information treatment and are asked to answer a few questions. Participants' responses are collected via Qualtrics, and they are recruited through Prolific.
We will first collect baseline characteristics of all respondents. After that, we will split the participants into two groups:
T1 Group: This group will receive the digital version of the voter guide that the government distributes.
T2 Group: This group will access a simple chatbot (in the form of a web application) that exclusively provides official information from the voter guide and can only answer questions related to the referendum.
Then, we will ask the participants to answer two sets of questions:
– 5 compulsory closed questions on the Propositions’ content. To answer correctly, they can use the chatbot (T2) or the guide (T1), depending on what they are assigned to. We will measure (i) the number of correct answers (knowledge) and (ii) the time they need to answer (objective cost of generating knowledge).
– 1 voluntary closed question on the Propositions’ content. We will ask them what fee (compensation for a correct answer) they would be willing to accept for answering each question, in the style of a BDM task. We will collect (i) the answer, (ii) the time they need to answer, and (iii) the bid (perceived cost of generating knowledge).
– We will encourage them to voluntarily write a question of their interest and look for the answer using the device they were assigned to.
– Finally, we will ask a last set of questions about their willingness to use BallotBot in future elections.
One week later, we will contact the participants again and:
– Repeat some of the quiz questions to measure retained knowledge.
– Ask about their turnout behavior.
– Ask them to write a short text to motivate their vote choice.
See the PAP for further details.
Design details
Cited by
No citation data available for this registration yet.