BallotBot: Can Chatbots Strengthen Direct Democracy?

How to cite

Ash, Elliott; Galletta, Sergio; Opocher, Giacomo. 2024. "BallotBot: Can Chatbots Strengthen Direct Democracy?." AEA RCT Registry. AEARCTR-0013594. https://doi.org/10.1257/rct.13594-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Elliott Ash* , Sergio Galletta , Giacomo Opocher

* Primary investigator

Registry ID
AEARCTR-0013594
DOI
https://doi.org/10.1257/rct.13594-1.1
First registered
Last updated
Status
Completed
Trial dates
2024-05-17 – 2024-06-09
Countries
Switzerland ()
JEL codes
D72, D83, L82
Registry record
http://www.socialscienceregistry.org/trials/13594

Abstract

Are AI technologies complementary to direct democratic institutions? This study provides empirical evidence on this question in the context of Swiss Federal Referendums. The starting point is BallotBot, a newly built GPT-powered chatbot that has access to full-text source material for referendum actions in Switzerland. In a survey experiment to be run at the end of May, we will randomize respondents to gain access to BallotBot (treatment) or a government-provided voter guide (active control). In the survey, respondents undertake a set of survey tasks to answer the following questions: (1) Do chatbots reduce the cost of information acquisition on referendum issues? (2) Does chatbot access increase knowledge about referendum issues? (3) Does chatbot access increase (expected) referendum turnout? These results provide an early indication of how AI technologies will shape democratic institutions in the years to come.

Intervention

We randomly assign a random sample of Swiss voters to use a chatbot or a voter guide to answer a set of closed-ended questions about the issues they vote for on the 9th of June. We measure the time they need to complete the task, the perceived cost, the share of correct answers, how sure they are of each answer, and their willingness to ask further questions. We do this to understand why these two sources of information are different and which one works better in the context of mapping political information into knowledge. Ultimately, we aim to shed light on the potential beneficial role of AI in strengthening direct democracy.

Experimental design

It is a survey experiment. After collecting the baseline characteristics of the participants (demographics, socioeconomic conditions, attitudes toward AI, baseline knowledge of the election matters), we randomize the respondents into two groups. The first group is given the PDF version of the official voter guide (a document that the government distributes to all the voters), and the second group is given an internally developed chatbot trained on the voter guide's text to answer any query about election matters. Respondents answer 5 closed-answer questions concerning election matters using the resource they are randomized into. For each correct answer, they earn 0.2 CHF on top of their participation fee. Then, they can answer an additional question and bid for their remuneration. A random number is drawn. If the number is higher than the bid, the respondent does not take the additional question, otherwise, she takes it and gets paid her bid in case of correct answer. Then, respondents are encouraged to ask a question of their choice to the resource. This is not incentivized. Finally, we collect feedback on the functioning of the chatbot to the chatbot group.

Design details

Randomization method
Qualtrics randomization mechanism.
Randomization unit
Individual.
Clusters
/
Observations
1000 individuals
Arms
/
Primary outcomes
Share of correct answers, degree of confidence, objective cost of answering the questions, perceived cost of answering, willingness to ask further questions, and time spent on the resource for answering further questions.
Secondary outcomes
We collect some feedback on the functioning of the chatbot, from the experimental group that received it to explore how they compare it with the voter guide.

Cited by

No citation data available for this registration yet.

Keywords