BallotBot: Can Chatbots Strengthen Direct Democracy?
How to cite
Ash, Elliott; Galletta, Sergio; Opocher, Giacomo. 2024. "BallotBot: Can Chatbots Strengthen Direct Democracy?." AEA RCT Registry. AEARCTR-0013594. https://doi.org/10.1257/rct.13594-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Are AI technologies complementary to direct democratic institutions? This study provides empirical evidence on this question in the context of Swiss Federal Referendums. The starting point is BallotBot, a newly built GPT-powered chatbot that has access to full-text source material for referendum actions in Switzerland. In a survey experiment to be run at the end of May, we will randomize respondents to gain access to BallotBot (treatment) or a government-provided voter guide (active control). In the survey, respondents undertake a set of survey tasks to answer the following questions: (1) Do chatbots reduce the cost of information acquisition on referendum issues? (2) Does chatbot access increase knowledge about referendum issues? (3) Does chatbot access increase (expected) referendum turnout? These results provide an early indication of how AI technologies will shape democratic institutions in the years to come.
Intervention
We randomly assign a random sample of Swiss voters to use a chatbot or a voter guide to answer a set of closed-ended questions about the issues they vote for on the 9th of June. We measure the time they need to complete the task, the perceived cost, the share of correct answers, how sure they are of each answer, and their willingness to ask further questions. We do this to understand why these two sources of information are different and which one works better in the context of mapping political information into knowledge. Ultimately, we aim to shed light on the potential beneficial role of AI in strengthening direct democracy.
Experimental design
It is a survey experiment. After collecting the baseline characteristics of the participants (demographics, socioeconomic conditions, attitudes toward AI, baseline knowledge of the election matters), we randomize the respondents into two groups. The first group is given the PDF version of the official voter guide (a document that the government distributes to all the voters), and the second group is given an internally developed chatbot trained on the voter guide's text to answer any query about election matters. Respondents answer 5 closed-answer questions concerning election matters using the resource they are randomized into. For each correct answer, they earn 0.2 CHF on top of their participation fee. Then, they can answer an additional question and bid for their remuneration. A random number is drawn. If the number is higher than the bid, the respondent does not take the additional question, otherwise, she takes it and gets paid her bid in case of correct answer. Then, respondents are encouraged to ask a question of their choice to the resource. This is not incentivized. Finally, we collect feedback on the functioning of the chatbot to the chatbot group.
Design details
Cited by
No citation data available for this registration yet.