Humans versus Bots

How to cite

Ajzenman, Nicolas; Elacqua, Gregory; Jaimovich, Analia; Hincapie, Diana; Perez, Graciela. 2021. "Humans versus Bots." AEA RCT Registry. AEARCTR-0008269. https://doi.org/10.1257/rct.8269-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Nicolas Ajzenman* , Gregory Elacqua , Analia Jaimovich , Diana Hincapie , Graciela Perez

* Primary investigator

Registry ID
AEARCTR-0008269
DOI
https://doi.org/10.1257/rct.8269-1.0
First registered
Last updated
Status
In development
Trial dates
2021-10-31 – 2022-03-31
Countries
Chile (Country)
JEL codes
D9, I2
Registry record
http://www.socialscienceregistry.org/trials/8269

Abstract

Qualified teachers are a fundamental input for any education system. This paper presents the results of a large-scale intervention to attract high-performing high-school students into the teaching profession in Chile. The intervention was a three-arm campaign that made salient three types of motivations: intrinsic/altruistic, extrinsic, and prestige-related. There are two treatment arms. In the first one, a team of promoters from a specialized NGO implemented an intensive campaign based on a structured script implemented through WhatsApp messages and phone calls. In the second one, the campaign was delivered using a pre-programmed Bot that operates through WhatsApp. The third arm is a passive control that does not receive any message. We use real-world behavioral outcomes to measure the effectiveness of each type of intervention. These include the probability that a high-school student applies to a teaching degree, the probability that she includes a teaching degree among the most preferred options, and the probability that a high-school student is finally offered (and accepts) a position in a university to pursue a teaching major.

Intervention

This experiment will be carried out with approximately 24,000 high-school students in their final year who declared on a survey conducted by Elige Educar (an NGO) that they might be interested in pursuing a career in teaching or in social sciences. Students were randomly assigned to one of the two treatment arms (“humans”, “bots”) or to the control group, stratifying by gender, geographic region, and type of school they attended (public, private subsidized, and private non-subsidized).

The content of the messages delivered by each treatment arm was designed to be as similar as possible. The messages are intended to motivate students to pursue a degree in education by appealing to three types of potential motivations: intrinsic related to altruism (related to how teachers can make a difference in the world), extrinsic (teachers have good conditions, low unemployment rate, a good salary) and prestige (teachers are respected in the society). The difference between the treatment arms is how they were delivered and how intense the treatment is.

The "humans" treatment arm is an intensive campaign that relies on the work of human tutors, who send WhatsApp messages, offer students to talk by phone and answer questions. The tutors follow a script, although there is some flexibility based on the questions they receive.

The "bots" treatment arm is a campaign that relies on the work of bot "tutors", who send WhatsApp messages and answer questions using a pre-programmed algorithm. The bots intend to follow a script similar to the one used in the "human" arm, although there is less flexibility.

The control group does not receive any message.

The sample is composed of high-school students that reported to have some interest in Education as a career.

The timeline of the campaign is the following:

- "Humans": they start the campaign in mid-October until the first week of December. They stop for a week and restart for four weeks until mid-January. In total, this campaign lasts for 12 weeks.
- "Bots": they start the campaign in the first week of November until the first week of December. They stop and restart in the first week of January until the second week of January. In total, this campaign lasts for seven weeks.

Experimental design

This experiment will be carried out with approximately 24,000 high-school students in their final year who declared on a survey conducted by Elige Educar (an NGO) that they might be interested in pursuing a career in teaching or in social sciences. Students were randomly assigned to one of the two treatment arms (“humans”, “bots”) or to the control group, stratifying by gender, geographic region, and type of school they attended (public, private subsidized, and private non-subsidized).

The content of the messages delivered by each treatment arm was designed to be as similar as possible. The messages are intended to motivate students to pursue a degree in education by appealing to three types of potential motivations: intrinsic related to altruism (related to how teachers can make a difference in the world), extrinsic (teachers have good conditions, low unemployment rate, a good salary) and prestige (teachers are respected in the society). The difference between the treatment arms is how they were delivered and how intense the treatment is.

The "humans" treatment arm is an intensive campaign that relies on the work of human tutors, who send WhatsApp messages, offer students to talk by phone and answer questions. The tutors follow a script, although there is some flexibility based on the questions they receive.

The "bots" treatment arm is a campaign that relies on the work of bot "tutors", who send WhatsApp messages and answer questions using a pre-programmed algorithm. The bots intend to follow a script similar to the one used in the "human" arm, although there is less flexibility.

The control group does not receive any message.

The sample is composed of high-school students that reported to have some interest in pursuing a career in education.

The timeline of the campaign is the following:

- "Humans": they start the campaign in mid-October until the first week of December. They stop for a week and restart for four weeks until mid-January. In total, this campaign lasts for 12 weeks.
- "Bots": they start the campaign in the first week of November until the first week of December. They stop and restart in the first week of January until the second week of January. In total, this campaign lasts for seven weeks.

Design details

Randomization method
Randomization done in an office by a computer
Randomization unit
Students (no clusters)
Clusters
No clusters.
Observations
Every arm will have approximately 8,000 individuals. No clusters.
Arms
Every arm will have approximately 8,000 individuals. No clusters.
Primary outcomes
Ranked an education major as the first option: takes a one if the student ranked a major in education as the first preference of degree she/he wants to pursue. The proportion of education majors included in the choice set: proportion of options that are majors in education out of the total of degree options selected by the student. Applied to at least one education major: takes a one if the student applied to at least one major in education in her/his set. All outcomes are real-world, high-stakes choices.

Cited by

No citation data available for this registration yet.

Keywords