Developing Simple Methods for Artificial Intelligence Learning and Education (SMaILE) through a gamified educational app: an RCT with middle school pupils in Italy

How to cite

Bernardini, Sara; Como, Giacomo; Tondini, Alessandro; Ballatore, Maria Giulia. 2023. "Developing Simple Methods for Artificial Intelligence Learning and Education (SMaILE) through a gamified educational app: an RCT with middle school pupils in Italy." AEA RCT Registry. AEARCTR-0011192. https://doi.org/10.1257/rct.11192-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Sara Bernardini* , Giacomo Como , Alessandro Tondini , Maria Giulia Ballatore

* Primary investigator

Registry ID
AEARCTR-0011192
DOI
https://doi.org/10.1257/rct.11192-1.0
First registered
Last updated
Status
On going
Trial dates
2022-11-22 – 2023-12-31
Countries
Italy (Piemonte)
Registry record
http://www.socialscienceregistry.org/trials/11192

Abstract

Artificial Intelligence (AI) has increasingly become part of our everyday life. Therefore, it is crucial that young adults start familiarizing themselves with it early. The successful use of AI for humanity strongly relies on the abilities and competencies of the people who develop, implement, and use it. Thus, a fundamental prerequisite to addressing the profound changes that our society faces is educating people with solid digital competencies, among which AI plays a central role. Progressively it will become crucial to understand the mechanisms behind it, and attitudes and knowledge of how AI works. We evaluate a randomized educational intervention that aims to familiarize middle school pupils with AI concepts and stimulate interest in STEM related subjects and careers. A randomly selected sub-sample of classes are granted access to an innovative app for mobile devices (phones and tablets) - “SMaILEApp”. The app aims to teach complex AI techniques to pupils through gamification, breaking down complex concepts into straightforward applications, accessible to a broad spectrum of young users. In line with gamification techniques, the App uses points, credits, rewards, challenges and virtual goods to ensure maximum engagement of the users. The SMaILEApp thus allows students to learn AI by playing. The app is an educational macro-game that contains micro-games, each of which focuses on a specific AI topic, e.g. machine learning, planning, optimization, etc.

A primary objective of this app is to change attitudes towards AI: the students learn that there are multiple AI sub-areas, each with its strengths and weaknesses, instead of treating AI as a monolithic magic “black box”.

SMaILE app as an educational device has been tested on 57 middle-school classes (second grade of middle school) in 20 schools in the Piemonte region, in Northern Italy. Within each school, half of the classes are randomly assigned the app from the 2022-23 academic year, a pre and post test are carried out, and administrative information from school registries will be collected at the end of the school year to complement the analysis. We plan to measure how this app changes attitudes and perceptions towards AI, and, in particular, whether students show a more informed view of the role of AI in society, after understanding it better.

Intervention

The SMaILEApp aims to teach complex AI techniques to pupils through gamification, breaking down complex concepts into straightforward applications, accessible to a broad spectrum of young users. SMaILEApp starts from simplified real-world problems that could benefit from AI solutions and assists users in gradually developing the required building blocks required to design and implement such a solution. The main modality of the game is the city building. Users are given the role of the mayor and are tasked with building a city through a game of positioning tiles. The aim is to make the city as sustainable as possible by building efficient parks, roads, schools, hospitals and other facilities. Once the city is built and populated, a high level of sustainability needs to be maintained. The mayor can do this by instructing the various aldermen, who are the AI engines that assist him/her. In line with gamification techniques, the app uses points, credits, rewards, challenges and virtual goods to ensure maximum engagement of the users. The SMaILEApp thus allows students to learn artificial intelligence by playing. The app is an educational macro-game that contains micro-games, each of which focuses on a specific AI topic, e.g. machine learning, planning, optimization, etc.

Experimental design

The treatment was performed at the class-level within each school, with at least one control and at least one treated class per school. No school participated with less than two classes. The sample of schools was recruited on a voluntary basis by the Ufficio Scolastico Regionale for the Piemonte Region from the population of middle schools in the region. The treated classes received the SMaILE App starting from December 2022 and no later than January 2023, while the control schools did not receive anything, although the teachers were made aware that these classes would also receive the app at the end of the school year.

Design details

Randomization method
Randomization done in office by computer.
Randomization unit
Class-level randomization stratified by school.
Clusters
57 classes, 20 schools. The cluster is the class (unit of randomization).
Observations
The classes participating in the evaluation have a total of 1192 students; 1031 students obtained parental consent to participate in the evaluation; 995 filled in the baseline survey.
Arms
28 control classes (504 control students), 29 treated classes (527 treated students)
Primary outcomes
AI attitudes AI theoretical knowledge AI reasoning Heterogeneous treatment effects will be systematically explored by gender.
Secondary outcomes
Attitudes towards a STEM career High school track intentions Mental rotation abilities Middle school grades in math and science (if made available by the school) Environmental identity
Sponsors
Name: Fondazione Compagnia di San Paolo Location: Torino, Italy Url: https://www.compagniadisanpaolo.it/
Partners
Name: Ufficio Scolastico Regionale per la Regione Piemonte Partner_type: government Url: http://www.istruzionepiemonte.it/

Cited by

No citation data available for this registration yet.

Keywords