Assess@Learning

How to cite

Azzolini, Davide; Marzadro, Sonia; Rettore, Enrico; Tondini, Alessandro. 2022. "Assess@Learning." AEA RCT Registry. AEARCTR-0009314. https://doi.org/10.1257/rct.9314-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Davide Azzolini* , Sonia Marzadro , Enrico Rettore , Alessandro Tondini

* Primary investigator

Registry ID
AEARCTR-0009314
DOI
https://doi.org/10.1257/rct.9314-1.0
First registered
Last updated
Status
On going
Trial dates
2021-03-01 – 2022-12-31
Countries
Private
JEL codes
I20, C93
Registry record
http://www.socialscienceregistry.org/trials/9314

Abstract

A growing number of studies highlight the potential of formative assessment for improving student achievement and increasing students' self-regulated learning. New technologies offer the possibility to enhance the scope and effective employment of formative assessment practices in schools. However, little is known about what interventions effectively increase whole-schools' systematic and purposeful adoption of digital formative assessment (DFA) practices. The A@L trial evaluates a newly developed online toolkit aimed at increasing lower secondary education schools' readiness to implement digital formative assessment. The toolkit is made available to a sample of school heads, teachers and students across five European Union Member States (Estonia, Finland, Greece, Portugal and Spain). A sample of 208 schools have been sampled and recruited across the five countries and randomly assigned to either the treatment or the control group. Participants in the treatment group are invited to use the toolkit between December 2021 and April 2022. The trial evaluates the effects of the intervention on a set of short-term primary outcomes collected on teachers and related to their awareness, attitudes and knowledge about DFA. Secondary outcomes analyzed include school heads’ awareness, attitudes and intention to support DFA practices; teachers’ actual use of DFA; students’ attitudes toward DFA, learning experience, motivation and self-perceived performance. Outcome data are collected through online surveys administered at baseline and at follow-up (May 2022).

Intervention

The treatment consists of an online toolkit (A@L Systemic Toolkit), which is made available to school heads, teachers, students in lower secondary schools, as well as to their parents, according to their different needs. The toolkit is also made available to policy makers in participating countries, if they requested to explore its content. The goal of the Toolkit is to show how digital formative assessment (DFA) can be integrated in school, teaching and learning practices in order to empower students to self-direct their learning. For each target group, a specific landing page helps to get introduced to what DFA is and why one should use it and what the role of each target group in DFA is (including policy makers). These landing pages are meant to be concise and engaging, they include interview videos, animations, infographics and cartoons. Teachers can find inspiration in the teaching scenarios section, a gallery with a wide range of DFA practices, ranging from online quizzes/classroom polling to concept maps and e-portfolios and instructional design models for a given subject and a given situation. School heads are invited to check out the Case Studies section where numerous challenges to DFA and the enablers that make it possible are illustrated. School heads can also browse the Theory of change section that is designed to help them in introducing new approaches to teaching and learning by helping them define goals and the steps to achieve them. Finally, there are glossaries for tools that support DFA and definitions with many terms related to that topic.

Experimental design

The trial follows a parallel two-arms design.

Schools assigned to the treatment group receive access to the A@L Toolkit, while those assigned to the control condition do not receive access. However, the Toolkit will be accessible to them after the follow-up survey has taken place.

The unit of randomization is the single school. The allocation ratio is .49 (T= 103; C= 105).

Design details

Randomization method
Randomization has been done using the statistical package Stata. To improve statistical precision, a stratified randomization design is implemented. Two variables, on top of country, are used in the stratified randomization procedure. Given that our primary focus is on teachers, we investigated what variables seem to be most correlated with our outcomes of interest at the teacher level: i.e., the share of teachers who make use of FA as indicated by the principal; and the number of devices available to students and teachers over the population of students and teachers in the school. Hence, the randomization was performed for 4 groups within each country: above/below median of digital devices and above/below median of share of teachers who perform FA. Median values are calculated within countries.
Randomization unit
The unit of randomization is the single school.
Clusters
208 schools, 188 with at least one teacher, 174 with at least one student.
Observations
208 school heads 894 teachers in 188 schools 2,589 students in 174 schools
Arms
T= 103; C= 105
Primary outcomes
The following 8 teacher outcomes are considered as primary outcomes: Dimension 1 Awareness about DFA Outcome 1.1 Awareness about assessment approaches and roles in assessment Outcome 1.1 Conception of assessment Dimension 2 Attitudes toward DFA Outcome 2.1 Attitudes toward FA Outcome 2.2 Attitudes toward DFA Dimension 3 Knowledge about DFA Outcome 3.1 Self-reported DFA competence Outcome 3.2 DFA-specific knowledge (quiz) Dimension 4 Teachers' Use of DFA Outcome 4.1 Adoption of (and intention to adopt) FA practices Outcome 4.2 Use of (and intention to use) digital tools in assessment
Secondary outcomes
The following secondary outcomes are analyzed on school heads, teachers and students: School head outcomes (Awareness about assessment approaches and roles in assessment; Recognition of DFA potential; presence of a DFA strategy in the school; professional development opportunities in DFA offered by the school) Teachers' participation (and intention to participate) in training on DFA Student outcomes (Attitudes toward DFA, Attitudes toward FA, Learning experience, Study motivation, Self-perceived performance)

Cited by

No citation data available for this registration yet.

Keywords