Digital Literacy and Well-Being: A Randomized Controlled Trial on Adolescents’ Use of Social Media and Algorithms

How to cite

Sanz, Ismael; Sainz, Jorge; Ballestar, Maria Teresa; Delgado, Manuel. 2025. "Digital Literacy and Well-Being: A Randomized Controlled Trial on Adolescents’ Use of Social Media and Algorithms." AEA RCT Registry. AEARCTR-0016923. https://doi.org/10.1257/rct.16923-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Ismael Sanz* , Jorge Sainz , Maria Teresa Ballestar , Manuel Delgado

* Primary investigator

Registry ID
AEARCTR-0016923
DOI
https://doi.org/10.1257/rct.16923-1.0
First registered
Last updated
Status
In development
Trial dates
2025-10-01 – 2026-02-08
Countries
Private
JEL codes
C93, I21, I12, J13, I31
Registry record
http://www.socialscienceregistry.org/trials/16923

Abstract

Abstract

This Pre-Analysis Plan describes a randomized controlled trial (RCT) evaluating the impact of a digital literacy and well-being intervention among approximately 700 adolescents aged 12–17 in Spain. The program consists of classroom-based workshops on responsible social media use, algorithmic awareness, privacy management, emotional regulation, and bioethical reflection. Randomization follows a dual design: block randomization at the classroom level in 1º–3º ESO (groups A–D), and individual-level randomization within classrooms in 1E–2º Bachillerato. Primary outcomes include measures of mental health (depression symptoms, self-esteem, psychological well-being, sleep quality) and digital literacy (understanding of algorithmic recommendations, ability to identify risky online behaviors, and privacy awareness). Secondary outcomes capture behavioral patterns of online activity, risk perception, and ethical awareness. With 700 students, the study is powered to detect effect sizes as small as 0.12–0.13 SD for pooled analyses with baseline controls. Pre-specified heterogeneity analyses will examine treatment effects by prior internet use, social media habits, gender, and grade level. The findings will provide causal evidence on the effectiveness of digital literacy interventions in strengthening resilience and promoting healthier online behaviors among adolescents.

Intervention

Intervention (Public)

The intervention is a school-based training program for adolescents aged 12–17 in Spain. Students participate in 3–4 classroom workshops (60 minutes each) on responsible social media use, algorithmic awareness, privacy management, emotional regulation, and digital well-being. Sessions combine interactive activities, real-life vignettes, and bioethical reflection to strengthen resilience and build critical digital literacy. Materials are adapted from the European DigComp 2.0 framework and bioethics principles.

The intervention will be implemented during the fall term of 2025 in participating secondary schools. Approximately 700 students are randomized either at the classroom level (block randomization in 1º–3º ESO, groups A–D) or within-classroom (1E, 2E, 3E, 4º ESO, 1º Bachillerato, 2º Bachillerato). Training is delivered by facilitators supported by student assistants from URJC.

The program consists of four modules:

Algorithmic awareness: how platforms like TikTok, Instagram, and Snapchat use recommendation systems to maximize engagement, covering echo chambers, attention manipulation, and privacy concerns.

Digital well-being and emotional regulation: strategies for screen-time management, sleep hygiene, coping with FOMO, and preventing compulsive use; reflection on mental health risks (depression, anxiety, self-esteem).

Privacy and online safety: understanding digital footprints, preventing cyberbullying, managing personal data, and addressing addictive design features.

Bioethics and responsible digital behavior: applying principles of autonomy, beneficence, non-maleficence, and justice to digital contexts, using case studies and classroom debates.

Evaluation is conducted through a randomized controlled trial (RCT) with pre- and post-surveys (Nov 2025 baseline, Jan 2026 follow-up). Validated instruments include ERA-RSI, PHQ-9, Rosenberg Self-Esteem, Ryff Scales, and Pittsburgh Sleep Quality Index. Outcomes cover mental health, digital literacy, risk perception, and ethical awareness

Experimental design

The evaluation follows a Randomized Controlled Trial (RCT).
• Sample: Approximately 700 students aged 12–17, spanning 1º–4º ESO and 1º–2º Bachillerato.
• Intervention: Classroom-based training modules on responsible social media use, algorithmic awareness, emotional regulation, privacy, and AI ethics. Materials are adapted from DigComp 2.0 and bioethics frameworks (Beauchamp & Childress, 2013).
• Duration: 3–4 workshops of 60 minutes, delivered during the fall term 2025.
Empirical Method
Main specification:

Y_ig = α + β * Treatment_ig + γ * X_ig0 + λ_g + ε_ig

where:

Y_ig = outcome of student i in group g
Treatment_ig = indicator for assignment to treatment
X_ig0 = baseline covariates
λ_g = grade (or strata) fixed effects
ε_ig = error term
The main ITT specification will strictly use the full randomization strata fixed effects (λs) which fully capture the classroom-by-grade stratification, instead of just the grade fixed effects (λg) presented for simplicity in Equation (62).
• Balance Checks and Adjustment: Baseline balance will be checked on all covariates (Xig0) across treatment and control groups. We will flag any variable with a standardized difference greater than 0.2 or a p<0.10 from an F-test. If a significant imbalance is detected, the full model will include the unbalanced covariate as a control variable (Xig0) to absorb the pre-treatment difference.
• Missing Covariate Data: For missing data in the baseline covariates (Xig0), we will use mean imputation and include a dummy variable indicating whether the covariate was imputed. Robustness checks will employ listwise deletion or multiple imputation if the rate of missingness exceeds 5%.
• Multiple Hypothesis Correction: The pre-specified families for Holm-Bonferroni control are:
o Family 1 (Mental Health): Depression symptoms (PHQ-9), Self-esteem (Rosenberg), Psychological well-being (Ryff Scales; BSPWB-A), and Sleep quality (Pittsburgh Index).
o Family 2 (Digital Literacy): Understanding of algorithmic recommendations, Ability to identify risky online behaviors, Privacy awareness, and Digital time management (DigComp 2.0 scales).
• Estimation: ITT (intention-to-treat). The choice of an MDE of 0.20 SD is based on meta-analyses of similar digital literacy and mental health interventions in field settings, which typically report effect sizes ranging from 0.10 to 0.35 SD. An effect of 0.20 SD is considered the minimum effect size that is substantively and practically meaningful for educational policy and behavioral change in adolescents
• Robustness: Per-protocol (≥80% participation); alternative specifications with randomization strata fixed effects.
• Multiple hypotheses: Family-wise error control using Holm-Bonferroni within outcome families (mental health, digital literacy, risk perception).
• Heterogeneity: Interaction terms for gender, grade, and baseline digital habits.
• Exploratory analysis: Machine learning classifiers to detect latent subgroups with differential response to treatment.

Design details

Randomization method
Randomization strategy: • 1º, 2º, 3º ESO (groups A–D): Block randomization by class, with two classes per grade in treatment and two in control. • 1E, 2E, 3E, 4º ESO, 1º Bachillerato, 2º Bachillerato: Within-class randomization, where students are individually assigned to treatment or control inside their own classroom. This dual design allows us to compare both between-classroom spillovers and within-classroom peer effects.
Randomization unit
Randomization Unit The unit of randomization varies by grade: For 1º–3º ESO (groups A–D), randomization is at the classroom level (clustered). For 1E, 2E, 3E, 4º ESO, 1º Bachillerato, and 2º Bachillerato, randomization is at the individual level within classrooms. This dual design allows us to study both classroom-level spillovers and within-class peer effects.
Clusters
12 classrooms (cluster-randomized arm). Additionally, we will run individual-level randomization within ~16 classrooms (these are not treatment clusters).
Observations
700 pupils (secondary schools in Spain, ages 12–17).
Arms
Cluster-randomized arm: 6 classrooms treatment (~150 pupils) and 6 classrooms control (~150 pupils). Individually randomized arm (within classrooms): ~200 pupils treatment and ~200 pupils control across ~16 classrooms. Total: ~350 treatment, ~350 control (≈50/50 split).
Primary outcomes
1. Mental health o Social Media addiction (ERA-RSI) o Sleep quality (Pittsburgh Index) o Self-esteem (Rosenberg) o Depression symptoms and psychological well-being (EIDAN) 2. Digital literacy o Understanding of algorithmic recommendations o Ability to identify risky online behaviors o Privacy awareness and digital time management (DigComp 2.0 scales)
Secondary outcomes
Secondary Outcomes • Risk perception: cyberbullying awareness, addictive patterns, perceptions of algorithmic manipulation • Behavioral outcomes: self-reported time online, number of hours on TikTok/Instagram, self-monitoring logs • Ethical awareness: responses to vignettes on beneficence, autonomy and justice in digital contexts

Cited by

No citation data available for this registration yet.

Keywords