Digital Literacy and Well-Being: A Randomized Controlled Trial on Adolescents’ Use of Social Media and Algorithms
How to cite
Sanz, Ismael; Sainz, Jorge; Ballestar, Maria Teresa; Delgado, Manuel. 2025. "Digital Literacy and Well-Being: A Randomized Controlled Trial on Adolescents’ Use of Social Media and Algorithms." AEA RCT Registry. AEARCTR-0016923. https://doi.org/10.1257/rct.16923-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Abstract
This Pre-Analysis Plan describes a randomized controlled trial (RCT) evaluating the impact of a digital literacy and well-being intervention among approximately 700 adolescents aged 12–17 in Spain. The program consists of classroom-based workshops on responsible social media use, algorithmic awareness, privacy management, emotional regulation, and bioethical reflection. Randomization follows a dual design: block randomization at the classroom level in 1º–3º ESO (groups A–D), and individual-level randomization within classrooms in 1E–2º Bachillerato. Primary outcomes include measures of mental health (depression symptoms, self-esteem, psychological well-being, sleep quality) and digital literacy (understanding of algorithmic recommendations, ability to identify risky online behaviors, and privacy awareness). Secondary outcomes capture behavioral patterns of online activity, risk perception, and ethical awareness. With 700 students, the study is powered to detect effect sizes as small as 0.12–0.13 SD for pooled analyses with baseline controls. Pre-specified heterogeneity analyses will examine treatment effects by prior internet use, social media habits, gender, and grade level. The findings will provide causal evidence on the effectiveness of digital literacy interventions in strengthening resilience and promoting healthier online behaviors among adolescents.
Intervention
Intervention (Public)
The intervention is a school-based training program for adolescents aged 12–17 in Spain. Students participate in 3–4 classroom workshops (60 minutes each) on responsible social media use, algorithmic awareness, privacy management, emotional regulation, and digital well-being. Sessions combine interactive activities, real-life vignettes, and bioethical reflection to strengthen resilience and build critical digital literacy. Materials are adapted from the European DigComp 2.0 framework and bioethics principles.
The intervention will be implemented during the fall term of 2025 in participating secondary schools. Approximately 700 students are randomized either at the classroom level (block randomization in 1º–3º ESO, groups A–D) or within-classroom (1E, 2E, 3E, 4º ESO, 1º Bachillerato, 2º Bachillerato). Training is delivered by facilitators supported by student assistants from URJC.
The program consists of four modules:
Algorithmic awareness: how platforms like TikTok, Instagram, and Snapchat use recommendation systems to maximize engagement, covering echo chambers, attention manipulation, and privacy concerns.
Digital well-being and emotional regulation: strategies for screen-time management, sleep hygiene, coping with FOMO, and preventing compulsive use; reflection on mental health risks (depression, anxiety, self-esteem).
Privacy and online safety: understanding digital footprints, preventing cyberbullying, managing personal data, and addressing addictive design features.
Bioethics and responsible digital behavior: applying principles of autonomy, beneficence, non-maleficence, and justice to digital contexts, using case studies and classroom debates.
Evaluation is conducted through a randomized controlled trial (RCT) with pre- and post-surveys (Nov 2025 baseline, Jan 2026 follow-up). Validated instruments include ERA-RSI, PHQ-9, Rosenberg Self-Esteem, Ryff Scales, and Pittsburgh Sleep Quality Index. Outcomes cover mental health, digital literacy, risk perception, and ethical awareness
Experimental design
The evaluation follows a Randomized Controlled Trial (RCT).
• Sample: Approximately 700 students aged 12–17, spanning 1º–4º ESO and 1º–2º Bachillerato.
• Intervention: Classroom-based training modules on responsible social media use, algorithmic awareness, emotional regulation, privacy, and AI ethics. Materials are adapted from DigComp 2.0 and bioethics frameworks (Beauchamp & Childress, 2013).
• Duration: 3–4 workshops of 60 minutes, delivered during the fall term 2025.
Empirical Method
Main specification:
Y_ig = α + β * Treatment_ig + γ * X_ig0 + λ_g + ε_ig
where:
Y_ig = outcome of student i in group g
Treatment_ig = indicator for assignment to treatment
X_ig0 = baseline covariates
λ_g = grade (or strata) fixed effects
ε_ig = error term
The main ITT specification will strictly use the full randomization strata fixed effects (λs) which fully capture the classroom-by-grade stratification, instead of just the grade fixed effects (λg) presented for simplicity in Equation (62).
• Balance Checks and Adjustment: Baseline balance will be checked on all covariates (Xig0) across treatment and control groups. We will flag any variable with a standardized difference greater than 0.2 or a p<0.10 from an F-test. If a significant imbalance is detected, the full model will include the unbalanced covariate as a control variable (Xig0) to absorb the pre-treatment difference.
• Missing Covariate Data: For missing data in the baseline covariates (Xig0), we will use mean imputation and include a dummy variable indicating whether the covariate was imputed. Robustness checks will employ listwise deletion or multiple imputation if the rate of missingness exceeds 5%.
• Multiple Hypothesis Correction: The pre-specified families for Holm-Bonferroni control are:
o Family 1 (Mental Health): Depression symptoms (PHQ-9), Self-esteem (Rosenberg), Psychological well-being (Ryff Scales; BSPWB-A), and Sleep quality (Pittsburgh Index).
o Family 2 (Digital Literacy): Understanding of algorithmic recommendations, Ability to identify risky online behaviors, Privacy awareness, and Digital time management (DigComp 2.0 scales).
• Estimation: ITT (intention-to-treat). The choice of an MDE of 0.20 SD is based on meta-analyses of similar digital literacy and mental health interventions in field settings, which typically report effect sizes ranging from 0.10 to 0.35 SD. An effect of 0.20 SD is considered the minimum effect size that is substantively and practically meaningful for educational policy and behavioral change in adolescents
• Robustness: Per-protocol (≥80% participation); alternative specifications with randomization strata fixed effects.
• Multiple hypotheses: Family-wise error control using Holm-Bonferroni within outcome families (mental health, digital literacy, risk perception).
• Heterogeneity: Interaction terms for gender, grade, and baseline digital habits.
• Exploratory analysis: Machine learning classifiers to detect latent subgroups with differential response to treatment.
Design details
Cited by
No citation data available for this registration yet.