Competing Behavioral Mechanisms for Notification Timing in a Digital Health Experiment

How to cite

Kapoor, Anuj; Deolankar, Varad; Goli, Ali. 2025. "Competing Behavioral Mechanisms for Notification Timing in a Digital Health Experiment." AEA RCT Registry. AEARCTR-0017273. https://doi.org/10.1257/rct.17273-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Anuj Kapoor* , Varad Deolankar , Ali Goli

* Primary investigator

Registry ID
AEARCTR-0017273
DOI
https://doi.org/10.1257/rct.17273-1.0
First registered
Last updated
Status
In development
Trial dates
2025-12-11 – 2026-01-08
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/17273

Abstract

This study examines whether sending app reminders at the same time every day helps people use a health app more consistently. Before starting the main experiment, we first run a short pilot to make sure everything works smoothly. After that, participants are randomly assigned to one of several groups: some receive no reminders, while others receive one reminder each day either in the morning, afternoon, or evening. A final group receives reminders at different times each day, chosen at random. For everyone, the reminders contain the same simple, neutral message about the benefits of daily monitoring.

The study lasts several weeks and includes a period before and after the reminders so we can compare changes in app usage over time. By tracking how often people use the app in each group, we aim to understand whether timing consistency—receiving reminders at the same time every day—leads to better engagement than reminders sent at varying times or not at all. This evidence can help improve how digital health apps design their notification systems to support healthier habits.

Intervention

The study evaluates a simple question: does the timing of a daily reminder shape how regularly people engage with a health-monitoring app? To answer this, we vary only when a single neutral reminder is delivered, holding the content and frequency constant.

After a brief quiet period with no notifications, participants are randomly assigned to one of five groups:

No-notification control, to benchmark natural usage.

Morning, afternoon, or evening fixed-time reminders, where the reminder arrives at the same time every day.

A variable-time group, where the reminder time shifts daily and is randomly drawn from the three fixed-time options.

Each reminder contains the same short, neutrally phrased message encouraging daily monitoring—nothing tailored or behaviorally charged. Every participant in the reminder conditions receives exactly one message per day for four weeks.

We then return to a period without notifications to see how engagement patterns persist once the reminders stop. This structure allows us to isolate whether consistency or variability in timing plays a meaningful role in sustaining healthy app habits.

Experimental design

This study examines whether delivering a single daily reminder at a consistent time of day, versus a time that varies from day to day, influences how regularly people use a health-monitoring app.

After a short period with no notifications, users are randomly assigned to one of several groups. One group receives no reminders, while the others receive one neutral, non-personalized push notification each day, either at a fixed time (morning, afternoon, or evening) or at a time that changes daily. All users in reminder groups receive the same number of notifications over the intervention period, and the message content remains identical across groups.

The experiment runs for several weeks and includes observation windows before and after the reminder period. This structure allows us to measure baseline behavior, immediate responses to reminder timing, and whether any effects persist once reminders stop. Because the intervention varies only the timing of a simple daily reminder, the design cleanly isolates the role of timing consistency in shaping digital-health engagement.

Design details

Randomization method
Users are randomly assigned to experimental groups using a computerized, reproducible randomization procedure. The allocation is automated within the platform’s backend, ensuring that each eligible user has an equal chance of being placed into any of the study arms. No manual assignment is involved.
Randomization unit
The randomization is conducted at the individual user level. Each user is independently assigned to one of the experimental arms. There are no group- or cluster-level assignments in this study.
Clusters
The study randomizes at the individual user level. We therefore expect approximately 50,000 users to serve as individual clusters.
Observations
Because randomization occurs at the individual user level, the planned number of observations corresponds to the number of participating users. We anticipate approximately 50,000–150,000 users contributing daily engagement data throughout the study period.
Arms
Randomization occurs at the individual-user level. We plan to divide the sample evenly across the five arms: Control (no notifications): ~20% of users Fixed Morning Reminder: ~20% of users Fixed Afternoon Reminder: ~20% of users Fixed Evening Reminder: ~20% of users Variable-Timing Reminder: ~20% of users Exact counts will depend on the final pool of eligible active users at the start of the experiment, but the allocation will be balanced across the five groups.
Primary outcomes
The main outcome of interest is daily engagement with the health-monitoring app, measured through objective behavioral traces recorded by the app. Specifically, we focus on: Probability of taking at least one reading on a given day. This is the core behavioral metric used in the prior experiment (e.g., daily counts of readings shown in the charts on pp. 1–2 of the descriptives report) and provides a clear, interpretable measure of engagement Number of readings taken per day. This captures intensity of engagement, complementing the extensive-margin measure above. The earlier analysis similarly relied on daily reading counts to detect lift during interventions and decay afterward Timing consistency of user engagement. For users exposed to fixed-time versus variable-time nudges, we track whether readings cluster more tightly around the assigned nudge time, relative to baseline patterns. All outcomes are observed continuously throughout the pre-intervention quiet period, the 4-week randomized timing intervention, and the post-intervention washout window. This structure allows us to test whether predictable versus unpredictable reminder timing differentially supports habitual, sustainable app use.

Cited by

No citation data available for this registration yet.

Keywords