About TIME (Technology in Math Education): Experiments 1 & 2

How to cite

Liu, Haoxuan; Kalil, Ariel; Mayer, Susan. 2022. "About TIME (Technology in Math Education): Experiments 1 & 2." AEA RCT Registry. AEARCTR-0009345. https://doi.org/10.1257/rct.9345-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Haoxuan Liu* , Ariel Kalil , Susan Mayer

* Primary investigator

Registry ID
AEARCTR-0009345
DOI
https://doi.org/10.1257/rct.9345-1.1
First registered
Last updated
Status
Completed
Trial dates
2022-04-18 – 2022-09-30
Countries
United States of America (Illinois)
Registry record
http://www.socialscienceregistry.org/trials/9345

Abstract

About TIME (Technology in Math Education) explores how technology can best be used to promote the acquisition of math skills among young children and reduce the gap in skill development between high- and low-income children. We hypothesize that digital apps may improve young children’s math skills over the long run compared to similar non-digital learning materials both because children are likely to spend more time using digital apps compared to non-digital learning materials and because digital apps may be more efficient at conveying math skill. About TIME will proceed in two parts. The first is a series of field experiments to help us understand how technology works to promote learning (corresponding to this registration). The second is a large-scale randomized controlled trial intended to compare the increase in math skills among preschool-age children who are provided with a tablet containing high-quality math apps and a group of children provided with high-quality non-digital materials of the sort recommended by experts in math learning and intended to promote the same skills as the digital apps. The results of the About TIME Project will: 1. compare the increase in math skills for children provided with a digital tablet containing high-quality math apps to the increase for children provided with high-quality non-digital materials intended to convey the same skills, 2. estimate the extent to which parent characteristics and barriers to parent engagement influence the effect of both digital and non-digital learning tools in building child math skill, 3. estimate how parent involvement moderates the effect of digital apps relative to non-digital materials on children’s math skills, 4. estimate how digital apps affect children’s and parents’ enthusiasm for math, 5. describe which apps are used the most, when are they used, and which features of apps are most often used by children.

Intervention

Experiment 1 (4 treatment arms):
Parents will be told that the goal of the session is for children to learn a specific skill, for example matching number symbols to quantities or naming large number symbols. Parent-child pairs will be randomly assigned to receive apps or non-digital learning materials. Parents will then be randomly assigned within the two modalities, either to be present or absent for the experiment. Holding the time constant, any difference in skill acquisition between the beginning and end of the experiment will be due to differences in the efficiency of the materials in the two scenarios where the parent is either present or absent.

Experiment 2 (2 treatment arms):
In this experiment, we randomly assign parents to a treatment in which we temporarily increase their cognitive load or to a treatment in which we do not increase cognitive load. There are standard methodologies for temporarily increasing cognitive load, such as asking participants to count backward by 7s from 100 and other similar tasks that tax cognition. As in experiment 1, we test how much children learn from digital or non-digital materials. Because low-income parents are more likely to have a high cognitive load, this can also potentially explain why low-income parents engage less with their children in math activities.

Experimental design

The intent of the field experiments is to test several potential explanations for how digital apps and non-digital materials influence math learning. There are two main experiments. Experiment 1 asks whether digital apps are more efficient at conveying math skills and what role the parent plays in skill development. Experiment 2 asks how parents’ cognitive load affects children’s learning with digital and non-digital materials.

The experiments have the same general format in that parent-child pairs will come to the experimental site, consent to the experiment, be randomized to a treatment group or a control group, and participate in a 30–45-minute experiment. Children will be given a brief assessment of a specific math skill at the beginning and end of the experiment and the difference in score will be the main outcome for each experimental comparison. Each experiment will also assess children’s enthusiasm for learning, measured by whether when given the chance children choose to spend more time on another math activity. All experiments will be live coded by researchers and video recorded for auditing purposes.

We will survey parents at baseline and at the end of the intervention to collect data on family demographic characteristics, parents’ reports of the home learning environment, parents’ beliefs and expectations about their children’s math skill development, and parents’ expected return on their time and material investments in children’s math skill development. We will also ask questions that measure parents’ math anxiety and growth mindset and parents’ reports of the structural, behavioral, and cognitive barriers that stand in the way of their child’s math skill development. These will be used as covariates in the analysis of changes in assessment scores. We will manage the equal distribution of low vs. high income into those groups by stratifying on income within the randomization procedure.

Design details

Randomization method
Parent-child dyads recruited across 15 institutional sites were randomly assigned to a treatment condition immediately upon recruitment.
Randomization unit
Our unit of randomization is parent-child dyads
Clusters
We don't design clusters, so the planned number of clusters is 900 parent-child dyads.
Observations
900 parent-child dyads
Arms
75 low/75 high-income parent-child dyads in each of the six treatment arms mentioned in the Intervention section
Primary outcomes
The efficiency of digital v. non-digital math learning materials in conveying numeracy skill to preschool-aged children analyzed by parent socio-economic status; Role of parent engagement in conveying math skill with digital v. non-digital math learning materials. Both outcomes hold time, discreet skill being conveyed, and pedagogy constant
Secondary outcomes
Qualitative analysis of parent-child interactions; math enthusiasm

Cited by

No citation data available for this registration yet.

Keywords