Matching Children with Level-Appropriate Books and Engaging Families

How to cite

Aguilar, Arturo; Ortega, Maria Elena. 2017. "Matching Children with Level-Appropriate Books and Engaging Families." AEA RCT Registry. AEARCTR-0001034. https://doi.org/10.1257/rct.1034-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Arturo Aguilar* , Maria Elena Ortega

* Primary investigator

Registry ID
AEARCTR-0001034
DOI
https://doi.org/10.1257/rct.1034-1.0
First registered
Last updated
Status
On going
Trial dates
2015-12-14 – 2017-06-15
Countries
Private
JEL codes
D61, I25, I28, O31
Registry record
http://www.socialscienceregistry.org/trials/1034

Abstract

Reading skills are an important predictor of educational attainment and, consequently, of other long-term outcomes. School curriculum often follows the “one size fits all” rule, which means that every student within a classroom is assigned the same reading materials, regardless of their reading skills. This practice potentially leads to adverse effects for students in the extremes of the reading abilities’ distribution, which is particularly negative for those falling behind. In this project, we test an alternative system to select reading material for early grade readers: we build a web-based platform that gives book recommendations tailored to readers’ abilities and interests; participants can borrow books for free at the community library. We also test whether parental involvement could complement this strategy. We implement a randomized control trial to assess the potential benefits of this intervention on students’ reading proficiency and habits.

Intervention

The presented intervention, hereon "Mundo de Libros" (MdL), program aims to improve reading skills and habits of primary students enrolled in Grades 1 to 3 in Spanish-speaking countries. This free program is independent of their school curriculum and runs in community libraries. The intervention has three core components:
1. Access to children’s books at the library and that can be taken home. Children have a program passport – similar to a library card – that allows them to borrow books and keep track of due-dates. Each library is equipped with a stock of 720 children’s books. The book collection is diverse in terms of difficulty and topics, thus ensuring that every child will have a wide spectrum of choices. All books in the project are new at the time of the start of the intervention.
2. Access to an individual profile (with login and password) through the web-based platform (www.mundodelibros.mx). Each child’s profile on the platform gives them personalized book recommendations according to their assessed level of vocabulary and reading skills. These recommendations are determined by QfD’s MATCH algorithm that takes into account the reading level of each child and the characteristics of each book. The reading level of the child is determined with an initial test that includes the EGRA and Peabody Picture Vocabulary standardized tests. After logging into the site, participants see recommendations, are able to filter titles according to interests, and search for specific titles, authors or words. The website also allows users to rate the books (on a scale of one to five) after returning them, which is later used to give recommendations based on their interest.
3. Workshops and materials for parents or caregivers. The main objectives of these workshops and materials is to: (i) promote parental engagement, (ii) provide information and strategies to scaffold their children’s reading practices, and (iii) advise how to create a rich literacy environment at home. Workshops for parents or caregivers take place every two months.

Experimental design

To evaluate the effect of the components of the “Mundo de Libros” program, the implementation sites (libraries) were divided in two groups based on parental workshops. Five libraries were selected to receive bi-monthly workshops for parents/guardians and five were selected to act as control libraries regarding the parental workshop component. Then, within each library, half of the participating children were randomly assigned to have access to the adequate-level book recommendations (using the matching algorithm) through the Internet-based platform. The other also had access to the platform but were selected to receive random recommendations (i.e. not matched to their abilities and interests). All the libraries and students have access to the same stock of 721 books and to the same web-based platform, including the loan history and book search tools. To summarize, 4 experimental groups are:
[A] Parent workshop & adequate-level book recommendations
[B] Parent workshop & random book recommendations
[C] No parent workshop & adequate-level book recommendations
[D] No Parent workshop & random book recommendations

Using this design, the effect of receiving adequate-level book recommendations will be assessed through a weighted average of the comparisons [A] to [B] and [C] to [D] groups. Similarly, the workshop component (research question 2) could be evaluated through a weighted average of the comparisons between [A] - [C] and [B] - [D]. Assignment in this case is not individual since the likelihood of contamination is higher. If complementarities between the parent workshop and the MATCH algorithm exist, it is likely that the effect of comparing [A] to [B] will be higher than that comparing [C] to [D].

Qualitative analysis and fidelity of implementation (FOI) will complement the quantitative assessment and will be used as the main strategy to evaluate potential benefits of the different components of the program.

Design details

Randomization method
Randomization of libraries to parental workshop condition was done in office using a computer random number generator. Within each library, children were randomly assigned to treatment (adequate-level recommendations) or control in office using coarsened exact matching. Children were matched based on two variables: (i) sex and (ii) mother's level of education. Pairs of children were formed and within each pair, one of the children was assigned treatment and the other control. Unassigned children were randomly assigned to treatment or control.
Randomization unit
We conducted randomization at 2 different levels: For the parental workshop treatment, the unit of randomization were libraries (cluster). For the adequate-level book recommendations treatment, the unit of randomization within each library were children (individual).
Clusters
10 libraries
Observations
786 children in two registration rounds* * Preliminary numbers. This will be modified in a subsequent round where we estimate a last round of close to 80 children to be added.
Arms
Parent Workshop: 5 treatment libraries, 5 control libraries. By treatment arm*: [A] Parent workshop & adequate-level book recommendations: 235 children [B] Parent workshop & random book recommendations: 241 children [C] No parent workshop & adequate-level book recommendations: 158 children [D] No Parent workshop & random book recommendations: 152 children * Preliminary numbers. This will be modified in a subsequent round where we estimate a last round of close to 80 children to be added.
Primary outcomes
To measure the impact of the program we use three different instruments, to be collected at beseline and endline: EGRA, TVIP, and a Reading Habits and Attitudes survey. a) EGRA. To measure different reading skills, from phonemic awareness to reading comprehension, the following EGRA standard subtasks, adapted into Spanish, were used: (1) Letter-sound Knowledge, (2) Initial Sound Identification, (3) Familiar Word Reading, (4) Non-word Reading, (5) Oral Reading Fluency (ORF), (6) Reading Comprehension. In addition, we determined it would be valuable to make two modifications. The first was to add two inferential questions to the subtask associated with the timed ORF subtask, which normally contains five Reading Comprehension questions. The second modification was to include two additional untimed subtasks: Adaptive Oral Reading Fluency (AORF) and Adaptive Reading Comprehension. These two subtasks help to better differentiate the reading comprehension level of the sample population. The adaptation of EGRA to the linguistic context and the pilot testing of the AORF and Adaptive Reading Comprehension subtasks were conducted by MetCuantus, a psychometrics consulting firm. The adapted EGRA was piloted by QFD with 225 children in June 2015 in one preschool and three public primary schools in the State of Mexico with children enrolled in the last grade of preschool through Grade 3. b) TVIP. To measure receptive (hearing) vocabulary acquisition, QFD administered the Pearson’s Spanish version of the Peabody Picture Vocabulary Test. Known as the Test de Vocabulario en Imagenes Peabody (TVIP) in Spanish, the test is applied in the following manner: an assessor says a stimulus word, and the child responds by pointing to one of the full-color pictures displayed on the test easel. A benefit of using the TVIP is that it can be individually administered from the age of two because it does not require reading or verbal or written responses. The TVIP used during the baseline assessment contains 125 stimulus words specific to Spanish vocabulary and norms in Mexico. c) SURVEY. To measure reading habits and attitudes of children, QFD designed and piloted a short Reading Habits and Attitudes survey. The survey includes yes/no questions and four-Likert scale questions focused on personal and family literacy activities and behaviors. For the endline, this survey was complemented with questions focused on the implementation of the program and basic context questions.

Cited by

No citation data available for this registration yet.

Keywords