Learning MORE (Model of Reading Engagement) Science and Social Studies Content and Vocabulary to Improve Third-Graders' Reading Comprehension at Scale: A Randomized Controlled Trial
How to cite
kim, james. 2021. "Learning MORE (Model of Reading Engagement) Science and Social Studies Content and Vocabulary to Improve Third-Graders' Reading Comprehension at Scale: A Randomized Controlled Trial." AEA RCT Registry. AEARCTR-0008337. https://doi.org/10.1257/rct.8337-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
With a randomized field trial involving 112 Kindergarten to grade 5 elementary schools (N = 9,844 third-graders), we test the effectiveness of the MORE (Model of Reading Engagement) science and social studies intervention. We examine the hypothesis that building domain and topic knowledge schemas, through thematic content literacy instruction and a personalized literacy app, could improve third-graders’ ability to comprehend science and social studies passages. The literacy app is designed to improve students’ knowledge of words related to the themes in the teacher-directed lessons and to build students’ meta-linguistic skills. The study is implemented at scale in a southeastern US urban school district. Schools are first grouped into randomization blocks based on prior experience with the intervention, school size, and terciles of baseline reading scores and then randomly assigned to a control group which implemented science lessons and a literacy app with science content only or a treatment group which implemented science lessons and literacy app with science and social studies content and vocabulary. Students will be administered transfer measures of domain specific (science and social studies) reading comprehension, background knowledge, and reading motivation. Digital data from the literacy app will generate information on measures of behavioral engagement (total unique logins, total time on task, total activities completed) and cognitive engagement (average accuracy across all digital activities). We will conduct both confirmatory and exploratory analyses to estimate intent-to-treat impacts, meditational effects, and moderator effects. Our first confirmatory aim is to fit multi-level models nesting students within classrooms, and classrooms within schools, to generate intent-to-treat estimates of impact on transfer measures of domain specific reading comprehension. In addition, to estimate intent-to-treat estimates of impact on growth in reading, we will fit piecewise growth models across three time points to assess fall to spring growth on a vertically scaled, computer adaptive, domain general reading comprehension measure. Second, we will conduct mediational analyses to examine whether effects on reading comprehension outcomes are mediated via improvements in the vocabulary, background knowledge, reading motivation, and cognitive and behavioral engagement with a literacy application. Finally, we will examine whether treatment effects on transfer measures of reading comprehension are moderated by prior reading level (both baseline beginning of grade 3 reading), and school attendance from grade 1 to grade 3.
Intervention
To read for understanding and to prepare for college, young children must acquire the domain and topic knowledge needed to read complex texts. Despite the growing literacy demands in the 21st century, fewer than 5% of US schoolchildren in the elementary grades can evaluate complex nonfiction texts with high background knowledge and vocabulary demands. Furthermore, there are larger gaps based on family income and student ethnicity on reading comprehension outcomes rather than basic word reading skills (D’Agostino & Rodgers, 2017; Fryer & Levitt, 2006; Reardon et al., 2012). During COVID-19, there is also increasing evidence that gaps in early grade reading comprehension tests have grown wider (Kuhfeld et al., 2021). Importantly, recent evidence suggests that the over-emphasis on basic literacy and math skills have contributed negative effects on science and social studies texts and exacerbated achievement disparities based on students’ racial and ethnic background, as well as their socioeconomic status (Arold & Shakeel, 2021; Song et al., 2019).
In many ways, US schools are increasingly focused on technology-based efforts to quickly personalize remediation efforts in literacy and math to help recover students' learning losses during COVID-19 (TNTP, 2021). At the same time, many states are enacting policies designed to improve K-3 literacy through ongoing investments in instructional approaches aligned with the science of reading (Burk, 2020; Folsom et al., 2017). Thus, there is a great need to identify evidence-driven literacy interventions that integrate teacher-directed instruction with educational technology that can help build students’ vocabulary and content knowledge while simultaneously providing more targeted and personalized activities to build more foundational literacy skills.
In response to this urgent national challenge, we designed this clustered randomized controlled trial to evaluate the replicability and scalability of the Model of Reading Engagement (MORE) content literacy intervention. In essence, the MORE intervention lessons and literacy apps are designed to build students’ domain knowledge and promote engagement with thematically related vocabulary (Connor et al., 2017; Guthrie & Klauda, 2014; Romance & Vitale, 2001; Vaughn et al., 2013; Williams et al., 2016). It also provides children with digital activities through a literacy app that supports the development of meta-linguistic awareness (MA). Researchers have defined MA as a child’s ability to analyze and play with language as an object independent from its meaning (Cazden, 1974; Roth et al., 1996; Tunmer & Bowey, 1984). MA is multi-dimensional and includes phonological and morphological awareness, syntactic awareness, and the ability to resolve semantic ambiguities in figurative language, such as riddles, jokes, and puns (Cairns et al., 2004; Yuill, 1996). There is emerging experimental evidence that brief interventions focused on building MA can support improvements in elementary grade students’ reading comprehension outcomes (Zipke, 2008; Yuill, 1996, 2009).
In this study, we developed a literacy app that included games to help students build their MA as they played games to build their knowledge of domain-specific science and social studies words and appropriately challenging single syllable words. In a previous validation study, we found that student performance, as measured by accuracy on the literacy app games, predicted improvement on both domain-specific and domain-general measures of reading comprehension, controlling for student covariates such as prior reading and math skills, demographic characteristics, and school and neighborhood contexts. These analyses found that a one standard deviation increase in accuracy predicted a 0.10 improvement in Spring formative assessments and a 0.15 increase in the North Carolina End of Grade exam. The magnitude of the association between cognitive engagement with the literacy app and student outcomes is consistent with recent meta-analyses of educational apps in literacy and math that show mean effect sizes of approximately 0.11 standard deviations on unconstrained skills like vocabulary and comprehension (Kim et al., 2020).
Experimental design
Researchers at the READS Lab at the Harvard Graduate School of Education will conduct a cluster randomized controlled trial. We stratified at the school level by prior experience with MORE lessons, student 3rd grade enrollment, and Fall achievement on the North Carolina Beginning of Grade (NC BOG) assessment in reading. For schools that had prior experience with MORE, there are five schools in each block, while for schools with no prior experience with MORE there are fourteen schools in each block. The treatment is clustered at the school-level. Next, we randomly assign whether the schools will receive treatment within each block. For blocks with an odd number of schools we randomize whether the first school is treatment or control.
Design details
Cited by
No citation data available for this registration yet.