Effectiveness of Teacher Adaptations to an Online Science Content Literacy Intervention to Improve Third-Graders' Reading Comprehension: A Randomized Controlled Trial During COVID-19 Pandemic

How to cite

kim, james. 2020. "Effectiveness of Teacher Adaptations to an Online Science Content Literacy Intervention to Improve Third-Graders' Reading Comprehension: A Randomized Controlled Trial During COVID-19 Pandemic." AEA RCT Registry. AEARCTR-0006718. https://doi.org/10.1257/rct.6718-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

james kim*

* Primary investigator

Registry ID
AEARCTR-0006718
DOI
https://doi.org/10.1257/rct.6718-1.0
First registered
Last updated
Status
In development
Trial dates
2020-10-29 – 2022-08-01
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/6718

Abstract

The purpose of this study is to test the effectiveness of allowing teachers to use local context to make structured adaptations to an evidence-based online science content literacy curriculum, Model of Reading Engagement (MORE). The MORE lessons are an in-person based curriculum organized thematically to provide a framework for helping student connect new learning to a meaningful domain-specific schema. We adapt the in-person MORE curriculum to an online version to be delivered both synchronously and asynchronously. To support the students at-home learning we also provide print books and reading comprehension activities. Then within our 26 schools, we randomize teachers into two conditions: a condition where we devise a core set of implementation procedures based upon the in-person MORE and common-sense adaptation needed for an online setting, or a condition with structured adaptations. In the adaptations condition, teachers can collaborate within their school or across schools to make practical improvements using their knowledge, experience, and local context. The randomized trial is designed to test the effectiveness of the adaptive implementation on third-graders' reading comprehension using researcher-developed transfer measures and standardized measures of broad reading comprehension.

Intervention

The randomized controlled trial aims to test the efficacy of the allowing teachers to make structured adaptations to an evidence-based online science content literacy intervention, Model of Read Engagement (MORE). To prepare teachers to make these adaptations as well as increase their knowledge of the science of reading, the adaptive MORE teachers will participate in a series of 4 asynchronous modules in November. The modules are based on Team-Based Learning (TBL) which aims to tightly couple knowledge acquisition with knowledge application (Michaelsen & Sweet, 2008). The November modules focus on helping teachers acquire knowledge about the core components of MORE and the science of reading research on effective instruction to improve vocabulary, morphology, and science content comprehension.

The knowledge application phase occurs in December as teachers apply their learnings to make structured and productive adaptations to MORE for their contexts. Building upon the asynchronous work in November, the December sessions will work with teams of teachers synchronously to use data to address potential barrier to student and family engagement. These teachers will have the flexibility to make both procedural-based and content-based changes to implementation, while keeping the core components of the intervention unchanged. Then in early January all teachers will be provided training for the MOREOnline curriculum with a focus MORE’s theory of change, the details of the lessons, and some best practices based upon previous years in-person results.

The Online MORE curriculum provides teachers with 5-weeks of science lessons including 10 synchronous lessons (2x week) as well as 3 days per week of 30 minutes asynchronous lesson that the students are expected to complete on their own. Science lessons will focus on human body systems (muscular, skeletal, nervous systems). The Online version of the curriculum adapts the in-person version of the science MORE curriculum. The in-person curriculum (a) give students access to complex and conceptually-related science content through engaging texts, (b) extend content learning through integrated, standards-aligned reading and writing activities, (c) attend to student motivation and engagement through a variety of strategies (e.g., choice, collaboration), and (d) increase exposure to content vocabulary. Student families will be provided with access to the application, two print books, as well as approximately 10-15 print reading comprehension activities in mid-January to complete prior to the beginning of the teacher-led portion of the intervention. Teachers will implement the lessons during February and March of 2021, and students will be sent a second batch of books and print activities during this time tied to the school lessons.

Experimental design

Researchers at the READS Lab at the Harvard Graduate School of Education will conduct the randomized controlled trial. Grade 3 teachers will be blocked by school and randomly assigned to Adaptive MORE or Core MORE. The number of teachers within each cell varies, but there is a minimum of two teachers within each block. The treatment is clustered at the classroom-level. Grade 3 parents whose parents provide consent will be study participants and the students will all receive the MORE lessons, but some will be taught by adaptive MORE teachers while others will be taught by CORE MORE teachers.

Baseline tests on mean math and reading scores on Measure of Academic Progress and demographic comparisons revealed no statistically significant differences in. For all tests, we ran multilevel models nesting students in classrooms and classrooms in schools, block on the school, and show p-values greater than .05. The standardized mean differences of MAP were 0.06 RIT points (p > .05) comparing Adaptive MORE with CORE MORE. A slightly larger proportion of students was gifted, but all other demographic covariates were (p>0.05)

Design details

Randomization method
Randomization was conducted in an office by a computer and implemented using STATA code.
Randomization unit
The unit of randomization is the classroom, blocked by school, in the 2020-21 school year.
Clusters
97 classroom clusters
Observations
Of the total of 1,748 students in participating classrooms, we expect approximately 80% of students to receive parental consent, yielding an analytic sample around 1,398 Grade 3 students.
Arms
48 Classrooms with 855 students in the Adaptive MORE 49 Classrooms with 893 students in the CORE MORE
Primary outcomes
There are two primary outcomes. First, we will administer a researcher-developed measures of students’ knowledge of science concepts taught in the MORE lessons. Second, we will use administrative data on a vertically scaled reading test – the MAP
Secondary outcomes
There are four secondary measures of student/family engagement and teacher adaptation including: student engagement with educational apps, student engagement with paper books, parent and family use of apps and books, and teacher knowledge of MORE core components and science of reading research on vocabulary and morphology instruction.

Cited by

No citation data available for this registration yet.

Keywords