Using a SMART (Sequential Multiple Assignment Randomized Trial) Design to Develop a Personalized Literacy Intervention to Improve Reading Engagement and Comprehension

How to cite

kim, james. 2018. "Using a SMART (Sequential Multiple Assignment Randomized Trial) Design to Develop a Personalized Literacy Intervention to Improve Reading Engagement and Comprehension." AEA RCT Registry. AEARCTR-0002983. https://doi.org/10.1257/rct.2983-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

james kim*

* Primary investigator

Registry ID
AEARCTR-0002983
DOI
https://doi.org/10.1257/rct.2983-2.0
First registered
Last updated
Status
On going
Trial dates
2018-05-09 – 2019-07-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/2983

Abstract

The READS Lab at the Harvard Graduate School of Education will conduct a Sequential Multiple Assignment Randomized Trial (SMART) Design to replicate the effects of a Model of Reading Engagement (MORE) on concept knowledge, expository writing, and reading comprehension outcomes for students in Kindergarten, Grade 1, and Grade 2. MORE is a literacy intervention that provides elementary students with (1) access to complex and connected science concepts; (2) comprehension instruction that integrates reading and writing; (3) support for reading at home; and (4) motivational supports. The SMART trial aims to test whether each of the following levers is more effective: support for deep vs wide reading at home; identification of non-responders at 2 weeks or 4 weeks; additional engagement supports via gamification or gamification and text messaging.

Intervention

A Model of Reading Engagement (MORE) is a literacy intervention that provides elementary students with (1) access to complex and connected science concepts; (2) comprehension instruction that integrates reading and writing; (3) support for reading at home; and (4) motivational supports. The two-week school-based curriculum provides 1) Access to complex, coherent, and connected science concepts, 2) Books and texts that motivate students and build self-efficacy, and 3) Comprehension instruction that integrates reading and writing. Teachers will implement the 9 MORE lessons and lead their students in each lesson’s corresponding reading and writing activities at the end of the 2017-2018 school year. The home-based intervention will give students the opportunity to continue reading over summer break with specialized activities tailored to the 10 free books they will receive as part of study participation. Activities will be delivered through the MORE@Home app and 4) Developmentally matched to students reading needs. The intervention also includes additional supports for students who are “not responding,” i.e. not engaging with the MORE@Home app. Two types of supports will be available: the first is “gamification” of the reading activities. Students will be trying to collect animals for their virtual zoo, and responding to the questions about each book will allow them to add one more animal to their virtual zoo. The second support is text messaging to parent/guardians to nudge them toward using the app with their student. In the DEEP condition, the home reading component will provide students with books connected to the science concepts learned in the school lessons and to provide additional exposure to key concepts and vocabulary taught during the two-week lesson cycle. In the WIDE condition, the home reading component will include a wide variety of fiction and non-fiction books.

Experimental design

Researchers at the READS Lab at the Harvard Graduate School of Education will conduct the SMART trial. Kindergarten, Grade 1, and Grade 2 teachers at one school will be blocked by grade and randomly assigned to one of two experimental conditions: DEEP MORE or WIDE MORE. The school-based intervention is clustered at the classroom level. Students in these grades whose parents provide consent and agree to participate will be study participants will receive the MORE lessons based on the assignment of that student’s primary reading teacher. Additionally, students will be randomized into a “typical support” and “early support” group. The typical support group’s participation in Home activities will be assessed at 4 weeks and the early support group’s participation in Home activities will be assessed at 2 weeks. Finally, the non-responders (based on assessment at either 2 or 4 weeks) will be randomly assigned to additional supports. One group will receive gamification of reading activities and the other group will receive gamification of reading activities plus motivational texts.

Design details

Randomization method
Randomization will be conducted in an office by a computer and implemented using STATA code.
Randomization unit
The unit of randomization for the DEEP vs. WIDE conditions is the classroom, blocked by grade, in the 2017-18 school year. The unit of randomization for the TYPICAL vs. EARLY support conditions is the student, blocked by grade and initial treatment condition. The unit of randomization for the GAME vs. GAME+TEXT conditions is the student, blocked by grade, and initial 2 treatment conditions.
Clusters
16 classroom clusters (6 Kindergarten, 5 Grade 1, 5 Grade 2)
Observations
Of the total of 295 students in participating classrooms, we expect approximately 80% of students to receive parental consent, yielding an analytic sample around 240 students, although final sample sizes could be smaller (N = 206) if consent rates are 70% or larger (N = 265) if consent rates are 90%.
Arms
o 8 classrooms with 150 students in the DEEP group ♣ 74 of these students will be in the EARLY support (assuming 100% consent) ♣ 76 of these students will be in the TYPICAL support (assuming 100% consent) ♣ The number receiving GAME or GAME+TEXT will be based on the number of non-responders, but assignment ratio will be 50/50 between conditions. o 8 classrooms with 145 students in the WIDE group ♣ 72 of these students will be in the EARLY support (assuming 100% consent) ♣ 73 of these students will be in the TYPICAL support (assuming 100% consent) ♣ The number receiving GAME or GAME+TEXT will be based on the number of non-responders, but assignment ratio will be 50/50 between conditions.
Primary outcomes
There are three primary outcomes. First, we will administer a researcher-developed measures of student knowledge of science concepts that were taught in the DEEP and WIDE lessons. Second, we will use administrative data on a vertically scaled reading test – the Measure of Academic Progress, Primary Grade Reading. Third, we will use administrative data on a measure of early literacy skills (MCLASS and Text Reading Comprehension).
Secondary outcomes
A secondary outcome is student engagement based on teacher ratings on a previously developed measure (Leppola et al., 2005).

Cited by

No citation data available for this registration yet.

Keywords