Using Teacher Messaging to Scale Programs: How does targeting messages improve teacher implementation and student comprehension
How to cite
Scherer, Ethan. 2023. "Using Teacher Messaging to Scale Programs: How does targeting messages improve teacher implementation and student comprehension." AEA RCT Registry. AEARCTR-0012754. https://doi.org/10.1257/rct.12754-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Introducing a new curriculum or program often raises a lot of questions. Whether it is a new reading curriculum in New York City or tutoring programs throughout the country to accelerate student learning post-COVID, principals and teachers may resist uprooting old practices and procedures. Most individuals are skeptical of something new even if there is evidence supporting its effectiveness. Thus, there is a need to complement evidence-based solutions with rigorous and research-based policies and procedures to scale a solution within a district.
The Model of Reading Engagement (MORE) equips teachers with standards-aligned and evidence-based tools – lessons, digital activities, and formative assessments of transfer – designed to boost students’ reading comprehension and literacy skills through science and social studies units. In addition, MORE supplies districts with a set of scaling tools to advance equitable access to MORE across the district. These tools focus on three levers of change involving (1) the Summer Leadership Institute (SLI) to train select district leaders and teachers, (2) a district-wide communication and implementation plan developed by district learning communities, and (3) MORE Teacher Innovators (TIs) to implement MORE principles outside of the lesson time, which collectively help build district capacity and buy-in at multiple levels of the system. During the school year the MORE team also provides professional learning and support to suggest adjustments to the communication plan, as needed, and provide feedback to TIs on adaptations.
Our study will assess the implementation and impact of the district-wide communication element of the of the scaling strategy. We will randomize the timing of messages sent to teacher and analyze secondary aggregate classroom data to assess implementation and impact based upon the content of the messages. While these principles that support the tools were developed with the MORE program in mind, many of these strategies could be applied in other contexts.
Intervention
Whether you are trying to diffuse hybrid corn in Iowa or a rigorous evidence-based reading curriculum, disseminating the innovation does not guarantee that it will be widely and deeply adopted. Diffusion is the social process that occurs among people in response to learning about an innovation – something new. More than sixty years of research has identified several factors that affect the rate of adoption of innovations (Rogers, 2003). First, it is important to clearly articulate the attributes of the innovation, its: (1) relative advantage, (2) simplicity (e.g., is it easy to learn), (3) compatibility with other tools (e.g., does it complement other programs offered by a district), (4) cost (in time), (5) observability (e.g., can we see the results) and (6) trialability (e.g., your ability to try it before fully adopting it). Empirical work has found that attributes one through three are the most important for wide adoption. While often there is significant deliberation and discussion before any significant change from district leadership, clearly and concisely distilling and communicating these deliberations to all teachers, principals and staff often is not allocated sufficient time. This lack of time can reduce the likelihood that potential solutions are widely and deeply adopted.
A second important result of the research is that most individuals do not evaluate innovations based upon whether the solution is evidence-based. Instead, most people depend upon subjective evaluations of the innovations from people like themselves (Rogers, 2003). This often means that, for example, a teacher who has not adopted a particular innovation would find the recommendation from another teacher far more meaningful than information from an "expert," central office staff, or even a principal. Specifically, opinion leaders similar to a potential adopter can rapidly accelerate adoption and use (Dearing & Cox, 2018). Yet, often these forms of communications are underutilized.
The Model of Reading Engagement (MORE) utilizes a top-down and bottom-up communication strategy focuses on three levers of change involving (1) the Summer Leadership Institute (SLI), (2) a district-wide communication and implementation plan, and (3) MORE Teacher Innovators (TIs) which helps build district capacity and buy-in at multiple levels of the system. This trial focused on the second component, a district-wide communication and implementation plan. MORE’s online platform supports sending personalized messages to teachers so district leaders can leverage a tiered communication strategy: general messages to all employees, behavior science informed scalable and targeted follow-up for those yet to engage with the content (Damgaard & Nielsen, 2018; Jackson & Makarin, 2018), and finally physical visits to schools. This tiered strategy allows leaders to better understand how they can target their limited time and resources.
We randomize the timing of when these messages are sent to test the efficacy of the MORE of implementation and impact. The MORE portal provides real-time data on implementation (e.g., students logins, teacher logins, teacher completion of modules, student progress in activities, completion of lessons, and formative assessments) allowing us to test the efficacy different content of messages using the data to teachers.
Experimental design
The district will randomize the timing of when schools receives particular messages with different content. The message will be received at least 1-week apart from one another and personalized to teacher using the real-time data available in the portal. Messages will vary by teacher/student participation levels and will focus on information, reminders, and social comparison nudges.
Design details
Cited by
No citation data available for this registration yet.