Impact and Mechanisms: Why Consulting Matters in Human Capital Intensive Organizations. Evidence from a Field Experiment on Teacher Coaching.

How to cite

Michel, Bastien; Andersen, Simon; Nielsen, Helena. 2017. "Impact and Mechanisms: Why Consulting Matters in Human Capital Intensive Organizations. Evidence from a Field Experiment on Teacher Coaching.." AEA RCT Registry. AEARCTR-0002419. https://doi.org/10.1257/rct.2419-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Bastien Michel* , Simon Andersen , Helena Nielsen

* Primary investigator

Registry ID
AEARCTR-0002419
DOI
https://doi.org/10.1257/rct.2419-1.1
First registered
Last updated
Status
Completed
Trial dates
2017-08-09 – 2019-12-31
Countries
Denmark (Jutland)
JEL codes
I21; O31
Registry record
http://www.socialscienceregistry.org/trials/2419

Abstract

Recent studies have demonstrated substantial effects of consultancy on learning and productivity (Allen et al., 2011; Bloom et al., 2013; Hanna et al., 2014). However, the channels through which the effect of consultancy materializes and the relative strength of the different channels remain to be clarified. In order to study this question, we implement a randomized controlled trial investigating the impact of consultancy in the form of a one-year coaching program aiming to improve teacher skills through peer coaching. It will allow us to measure the relative strength of various channels through which the effects of consulting can materialize.

Intervention

As part of this study, we aim to measure the impact of a teacher-coaching intervention. We will do so in a context where all teachers (whether or not they receive the coaching intervention) will benefit from a professional development course designed to improve their teaching skills. Half of those teachers will be selected to benefit from a coaching intervention as well.

3.1 Professional development course
We will implement an intervention where all participating teachers and coaches attend five one-day course modules on the following education production factors: i) Physical surroundings, ii) Structuring schoolwork, iii) Self-regulation iv) The teacher's focus, and v) The teacher's communication. As part of these course modules, teachers will learn to use both technological input factors (e.g. timer, checklist, five-point scale, etc.) and behavioral input factors (e.g. support in transition phases, use calm voice, mentalizing, etc.). Furthermore, during each module, teachers will be asked to reflect on their own students and to set an agenda for themselves detailing how they intend to change their teaching practices so as to take into account what they will have learnt during the course modules. In addition, each course module will start out with rehearsal and reflection on what they have learnt in order to consolidate knowledge.

The course modules will be held on the following dates:
- Course module 1: August 9th, 2017
- Course module 2: September 14th, 2017
- Course module 3: October 31st, 2017
- Course module 4: November 28th, 2017
- Course module 5: February 7th, 2017

3.2 Treatment: Teacher-coaching program
Teachers selected to receive the teacher-coaching program will be paired with a co-teacher, their "coach", with whom they will spend 2.5 hours per week during an entire school year. Co-teachers will help teachers get closer to the production-possibility frontier through "peer coaching", as part of which the teacher receives feedback on how to improve teaching practice. These sessions will be dedicated to the implementation of what will have been covered during the professional development courses.

Experimental design

As there are strong reasons to expect significant selection with respect to the type of teachers who enroll in a coaching program, we will measure its impact using a randomized controlled trial design.

The coaching intervention will be randomized at the teacher-level. For implementation-related reasons, it was decided that the draw would be stratified by school. As a consequence, we will randomly select in each school half of the language arts teachers to benefit from the coaching intervention. In schools where an uneven number of teachers will be enrolled in the experiment, the number of teachers who should benefit from the coaching intervention was decided in advance for logistical reasons. We will adjust for the ensuing variation in teachers' probability to receive the coaching intervention across schools in our estimation strategy.

When schools will have appointed more than one coach, the teacher-coach pairs will be randomly generated.

Design details

Randomization method
Randomization done in office by a computer.
Randomization unit
The coaching intervention will be randomized at the language arts teacher-level. Again, for implementation-related reasons, it was decided that the draw would be stratified by school.
Clusters
17 schools
Observations
68 teachers 1490 students
Arms
In each school, 50% of the teachers enrolled in the experiment will benefit from the intervention.
Primary outcomes
In order to answer our research questions, several rounds of data collection will be carried out (for each batch) at different points in time: 1/ At the beginning of the school year, during the first 20 minutes of the first course module (baseline data collection, August 2017 - before the results of the randomization are shared with the schools) 2/ During the first 20 minutes of each remaining course module (follow-up surveys) 3/ After the first coaching session carried out after each course module (monitoring data collection) 4/ At the end of the school year, upon completion of the course modules (endline data collection, June 2018) The data will be gathered through different data sources: - Administrative registers: student, parent and teacher background, tracking of students and teachers, output measures (e.g. national test scores and grades for the teacher's past, current, and future students) - Observation data: classroom instrument developed from the course material and/or CLASS-UE measurement - Teachers and students surveys: they will be collected online using the SurveyXact platform Monitoring data We will collect information depicting how the intervention is carried out. In particular, we will collect information on the identity of the teachers attending the professional development course and on those actually receiving the coaching intervention. Furthermore, descriptive information will also be collected so as to understand how the coaching intervention is implemented. Teachers will be asked to answer (together with their coach) a short questionnaire after the first coaching session carried out after each course module. This questionnaire will collect information on their experience with the co-teaching intervention. Outcome variables Our primary outcome is teachers' teaching skills and, more specifically, whether or not they use the elements taught during the professional development course in class. This will be assessed by surveyors who will attend teachers' class and will rate the extent to which they use each of the techniques taught during the course. We will supplement these class observations by a measure of teaching skills as reported by the students using a validated index of teaching quality (TRIPOD). Our secondary outcome is students' achievements and, in particular, their reading skills (as measured by national standardized reading tests) and socio-emotional skills (as measured by national standardized wellbeing surveys).

Cited by

No citation data available for this registration yet.

Keywords