Supervisor Screening and Training

How to cite

Nyshadham, Anant; Adhvaryu, Achyuta; Tamayo, Jorge; Gauthier, Jean-François; Murathanoglu, Emir. 2023. "Supervisor Screening and Training." AEA RCT Registry. AEARCTR-0011288. https://doi.org/10.1257/rct.11288-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Anant Nyshadham* , Achyuta Adhvaryu , Jorge Tamayo , Jean-François Gauthier , Emir Murathanoglu

* Primary investigator

Registry ID
AEARCTR-0011288
DOI
https://doi.org/10.1257/rct.11288-1.0
First registered
Last updated
Status
In development
Trial dates
2023-08-31 – 2024-12-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/11288

Abstract

The experiment follows Adhvaryu et al. (2023) which demonstrates a critical role of certain managerial dimensions in determining firm productivity, and shows that intervening to improve management practices can generate meaningful impacts. We find that several key dimensions of managerial quality, such as attention, autonomy and locus of control are important for productivity but are not appropriately priced into market pay. Firms could substantially improve productivity at a low cost via psychometric measurement and screening of potential hires, and by providing training to improve managerial deficiencies. Adhvaryu et al. (2023a) follows up on this insight by evaluating a managerial training program based on the learnings from Adhvaryu et al. (2023b) identifying which managerial skills and practices contribute most to the productivity of production lines of India’s largest garment producer, Shahi Exports. The resulting program, called Supervisors Transform Into Change Holders (STITCH), was evaluated via a randomized controlled trial across hundreds of supervisors at Shahi Exports (our partner organization). The training achieved large and persistent impacts: on average efficiency (produced quantity/target quantity) of lines managed by trained line supervisors increased by 7.3% during training and 5.8% over the six months after the training. The proposed study empirically tests a screening and training technology that we have developed to hire and/or promote (garment) production line supervisors based on their psychometric traits and/or managerial practices to improve firm productivity and worker wellbeing. The trial will take place in 62 garment manufacturing units in India among upper- level managers and production line supervisors overseeing more than 70,000 workers of Shahi Exports. We will randomly select roughly a quarter of the units to get the screening technology, a quarter to get the training technology, and a quarter to get both while the remaining units will serve as the control group.

Intervention

This study will be conducted in 62 factories of Shahi Exports Pvt Ltd. with a self-administered screening tool and training tool consisting of managerial and leadership skills. We will be doing 3 follow ups.

The factories in the first treatment arm will receive only the screening and training tools (T1)
The factories in the second treatment arm will receive the screening and training tools and the technical tool (T2)
The factories in the third treatment arm will receive only the technical tool (T3)
The control units will not receive any of the tools apart from using the screening tool. We will use the survey app embedded in the survey tool to measure characteristics, traits, and practices at baseline and endline for the control group. The information from these surveys will not be shared with HR as they would in the other treatment arms.


Introduction to the tools:

1. Screening tool: The screening tool contains a series of psychometric questions, questions about management skills, leadership skills and soft skills. The screening tool is app based and self-administered through tablets. The results from this tool will be recorded on an online portal that can be accessed by the HR manager in-charge of hiring. They will see a display of results based on the screening score. At the end of the study, we will also see to what extent the HR department has adhered to our recommendations. This will be a secondary result. At the backend of the screening tool, there is a proprietary formula that scores each section and then weights those scores to decide which training the user would need to undergo. The overall time to undertake the screening tool is 60-90 minutes. This must be done at a stretch.

2. Training Tool: The training tool is a collection of modules and lessons mapped to the screening tool. These lessons are based on leadership, stress management, conflict resolution and other topics that would help make participants into better supervisors. Based on their screening result, each supervisor would have a unique learning path carved out for them in the training tool. Once they have completed their targeted training modules, they will have access to the entire tool including modules that they weren’t recommended. They can do these at their own pace. The training tool has a gamified interface that rewards participants in the form of coins and stars based on their progress in the app. The training app, along with being self-administered, is self-paced. The participants can take their time to complete the modules.

3. Technical tool: The technical tool is a digitized screening test internally developed by Shahi based on the technical assessment done in the existing hiring framework. It has been developed with the objective to assess nine key competencies required for a line supervisor. The tool aims to establish unbiased and standardized hiring and promotion processes across all units in Shahi. This tool is being used for the hiring/promotion process in Shahi currently. The test is divided into two primary sections: Selection and Promotion. The test consists of 50 multiple choice questions broadly covering themes such as professional skill, managerial skill, technical knowledge, and organizational context. This standardized test will administer a set of questions to prospective supervisors, calculate their scores based on their responses, and rank them by their predictive productivity at a supervisory role.
The trial allows us to test for both the effect of the different tools and their complementarity.

Experimental design

For this project, we are going to do an RCT in 62 factories of Shahi Exports Pvt Ltd. with a self-administered screening tool and training tool consisting of managerial and leadership skills. We will be doing 3 follow ups. Our unit of treatment is the entire factory. The unit of analysis differs between different outcomes. For line-level productivity outcomes, the unit of analysis is the production line. For retention, absenteeism, and survey outcomes, the unit of analysis is the individual worker/supervisor.

Experimental Design Details
Surveys will happen for 3 different groups - supervisors, workers and floor-in-charges. Each group will be surveyed 3 times. For supervisors, this will mean interacting with the same supervisor 3 times. For workers, this will mean interacting with each worker in the sample less than or equal to 3 times at random.

Supervisor Surveys: These will be done using an app designed by GBL to screen supervisors for managerial skills, soft skills and leadership skills. These will be self-administered.
FIC Surveys: These will be done by an enumerator using a SurveyCTO form
Worker Surveys: These will be done by an enumerator using a SurveyCTO form

Design details

Randomization method
We pre-randomize by computer using Stata’s pseudo-random number generator.
Randomization unit
Unit of randomization is the factory
Clusters
62
Observations
1200 supervisors and 4000 workers
Arms
16 factories in screening and training tools arm, 16 factories in the screening, training and technical tool arm, 15 factories in the technical tool arm, 15 factories control
Primary outcomes
1. Managerial Composition We will use data from the baseline and the endline survey to create measures of the managerial skills and traits of supervisors. 2. Productivity Productivity is calculated at the production line level, as the ratio of actual pieces of garment produced to the target number of pieces that could be produced under optimal conditions in a given day. This number is an exogenous industry standard. 3. Absenteeism A worker is defined as absent if they were not present for work on a given working day. We will have daily absenteeism data for each worker and supervisor. 4. Worker Satisfaction All survey outcome variables, like worker satisfaction, are indices generated by normalizing each component of the index to mean zero and standard deviation one, and then averaging across all components.

Cited by

No citation data available for this registration yet.

Keywords