Understanding Skill Gap

How to cite

Vecci, Joe; Ip, Edwin; Feld, Jan; Leibbrandt, Andreas. 2019. "Understanding Skill Gap." AEA RCT Registry. AEARCTR-0004227. https://doi.org/10.1257/rct.4227-3.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Joe Vecci* , Edwin Ip , Jan Feld , Andreas Leibbrandt

* Primary investigator

Registry ID
AEARCTR-0004227
DOI
https://doi.org/10.1257/rct.4227-3.1
First registered
Last updated
Status
Completed
Trial dates
2019-09-02 – 2020-01-15
Countries
United States of America ()
JEL codes
J71, C93
Registry record
http://www.socialscienceregistry.org/trials/4227

Abstract

In this project, we study whether employers believe women are less skilled programmers and whether these beliefs are justified. We also study different ways of reducing potential misconceptions about women’s skill.

Experimental design

Our design aims to measure perceived and actual gender skill gap in the programming field between male and female programmers.

Design details

Randomization method
Randomization will be carried out by a computer.
Randomization unit
The randomization unit will be the individual for all treatments.
Clusters
For stage 1, we plan to invite up to 360 applicants to complete the Python skills assessment. All applicants will be given the test OR if there are more than 360 applicants a conditional random sample of applicants will be given the test (depending on the number and gender composition of the applicants). The number of observations depends on the number of applicants for the programming job as well as the share of invited applicants who will actually complete the assessments. For stage 2, we plan to have between 450 and 780 potential employers (50% programmers and 50% HR professionals) depending on availability and funding. We will decide on the exact number of potential employers before the start of stage 2. This will be stated in Appendix 2.
Observations
For the applicant sample (stage 1), the number of observations is the same as the number of clusters. We hope to have up to 360 observations (180 male applicants and 180 female applicants). For the potential employer sample (stage 2), each employer will guess 20 different profiles (10 for each gender). This leaves us with between 9,000 and 15,6000 employer-applicant-profile observations (depending on the number of potential employers). For parts of the analysis where we focus on across employer guesses (due to order effects), we will only use the between-gender-profile observations from the guesses of the first gender which leaves us with a maximum 7,800 employer-applicant-profile observations.
Arms
In the first stage we plan to have up to 180 applicants (90 male and 90 female) who complete the Python test and the personality test while the same number will also answer the Python test and the aptitude test. In the second stage, in each randomization block will have between 75 and 130 potential employers who will make 20 guesses each. The final number will depend on availability of participants and budget [1]. The number of observations will be the same in each randomization block. These are the 6 randomization blocks. 1. Control / Male first 2. Control / Female first 3. Information (Personality) / Male first 4. Information (Personality) / Female first 5. Information (Aptitude) / Male first 6. Information (Aptitude) / Female first Notes: [1] Further details about the information given to employers, including the details on the information treatment, will be provided in Appendix 2, after stage 1 data has been collected but before stage 2
Primary outcomes
Perceived gender skill gap: difference in guesses of programming test scores of randomly selected male and female applicants. These guesses come from our sample of potential employers in the control group. Actual gender skill gaps: difference in programming test scores of randomly selected male and female applicants. Gender bias: Difference between perceived and actual gender skill gap (as described above). Perceived Gender Gap with Second Order Beliefs The average difference in second order beliefs (i.e. what employers believe other employers guessed for that profile) of male profiles and female profiles. These second order beliefs can also be considered as a descriptive norm. Gender Bias in Second Order Beliefs Difference between gender gap in second order beliefs and actual gender skill gap (as described above).
Secondary outcomes
We will explore secondary questions about whether perception of skill differs depend on whether the potential employers’ job functions are human resource related or programming oriented. Human resource professionals may have more experience and training in hiring, however, fellow programmers may have better understanding and experience with regards to evaluating programmers’ actual skills and on the job performance. Further, we will investigate heterogeneity in the perceived and actual skill gaps. For instance, we will study the age skill gap gradient and the number of years of experience skill gap gradient. We will collect other predictors of performance in the applicants’ job applications such as; CVs (education, experience), university grade point average, competitiveness and risk, and results on aptitude and personality tests which they complete as part of the application process. Finally, to measure experimenter demand effect, we will follow de Quidt et al., (2018) and include a few questions at the end of the employer survey to estimate a demand effect. References: de Quidt, Jonathan, Johannes Haushofer, and Christopher Roth. 2018. "Measuring and Bounding Experimenter Demand." American Economic Review, 108 (11): 3266-3302.

Cited by

No citation data available for this registration yet.

Keywords