What Drives Different Assessments of Performance on Coding Tasks?
How to cite
Craig, Ashley; Effenterre, Clémentine Van. 2022. "What Drives Different Assessments of Performance on Coding Tasks?." AEA RCT Registry. AEARCTR-0009816. https://doi.org/10.1257/rct.9816-1.2
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This study focuses on evaluations of in coding interviews, which are used to hire computer programmers. We aim to shed light on the mechanisms underlying these differences in these ratings, including differences in coding quality and style, and coder effort.
Intervention
We aim to assess how software developers evaluate pieces of code written by computer programmers. Our experiment uses a large set of de-identified code blocks from an online coding platform, which span coders of different skill and problems of different levels of difficulty. For each code block, we will have access to the code's objective measures of performance including sub-test results (e.g., whether it runs, whether it produces correct answers to unit tests etc.). Using these data, we will ask evaluators to judge the quality of the code using the same Likert scales as on the platform.
Experimental design
Our design relies on multiple observations per subject. Each participant will evaluate 4 code blocks.
Design details
Cited by
No citation data available for this registration yet.