Video Assessments

How to cite

Bapna, Sofia; Gutt, Dominik; Ravindran, Kiron; Tsekouras, Dimitrios. 2025. "Video Assessments." AEA RCT Registry. AEARCTR-0016929. https://doi.org/10.1257/rct.16929-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Sofia Bapna* , Dominik Gutt , Kiron Ravindran , Dimitrios Tsekouras

* Primary investigator

Registry ID
AEARCTR-0016929
DOI
https://doi.org/10.1257/rct.16929-1.0
First registered
Last updated
Status
In development
Trial dates
2025-10-15 – 2026-10-15
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/16929

Abstract

Recently, many organizations have adopted 0:1 video-recorded interviews, particularly in job recruitment. In this format, candidates enter a virtual meeting room without a live interviewer, are presented with questions (typically in written form on the screen), and must respond verbally while being video-recorded. Increasingly, these asynchronous 0:1 interviews are replacing traditional 1:1 live interviews, especially in the initial stages of the hiring process.

A common concern is that individuals may feel uncomfortable with this format, as it lacks a real interviewer with whom they can make eye contact, build rapport, interpret nonverbal cues etc.—factors that can affect interview performance.

Thus, we conduct a randomized field experiment testing whether educating interviewees about the fairness and equity advantages of 0:1 video-recorded interviews influences their performance. During the interview, the control group is shown only the basic format description, whereas the treatment group is shown the basic format description and also informed about the format’s fairness and equity benefits. Furthermore, we explore heterogeneity in interview performance based on individuals’ personality traits and their anxiety related conditions.

Intervention

The sample comes from students enrolled in a master's level course on Research Methods. One of their graded assignments (20% of the course final grade) is a viva voce test which takes the form of a 0:1 video-recorded interview. All of the students are required to take part in the 45-minute interview at the same time. The students are randomly assigned into one of two groups—Control or Treatment.

During the interview, students in the Control group are shown information about the interview format only. The text displayed to them reads:
"Assignment format: no live examiner; written presentation of questions; preset questions; recorded answers."

During the interview, students in the Treatment group are shown the same format information as the Control group, supplemented with explanations of the format’s fairness and equity advantages. The text displayed to them reads:
"Assignment format: no live examiner, ensures reduced interpersonal bias (EQUITY); written presentation of questions, ensures uniform delivery across candidates (EQUITY); preset questions, ensures consistent examination across individuals (FAIRNESS); recorded answers, ensures consistent rubric application via replay (FAIRNESS)."

Experimental design

The sample consists of students enrolled in a master’s-level course on research methods. One of their required graded assignments is a viva voce examination conducted as a 0:1 video-recorded interview. All enrolled students simultaneously participate in the 45-minute interview. Students are randomly assigned to one of two groups: Control or Treatment. During the interview, the Control group is shown only a basic description of the interview format, whereas the Treatment group is shown the same description along with additional information emphasizing the format’s fairness and equity benefits.

The primary outcomes are interview performance—assessed through graded scores, non-graded behavioral indicators, and self-reported confidence or satisfaction with interview performance. Furthermore, we examine heterogeneity in performance based on individuals’ personality traits and anxiety-related conditions.

Design details

Randomization method
By a computer.
Randomization unit
Level of randomization: student Randomization method: block randomization, with gender as the block
Clusters
N/A
Observations
The sample includes all students enrolled in the Master’s-level BIM Research Methods course. We expect close to 200 enrolled students.
Arms
The enrolled students will be randomly assigned to the two groups (control and treatment); approximately half the students will be in each group.
Primary outcomes
Interview performance (i.e., grade of the student based on the instructors' evaluation of the interview). Note: We will examine heterogeneity in performance based on personality traits (Big Five and narcissism) and anxiety-related conditions (social interaction anxiety and social phobia). Personality traits and anxiety-related conditions will be measured through surveys administered separately from the viva voce interview.
Secondary outcomes
1) Self-reported confidence/satisfaction in interview performance, measured using the following question administered at the end of the viva voce assignment: To what extent do you agree with the statement, “I feel positive about my performance on the assignment. (Not at all, To a small extent, To some extent, To a great extent, To a very great extent)” 2) Non-grade related markers of performance. For example: -Response fluency (fewer filler words, smoother delivery) -Lexical richness/vocabulary -Emotional markers in voice -Facial tension/camera avoidance (For #2, the scope of voice and video analysis will depend on the permissions granted by the university concerning the use of student-related materials and recordings.) 3) As a robustness test, we will examine whether the likelihood of cheating varies across conditions, to verify that any observed performance differences are not attributable to cheating.

Cited by

No citation data available for this registration yet.

Keywords