The impact of psychometric matching on student outcomes and student satisfaction in online 1-1 learning environments

How to cite

Beaton, Jamie; Bowes, Lucy. 2019. "The impact of psychometric matching on student outcomes and student satisfaction in online 1-1 learning environments." AEA RCT Registry. AEARCTR-0004444. https://doi.org/10.1257/rct.4444-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Jamie Beaton* , Lucy Bowes

* Primary investigator

Registry ID
AEARCTR-0004444
DOI
https://doi.org/10.1257/rct.4444-1.1
First registered
Last updated
Status
On going
Trial dates
2019-06-24 – 2020-12-10
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/4444

Abstract

This randomized control trial evaluates the impact of psychometric matching on student outcomes and student satisfaction. All students and tutors in the study are evaluated based on a psychometric assessment known as HEXACO. From there, half of the students are matched to tutors based on an algorithm which evaluates the HEXACO traits of the student and the tutor and half of the students are matched randomly. Student self-reported session ratings of tutors is used as a measure of satisfaction and student SAT score improvement is used as a measure of change in outcomes. This trial takes place in an online school environment in the context of 1-1 synchronous learning via video call. Randomization occurs through a random number generator function which assigns people below a certain cut-off value (0.5) to one trial and those at or above the cut-off value to the other trial. Blinding occurs as students and tutors do not know if they are matched through psychometrics or not. Additional blinding occurs as the data analysis of results will be conducted blind as well.

Intervention

The intervention is assignment of online 1-1 student-tutor pairs based on HEXACO through a matching algorithm compared to a random match. HEXACO includes six factors, or dimensions: Honesty-Humility (H), Emotionality (E), Extraversion (X), Agreeableness (A), Conscientiousness (C), and Openness to Experience (O). Each factor has multiple sub-dimensions and is evaluated on a 0 - 5 scale after completion of a psychometric assessment which lasts approximately twenty minutes.

Experimental design

Students will be enrolled into the trial over a rolling six month period. Students will participate in the trial over a six month period. Upon entry into the program, students will be assigned into either the control or intervention group based on a random number generator. Students in the control group will have no tutors matched to them based on the psychometric matching algorithm. Students in the intervention group will have all tutors matched to them based on the psychometric matching algorithm.

Design details

Randomization method
Randomization done by excel random number generator function.
Randomization unit
Individual randomization.
Clusters
816 students
Observations
816 students
Arms
408 students in control, 408 students in intervention group
Primary outcomes
SAT score improvement and student self-reported satisfaction score.
Secondary outcomes
Cancellation Rate (Student) Cancellation Rate (Tutor)

Cited by

No citation data available for this registration yet.

Keywords