Effectiveness of Peer-Assisted Learning

How to cite

Cort, Willem De. 2024. "Effectiveness of Peer-Assisted Learning." AEA RCT Registry. AEARCTR-0014321. https://doi.org/10.1257/rct.14321-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Willem De Cort*

* Primary investigator

Registry ID
AEARCTR-0014321
DOI
https://doi.org/10.1257/rct.14321-1.0
First registered
Last updated
Status
In development
Trial dates
2024-10-02 – 2026-09-18
Countries
Private
JEL codes
I23
Registry record
http://www.socialscienceregistry.org/trials/14321

Abstract

The main aim of this experiment is to measure the effectiveness of a senior peer tutoring intervention in higher education at improving students' educational performance. Student organizations in law and engineering organize peer-assisted learning sessions (PAL) to first year students taught by senior peers for the most difficult courses. Because these tutoring sessions are oversubscribed, we randomly allocate students to treatment and control groups after initial enrolment. We vary the treatment by group size, and, potentially, in the second semester, by ability grouping or a different variable. The primary outcome variables is students' performance on that course using administrative data. We also consider some heterogeneity in treatment effect and secondary outcome variables, though this will depend on the obtained sample size and the level of over subscription. The final pre-analysis plan will be uploaded after students have registered for the tutoring intervention, but before outcome variables are collected. In this way, no p-hacking is possible, while still allowing me to demonstrate my initial intentions.

Intervention

The intervention consists of (bi-) weekly peer-assisted learning sessions taught by senior peers in law or engineering. The tutees are first year students in engineering, law, or in some other programs that have an intro to law course such as economics (though it is possible that this might still change as the student organizations decide quite late for which extra courses they will provide tutoring sessions). The tutors receive a short training sessions on how to deliver these PAL-tutoring sessions. This usually consists of both training on didactics specific to PAL as well as a refresher with respect to content for the specific course.

One treatment arm will have tutoring sessions in smaller groups than the other treatment arm. The precise number will depend on the amount of tutees and tutors wanting to participate in the program. We expect that the small groups will be 4 to 5 tutees per tutor, while the large groups will have 8 to 10 tutees per tutor. The law program usually has one tutor per group, while the engineering program has two tutors per group. The level of oversubscription will also influence the size of the control group, and perhaps if there is no oversubscription, will determine whether there is a control group.

We might update these details once the tutoring program's details have been decided by the student organizations and once the treatment arms have been decided based on the number of tutees and tutors registering, but before the tutoring program starts and thus before outcome variables are collected. In this way, no p-hacking is possible, while still allowing me to demonstrate my initial intentions.

Experimental design

The student organizations motivate students to enroll for the tutoring sessions. They do so by going to the classes and advertising the program during their activities and on their social media. Interested students fill in a survey stating they want to participate in the tutoring program and whether they want to give the researchers their informed consent to later retrieve their student records from the university's administrative data. The same procedure is used for recruiting tutors.

After about one to two weeks (the precise date is yet to be determined by the different student organizations), enrolment possibilities closes. I then randomize the enrolled students into two or three different groups in negotiation with the student organizations. A first treatment group will consist of students being able to follow PAL sessions in a small group of 4 to 5 tutees per tutor. A second treatment group will consist of students being able to follow PAL sessions in a larger group of 8 to 10 tutees per tutor. If there is a sufficient level of oversubscription, remaining students will be allocated to the control group. Once first-year students are allocated to such groups, we randomly allocate tutors to the different treatment groups. We might also create different treatment arms by pre-test ability in the second semester, but this is conditional on having a sufficient power in the first semester.

We will update these details once the tutoring program's details have been decided by the student organizations and once the treatment arms have been decided based on the number of tutees and tutors registering, but before outcome variables in the first semester are collected. In this way, no p-hacking is possible.

Design details

Randomization method
Randomization will be stratified by gender, course and pre-test ability (if strata are sufficiently large).These details might change based on the actual enrolment numbers. For example the low number of female students in engineering might make stratification by gender difficult. Again, adjustments might be made and reported after baseline data has collected but before the intervention starts, such that no p-hacking is possible while still allowing me to demonstrate my initial intentions..
Randomization unit
The unit of randomization is students, given that students in the same program can be allocated to different experimental arms.
Clusters
The treatment is not clustered. Students are nested into different programs, but the level of randomization is the student.
Observations
1500 students. Over the two semesters, the law PAL program had about 1100 students in the past year. The engineering PAL program had about 400 students. This varies quite a lot per year, because interest in the program and the amount of PAL programs offered varies.
Arms
This will depend on the level of oversubscription, which might differ between PAL programs for the different courses. The Law PAL programs had a significant level of oversubscription each year. We hope to be able to allocate about 1/4th or 1/3rd of the enrolled students into the control group, though again this will be negotiated with the student organization once the registration is over. The engineering PAL program only has some minor oversubscription in some years. If we are not able to create a control group, we will still compare the outcome variables between the two randomized treatment groups.
Primary outcomes
Grades on course for which the first year student wanted to enrol for PAL sessions
Secondary outcomes
Sense of belonging, measured using the same six-item scale as used in Dekker et al. (2023) and Meeuwissen et al. (2010) Self-perceived understanding of course (asked in the post-test in a single question on a scale of 0 to 10) Course-specific exam anxiety (asked in the post-test in a single question on a scale of 0 to 10) Individual study time for the course (asked in the post-test in terms of average hours per week) We might also consider long-term positive effects and spillovers to other courses conditional on finding significant and sizeable effects on the main course. We assume that long-term effects and spillovers will only occur if the tutoring is effect for the main course. Long term effects might however be published in a separate paper. We might update these outcomes updated once the tutoring program's details have been decided by the student organizations and once the treatment arms have been decided based on the number of tutees and tutors registering, but before the tutoring program starts and thus before outcome variables are collected. In this way, no p-hacking is possible. For example, a small control group might imply that we test less hypotheses.

Cited by

No citation data available for this registration yet.

Keywords