Belief Elicitation in Project Course

How to cite

Steinle, Dennis; Rieber, Alexander; Schudy, Simeon. 2025. "Belief Elicitation in Project Course." AEA RCT Registry. AEARCTR-0017300. https://doi.org/10.1257/rct.17300-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Dennis Steinle* , Alexander Rieber , Simeon Schudy

* Primary investigator

Registry ID
AEARCTR-0017300
DOI
https://doi.org/10.1257/rct.17300-1.0
First registered
Last updated
Status
In development
Trial dates
2025-11-21 – 2028-10-30
Countries
Private
JEL codes
D81, D83, D84, C93, I26
Registry record
http://www.socialscienceregistry.org/trials/17300

Abstract

This pre-analysis plan outlines a multi-year field experiment designed to study belief formation and belief updating in an undergraduate data-science course. Across four team-based projects per cohort, students repeatedly form expectations about their project grades, evaluate the work of peers, receive structured peer feedback, and revise their beliefs. Using randomized assignment of reviewers and experimentally manipulated feedback, we identify how exposure to exclusively positive versus exclusively negative feedback affects students’ grade expectations, learning behavior, and subsequent academic performance. The experiment follows an A–B / B–A design across two semesters: in the first semester, teams are randomly assigned to receive only positive or only critical elements of the peer reviews; in the second semester, the feedback condition is reversed at the individual level to study both immediate and longer-run effects as well as order effects. Primary outcomes include belief updating after reviewing another team’s project, belief updating after receiving manipulated feedback, and final exam performance. Secondary analyses examine heterogeneity by demographic characteristics, prior academic preparation and team composition. With approximately 60–80 students per cohort and repeated random assignment of teams, the design yields around 300 team-level observations across three years, providing adequate power to detect moderate effect sizes. This study contributes to the literature on belief updating, feedback processing, and learning in team-based educational environments by providing the first large-scale field evidence on how curated positive versus negative feedback shapes students’ expectations and downstream performance.

Intervention

Once peer evaluations are submitted, each participant receives three assessment reports on their own project (identical within teams). Participants are then required to rank these reports (1–3) based on perceived usefulness and clarity of feedback (general guidance for ranking is provided in Appendix B). To incentivize the provision of careful and meaningful evaluations, individuals must obtain at least one ranking within the top two positions across the reviews they provide to qualify for the final exam. Rankings are revealed only after all review rounds are concluded, ensuring sustained effort throughout the process.

After report submission, we extract evaluative elements from the reviews. For the intervention, teams are randomly allocated to receive either exclusively positive or exclusively critical components of the feedback. Only the respective subset of comments is shared, depending on assigned treatment.

Experimental design

NA

Design details

Randomization method
Done by the computer. For more details see our pre-analysis plan.
Randomization unit
Randomization on the group level. Each group consists of three students. We re-randomize after each group project but students stay in the same treatment for the semester. For more information see the detailed pre-analysis plan.
Clusters
300 project groups
Observations
300 observations on the group level and when looking at satisfaction we have 900 observations, as we have 300 project groups and in each project group are 3 students.
Arms
150 project groups in the control group and 150 project groups in the treatment group
Primary outcomes
Expected Points, Team Performance, Individual Performance

Cited by

No citation data available for this registration yet.

Keywords