Feedback, Motivation, and Performance
How to cite
Saygin, Perihan; Pollard, Garrison; Knight, Thomas; Rush, Mark. 2023. "Feedback, Motivation, and Performance." AEA RCT Registry. AEARCTR-0010898. https://doi.org/10.1257/rct.10898-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Individuals often make educational and professional decisions conditional on feedback of their performance. In educational settings, students receive grades as the primary source of feedback. These grades may impact how students choose to allocate their efforts, as well as influence future educational decisions (major, course enrollment, etc.). The most challenging issue in this line of research is the lack of a exogenous variation in feedback. We aim to overcome this selection issue by adopting an experimental design in which we randomly assign graders of varying innate harshness (despite consistent guidelines) to blindly grade short essays in two large introductory economics courses. With this, we are able to analyze the causal effect of a grade’s harshness/lenience on students’ subsequent decisions and outcomes. We also measure students’ grade expectations (confidence) and track grade challenges, enabling us to characterize a potential casual channel (confidence) and identify heterogeneities in the resistance to feedback.
Experimental design
This is a field experiment conducted in two large principles of economics courses at a large, comprehensive university. Students will be asked to complete 3 short writing assignments (SWAs), which are short essays answering some content-relevant prompt. All prompts have an objectively correct answer in the rubric provided to all graders, but partial credit can be awarded at the discretion of the grader. For each assignment, students are also asked to provide a grade expectation, the honesty of which is incentivized by providing extra credit for accuracy.
Once the assignment due date has passed, students’ assignments are randomly assigned to a grader. Graders are responsible for only grading and do not interact with students in any way. Grading is double-blind such that neither student nor grader is aware of the identity of the other during grading period until grades are posted. Graders are provided a standard rubric of ten elements, all valued at ten points each. The first seven are objective and related to the SWA prompt. The final three elements are concerned with grammar, typos, and writing quality. All graders are given guidelines on the grading of these elements at the start of the semester. However, all entered grades are at the sole discretion of the grader and are never altered by anyone other than the grader or the instructor. Once all assignments have been graded, all detailed rubrics are downloaded from the LMS before being posted. Students are immediately notified of posted grades by the LMS. At the same time, the rubric is available to students so that they can review the points they lost, if any. Once posted, the assigned grade is no longer double-blind as the student is now able to view the grader’s name. Grader's names seen by students are randomly assigned to a generic female and male sounding names (Amanda, Emma, Eric, James, Jessica, John, Katherine, and Michael).
Students are permitted to contest a grade only by contacting the course instructor. Graders are never permitted to interact with students and are instructed to forward any student communication to the study coordinator. The instructor has the sole discretion to alter any submitted grade. Instructors maintain a record of all grade contests, and any alterations are flagged when rubrics are downloaded at the end of the semester.
Several days after each short writing assignment, students complete a 40-question multiple choice exam covering the same material as the preceding SWA. The final question of each exam asks students to report the number of questions they expect to get correct. Similar to the SWAs, honesty is incentivized by providing extra credit for accuracy.
After the assignment grades have been posted for the first and second assignments, students will be asked to provide a rating of their perceived fairness of the each grade they were assigned (1 to 5 where 1 is “Very Unfair” and 5 is “Very Fair”). This response is not incentivized. On the final exam, students will be asked to provide a rating of their perceived fairness of the overall grading in the course (1 to 5 where 1 is “Very Unfair” and 5 is “Very Fair”).
At the end of the semester, each SWA will be regraded blindly by two additional randomly-assigned graders.
Design details
Cited by
No citation data available for this registration yet.