The effect of failure in a high-stakes application procedure on perceived procedural efficiency
How to cite
Fischer, Mira; Stueber, Robert; Kuebler, Dorothea. 2022. "The effect of failure in a high-stakes application procedure on perceived procedural efficiency." AEA RCT Registry. AEARCTR-0009635. https://doi.org/10.1257/rct.9635-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
In this project, we analyze how the experience of success or failure in a high stakes environment influences beliefs and perceptions. Using the central application procedure for places in the medical fields in Germany, we study how the (non-)assigment of a study place through this procedure influences perceived procedural efficiency. We attempt to follow two identification strategies. Our first identification strategy relies on a regression discontinuity design that uses application scores in the three admission quotas as running variable and incentivized survey measures administered after applicants are informed about the outcome of their application as outcomes. Our second identification strategy relies on a difference-in-differences analysis of incentivized survey measures administered before and after applicants are informed.
Intervention
Using the central application procedure for places in the medical fields in Germany, we study how the (non-)assigment of a study place through this procedure influences perceived procedural efficiency. Medical studies are some of the most prestigious study fields in Germany. Each year, only about 1 out of 6 applicants receives a study place. This overdemand has lead to ongoing public debates about and reforms of the application procedure. At the core of these debates and reforms is the question of which applicant characteristics should play the biggest role in the application decision. Proponents of the application procedure – as currently implemented – advocate that an applicant’s high school GPA receives a high weight in the procedure, whereas opponents of the current procedure emphasize that other factors, such as the applicant’s emphathy, are more important predictors of success as a physician, and should therefore receive more weight in the admission decision. We attempt to understand how the apparent differences in beliefs between the proponents and opponents of the current system come about. In particular, we analyze whether they reflect fundamental differences in beliefs between admitted and non-admitted applicants and whether beliefs are motivated, that is, whether the admission itself influences these beliefs.
We consider four primary outcomes. The first two outcomes relate to the relationship between a selection criterion (GPA or empathy) and the job performance of medical doctors. We elicit participants’ beliefs regarding these relationships as estimated in two academic studies. The other two outcomes elicit participants’ second-order beliefs and refer to the participants’ assessment of the perception of physicians in society. We elicit participants’ beliefs regarding the perceived competencies and trustworthiness of physicians.
For wave 2, we intend to slightly alter these outcomes such that (i) the effects of a potential preference for making consistent responses in wave 2 are reduced and (ii) the wave-1 measures of beliefs still constitutes a meaningful control for these beliefs. In particular, we employ different measures of job performance and elicit the beliefs about the perception of competencies and trustworthiness in the society based on a different survey year.
We ask the following questions: (i) Does failure (relative to success) in the application procedure weaken the perceived relationship between physicians’ high school GPA and their job performance (measured by their attainment of further professional medical qualifications)? (ii) Does failure (relative to success) in the application procedure strengthen the perceived relationship between physicians’ empathy and their job performance (measured by their success in treating their diabetes patients, given beliefs about the importance of empathy for a successful application)?(iii) Does failure (relative to success) in the application procedure decrease beliefs about the competencies of physicians? (iv) Does failure (relative to success) in the application procedure decrease beliefs about the trustworthiness of physicians?
Experimental design
About 3850 individuals will be invited to participate in both survey waves before and after the reception of feedback and about 3850 will be invited to participate only in the survey wave taking place after the reception of feedback. We expect a response rate of about 65% each, amounting to about 2500 individuals in each group. The survey is conducted in cooperation with the Studierendenauswahl-Verbund (stav). They send out invitations to participate in the survey containing individualized links. The polling is managed by the research team and hosted by the WZB Berlin Social Science Center, who collect the data via LimeSurvey. We guarantee full anonymity to our survey participants who answer the survey questions autonomously on their own digital devices. We ask for permission to recontact them and collect their e-mail addresses to do so. We also collect their bank and/or PayPal information to send them the participation rewards. Randomization is carried out using the survey software LimeSurvey at the individual level.
Our experiment is structured as follows:
Subjects will conduct the experiment via an online platform. Each subject takes decisions in four stages.
Stage 1: perceived relationship between physicians’ high school GPA and job performance
Stage 2: perceived relationship between empathy and job performance
Stage 3: beliefs about competencies of physicians
Stage 4: beliefs about trustworthiness of physicians
Design details
Cited by
No citation data available for this registration yet.