Blinded by the Person: Field Experimental Evidence from Idea Evaluation in a Multinational Company

How to cite

Dahlander, Linus; Thomas, Arne; Wallin, Martin; Ångström, Rebecka. 2020. "Blinded by the Person: Field Experimental Evidence from Idea Evaluation in a Multinational Company." AEA RCT Registry. AEARCTR-0005439. https://doi.org/10.1257/rct.5439-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Linus Dahlander* , Arne Thomas , Martin Wallin , Rebecka Ångström

* Primary investigator

Registry ID
AEARCTR-0005439
DOI
https://doi.org/10.1257/rct.5439-1.0
First registered
Last updated
Status
On going
Trial dates
2019-12-18 – 2020-02-22
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/5439

Abstract

To study idea evaluation and selection within firms, we conduct a field experiment in cooperation with a large multinational company in the information and communication technology (ICT) sector. The field experiment aims to provide causal evidence on how the relationship between idea proposers and idea evaluators affects evaluation and ultimately innovation outcomes. To do so, we manipulate the idea evaluation process where innovation managers rate ideas proposed by their colleagues. Specifically, we experimentally vary the amount of information that innovation managers receive on the idea proposer. There are two conditions: (1) non-blind evaluation, where innovation managers receive information on the name, organizational unit, and location of the idea proposer, and (2) blind evaluation, where innovation managers receive no information on the idea proposer. We intend to analyze how blind evaluation changes evaluation scores (H1 - Rating) and how the effect depends on order (H2 - Order). Moreover, we want to test how blind evaluation changes evaluation scores when the idea proposer is female (H3 - Gender), when the idea proposer comes from the same organizational unit as the evaluator (H4 - Subunit), and when the idea proposer is located in the same country as the evaluator (H5 - Location).

Experimental design

In this field experiment, we focus on the evaluation of ideas within companies. Therefore, we recruited innovation managers in our partner company and asked them to evaluate real ideas that other employees of our partner firm had proposed. We designed an online evaluation interface with the survey tool Qualtrics, where we will also collect the data.

Design details

Randomization method
Randomization done by Qualtrics
Randomization unit
Individual-idea pair
Clusters
Each innovation manager evaluates 48 ideas
Observations
40 innovation managers * 48 ideas = 1920 observations at the evaluator-idea level
Arms
See the full pre-analysis plan
Primary outcomes
Dependent variables. We will employ Evaluation score as our main dependent variable to test all hypotheses. For each idea, innovation managers are asked to rate the overall quality of an idea on a 7-point scale (1=very low to 7=very high; “On a scale of 1 to 7 (1 lowest to 7 highest), please assess the overall quality of the idea.”). In addition, we ask innovation managers to rate the ideas along three specific dimensions, which are regularly used at our partner firm: 1. Desirability (defined as “The degree to which a solution to a problem addresses someone’s needs.”), 2. Feasibility (defined as “The degree to which a solution is possible and suitable for [name of partner firm] to implement.”), 3. Viability (defined as “The degree to which an idea makes business sense for [name of partner firm].”). Innovation managers will be asked to rate the ideas along those three dimensions on a 7-point scale (1=very low to 7=very high; “On a scale of 1 to 7 (1 lowest to 7 highest), please rate different aspects of this idea. You can scroll over the items to see a short definition.”). Finally, we will ask innovation managers whether an idea should be promoted to the next round, which they can answer with yes or no (“Would you like to promote this idea to proceed to the next round?”). The answer to this question will be recorded in the dummy variable Next round. We will use the variables Desirability, Feasibility, Viability and Next Round as alternative dependent variables in stacked regressions, e.g. to investigate whether their systematic differences on these dimensions with respect to H3 to H5 (see “Additional Analyses”).

Cited by

No citation data available for this registration yet.

Keywords