Understanding Collective Intelligence

How to cite

Weidmann, Ben; Deming, David. 2019. "Understanding Collective Intelligence." AEA RCT Registry. AEARCTR-0002896. https://doi.org/10.1257/rct.2896-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Ben Weidmann* , David Deming

* Primary investigator

Registry ID
AEARCTR-0002896
DOI
https://doi.org/10.1257/rct.2896-2.0
First registered
Last updated
Status
Completed
Trial dates
2019-08-19 – 2020-04-17
Countries
United States of America ()
Registry record
http://www.socialscienceregistry.org/trials/2896

Abstract

Most jobs require teamwork. Are some people good team players? In this paper we design and test a new method for identifying individual contributions to group performance. We randomly assign people to multiple teams and predict team performance based on previously assessed individual skills. Some people consistently cause their group to exceed its predicted performance. We call these individuals “team players”. Team players score significantly higher on a well-established measure of social intelligence, but do not differ across a variety of other dimensions, including IQ, personality, education and gender. Social skills – defined as a single latent factor that combines social intelligence scores with the team player effect – improve group performance about as much as IQ. We find suggestive evidence that team players increase effort among teammates.

Intervention

Our study explores the performance of teams. Why are some groups ‘more than the sum of their parts’ while others fail live up to expectations? Is this predictable based on the characteristics of individuals in each group? We study these questions in the context of group problem solving. We pay particular attention to the causal impact that individuals may have on groups. As such, we will have as many ‘treatments’ as we have individuals who complete the study.

Experimental design

Participants are asked to complete 6 batteries of tasks, labelled: A, B1,B2,B3,B4 & C. These batteries are described in detail in the Statistical Analysis Plan. We present them here in brief.

First, participants work as individuals to complete a set of online tasks (Battery A). Battery A consists of:
- Big5 inventory;
- Reading the Mind in the Eyes;
- Ravens Advanced Progressive Matrices
- Three short tests of short-term memory (images; words; stories);
- A numerical optimization task (developed for the purpose of this study)

After completing individual assessments, participants come into the Harvard Decision Science Lab (HDSL) at least twice. During these visits, they are randomly allocated into groups of three people. Each participant will be a member of 4 groups. The first three groups complete batteries B1, B2, B3. The final group completes B4 and C. Batteries B1, B2, B3 and B4 consist of the following:
- a CFIT form (OR an advanced Ravens Progressive Matrices form)
- a Group Memory task (developed for this study);
- the Group Optimization task (developed for this study).

Finally, participants complete a validation task in their final group (Battery C; a cryptography task).

Overall, we anticipate that participants will spend 3-4 hours on our experiment.

Design details

Randomization method
Participants are asked to come to the lab in groups. To run a study ‘session’ there must be at least n=6 participants. Ideally, there will be n=9 or n=12 participants at a time. Allocation to groups is random, using a blocked design. The study managers (either Ben Weidmann or Research Assistants at the HDSL) will apply the following principles: - Avoid allocations where people are in a group with the same person more than once. In cases where n=6, this will not be possible, but will be adjusted for the analysis. - Avoid where people are working with people they know - Attempt to balance groups so that the overall level (and variance) of skill in each group is as-similar-as-possible across groups To achieve these aims, we apply the following procedures - The n participants in each session are divided into three equal blocks of size n/3, based on their overall score in Battery A [if n is not a multiple of three, excess participants are paid for their time and asked to return to another session]. Blocks can loosely be thought of as ‘high’, ‘medium’, and ‘low’ in terms of individual skill. - If people arrive at the lab with friends, they are automatically placed in the same block. - There are three bags, one for each block. Each bag has n/3 balls, marked with a letter. Bags have consecutive letters. For example, in a session of 9 people, the bag for ‘high’ performance will have balls with A,B,C; the ‘medium’ bag will have D,E,F; and the final bag has G,H,I. - Each participant draws a ball. This ball defines their 2 groups for that session. To return to the example of a 9-person session, allocations would be as follows: first set of groups: {ADG,BEH,CFI}; second set of groups: {AEI,BFG,CDH}.
Randomization unit
We randomize individuals to groups.
Clusters
Given the novelty of our design – coupled with the fact that we are exploring new measurement instruments – our power calculations are extremely uncertain. Moreover, recruitment may be challenging given the demands of our design in which people are required to come to the lab twice, and sessions require a minimum of 6 people to proceed. As such, we have the following stopping rules. We will proceed until we meet one of the following conditions: - Reached 450 participants [and exhausted our planned budget]; OR - Exceeded our target minimum of 207 participants AND achieved a standard error for σ_β of 0.01, or less (see Statistical Analysis Plan); OR - Exhaust the HDSL participant pool and are no longer able to schedule new groups
Observations
Our target is 207 individuals [which corresponds to 276 groups]
Arms
Our target is 207 individuals
Primary outcomes
We define a new measure: the Teamplayer Index. For each participant, we estimate this index. For more details, see the attached Statistical Analysis Plan.
Secondary outcomes
We explore a group-level outcome: “group efficiency”. This can be conceptualized as whether a group over- or under-performed controlling for the individual skill level of its members. See the Statistical Analysis Plan for details.

Papers from this trial

  1. Weidmann, B., & Deming, D., Team Players: How Social Skills Improve Group Performance, NBER Working Paper 27071 https://www.nber.org/papers/w27071

Cited by

No citation data available for this registration yet.

Keywords