Revealed work preferences of MTurkers

How to cite

Nagel, Lisa; Hornuf, Lars. 2021. "Revealed work preferences of MTurkers." AEA RCT Registry. AEARCTR-0007466. https://doi.org/10.1257/rct.7466-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Lisa Nagel* , Lars Hornuf

* Primary investigator

Registry ID
AEARCTR-0007466
DOI
https://doi.org/10.1257/rct.7466-1.0
First registered
Last updated
Status
On going
Trial dates
2021-08-03 – 2021-10-01
Countries
Private
Secondary IDs
DFG HO 5296/3-1
Registry record
http://www.socialscienceregistry.org/trials/7466

Abstract

This study will contribute to a better understanding of the motivation of workers active on microtask online platforms. Recent literature has focused mainly on self-reported motivation and thus measured stated preferences (SP), but little has been done to investigate revealed work preferences (RP). Therefore add to the existing literature by investigating RP of MTurkers in the US and India, the two largest MTurker groups.
Because of the higher average income in the USA (Prydz & Wadhwa, 2019), the amount of payment on MTurk constitutes a low stake in the USA and a high stake in India. Since low stakes and high stakes can have an impact on worker motivation, we examine possible differences between RP of workers from the USA and India.
In this study we will run a partial profile design conjoint analysis in which MTurkers can choose the job they later perform. In particular, we will investigate (1) whether the RP of MTurkers are in line with the findings of previous research; (2) whether SP and RP of MTurkers are inline in our study; and (3) whether high or low stakes in terms of payment affect SP and RP of MTurkers.

Intervention

In our choice experiment there is one intervention:
Each participant is presented with 14 choice tasks. In each task the participant has to choose the preferred of two alternatives. Each alternative is characterized by a set of four mostly discretely valued job attributes. In advance, a total of 300 combinations with four variables each were generated. Each participant is randomly shown 14 choice tasks, meaning the participants will not see the same alternatives.

Experimental design

We will collect the data by creating a task on MTurk. MTurkers will see our task in the same manner as any other tasks and have to decide if they want to participate based to the task description and conditions (e.g., duration, payment). We conduct an incentivized decision experiment in the field. To identify RP of workers we use a partial profile conjoint analysis that determines the task they are later going to perform. We measure the SP based on a ranking based conjoint analysis that does not determine the task workers are going to perform.
We will run the identical study on MTurk for workers based in the USA and India.
Participant will start filling out a short questionnaire. Second, MTurkers are going to conduct the decision experiment. MTurkers will be informed that they can influence the task they have to do at the end of the study, based on their decisions in this part of the study. As part of a partial profile conjoint analysis workers decide which work characteristics the task they are going to work on should have, which enables us to investigate the RP of the MTurkers. Participants will several times see two job descriptions and have to decide which job they prefer. Overall, there will be 14 rounds with two job descriptions each. In every round, participants will choose which of the jobs presented they like better. Each job contains four measures of motivation, which were chosen based on the existing literature on motivation of workers. Hence, we have formulated 17 short statements of job attributes based upon the literature, which measure the different characteristics of work motivation. The combination of the variables and levels shown in the partial profile design conjoint analysis will be randomly assigned to the participants. Each choice set contains two job descriptions, each consisting of four statements. Each round both jobs show the same set of motivational factors but levels might differ (e.g. fun high vs. low).
After finishing the task of the partial profile conjoint analysis, participants will answer a survey.
Then Participants are asked to simply rank the job characteristics they saw at the beginning of the experiment according to the attractiveness of the characteristics, without having an influence on the job workers will perform. This ranking-based conjoint analysis allows us to capture the SP of participants, which we can then compare with the RP from the incentivized partial profile design conjoint analysis.
The last part of this experiment is a task based on the decisions the MTurkers made during the
study. Based on their statements they will either perform a real effort task that has a high degree of intrinsic motivation (fun) or a real effort task that has a high degree of extrinsic motivation (payment). If for example a participant stated that he or she would prefer a job that is fun over a job that has a high monetary reward the participant will be able to work on the fun task. In our case participant will see eight pictures of colorful landscapes. In one of the pictures is a hidden animal they have to find. If participants choose payment over fun the workers are paid 10% extra and will see eight pictures containing each three rows with numbers. Participants should find the picture that contains the number zero at least three times. The participants will see those eight pictured for seven rounds. To check whether our incentivization has worked, we are going to ask the participant, whether they believe that the task they just did was chosen for them because of the statements they made during the partial profile conjoint analysis.

Design details

Randomization method
The randomization is based on a model we generated with the software Sawtooth. Using randomized combinations of our variables, 300 decision tasks were generated. We tested our model of decision tasks, measuring different concepts of extrinsic and intrinsic motivation, in Sawtooth and a good fit was predicted, which implies that the occurrence probability of the items is equally distributed for our targeted number of participants. From the pool of 300 decision tasks created in this way, 14 tasks are randomly displayed to each subject in our experiment.
Randomization unit
The randomization is on individual level
Clusters
Since the experiment is not clustered, the number of clusters is the same as the sample size (see below).
Observations
1500 individual workers from the crowdsourcing platform Amazon Mechanical Turk. 750 workers from the USA 750 workers from India
Arms
There is only one treatment. The randomization of the shown choice sets is the basis for this decision experiment. As our model prediction could show, the occurrence probability of the items is equally distributed for our targeted number of participants.
Primary outcomes
Our first outcome of interest is if the RP of MTurkers are in line with the findings of previous research. Our second outcome of interest is whether SP and RP of MTurkers are identical in our study. Our third outcome of interest is whether high or low stakes in terms of payment affect SP and RP of MTurkers.
Secondary outcomes
Our first secondary outcome of interest is whether age has an influence on work motivation (intrinsic, extrinsic). Our second secondary outcome of interest is whether sex has an influence on work motivation (intrinsic, extrinsic).

Cited by

No citation data available for this registration yet.

Keywords