Motivating metadata contributions for data re-use and reproducibility

How to cite

Chen, Yan; Li, Linfeng; Vilhuber, Lars; Levenstein, Margaret. 2020. "Motivating metadata contributions for data re-use and reproducibility." AEA RCT Registry. AEARCTR-0006159. https://doi.org/10.1257/rct.6159-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Yan Chen* , Linfeng Li , Lars Vilhuber , Margaret Levenstein

* Primary investigator

Registry ID
AEARCTR-0006159
DOI
https://doi.org/10.1257/rct.6159-1.1
First registered
Last updated
Status
Completed
Trial dates
2020-08-03 – 2020-09-30
Countries
United States of America ()
JEL codes
H49
Registry record
http://www.socialscienceregistry.org/trials/6159

Abstract

Metadata is data about the data. It is introduced for archival purposes but now serves a more active role: machine-readable metadata is crucial for making the documented studies "findable" by modern search engines. We investigate different motivational treatments for soliciting contribution to metadata. In our experiment, we approach the authors who published papers with online data sets and ask them to provide study-level metadata for the studies that they have published. Our investigation will also shed light on the crowdsourcing approach to supply high-quality metadata at scale.

Intervention

Participants will receive personalized emails generated using different templates corresponding to different experimental conditions, all of which encourage them to contribute metadata to their published studies.

Experimental design

Subjects will receive a personalized email inviting them to provide metadata for one of their published studies. We introduce one control message and two variants of treatment message. In the baseline template, we describe the task and invite subjects to provide metadata in a customized survey interface. In treatment 1 and treatment 2, we insert one extra paragraph into the baseline template that explains the findability implication of metadata. Treatment 1 and treatment 2 are different in how the new paragraph concludes.

Design details

Randomization method
Randomization was done in office by a computer. Relying on the network of coauthors, we extract a set of independent articles such that none of the articles we include in the experiment shall have any common authors. In practice, we take out "bridging" edges from the original network and generate a reduced network with fewer edges. We randomly pick one article from each component of the network in the reduced network and proceed to the randomization step. As the articles are chosen from components in the reduced network, we block by the "origin" of the components as well as the year of publication and number of authors for the articles. Within each block, we reshuffle the list and stack the list of articles in the same block on top of another list of articles from another block. Lastly, in the master list where blocks are stacked on top of each other, we divide the index by 3 and assign experimental conditions based on the residual from the division.
Randomization unit
We cluster treatment at the article level, where all authors of the same article are assigned to the same experimental condition.
Clusters
1,458 articles
Observations
3,044 participants
Arms
Each treatment arm has 486 clusters (articles).
Primary outcomes
Individual's willingness to participate; Article-level metadata contribution.
Secondary outcomes
willingness to directly update metadata; self-reported motivation for contributing to metadata
Sponsors
Name: NSF Location: United States Url: https://www.nsf.gov/

Cited by

No citation data available for this registration yet.

Keywords