Reproducibility Tutorial

Lars Vilhuber

2025-06-22

Who am I?

Lars Vilhuber

Executive Director of the Labor Dynamics Institute and Senior Research Associate in the Economics Department at Cornell University, and the American Economic Association’s Data Editor.

Journals

Data Editor of the AEA

2500 Manuscripts and 4500 Reports, approx. 4400 authors reached.

DCAP

Data and Code Transparency in Economics

Transparency

  • Provenance of the data
  • Processing of the data, from raw data to results (code)

It is the policy of the American Economic Association to publish papers only if the data used in the analysis are clearly and precisely documented and access to the data and code is clearly and precisely documented and is non-exclusive to the authors.

Completeness

  • All data needs to be identified and and access described
  • All code needs to be described and provided
  • All materials must be provided (survey forms, etc.)

Authors … must provide, prior to acceptance, the data, programs, and other details of the computations sufficient to permit replication

Preservation

  • All data needs to be preserved for future replicators
    • Ideally, within the replication package, subject to ToU, for convenience
    • Otherwise, in a trusted repository

Preservation

  • Code must be in a trusted repository
    • Usually, within the replication package
    • Websites, Github, are not acceptable

Historically

AER 1911 thanks to Stefano Dellavigna

Modern preservation

Exceptions to the Policy

None

… there is a grey zone:

  • When data do not belong to researcher, no control over preservation, access!
  • Sometimes, ToU prevent researcher from revealing metadata (name of company, location)

Transparency again

  • However:
    • No exception for need to describe access (own and other)
    • No exception for need to fully describe processing (possibly with redacted code)

Reproducibility in Economics and beyond

Data Editors

Common policies

https://social-science-data-editors.github.io/

Elsewhere: Political Science

APSR

AJPS

Elsewhere: Sociology

Sociological Science