Reproducibility Tutorial

Lars Vilhuber

2025-06-22

Who am I?

Lars Vilhuber

Executive Director of the Labor Dynamics Institute and Senior Research Associate in the Economics Department at Cornell University, and the American Economic Association’s Data Editor.

Data Editor of the AEA

2500 Manuscripts and 4500 Reports, approx. 4400 authors reached.

Data and Code Transparency in Economics

Transparency

Provenance of the data
Processing of the data, from raw data to results (code)

It is the policy of the American Economic Association to publish papers only if the data used in the analysis are clearly and precisely documented and access to the data and code is clearly and precisely documented and is non-exclusive to the authors.

Completeness

All data needs to be identified and and access described
All code needs to be described and provided
All materials must be provided (survey forms, etc.)

Authors … must provide, prior to acceptance, the data, programs, and other details of the computations sufficient to permit replication

Preservation

All data needs to be preserved for future replicators
- Ideally, within the replication package, subject to ToU, for convenience
- Otherwise, in a trusted repository

Preservation

Code must be in a trusted repository
- Usually, within the replication package
- Websites, Github, are not acceptable

Historically

AER 1911 thanks to Stefano Dellavigna

Modern preservation

Exceptions to the Policy

None

…

… there is a grey zone:

When data do not belong to researcher, no control over preservation, access!
Sometimes, ToU prevent researcher from revealing metadata (name of company, location)

Transparency again

However:
- No exception for need to describe access (own and other)
- No exception for need to fully describe processing (possibly with redacted code)

Reproducibility in Economics and beyond

Data Editors

American Economic Association (8)
Econometric Society (3)
Canadian Journal of Economics (1)
Royal Economic Society (2)
Western Economic Association International (1)
European Economic Association (1)
Review of Economic Studies (1)

Common policies

https://social-science-data-editors.github.io/

Elsewhere: Political Science

Elsewhere: Sociology