Reproducibility in research – Ensuring the transparency and credibility of your work
- May 26 (1:00 p.m. to 4:00 p.m.)
- May 27, 2025 (9:00 a.m. to 4:00 p.m.)
Location: DS-1520, pavillon DS, UQAM, 320, rue Sainte-Catherine Est, Montreal, Québec (see details on last tab)
Registration required, see registration link.
Sponsored by QICSS, part of the Canadian Economics Association meetings pre-program.
- 13:00 Welcome
- 13:05 Walkthrough
- 13:15 Goals
- 13:30 Technical setup, possible team formation
- 13:45 🔒Hands-on Exercise: A very imperfect example
- 14:00 Motivation: A tale of ineffective technical collaboration
- 14:05 Day 1: Setting yourself up for reproducibility
- 16:00 End of day one
- 9:00 Discussion of the “Very imperfect example”
- 9:30 Documenting it all: How to correctly document a replication package (and why!)
- 10:30 Break
- 10:45 How to run Stata! or R! (reproducibly)
- 11:00 Extra: How to install Stata packages
- 11:15 Topic A (see Survey)
- 12:00 Break
- 13:00 🔒When data cleaning is 🔒critical
- 13:30 Topic B (see Survey)
- 14:15 Break
- 14:30 Hands-on: Improving the replication package (very imperfect -> a lot better)
- 15:00 Hands-on: Testing it all
- 15:15 Wrap up
- 16:00 Fin.
For part of Day 2, we have a choice:
- Reproducibility when some data are confidential
- Preserving raw survey data
- Advanced self-checking of reproducibility
To identify what we should be speaking about on Day 2, please fill out this other survey:
https://cornell.yul1.qualtrics.com/jfe/form/SV_cNkhKL69K2Ob7o2
Results
| Preferred_topic | Frequency | Percent |
|---|---|---|
| Advanced self-checking of reproducibility | 1 | 16.67 |
| Preserving raw survey data | 2 | 33.33 |
| Reproducibility when some data are confidential | 3 | 50.00 |
- Discussions on concerns about ethics in data dissemination (privacy, scraping, copyright, etc.)
- Reproducibility for LLM and AI
Guidance
Some additional guidance can be found on the website of the Social Science Data Editors (URLs subject to change):
Examples of replication packages
With confidential data
- https://doi.org/10.3886/E154241V2 not only code, but faces the problem that IRS data cannot have variables revealed. Their workaround is not the same one as in this tutorial.
- https://doi.org/10.3886/E162581V1
Using containers:
- Kline et al (2024) “A Discrimination Report Card: primary replication package, with container specification, image on Docker Hub, and preserved image on Zenodo.
- Herbert et al (2024) “Reproduce to validate”: primary replication package, container specification and preserved image on Borealis.ca.
Extra info
- This document’s source: https://github.com/larsvilhuber/summer-school-qicss-2025
- Licensed under
