Reproducibility in research – Ensuring the transparency and credibility of your work

Details

• May 26 (1:00 p.m. to 4:00 p.m.)
• May 27, 2025 (9:00 a.m. to 4:00 p.m.)

Location: DS-1520, pavillon DS, UQAM, 320, rue Sainte-Catherine Est, Montreal, Québec (see details on last tab)

Registration required, see registration link.

Sponsored by QICSS, part of the Canadian Economics Association meetings pre-program.

Follow along

Day 1

• 13:00 Welcome
  
• 13:15 Walkthrough
  
• 13:30 Technical setup, possible team formation
• 13:45 🔒Hands-on Exercise: A very imperfect example (link coming)
• 14:00 Day One: Setting yourself up for reproducibility
  
• 16:00 End of day one

Day 2

• 9:00 Discussion of the “Very imperfect example”
• 9:30 Documenting it all: How to correctly document a replication package (and why!)
• 10:30 Break
• 10:45 How to run Stata! or R! (reproducibly)
• 11:00 Extra: How to install Stata packages
• 11:15 Topic A (see Survey)

---

• 12:00 Break

---

• 13:00 🔒When data cleaning is critical
• 13:30 Topic B (see Survey)
• 14:15 Break
• 14:30 Hands-on: Improving the replication package (very imperfect -> a lot better)
• 15:00 Hands-on: Testing it all
• 15:15 Wrap up
• 16:00 Fin.

Survey for Day 2

For part of Day 2, we have a choice:

• Reproducibility when some data are confidential
• Depositing a replication package
• Advanced self-checking of reproducibility

To identify what we should be speaking about on Day 2, please fill out this other survey:

https://cornell.yul1.qualtrics.com/jfe/form/SV_cNkhKL69K2Ob7o2

Results

No data yet. Check back later.

Possible extra

• Discussions on concerns about ethics in data dissemination (privacy, scraping, copyright, etc.)
• Reproducibility for LLM and AI

Extras

Guidance

Some additional guidance can be found on the website of the Social Science Data Editors (URLs subject to change):

• https://social-science-data-editors.github.io/guidance/DCAS_Restricted_data.html#us-census-bureau-and-fsrdc
• https://social-science-data-editors.github.io/guidance/Requested_information_hosting.html#self-generated-repositories

Examples of replication packages

With confidential data

• https://doi.org/10.3886/E154241V2 not only code, but faces the problem that IRS data cannot have variables revealed. Their workaround is not the same one as in this tutorial.
• https://doi.org/10.3886/E162581V1

Using containers:

• Kline et al (2024) “A Discrimination Report Card: primary replication package, with container specification, image on Docker Hub, and preserved image on Zenodo.
  
• Herbert et al (2024) “Reproduce to validate”: primary replication package, container specification and preserved image on Borealis.ca.

Extra info

• This document’s source: https://github.com/larsvilhuber/summer-school-qicss-2025
• Licensed under

Map

UQAM Pavillon DS