Reproducibility in research – Day 2

Lars Vilhuber

Marie Connolly

2026-09-09

Follow along

larsvilhuber.github.io/workshop-uppsala-2026/presentation/day2.html (PDF)

The program

Day 2

• 9:00 Welcome
• 9:05 Walkthrough
• 9:15 Goals
• 9:30 🔒 Hands-on Exercise: A very imperfect example
• 9:45 Discussion of the “Very imperfect example”
• 10:00 Setting yourself up for reproducibility from Day 1
• 10:30 Break
• 10:45 How to correctly document a replication package (and why!)
• 11:15 Reproducibility when some data are confidential
• 11:45 🔒 Explanation of “homework”

Day 2

• 9:00 Review of “homework”, assignment of reviewers, active review
• 9:30 Discussion of results from review
• 10:00 Advanced reproducibility setup
• 10:45 Break
• 11:00 Choices:
  • Provenance and preservation
  • APIs and principled retrieval (with SIVACOR)
• 11:45 Wrap up

Review of the “homework”

Review of the “homework”

• Assignment of reviewers
• Active review

🔒Explanation of “homework”

Active review

• Can you run the code?
• Can you find the data (based on the README!)
• Do the numbers match the “manuscript”?

• FRED
• IPUMS

Start

Discussion of results from review

Discussion

• What did you find?
• What was easy? What was hard?
• What would have made it reproducible?

Advanced reproducibility setup

Advanced reproducibility setup

.. how to make your replication package check itself.

➡️ Advanced reproducibility setup

How to run Stata (or R, or Python) reproducibly

➡️ How to create log files

How to install packages reproducibly

➡️ How to control your environment

Break

Choices

Choices

Pick one:

• ➡️ Provenance and preservation
• ➡️ APIs and principled retrieval (with SIVACOR)

Choice A: Provenance and preservation

➡️ Provenance and preservation

Choice B: APIs and principled retrieval

• ➡️ What to do when data come from an API?
• ➡️ SIVACOR

Wrap up

Any further thoughts?

Pearls Before Swine

Where to?

Choices

• Issue No. 1: Data Access and preservation 💥 [a]
• Issue No. 2: Confidential data - same ⬆️! [a] [b]
• Lifecycle checking: Self-checking reproducibility and presentation
• New challenges: AI and Big Data but same ⬆️!
• New methods: Transparency outsourced or certified
• Implementation in academia: Students!

Resources

README

Lars Vilhuber, Connolly, M., Koren, M., Llull, J., & Morrow, P. (2022). A template README for social science replication packages (v1.1). Social Science Data Editors. https://doi.org/10.5281/zenodo.7293838

You can download the Word, LaTeX, or Markdown version of the README with lots of examples.

Other guidance

• Presentation on “Self Checking Reproducibility” and its associated website

• Guidance when (some) data are confidential: https://labordynamicsinstitute.github.io/reproducibility-confidential/

• Guidance for citations: https://social-science-data-editors.github.io/guidance/addtl-data-citation-guidance.html

Extra info

• This document’s source: https://github.com/larsvilhuber/workshop-uppsala-2026
• Licensed under

Sources

• Images: NYT, Bluesky 1, 2, Ike Hayman/Wikimedia

Fin.