Introduction

This document describes a few possible steps to create reproducible research using LLM and AI systems. It is not meant to be exhaustive, and it is not meant to be prescriptive. This is a quickly evolving environment, and there may be newer methods since this document was published.

Computational Empathy

The key ingredient is what I call “computational empathy” - thinking about what an unknown person attempting to reproduce the results in your paper might face, what they might know and assume, and more importantly, what they might not know or know to assume. While the replication package might very well run on your computer, that is by no means evidence that it will run on someone else’s computer.

How to read this document

The table of contents goes initially from easy to more complex. Each section should be seen as one method of running, with varying levels of “trust” in how robust it is to replicators’ environments. Some can be combined, others may not work well together.

TL;DR

Techy lingo for “too long, didn’t read”. A summary of the most important takeaways will be at the top of each section.

A presentation that may be more up-to-date is available here.

How to contribute

Content is .