A simple example

A simple example#

Let’s consider a simple example in Python, where this functionality is (nearly) native. We will use the virtualenv package, which is a standard part of Python. First, let’s consider what happens when we install a package in the “base” environment, the default when you start Python.

# list installed packages
pip list

outputs:

Package            Version
------------------ ----------
certifi            2023.11.17
charset-normalizer 3.3.2
idna               3.6
pip                23.2.1
PyYAML             6.0.1
requests           2.31.0
urllib3            2.1.0
wget               3.2
zenodo-get         1.5.1

These are packages I have installed to work. Now let’s create a new environment, that activate it to see what happens:

POSIX (Linux, macOS, etc.)

# create a new environment
python -m venv myenv
# activate the new environment
source myenv/bin/activate

Windows/command

# create a new environment
python -m venv myenv
# activate the new environment
myenv\Scripts\activate.bat
# On Windows/Bash:
source myenv/Scripts/activate

Windows/bash

# create a new environment
python -m venv myenv
# activate the new environment
source myenv/Scripts/activate

Let’s see what happens when we list the installed packages:

# list installed packages
pip list

will output

Package Version
------- -------
pip     23.2.1

So where did all the other packages go? One hint is if we identify which Python we are using:

which python

/c/Users/lv39/Documents/GitHub/self-checking-reproducibility/myenv/Scripts/python

So the operating system is looking for Python in the myenv directory. And this is also where pip will look. In fact, all of the packages previously listed are still on my computer - but are not accessible as long as I have this environment active.

Note

Technically, we modified the operating system’s PATH variable. Every operating system has an ordered list through which it searches for commands and software.

Our environment got added to the top of the various directories listed there, so that the particular location for Python is found first. Python, in turn, then uses the location of python in sys.prefix and sys.exec_prefix to find the other packages. See Python documentation.

Installing packages when an environment is active#

TODO

Walk through the steps to install packages, and identify where they go.

Installing precise versions of packages#

TODO

Walk through the steps to install specific versions of packages, and where they come from.

Making environments portable and reproducible#

This is deferred to later.

Warning

While we used interactive commands to install the various packages here, that was only for illustrative purposes. Always using a requirements.txt file to specify the packages to be installed. We will address how and when to use that file in the next section.