Virtual Environments (in Python)

Isolating and documenting project dependencies.

Overview
pip
pipenv
venv
Using IDEs with virtual environments
Conclusions

Overview

Concept

Python comes with an ability to create virtual environments - specific areas of memory and disk space where you can install the dependencies and settings necessary to run a specific app in isolation from other apps on the same computer.

a virtual environment is a sandboxed areas of memory where modules can be installed without affecting the modules installed and available in other projects.
each project can have its own isolated environment with its own set of dependencies, which may be different from that of other projects.
the configuration of a given virtual environment can be saved into a simple file that can be shared; this allows multiple developers to set up the same environment on different machines.
when developing an application that depends upon a specific set of modules, it is good practice to install dependencies into a virtual environment.

Choices

There are multiple tools for setting Python virtual environments. We will focus on:

venv - virtual environment manager
pip - the standard Python package manager. Not used to create virtual environments, but used to install dependencies into them.
pipenv - a wrapper around venv and pip so you can use both through one interface

Another popular tool for the job is conda, but we will not discuss it here as it is most commonly used with the Anaconda distribution of Python, not the standard distribution.

pip

Overview

Before we discuss virtual environments, we must understand pip, the standard “package manager” used to download and install dependencies into a Python project.

pip comes with the standard distribution of Python…
on some systems, the command to run it is pip, on others it is pip3… if one doesn’t work, try the other. We use pip in the examples below.
The pip install command is used to install a module into the current Python environment, e.g. pip install Flask or pip install pymongo
By default, pip downloads modules from the Python Package Index (PyPi), but it can also install modules from other sources, such as git repositories.

Base environment

If no virtual environment is active, pip will install modules into the global base Python environment, which is shared by all projects on the computer.

However, if a virtual environment is active, dependencies installed with pip will be placed into the virtual environment for the current project, rather than the base environment.

Documenting dependencies

The pip freeze command is used to list all of the modules installed into the current Python environment.

To save the list of dependencies into a file commonly named requirements.txt, run:

pip freeze > requirements.txt
This file can be shared and used to re-install the same set of dependencies into a different computer or a different Python environment, e.g.

pip install -r requirements.txt

pipenv

Overview

The ability to make virtual environments with pipenv is relatively easy, but it does not come pre-installed with Python. It must be installed.

Install pipenv using pip:

pip install pipenv

Activate it:

pipenv shell

Your command line is now running within a virtual environment. Installing or uninstalling any modules will only affect the current environment.

Path to environment

Generally, you don’t need to worry about where pipenv stores the virtual environment files… it just works. However, in some cases, such as setting an IDE to use a specific environment, it may be helpful to know the path to the virtual environment files.

When running pipenv shell, the path to the virtual environment is printed to the console.

pipenv shell file path

Installing dependencies

Use the pipenv install command to download and install dependencies into the current virtual environment.

pipenv install Flask

pipenv install pymongo

As you install dependencies with pipenv, you will a file named Pipfile that is automatically created (if it doesn’t yet exist) and modified to list the dependencies you’ve installed.

Specifying a Python version

In cases where it is important, you can specify a specific version of Python to be used in the virtual environment:

For example:

pipenv --python 3.10

Configuration file

The file named Pipfile contains the list of dependencies for the current project.

dependencies are the other Python modules that this app depends upon to run.
the Pipfile lists each dependency and the version number that should be installed.
the Pipefile includes the version of Python, if any, that it has been specified for the virtual environment.
if your project already had a file named Pipenv when you activated the virtual environment, all dependencies listed therein would have been automatically installed. You can always ensure they have been installed by running pipenv install with no arguments.
more about pipenv

Naming virtual environments

If you have many virtual environments for many projects, it can be helpful to name them intuitive mnemonic names.

In pipenv, we do this by first creating an environment variable in the command shell (Terminal) named PIPENV_CUSTOM_VENV_NAME and setting it to our desired name.

export PIPENV_CUSTOM_VENV_NAME=my_custom_venv_name

Secondly, we create a new virtual environment which adopts that name:

pipenv shell

venv

Overview

If you are unable or refuse to use pipenv for some reason, you can use the more traditional venv instead. The ability to make virtual environments with venv comes included with standard Python distributions.

Using venv is heavily discouraged, since pipenv provides an easier-to-use interface for creating virtual environments and installing dependencies into them. However, you may encounter venv in other people’s projects and should be aware of it.

Create and activate

This command creates a new virtual environment with the name .venv:

python -m venv .venv # try 'python3' instead of 'python' if your system requires it

To activate the virtual environment named .venv…

On Mac:

source .venv/bin/activate

On Windows:

.venv\Scripts\activate.bat

Installing dependencies

Once a venv virtual environment has been activated, any dependencies installed with pip will be installed into the virtual environment, not the global environment.

pip install Flask

pip install pymongo

You can run pip freeze > requirements.txt to save the installed dependencies into a settings file named requirements.txt. This file can then be shared and used to install the same dependencies on other computers or in other virtual environments.

Note that more contemporary projects using pipenv instead of venv can create a backwards-compatible requirements.txt file (in addition to the Pipfile) by running pipenv lock -r > requirements.txt.

Using IDEs with virtual environments

Overview

When you have set a virtual environment using the command shell (i.e. terminal), the IDE you are using to edit the code may not be configured to use the same environment.

Any good Python IDE will allow you to select the virtual environment to use when running the code.

Visual Studio Code

For example, if you are using Visual Studio Code, you may notice that the linter in the IDE will flag code as incorrect that references modules that are indeed installed in the virtual environment. This is usually because the IDE is not set to use the same environment.

Python import not found

Visual Studio Code (continued)

To fix this, configure the IDE to use the same environment as the command shell. This can be done by selecting Python: Select Interpreter from the Command Palette and choosing the interpreter marked with the virtual environment you have set up.

If you don’t see your desired environment in the list, you may enter the path to it manually.

VSCode select Python interpreter

Visual Studio Code (continued again)

Once a virtual environment with the project’s dependencies installed has been selected, the code linter should not display any errors or warnings about the import statements.

Python import found

If the import statements are still flagged by the linter, make sure you have installed the dependencies into the virtual environment using pipenv install (if using pipenv to manage the environments) or pip install (if using venv instead, contrary to our advice).

Conclusions

Thank you. Bye