[![jazzband-image]][jazzband] [![pypi][pypi-image]][pypi] [![pyversions][pyversions-image]][pyversions] [![pre-commit][pre-commit-image]][pre-commit] [![buildstatus-gha][buildstatus-gha-image]][buildstatus-gha] [![codecov][codecov-image]][codecov] [![Matrix Room Badge]][Matrix Room] [![Matrix Space Badge]][Matrix Space] [![discord-chat-image]][discord-chat]

pip-tools = pip-compile + pip-sync

A set of command line tools to help you keep your pip-based packages fresh, even when you've pinned them. You do pin them, right? (In building your Python application and its dependencies for production, you want to make sure that your builds are predictable and deterministic.)

[![pip-tools overview for phase II][pip-tools-overview]][pip-tools-overview]

Installation

Similar to pip, pip-tools must be installed in each of your project's virtual environments:

$ source /path/to/venv/bin/activate
(venv) $ python -m pip install pip-tools

Note: all of the remaining example commands assume you've activated your project's virtual environment.

Example usage for pip-compile

The pip-compile command lets you compile a requirements.txt file from your dependencies, specified in either pyproject.toml, setup.cfg, setup.py, or requirements.in.

Run it with pip-compile or python -m piptools compile (or pipx run --spec pip-tools pip-compile if pipx was installed with the appropriate Python version). If you use multiple Python versions, you can also run py -X.Y -m piptools compile on Windows and pythonX.Y -m piptools compile on other systems.

pip-compile should be run from the same virtual environment as your project so conditional dependencies that require a specific Python version, or other environment markers, resolve relative to your project's environment.

Note: If pip-compile finds an existing requirements.txt file that fulfils the dependencies then no changes will be made, even if updates are available. To compile from scratch, first delete the existing requirements.txt file, or see Updating requirements for alternative approaches.

Requirements from pyproject.toml

The pyproject.toml file is the latest standard for configuring packages and applications, and is recommended for new projects. pip-compile supports both installing your project.dependencies as well as your project.optional-dependencies. Thanks to the fact that this is an official standard, you can use pip-compile to pin the dependencies in projects that use modern standards-adhering packaging tools like Setuptools, Hatch or flit.

Suppose you have a 'foobar' Python application that is packaged using Setuptools, and you want to pin it for production. You can declare the project metadata as:

[build-system]
requires = ["setuptools", "setuptools-scm"]
build-backend = "setuptools.build_meta"

[project]
requires-python = ">=3.9"
name = "foobar"
dynamic = ["dependencies", "optional-dependencies"]

[tool.setuptools.dynamic]
dependencies = { file = ["requirements.in"] }
optional-dependencies.test = { file = ["requirements-test.txt"] }

If you have a Django application that is packaged using Hatch, and you want to pin it for production. You also want to pin your development tools in a separate pin file. You declare django as a dependency and create an optional dependency dev that includes pytest:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "my-cool-django-app"
version = "42"
dependencies = ["django"]

[project.optional-dependencies]
dev = ["pytest"]

You can produce your pin files as easily as:

$ pip-compile -o requirements.txt pyproject.toml
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --output-file=requirements.txt pyproject.toml
#
asgiref==3.6.0
    # via django
django==4.1.7
    # via my-cool-django-app (pyproject.toml)
sqlparse==0.4.3
    # via django

$ pip-compile --extra dev -o dev-requirements.txt pyproject.toml
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --extra=dev --output-file=dev-requirements.txt pyproject.toml
#
asgiref==3.6.0
    # via django
attrs==22.2.0
    # via pytest
django==4.1.7
    # via my-cool-django-app (pyproject.toml)
exceptiongroup==1.1.1
    # via pytest
iniconfig==2.0.0
    # via pytest
packaging==23.0
    # via pytest
pluggy==1.0.0
    # via pytest
pytest==7.2.2
    # via my-cool-django-app (pyproject.toml)
sqlparse==0.4.3
    # via django
tomli==2.0.1
    # via pytest

This is great for both pinning your applications, but also to keep the CI of your open-source Python package stable.

Requirements from setup.py and setup.cfg

pip-compile has also full support for setup.py- and setup.cfg-based projects that use setuptools.

Just define your dependencies and extras as usual and run pip-compile as above.

Requirements from requirements.in

You can also use plain text files for your requirements (e.g. if you don't want your application to be a package). To use a requirements.in file to declare the Django dependency:

# requirements.in
django

Now, run pip-compile requirements.in:

$ pip-compile requirements.in
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile requirements.in
#
asgiref==3.6.0
    # via django
django==4.1.7
    # via -r requirements.in
sqlparse==0.4.3
    # via django

And it will produce your requirements.txt, with all the Django dependencies (and all underlying dependencies) pinned.

(updating-requirements)=

Updating requirements

pip-compile generates a requirements.txt file using the latest versions that fulfil the dependencies you specify in the supported files.

If pip-compile finds an existing requirements.txt file that fulfils the dependencies then no changes will be made, even if updates are available.

To force pip-compile to update all packages in an existing requirements.txt, run pip-compile --upgrade.

To update a specific package to the latest or a specific version use the --upgrade-package or -P flag:

# only update the django package
$ pip-compile --upgrade-package django

# update both the django and requests packages
$ pip-compile --upgrade-package django --upgrade-package requests

# update the django package to the latest, and requests to v2.0.0
$ pip-compile --upgrade-package django --upgrade-package requests==2.0.0

You can combine --upgrade and --upgrade-package in one command, to provide constraints on the allowed upgrades. For example to upgrade all packages whilst constraining requests to the latest version less than 3.0:

$ pip-compile --upgrade --upgrade-package 'requests<3.0'

Understanding output file behavior

When you run pip-compile, it reads any existing output file (e.g., requirements.txt) and uses the versions specified there as constraints. This ensures that the output is stable -- pip-compile will prefer to use existing pinned versions.

When upgrading packages or adding new packages to the input, it is possible for the existing pins to conflict with new packages. In such cases, pip-compile will update the pins and emit a warning (e.g., Discarding foo=1.2.3 to proceed).

In order to resolve dependencies afresh, pip-compile provides the following options:

• --upgrade / -U flag to upgrade all packages
• --upgrade-package <package>/-P <package> to upgrade specific packages

Using hashes

If you would like to use Hash-Checking Mode available in pip since version 8.0, pip-compile offers --generate-hashes flag:

$ pip-compile --generate-hashes requirements.in
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --generate-hashes requirements.in
#
asgiref==3.6.0 \
    --hash=sha256:71e68008da809b957b7ee4b43dbccff33d1b23519fb8344e33f049897077afac \
    --hash=sha256:9567dfe7bd8d3c8c892227827c41cce860b368104c3431da67a0c5a65a949506
    # via django
django==4.1.7 \
    --hash=sha256:44f714b81c5f190d9d2ddad01a532fe502fa01c4cb8faf1d081f4264ed15dcd8 \
    --hash=sha256:f2f431e75adc40039ace496ad3b9f17227022e8b11566f4b363da44c7e44761e
    # via -r requirements.in
sqlparse==0.4.3 \
    --hash=sha256:0323c0ec29cd52bceabc1b4d9d579e311f3e4961b98d174201d5622a23b85e34 \
    --hash=sha256:69ca804846bb114d2ec380e4360a8a340db83f0ccf3afceeb1404df028f57268
    # via django

Output File

To output the pinned requirements in a filename other than requirements.txt, use --output-file. This might be useful for compiling multiple files, for example with different constraints on django to test a library with both versions using tox:

$ pip-compile --upgrade-package 'django<1.0' --output-file requirements-django0x.txt
$ pip-compile --upgrade-package 'django<2.0' --output-file requirements-django1x.txt

Or to output to standard output, use --output-file=-:

$ pip-compile --output-file=- > requirements.txt
$ pip-compile - --output-file=- < requirements.in > requirements.txt

Forwarding options to pip

Any valid pip flags or arguments may be passed on with pip-compile's --pip-args option, e.g.

$ pip-compile requirements.in --pip-args "--retries 10 --timeout 30"

Configuration

You can define project-level defaults for pip-compile and pip-sync by writing them to a configuration file in the same directory as your requirements input files (or the current working directory if piping input from stdin). By default, both pip-compile and pip-sync will look first for a .pip-tools.toml file and then in your pyproject.toml. You can also specify an alternate TOML configuration file with the --config option.

It is possible to specify configuration values both globally and command- specific. For example, to by default generate pip hashes in the resulting requirements file output, you can specify in a configuration file:

[tool.pip-tools]
generate-hashes = true

Options to pip-compile and pip-sync that may be used more than once must be defined as lists in a configuration file, even if they only have one value.

pip-tools supports default values for all valid command-line flags of its subcommands. Configuration keys may contain underscores instead of dashes, so the above could also be specified in this format:

[tool.pip-tools]
generate_hashes = true

Configuration defaults specific to pip-compile and pip-sync can be put beneath separate sections. For example, to by default perform a dry-run with pip-compile:

[tool.pip-tools.compile] # "sync" for pip-sync
dry-run = true

This does not affect the pip-sync command, which also has a --dry-run option. Note that local settings take preference over the global ones of the same name, whenever both are declared, thus this would also make pip-compile generate hashes, but discard the global dry-run setting:

[tool.pip-tools]
generate-hashes = true
dry-run = true

[tool.pip-tools.compile]
dry-run = false

You might be wrapping the pip-compile command in another script. To avoid confusing consumers of your custom script you can override the update command generated at the top of requirements files by setting the CUSTOM_COMPILE_COMMAND environment variable.

$ CUSTOM_COMPILE_COMMAND="./pipcompilewrapper" pip-compile requirements.in
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    ./pipcompilewrapper
#
asgiref==3.6.0
    # via django
django==4.1.7
    # via -r requirements.in
sqlparse==0.4.3
    # via django

Workflow for layered requirements

If you have different environments that you need to install different but compatible packages for, then you can create layered requirements files and use one layer to constrain the other.

For example, if you have a Django project where you want the newest 2.1 release in production and when developing you want to use the Django debug toolbar, then you can create two *.in files, one for each layer:

# requirements.in
django<2.2

At the top of the development requirements dev-requirements.in you use -c requirements.txt to constrain the dev requirements to packages already selected for production in requirements.txt.

# dev-requirements.in
-c requirements.txt
django-debug-toolbar<2.2

First, compile requirements.txt as usual:

$ pip-compile
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile
#
django==2.1.15
    # via -r requirements.in
pytz==2023.3
    # via django

Now compile the dev requirements and the requirements.txt file is used as a constraint:

```console $ pip-compile dev-requirements.in

This file is autogenerated by pip-compile with Python 3.10

by the following command:

pip-compile dev-requirements.in

django==2.1.15 # via # -c requirements.txt # django-debug-toolbar django-debug-toolbar==2.1 # via -r dev-requirements.in pytz==2023.3 # via # -c requirements.txt # django sqlparse==0.4.3 # via django-debug-too