UniDep - Unified Conda and Pip Dependency Management

UniDep streamlines Python project dependency management by unifying Conda and Pip packages in a single system. Learn when to use UniDep in our FAQ.

Handling dependencies in Python projects can be challenging, especially when juggling Python and non-Python packages. This often leads to confusion and inefficiency, as developers juggle between multiple dependency files.

• 📝 Unified Dependency File: Use either requirements.yaml or pyproject.toml to manage both Conda and Pip dependencies in one place.
• ⚙️ Build System Integration: Integrates with Setuptools and Hatchling for automatic dependency handling during pip install ./your-package.
• 💻 One-Command Installation: unidep install handles Conda, Pip, and local dependencies effortlessly.
• ⚡️ Fast Pip Operations: Leverages uv (if installed) for faster pip installations.
• 🏢 Monorepo-Friendly: Render (multiple) requirements.yaml or pyproject.toml files into one Conda environment.yaml file and maintain fully consistent global and per sub package conda-lock files.
• 🌍 Platform-Specific Support: Specify dependencies for different operating systems or architectures.
• 🔧 pip-compile Integration: Generate fully pinned requirements.txt files from requirements.yaml or pyproject.toml files using pip-compile.
• 🔒 Integration with conda-lock: Generate fully pinned conda-lock.yml files from (multiple) requirements.yaml or pyproject.toml file(s), leveraging conda-lock.
• 🥧 Pixi Support: Generate pixi.toml files from your dependency files, enabling Pixi-based workflows while keeping UniDep as the single source of truth.
• 🤓 Nerd stats: written in Python, 100% test coverage, fully-typed, all Ruff's rules enabled, easily extensible, and minimal dependencies

unidep is designed to make dependency management in Python projects as simple and efficient as possible. Try it now and streamline your development process!

[!TIP] Check out the example requirements.yaml and pyproject.toml below.

:books: Table of Contents

• :rocket: Bootstrap from Scratch
• :package: Installation
• :memo: requirements.yaml and pyproject.toml structure
• Example
  • Example requirements.yaml
  • Example pyproject.toml
• Key Points
• Supported Version Pinnings
• Conflict Resolution
  • How It Works
• Platform Selectors
  • Supported Selectors
  • Usage
  • Implementation
• Custom Pip Index URLs
  • How It Works
  • Example Usage
  • Generated Output
• [project.dependencies] in pyproject.toml handling
• :jigsaw: Build System Integration
• Local Dependencies in Monorepos
• PyPI Alternatives for Local Dependencies
• Overriding Nested Vendor Copies with use
  • Example: Override foo's bundled bar with your PyPI build
• All use values
• Build System Behavior
• Example packages
• Setuptools Integration
• Hatchling Integration
• :desktop_computer: As a CLI
• unidep merge
• unidep install
• unidep install-all
• unidep conda-lock
• unidep pixi
  • What unidep pixi generates
  • Dependency reconciliation rules (important)
  • Channels/platforms precedence
  • Example (single-file)
• unidep pip-compile
• unidep pip
• unidep conda
• unidep doctor
• ❓ FAQ
• Q: When to use UniDep?
• Q: Just show me a full example!
• Q: Uses of UniDep in the wild?
• Q: How do I force PyPI instead of a local path for one dependency?
• Q: How do I ignore a local dependency entirely?
• Q: A submodule brings its own copy of package X. How do I avoid conflicts?
• Q: How is this different from conda/mamba/pip?
• Q: I found a project using unidep, now what?
• Q: How to handle local dependencies that do not use UniDep?
• Q: Can't Conda already do this?
• Q: What is the difference between conda-lock and unidep conda-lock?
• Q: What is the difference between hatch-conda / pdm-conda and unidep?
• :hammer_and_wrench: Troubleshooting
• pip install fails with FileNotFoundError
• :warning: Limitations

:rocket: Bootstrap from Scratch

To get started quickly with UniDep, run the following command. This will download and install micromamba (recommended for fast Conda environment management), uv (recommended for faster pip installations), and then install UniDep:

"${SHELL}" <(curl -LsSf raw.githubusercontent.com/conda-incubator/unidep/main/bootstrap.sh)

[!NOTE] Micromamba and uv are recommended to optimize your installation experience, but they are not required if you prefer to use your existing Conda and pip setup.

[!WARNING] NEVER! run scripts from the internet without understanding what they do. Always inspect the script first!

Pin the hash of the bootstrap script with:

"${SHELL}" <(curl -LsSf raw.githubusercontent.com/conda-incubator/unidep/8a4fa7f70063481164f3e630fd752a65cc7aaacb/bootstrap.sh)

:package: Installation

To install unidep, run one of the following commands that use pipx (recommended), pip, or conda:

pipx install "unidep[all]"  # Recommended (install as a standalone CLI)

or

pip install "unidep[all]"

or

conda install -c conda-forge unidep

:memo: requirements.yaml and pyproject.toml structure

unidep allows either using a 1. requirements.yaml file with a specific format (similar but not the same as a Conda environment.yaml file) or 2. pyproject.toml file with a [tool.unidep] section.

Both files contain the following keys:

• name (Optional): For documentation, not used in the output.
• channels: List of conda channels for packages, such as conda-forge.
• dependencies: Mix of Conda and Pip packages.
• local_dependencies (Optional): List of paths to other requirements.yaml or pyproject.toml files to include.
• optional_dependencies (Optional): Dictionary with lists of optional dependencies.
• platforms (Optional): List of platforms that are supported (used in conda-lock).
• pip_indices (Optional): List of custom pip index URLs for private or alternative package repositories.
• requires_unidep (Optional): Version specifier for the unidep version required to parse the file.

Whether you use a requirements.yaml or pyproject.toml file, the same information can be specified in either. Choose the format that works best for your project.

Example

Example requirements.yaml

Example of a requirements.yaml file:

name: example_environment
requires_unidep: ">=3.4.1"  # (Optional) require this unidep version or newer
channels:
  - conda-forge
dependencies:
  - numpy                   # same name on conda and pip
  - conda: python-graphviz  # When names differ between Conda and Pip
    pip: graphviz
  - pip: slurm-usage >=1.1.0,<2  # pip-only
  - conda: mumps                 # conda-only
  # Use platform selectors
  - conda: cuda-toolkit =11.8    # [linux64]
local_dependencies:
  - ../other-project-using-unidep     # include other projects that use unidep
  - ../common-requirements.yaml       # include other requirements.yaml files
  - ../project-not-managed-by-unidep  # 🚨 Skips its dependencies!
optional_dependencies:
  test:
    - pytest
  full:
    - ../other-local-dep[test]  # include its optional 'test' dependencies
platforms:  # (Optional) specify platforms that are supported (used in conda-lock)
  - linux-64
  - osx-arm64
pip_indices:  # (Optional) additional pip index URLs for private packages
  - https://pypi.org/simple/  # Main PyPI index (automatically included if not specified)
  - https://private.example.com/simple/  # Private index
  - https://${PIP_USER}:${PIP_PASSWORD}@private.example.com/simple/  # Authenticated index with env vars

[!IMPORTANT] unidep can process this during pip install and create a Conda installable environment.yaml or conda-lock.yml file, and more!

[!NOTE] For a more in-depth example containing multiple installable projects, see the example directory.

Example pyproject.toml

Alternatively, one can fully configure the dependencies in the pyproject.toml file in the [tool.unidep] section:

[tool.unidep]
requires_unidep = ">=3.4.1"  # (Optional) require this unidep version or newer
channels = ["conda-forge"]
dependencies = [
    "numpy",                                         # same name on conda and pip
    { conda = "python-graphviz", pip = "graphviz" }, # When names differ between Conda and Pip
    { pip = "slurm-usage >=1.1.0,<2" },              # pip-only
    { conda = "mumps" },                             # conda-only
    { conda = "cuda-toolkit =11.8:linux64" }         # Use platform selectors by appending `:linux64`
]
local_dependencies = [
    "../other-project-using-unidep",    # include other projects that use unidep
    "../common-requirements.yaml",      # include other requirements.yaml files
    "../project-not-managed-by-unidep"  # 🚨 Skips its dependencies!
]
optional_dependencies = {
    test = ["pytest"],
    full = ["../other-local-dep[test]"]  # include its optional 'test' dependencies
}
platforms = [ # (Optional) specify platforms that are supported (used in conda-lock)
    "linux-64",
    "osx-arm64"
]
pip_indices = [ # (Optional) additional pip index URLs for private packages
    "https://pypi.org/simple/",  # Main PyPI index (automatically included if not specified)
    "https://private.example.com/simple/",  # Private index
    "https://${PIP_USER}:${PIP_PASSWORD}@private.example.com/simple/"  # Authenticated index with env vars
]

This data structure is identical to the requirements.yaml format, with the exception of the name field and the platform selectors. In the requirements.yaml file, one can use e.g., # [linux64], which in the pyproject.toml file is :linux64 at the end of the package name.

See Build System Integration for more information on how to set up unidep with different build systems (Setuptools or Hatchling).

[!IMPORTANT] In these docs, we often menti