> ## Documentation Index
> Fetch the complete documentation index at: https://developer.watson-orchestrate.ibm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Python Dependency Management in watsonx Orchestrate

Understanding how Python dependencies work in watsonx Orchestrate is critical for building secure, repeatable deployments across environments. This guide clarifies what gets packaged during export, when dependencies are installed, and which patterns are officially supported for enterprise deployments.

## What are "dependencies" in watsonx Orchestrate?

<Warning>
  **Important**: *Dependencies* in watsonx Orchestrate are the **source specifications** listed in your `requirements.txt` file, **not** the compiled binary packages or installed libraries.
</Warning>

When you export a Python tool or agent, the export includes:

* ✅ Your Python source code files (`.py` files)
* ✅ Your `requirements.txt` file (dependency specifications)
* ✅ Any additional resources (data files, configuration files)
* ❌ **Does not include** the installed binary packages or virtual environment

This is a **source-only export model**. The actual Python packages (wheels, compiled binaries) are downloaded and installed separately during deployment.

### Example: What gets exported

```bash theme={null}
# When you export an agent, you get:
agent/
├── tools/
│   └── my_tool/
│       ├── my_tool.py              # ✅ Source code
│       ├── requirements.txt        # ✅ Dependency specifications
│       └── utils.py                # ✅ Additional source files
├── agents/
│   └── my_agent.yaml               # ✅ Agent configuration
└── connections/
    └── my_connection.yaml          # ✅ Connection definitions

# What is NOT included:
# - site-packages/ (installed libraries)
# - .venv/ (virtual environment)
# - Binary wheels or compiled packages
```

## When and where are dependencies installed?

Dependencies are installed **server-side** at **agent deployment time**, not during export or import.

### Installation timeline

1. **Development**: You create a tool with a `requirements.txt` file
2. **Import**: You import the tool using `orchestrate tools import`
3. **Deployment**: When the tool is deployed to an environment (Draft or Live):
   * watsonx Orchestrate reads your `requirements.txt`
   * Runs `pip install` server-side
   * Creates an isolated virtual environment (venv)
   * Installs all specified packages

<Note>
  This installation happens **in every environment** where you deploy the tool—DEV, QA, and PROD instances all perform their own dependency installation.
</Note>

### Virtual environment caching

To optimize performance and reduce redundant installations, watsonx Orchestrate uses a **canonical hash-based deduplication mechanism**:

1. **Hash computation**: When you deploy a tool, the system computes a unique fingerprint (hash) of your `requirements.txt` file
2. **Cache check**: The system checks if a virtual environment with this exact hash already exists
3. **Reuse or create**:
   * **Hash match found**: Reuses the existing cached venv (no installation needed)
   * **No match**: Creates new hash, builds new venv, stores in cache

This means packages are downloaded **once per unique requirements.txt** within a cluster, not once per tool or tenant.

#### When a new virtual environment is created

A new venv will be created when the dependency hash changes due to:

* **Dependency version changes**: Updating from `requests==2.32.3` to `requests==2.32.4`
* **Adding/removing packages**: Adding `numpy==1.24.0` to existing dependencies
* **Different registry URLs**: Changing `--index-url` or `--extra-index-url`
* **ADK version changes**: If you re-import with a different ADK version and have unpinned dependencies, ADK may add different "latest" versions

<Tip>
  **Best practice**: Use pinned versions in `requirements.txt` to ensure consistent hash generation and maximize venv reuse across deployments.
</Tip>

## Enterprise dependency management patterns

### Using private registries (Artifactory, Nexus)

watsonx Orchestrate supports pulling packages from private PyPI registries like Artifactory or Sonatype Nexus. This is essential for organizations that:

* Require security scanning of all packages
* Cannot allow direct downloads from public PyPI
* Need to control which package versions are available

#### Configuration with `--index-url`

Use `--index-url` to specify your primary package registry:

```txt requirements.txt theme={null}
--index-url https://artifactory.company.com/api/pypi/python-virtual/simple
ibm-watsonx-orchestrate==2.4.0
requests==2.32.3
pydantic==2.8.2
```

<Warning>
  When using `--index-url`, pip will **only** check your specified registry. If a package is not found there, the installation will fail—it will NOT fall back to public PyPI.
</Warning>

#### Configuration with `--extra-index-url`

Use `--extra-index-url` to add your registry as an additional source while keeping PyPI as a fallback:

```txt requirements.txt theme={null}
--extra-index-url https://artifactory.company.com/api/pypi/python-virtual/simple
ibm-watsonx-orchestrate==2.4.0
requests==2.32.3
pydantic==2.8.2
```

<Note>
  With `--extra-index-url`, pip will check both your private registry AND public PyPI. This provides flexibility but may not meet strict security policies that prohibit any public PyPI access.
</Note>

#### Direct wheel URLs

You can also specify direct URLs to wheel files hosted in your registry:

```txt requirements.txt theme={null}
ibm-watsonx-orchestrate==2.4.0
pydantic @ https://artifactory.company.com/python-packages/pydantic-2.8.2-py3-none-any.whl
requests @ https://artifactory.company.com/python-packages/requests-2.32.3-py3-none-any.whl
```

This approach gives you maximum control over exactly which binary artifacts are used.

### Pinned dependencies for reproducibility

To ensure consistent builds across environments, always pin your dependencies to exact versions:

#### ✅ Recommended: Exact version pinning

```txt requirements.txt theme={null}
ibm-watsonx-orchestrate==2.4.0
requests==2.32.3
pydantic==2.8.2
numpy==1.24.0
```

#### ❌ Not recommended: Unpinned or range versions

```txt requirements.txt theme={null}
# Avoid these patterns for production deployments
ibm-watsonx-orchestrate>=2.4.0  # May pull different versions over time
requests                         # Will pull latest version
pydantic~=2.8                   # May pull 2.8.x updates
```

<Warning>
  Unpinned dependencies can lead to:

  * Different packages installed in DEV vs PROD
  * Unexpected behavior from minor/patch version changes
  * Hash mismatches causing unnecessary venv rebuilds
</Warning>

### Hash-based verification for maximum security

For the highest level of reproducibility and security, use `pip-compile` to generate hashes:

```bash theme={null}
# Generate requirements.txt with hashes
pip-compile --generate-hashes requirements.in -o requirements.txt
```

This produces a `requirements.txt` with cryptographic hashes:

```txt requirements.txt theme={null}
ibm-watsonx-orchestrate==2.4.0 \
    --hash=sha256:abc123... \
    --hash=sha256:def456...
requests==2.32.3 \
    --hash=sha256:789ghi... \
    --hash=sha256:012jkl...
```

When hashes are present, pip will **verify** that the downloaded package matches the expected hash, preventing:

* Compromised packages
* Registry tampering
* Accidental version mismatches

## Export/import vs Git-based promotion

Understanding the difference between these deployment approaches is crucial for choosing the right strategy.

### Export/import model

**What it provides**:

* Source code equivalence
* Configuration portability
* Quick environment setup

**What it does NOT provide**:

* Binary package equivalence (packages are re-installed in target environment)
* Guaranteed identical dependencies (unless using pinned versions + private registry)

**When to use**:

* Moving agents between different watsonx Orchestrate instances
* Sharing agents with other teams or organizations
* Backing up agent configurations

**Example workflow**:

```bash theme={null}
# Export from DEV environment
orchestrate agents export -n my-agent -k native -o my-agent.zip

# Import to QA environment (switch environment first)
orchestrate env set qa-environment
orchestrate agents import -f my-agent.zip
```

<Note>
  After import, the QA environment will run `pip install` using its own registry configuration, potentially pulling different package versions if dependencies are not pinned.
</Note>

### Git-based promotion

**What it provides**:

* Version control integration
* Commit-based traceability
* Code review workflows

**What it does NOT provide**:

* Binary package equivalence (packages are still re-installed)
* Protection against registry changes (unless using locked Artifactory)

**When to use**:

* Standard CI/CD pipelines
* Promoting tested code through environments
* Maintaining audit trails

**Example workflow**:

```bash theme={null}
# Tag a tested commit in DEV
git tag v1.2.3-qa-ready
git push origin v1.2.3-qa-ready

# Deploy from that commit in QA
git checkout v1.2.3-qa-ready
orchestrate env set qa-environment
orchestrate agents import -f agents/my-agent.yaml
```

### Achieving binary equivalence

Neither export/import nor Git-based promotion guarantees binary equivalence by default. To achieve it:

1. **Pin all dependencies** to exact versions in `requirements.txt`
2. **Use a locked private registry** (Artifactory/Nexus) with controlled package versions
3. **Optional**: Add hash verification with `pip-compile --generate-hashes`
4. **Ensure** the same registry configuration across all environments

**Example locked requirements.txt**:

```txt requirements.txt theme={null}
--index-url https://artifactory.company.com/api/pypi/python-locked/simple
ibm-watsonx-orchestrate==2.4.0 \
    --hash=sha256:abc123...
requests==2.32.3 \
    --hash=sha256:def456...
pydantic==2.8.2 \
    --hash=sha256:789ghi...
```

With this configuration:

* ✅ Same source code (via export/import or Git)
* ✅ Same dependency versions (pinned)
* ✅ Same binary packages (locked registry + hashes)
* ✅ Repeatable across all environments

## Air-gapped environments

For deployments in air-gapped or restricted network environments, see the dedicated guide: [Operating in an air-gapped environment](/environment/air_gap_environment).

Key considerations:

* ADK and all dependencies must be uploaded to your internal registry
* Use `--index-url` to point to your internal registry
* Ensure all transitive dependencies are available

## Best practices summary

<CardGroup cols={2}>
  <Card title="Pin all versions" icon="thumbtack">
    Use exact version pinning (`==`) in `requirements.txt` for all dependencies
  </Card>

  <Card title="Use private registries" icon="lock">
    Configure `--index-url` to pull from your organization's Artifactory or Nexus
  </Card>

  <Card title="Add hash verification" icon="shield-check">
    Use `pip-compile --generate-hashes` for maximum security and reproducibility
  </Card>

  <Card title="Lock your registry" icon="vault">
    Prevent package additions/removals in your Artifactory between deployments
  </Card>
</CardGroup>

### Recommended requirements.txt template

```txt requirements.txt theme={null}
# Use your organization's private registry
--index-url https://artifactory.company.com/api/pypi/python-virtual/simple

# Pin ADK version
ibm-watsonx-orchestrate==2.4.0

# Pin all dependencies with exact versions
requests==2.32.3
pydantic==2.8.2
python-dotenv==1.0.0

# Optional: Add hashes for verification (generated with pip-compile)
# ibm-watsonx-orchestrate==2.4.0 \
#     --hash=sha256:abc123...
```

## Troubleshooting

### Issue: Different packages installed in QA vs PROD

**Cause**: Unpinned dependencies or registry changes between deployments

**Solution**:

1. Pin all dependencies to exact versions
2. Use a locked Artifactory with controlled package versions
3. Verify registry configuration is identical across environments

### Issue: Tool fails to import due to missing packages

**Cause**: Package not available in private registry

**Solution**:

1. Verify package exists in your Artifactory/Nexus
2. Check `--index-url` configuration in `requirements.txt`
3. Ensure network connectivity from watsonx Orchestrate to your registry

### Issue: Slow deployment times

**Cause**: Virtual environment cache misses due to hash changes

**Solution**:

1. Use pinned versions to maximize cache hits
2. Avoid unnecessary changes to `requirements.txt`
3. Keep ADK version consistent across re-imports

### Issue: Security scan failures

**Cause**: Packages pulled from public PyPI without scanning

**Solution**:

1. Use `--index-url` (not `--extra-index-url`) to prevent PyPI fallback
2. Configure your Artifactory to scan packages before making them available
3. Implement package approval workflows in your registry

## Related resources

* [Authoring Python-Based Tools](/tools/create_tool)
* [Importing tools](/tools/deploy_tool)
* [Operating in an air-gapped environment](/environment/air_gap_environment)
* [Managing tools](/tools/manage_tool)