Authoring python tools

Authoring Python-Based Tools

Python tools are one of the most flexible and powerful ways to extend agent functionality in watsonx Orchestrate, especially when combined with Flows. A Python tool consists of one or more Python files. Within these files, you define one or more functions and annotate them with the @tool decorator to expose them to Orchestrate. Each Python tool runs inside an isolated container with its own virtual environment. This ensures that the tool and its dependencies remain sandboxed from the host operating system, providing a secure and consistent runtime. Python tool example

Python

#test_tool.py
from ibm_watsonx_orchestrate.agent_builder.tools import tool


@tool()
def my_tool(input: str) -> str:
    """Executes the tool's action based on the provided input.

    Args:
        input (str): The input of the tool.

    Returns:
        str: The action of the tool.
    """

    #functionality of the tool

    return f"Hello, {input}"

Show detailed @tool parameter descriptions

name

string

The unique name of the tool as it would be referenced by your agent.This defaults to the name of your python function.

display_name

string

The name of the tool as it shows up within the “Manage Agents” UI.

description

string

Provides a summary of the tool’s purpose and functionality. This description helps the LLM determine when and how to use the tool effectively.By default, this text is automatically generated from the tool’s docstring, but you can override it here if needed.

expected_credentials

List[ExpectedCredentials]

Specifies a list of connections along with their expected connection types. Any connection required during the tool’s execution must be included in this list. This ensures that all necessary connections are properly defined and bound to the tool at import time.

Show ExpectedCredentials

app_id

The app_id of the connection which this tool is expected to require.

type

ConnectionType | List[ConnectionType]

The type of connection(s) (BASIC_AUTH, BEARER_TOKEN, API_KEY_AUTH, OAUTH2_AUTH_CODE, …, KEY_VALUE) this tool is compatible with for this app_id.When compatible with more than one ConnectionType, please ensure the type provided at runtime is the expected ConnectionType before attempting to fetch it’s value. This can be done using the following:

from ibm_watsonx_orchestrate.run import connections
...
@tool(expected_credentials=[{'app_id': 'my_app', 'type': [ConnectionType.BASIC_AUTH, ConnectionType.OAUTH2]}])
def my_tool():
    headers = {}
    if connections.get_connection_type('my_app') == ConnectionType.BASIC_AUTH:
        conn = connections.basic('my_app')
        username = conn.username
        password = conn.password
        headers['Authorization'] = f"Basic {b64encode(f"{username}:{password}")}"
    elif  connections.get_connection_type('my_app') == ConnectionType.OAUTH2:
        conn = connections.oauth2_auth_code('my_app')
        headers['Authorization'] = f"Bearer {conn.access_token}"

kind

PythonToolKind

default:"PythonToolKind.TOOL"

The kind of Python tool. It can be either PythonToolKind.TOOL for general-purpose tools or PythonToolKind.JOIN_TOOL for tools which are intended to be used as join tools for the Planner-Act style of agent.This controls the type of schema validation that is done on the tool during import time.

input_schema

object

The JSON Schema representing the input of the tool conforms to. The JSON Schema is always defined as an object for each of the keyword arguments of the tool and is inferred automatically by the typings of the Python function.The description of each argument will be extracted from the docstring (see below note about compatible docstring types).The type of each argument will be inferred from the field’s typings or pydantic type.The generated value can be inspected by running orchestrate tools list -v.

output_schema

object

The JSON Schema representing the output of the tool conforms to. This is automatically inferred from the return type of the function using either typings or pydantic.The generated value can be inspected by running orchestrate tools list -v.

permission

ToolPermission

default:"ToolPermission.READ_ONLY"

deprecated

This field is used to determine whether or not an agent would confirm each action before performing it, if the type was not ToolPermission.READ_ONLY.This behavior has been deprecated and removed; it no longer occurs.

Note:
Python tools must use Google-style docstrings to apply descriptions and document the tool.

Importing Python-Based Tools

To import a Python tool use the orchestrate tools import command using the -f flag to specify which python file contains your tool definitions. Each @tool annotated in the given file will be exposed as a tool within your active watsonx Orchestrate environment. It is recommended to include only a single @tool annotated function per file so that tools can be imported independently.

BASH

orchestrate tools import -k python -f my-tool.py -r requirements.txt -a app1 -a app2

Show command flags

Flag	Type	Required	Description
`--kind` / `-k`	string	Yes	The kind of tool to import, this is always `python` for Python based tools.
`--file` / `-f`	string	Yes	The path to the file of the openapi spec you want to import (or a URL containing the file).
`--package-root` / `-p`	string	No	For multi file Python tools, this is the root folder of the package to upload. All files within this package root folder will be included in the Python tool imported into the runtime.
`--app-id` / `-a`	string	No	One or more application ids of a connection which can be used to authenticate against this endpoint. To bind this tool to more than one application id, repeat this flag for each app_id. Additionally, if a tool was written expecting one app_id but your version of orchestrate expects another, you can remap the application name by using `-a app_name_in_tool=app_name_in_wxo_instance`.

Additional features of Python tools

Adding Dependencies

If your Python function relies on external libraries or packages, you can ensure your tool works correctly by specifying these dependencies in the requirements.txt file.Python tool example

Python

import requests
from ibm_watsonx_orchestrate.agent_builder.tools import tool


@tool
def fetch_url(endpoint: str) -> str:
    """Executes a GET request against a given endpoint to fetch its contents.

    Args:
        endpoint (str): the url to fetch

    Returns:
        str: the contents of the page
    """

    #functionality of the tool

    return requests.get(endpoint).text

requirements.txt example

TEXT

requests==2.32.4

For more information on the format of a requests file, please see the official pip documentation.

When importing your tool, specify the requirements.txt file by using the -r flag.

BASH

orchestrate tools import -k python -f my-tool.py -r requirements.txt

Using complex input and output argument types with Pydantic

Python tools support input and output arguments based on any native Python type from the typings package and classes which extend BaseModel from the popular library Pydantic.Example

PYTHON

import requests
from ibm_watsonx_orchestrate.agent_builder.tools import tool
from pydantic import BaseModel

class InputExample(BaseModel):
    sample: Optional[str] = Field(None, description="description of my field")

class OutputExample(BaseModel):
    result: Optional[str] = Field(None, description="description of my result field")

@tool
def fetch_url(example: InputExample, endpoint: str) -> OutputExample:
    """Executes a GET request against a given endpoint to fetch its contents.

    Args:
        example: An example input object that takes a sample
        endpoint: an example endpoint

    Returns:
        OutputExample: the example output
    """

    #functionality of the tool

    return OutputExample(result=requests.get(endpoint).text)

Securely providing credentials

Creating multi file Python tool packages

In addition to importing a single tool file, it is possible to import an entire folder (package) along with your tool file. This is useful when you wish to centralize logic into shared utility libraries, for example, shared authentication, data processing, or including static files such as CSVs or sqlite databases.Assuming a folder structure as follows:

─── my_agentic_project/
    └── tools/
        └── multi_file_tool/
            └── utils/
               └── csv_utils.py
               └── __init__.py
            └── my_tool.py
            └── my_data.csv
            └── requirements.txt

my_tool.py:

PYTHON

from ibm_watsonx_orchestrate.agent_builder.tools import tool
from os import path
import pandas as pd
from .utils.csv_utils import parse_csv # this is a relative import of my csv_utils.py file I included in this package

@tool
def return_file_contents() -> str:
    """
    Gets the contents of my super important csv file as a table.

    Returns:
        str: the contents of the csv
    """
    base_path = os.path.dirname(os.path.abspath(__file__))
    file_path = path.join(base_path, 'my_data.csv')
    dataframe: pd.DataFrame = parse_csv(file_path)

    return dataframe.to_markdown() # note dataframe output is unsupported, but dataframe.to_markdown will render a table in a format which the ui knows how to render

Importing your multifile Python tool:
Assuming your cli was currently in the my_agentic_project folder, this tool could be imported using the following:

BASH

orchestrate tools import -k python \
    -p tools/multi_file_tool \
    -f tools/multi_file_tool/my_tool.py \
    -r tools/multi_file_tool/requirements.txt

Notes:

Only tools defined in my_tool.py will be imported. Other Python files in the package will not be scanned for tools.
The system will only accept strings composed of alphanumeric characters and underscores (_) in the name attribute of the @tool decorator in my_tool.py.
The system will only accept tool file names composed of alphanumeric characters and underscores (_).
The package root folder and the tool file path CLI arguments MUST share a common base path.
The path of the tool file folder relative to the package root folder, must be composed of folder names which are only composed of alphanumeric characters and underscores (_).
Any whitespace like characters which prefix or suffix provided package root path will be stripped by the system.
A package root folder path that resolves to an empty string will make the system assume that no package root was specified. Then, the system falls back to single Python file tool import.

Limitations:

The max compressed tool size: 50Mb
File resolution for non-python packages must be done relative to the current file, as seen above
Imports to packages within your code must be resolved relatively

Creating tools that accept files

You can create Python tools that accept files or return files to download.To do that, you must comply with the following requirements:

To accept files as input, the tool must accept a sequence of bytes as arguments.
To return a file for download, the tool must return a sequence of bytes as output.

The following example accepts a file as input and returns the processed file for download:

from typing import Union
from io import BytesIO
from typing import Any
from ibm_watsonx_orchestrate.agent_builder.tools import tool
import base64

@tool
def create_image_with_filter(name: str, age: int, image_bytes: bytes) -> bytes:
    """Gets user information, applies a filter to the image, and returns the processed image.

    Args:
        name (str): The user's name.
        age (int): The user's age.
        image_bytes (bytes): The original image in bytes format.

    Returns:
        bytes: The processed image in bytes format.
    """

    import cv2
    import numpy as np
    from typing import Union
    from io import BytesIO
    # Convert bytes to numpy array
    image_array = np.frombuffer(image_bytes, np.uint8)
    image = cv2.imdecode(image_array, cv2.IMREAD_COLOR)

    if image is None:
        raise ValueError("Invalid image bytes provided")

    # Apply bilateral filter for smoothing while preserving edges
    smooth = cv2.bilateralFilter(image, d=9, sigmaColor=75, sigmaSpace=75)

    # Convert to grayscale and detect edges
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    edges = cv2.medianBlur(gray, 7)
    edges = cv2.adaptiveThreshold(
        edges, 255, cv2.ADAPTIVE_THRESH_MEAN_C,
        cv2.THRESH_BINARY, blockSize=9, C=2
    )

    # Convert edges to color image
    edges_colored = cv2.cvtColor(edges, cv2.COLOR_GRAY2BGR)

    # Combine smoothed image with edges
    cartoon = cv2.bitwise_and(smooth, edges_colored)

    # Enhance color saturation
    hsv = cv2.cvtColor(cartoon, cv2.COLOR_BGR2HSV)
    hsv[..., 1] = cv2.multiply(hsv[..., 1], 1.4)  # increase saturation
    cartoon_enhanced = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)

    # Convert back to bytes
    success, buffer = cv2.imencode('.jpg', cartoon_enhanced)
    if not success:
        raise RuntimeError("Failed to encode image")

    return buffer.tobytes()

Creating tools that return inline tables, images or links

The native watsonx Orchestrate webchat supports rendering markdown. To render output correctly, it is possible to instruct the LLM to render the output in this format, though in some cases it’s desirable to simply have the tool return the expected output and reduce the amount of transformation that the LLM has to do.Inline tables

PYTHON

from ibm_watsonx_orchestrate.agent_builder.tools import tool
import textwrap


@tool
def return_file_contents() -> str:
    """
    Renders a markdown table

    Returns:
        str: the markdown table
    """

    return textwrap.dedent('''
    | a | b | c |
    | -- | -- | -- |
    | value1 | value2 | value3 |
    | value1 | value2 | value3 |
    ''')

Links

PYTHON

from ibm_watsonx_orchestrate.agent_builder.tools import tool

@tool
def return_link() -> str:
    """
    Renders a link

    Returns:
        str: the link
    """

    return 'Navigate to [IBM.com](http://ibm.com)'

Images

PYTHON

from ibm_watsonx_orchestrate.agent_builder.tools import tool

@tool
def return_image() -> str:
    """
    Renders an image

    Returns:
        str: the image
    """

    return 'Check out this image\n ![alt text](https://mintlify.s3.us-west-1.amazonaws.com/ibm-2e3153bf/assets/agents/deploy-agent.png)'

Common pitfalls of Python tools

Host networking

On watsonx Orchestrate Developer Edition: Because Python tools run within a container, localhost within a Python tool refers to the container executing the tool, not the host machine. To call an endpoint on your host machine, use a URL of docker.host.internal or explicitly use your host machine’s IP address.
On SaaS: The SaaS version of watsonx Orchestrate does not have access to the internal network of your company. If this access is necessary it is possible to prototype with the Developer Edition and then deploy watsonx Orchestrate via the Cloud Pak for Data, on-premises, or by engaging with IBM for alternatives.
On CPD (On-premises): for a tool to be able to access the external internet resources, ensure that no firewall rules prevent access to that resource from the node pool responsible for tool execution.

Read only filesystem

For the safety and security of the watsonx Orchestrate runtime, the filesystem your tool executes in, your tool has read only access to its own filesystem. This means it is not possible to update files during tool execution.

Release Notes

Get Started

Build

Analyze

watsonx Orchestrate Developer Edition

Reference

Legal notices

Authoring Python-Based Tools

Importing Python-Based Tools

Additional features of Python tools

Common pitfalls of Python tools

Host networking

Read only filesystem

Release Notes

Get Started

Build

Analyze

watsonx Orchestrate Developer Edition

Reference

Legal notices

​Authoring Python-Based Tools

​Importing Python-Based Tools

​Additional features of Python tools

​Common pitfalls of Python tools

​Host networking

​Read only filesystem

Authoring Python-Based Tools

Importing Python-Based Tools

Additional features of Python tools

Common pitfalls of Python tools

Host networking

Read only filesystem