Choosing a tool type

​

Available tool types

• Standalone Python tools: Individual Python tools that run in isolated processes
• Python toolkits: Groups of related Python tools that run together in a single Python process
• Local MCP toolkits: MCP servers (Python or Node.js) that run inside the watsonx Orchestrate runtime
• Remote MCP toolkits: MCP servers hosted externally that watsonx Orchestrate connects to via HTTP
• OpenAPI tools: REST API endpoints defined by OpenAPI specifications

​

Decision framework

​

Detailed comparison

​

Standalone Python tools

• You have a single, independent tool
• The tool uses non-thread-safe operations (file I/O, global state, etc.)
• You need process isolation between tool invocations
• You’re prototyping or testing a new tool

• Each tool invocation starts a lightweight Python process
• Full process isolation ensures no state leakage between calls
• Supports non-thread-safe operations without modification
• Higher execution overhead due to process startup
• Simpler to develop and debug

• Draft environment: Runs in shared container per tenant (2 vCPUs, 2 GB memory, 5 workers)
• Live environment: Same shared container model
• Process overhead: ~100-300ms per invocation for process startup

• Tools that write to temporary files
• Tools using libraries with global state
• Infrequently called tools where startup overhead is acceptable
• Prototype tools during development

​

Python toolkits

• You have multiple related Python tools
• All tools are thread-safe and reentrant
• You expect high concurrency or frequent tool calls
• You want to minimize execution overhead

• Groups related tools in a single Python process
• Tools share the same process and can share dependencies
• Requires all tools to be thread-safe
• Significantly faster execution (no process startup overhead)
• Tools deploy and update together as a unit

• Draft environment: Runs in shared container per tenant (2 vCPUs, 2 GB memory, 5 workers)
• Live premium environment: Each toolkit gets dedicated container (2 vCPUs, 2 GB memory, 5 workers, 2 replicas)
• Process overhead: None after initial startup
• Concurrency: Up to 5 concurrent requests per toolkit

• Suite of data transformation tools
• Multiple tools accessing the same API or service
• High-frequency tools called repeatedly by agents
• Tools sharing common utility functions or dependencies

• All tools must be thread-safe
• No global mutable state
• No file system writes (read-only filesystem)
• Proper async/await patterns for I/O operations

​

Local MCP toolkits

• You have existing MCP servers (Python or Node.js)
• You want to use Node.js for tool development
• You need the MCP protocol’s features (resources, prompts, etc.)
• You want to share tools across multiple AI platforms

• Runs MCP servers inside watsonx Orchestrate runtime
• Supports both Python and Node.js implementations
• Can import from npm, PyPI, or local filesystem
• Behaves like standalone Python tools (process per invocation)
• Full MCP protocol support

• Similar to standalone Python tools
• Process startup overhead per invocation
• Supports stdio transport only (local execution)

• Existing MCP servers you want to reuse
• Node.js-based tools
• Tools that need MCP resources or prompts
• Cross-platform tool sharing

​

Remote MCP toolkits

• Tools must run in external infrastructure
• You need to integrate cloud services or third-party APIs
• You want centralized tool management across multiple Orchestrate instances
• You have existing remote MCP servers

• MCP server runs externally (your infrastructure or third-party)
• watsonx Orchestrate connects via HTTP (SSE or streamable HTTP)
• Supports multiple authentication methods
• Network latency added to each tool call
• Centralized updates without redeploying to Orchestrate

• Network latency: typically 50-200ms per call
• Depends on remote server’s performance
• No local resource consumption

• Integration with cloud services (AWS, Azure, GCP)
• Third-party MCP servers (GitHub Copilot, CoinGecko, etc.)
• Tools requiring specialized infrastructure
• Centrally managed tools used by multiple tenants

​

OpenAPI tools

• You have existing REST APIs
• You want to expose HTTP endpoints as tools
• You don’t need custom Python logic
• You want declarative tool definitions

• Defined by OpenAPI specifications
• No custom code required
• Automatic schema validation
• Supports various authentication methods
• Direct HTTP calls to your APIs

• Network latency depends on API location
• No local processing overhead
• Scales with your API infrastructure

• Existing microservices or REST APIs
• Third-party API integrations
• CRUD operations on databases
• Webhook-based integrations

​

Performance considerations

​

Execution overhead comparison

​

Concurrency and scaling

• 5 workers per toolkit
• 2 replicas per toolkit
• Up to 10 concurrent requests per toolkit
• Dedicated resources (2 vCPUs, 2 GB per toolkit)

• Share tenant-wide container
• 5 workers total across all tools
• 2 replicas
• Suitable for moderate concurrency

• Scaling depends on remote server
• No local resource limits
• Network becomes the bottleneck

​

Thread safety requirements

​

What makes a tool thread-safe?

# ✅ Good: No shared state
@tool()
async def calculate_sum(numbers: list[int]) -> int:
    return sum(numbers)

# ✅ Good: Immutable shared data
CONSTANTS = {"pi": 3.14159, "e": 2.71828}

@tool()
async def get_constant(name: str) -> float:
    return CONSTANTS.get(name)

# ✅ Good: Thread-local storage
import threading
thread_local = threading.local()

@tool()
async def process_data(data: str) -> str:
    thread_local.temp = data.upper()
    return thread_local.temp

# ❌ Bad: Global mutable state
counter = 0

@tool()
async def increment_counter() -> int:
    global counter
    counter += 1  # Race condition!
    return counter

# ❌ Bad: File system writes
@tool()
async def save_to_file(data: str) -> str:
    with open("/tmp/data.txt", "w") as f:  # Multiple threads conflict
        f.write(data)
    return "saved"

# ❌ Bad: Non-thread-safe library
import some_library  # Assume this has global state

@tool()
async def use_library(input: str) -> str:
    some_library.set_config(input)  # Affects all threads
    return some_library.process()

​

Making tools thread-safe

1. Remove global mutable state: Use function parameters instead
2. Use thread-safe libraries: Check library documentation
3. Add synchronization: Use asyncio.Lock() for critical sections
4. Keep as standalone tool: If thread-safety is too complex

​

Migration scenarios

​

From standalone Python tool to Python toolkit

• Tool is called frequently (>10 times per minute)
• You have multiple related tools
• You’ve verified thread-safety
• You want better performance in live environment

1. Verify all tools are thread-safe
2. Create toolkit folder structure
3. Move tool files into toolkit
4. Update imports if needed
5. Import as toolkit
6. Test in draft environment
7. Update agents to use toolkit_name:tool_name format
8. Deploy to live environment

​

From Python toolkit to standalone tools

• Discovering thread-safety issues in production
• Tools are called infrequently
• Tools have conflicting dependencies
• Need independent tool updates

1. Export each tool from toolkit
2. Import each as standalone tool
3. Update agent configurations
4. Test thoroughly
5. Remove old toolkit

​

From local MCP to Python toolkit

• Want better performance (eliminate process overhead)
• Don’t need MCP-specific features
• Tools are Python-based and thread-safe
• Want dedicated resources in live environment

1. Extract Python code from MCP server
2. Convert to @tool decorated functions
3. Ensure thread-safety
4. Import as Python toolkit
5. Update agent configurations
6. Test and validate

​

Choosing based on ownership and dependencies

​

Shared dependencies

• Multiple tools use the same libraries
• Dependencies are large (reduces memory usage)
• Tools need to share utility functions

• Tools have conflicting dependency versions
• Each tool has unique dependencies
• You want independent dependency updates

​

Ownership boundaries

• Different teams own different tools
• Tools have different update schedules
• You want independent deployment

• One team owns all tools
• Tools are tightly coupled
• You want atomic updates

​

Environment-specific considerations

​

Draft environment

• All Python toolkits and tools share one container per tenant
• Process overhead exists for all tool types
• Good for testing and development
• Resource limits: 2 vCPUs, 2 GB memory, 5 workers

​

Live environment (premium)

• Each Python toolkit gets dedicated container
• No process overhead for toolkit tools
• Better isolation and performance
• Up to 5 toolkits per tenant

​

Live environment (non-premium)

• Similar to draft environment
• Shared container model
• Contact support for toolkit access

​

Best practices summary

1. Start simple: Begin with standalone Python tools during development
2. Measure performance: Profile your tools to identify bottlenecks
3. Group related tools: Bundle tools that share dependencies or are called together
4. Ensure thread-safety: Test thoroughly before using Python toolkits
5. Use async patterns: Always use async/await for I/O operations
6. Monitor concurrency: Track concurrent requests to avoid worker exhaustion
7. Plan for scaling: Consider expected load when choosing tool type
8. Document dependencies: Clearly specify shared dependencies in toolkits

​

Next steps

• Authoring Python tools
• Importing Python toolkits
• Importing local MCP toolkits
• Importing remote MCP toolkits
• Migrating to Python toolkits
• Managing toolkits