llama-3.2-90B-vision-instruct to gpt-oss-120b. Based on extensive testing across 256 production agents, GPT-OSS-120B delivers superior performance in response time, error handling, conversational quality, and user experience when correctly configured.
GPT-OSS-120B is the default model for new agents created in watsonx Orchestrate.
GPT-OSS-120B is available through the following providers:
- Groq:
groq/openai/gpt-oss-120b - AWS Bedrock:
bedrock/openai.gpt-oss-120b-1:0 - watsonx.ai:
watsonx/openai/gpt-oss-120b(GovCloud only — available in the April mid-release)
Why migrate to GPT-OSS-120b
GPT-OSS-120B offers significant advantages over Llama models:- Faster response times across all scenarios
- Superior error handling with intelligent recovery and detailed explanations
- Enhanced conversational efficiency through single-turn parameter collection
- Better memory and context retention across multi-turn dialogues
- Improved safety with robust out-of-scope request handling
- More natural interactions that feel less robotic
Before you begin
Prerequisites
- Access to watsonx Orchestrate (SaaS or Developer Edition)
- Existing agents built with Llama models
- Familiarity with agent configuration and prompt engineering
Understanding the differences
GPT-OSS-120B behaves differently from Llama models in the following ways:Migration process
1
Review your agent configuration
Export your existing Llama-based agent to review its current configuration:Document the following elements:
- Agent instructions and tone
- Tool usage patterns
- Knowledge base dependencies
- Expected user interaction flows
2
Update the LLM configuration
Update your agent configuration to use GPT-OSS-120B. Choose the provider that matches your deployment:
3
Optimize agent instructions
GPT-OSS-120B requires explicit, model-specific instructions. Add the following blocks to your agent instructions as needed:
Essential instruction template
Use this template as a starting point:Prioritize knowledge bases (if applicable)
If your agent uses knowledge bases, add the following instruction:Optimize tool usage
For agents with multiple tools, add explicit guidance:Control agent routing (for supervisor agents)
If your agent delegates to other agents, use explicit action verbs in the instructions:4
Remove problematic patterns
Overly specific examples can constrain GPT-OSS-120B. Review your instructions and update accordingly:✅ Prefer:✅ Prefer:
❌ Avoid:❌ Avoid:5
Test and validate
After you update your agent configuration, run the following tests:
-
Test basic interactions:
- Test tool calling: Verify that tools are called correctly with the appropriate parameters.
- Test error scenarios: Confirm that error messages are clear and that recovery is handled correctly.
- Test multi-turn conversations: Confirm that context is retained across multiple exchanges.
- Test edge cases: Validate behavior with incomplete information, out-of-scope requests, and ambiguous queries.
6
Deploy and monitor
Import the updated agent:Monitor initial usage for:
- Response quality and accuracy
- Tool call precision
- User satisfaction
- Error rates and recovery success
Recommended config parameters for gpt-oss-120b
Use the followingconfig block as a starting configuration when you register gpt-oss-120b as a virtual model. You can adjust the values as needed. Refer to your LLM provider’s documentation for details on how max_completion_tokens, temperature, reasoning_effort, and seed affect model behaviour:
Common migration challenges
Challenge 1: Over-reliance on internal knowledge
Symptom: The agent provides answers from its training data instead of using tools or knowledge bases. Solution: Add explicit knowledge prioritization instructions (see Step 3).Challenge 2: Excessive parameter collection
Symptom: The agent asks for parameters before routing to specialized agents or calling tools. Solution: Use strong negations in instructions:Challenge 3: Literal example following
Symptom: The agent handles only scenarios exactly as shown in examples. Solution: Remove specific examples and use generic patterns instead.Challenge 4: Tool call precision issues
Symptom: The agent makes irrelevant tool calls or misses required calls. Solution:- Improve tool descriptions with clear use cases.
- Add explicit tool usage rules in instructions.
- Test iteratively and refine based on results.
Challenge 5: Agent routing confusion
Symptom: The agent returns JSON instead of executing agent calls. Solution: Replace implicit instructions with explicit ones:Prompt engineering best practices
Do
- ✅ Use explicit action verbs (“Call”, “Execute”, “Use”).
- ✅ Provide strong negations for unwanted behaviors.
- ✅ Keep examples generic and minimal.
- ✅ Trust the model’s reasoning capabilities.
- ✅ Use single-turn parameter collection.
- ✅ Test iteratively with real scenarios.
Do not
❌Rely on implicit instructions (“delegate”, “route”).❌Provide overly specific examples that constrain behavior.❌List exhaustive value options (the agent might limit itself).❌Use weak negations for critical constraints.❌Force turn-by-turn parameter collection.❌Over-constrain conversational patterns.
Performance optimization tips
Reduce response latency
Improve conversational flow
Enhance error recovery
Validation checklist
Before you consider the migration complete, verify the following:- Agent uses GPT-OSS-120B (via Groq, AWS Bedrock, or watsonx) as the LLM
- Instructions include model-specific optimizations
- Knowledge base prioritization is configured (if applicable)
- Tool usage rules are explicit and clear
- Agent routing uses explicit action verbs (if applicable)
- Overly specific examples have been removed
- Basic interactions work as expected
- Tool calling is accurate and efficient
- Error handling is clear and helpful
- Multi-turn conversations maintain context
- Edge cases are handled gracefully
Troubleshooting
Agent is too verbose
Agent is too verbose
Add output constraints to the agent instructions:
Agent ignores knowledge bases
Agent ignores knowledge bases
Strengthen the knowledge prioritization instruction:
Agent makes too many tool calls
Agent makes too many tool calls
Add reasoning constraints to the agent instructions:
Agent does not follow instructions
Agent does not follow instructions
Ensure that instructions are:
- Explicit and unambiguous
- Free from conflicting directives
- Written in strong, clear language
- Tested with multiple scenarios
Next steps
Managing agents
Update, export, and manage your migrated agents.
Agent descriptions and instructions
Write effective instructions for GPT-OSS-120B.
Managing custom LLMs
Explore advanced LLM configuration options.
Model policies
Set up fallback policies and load balancing for production resilience.