> ## Documentation Index
> Fetch the complete documentation index at: https://developer.watson-orchestrate.ibm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Choosing your LLM

When you create an agent, the choice of LLM determines how effectively the agent can complete your workflow. watsonx Orchestrate provides optimized support for specific model/provider combinations, and allows registration of additional models as virtual models. **\[1]**

### Available Models

watsonx Orchestrate provides optimized support for the following models:

| **Model**                                                                                                                                                                                                                                                                                                                            | **Description**                                                                                                                                                                                                                                                                                                                                                                                                              | **License**                                                            |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| ★ <Tooltip tip="This model is a non-IBM product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to the terms." cta="Read the terms" href="https://www.ibm.com/support/customer/csol/terms/?id=i126-9451&lc=en">**groq/openai/gpt-oss-120b**</Tooltip>        | <Tooltip tip="This model is a non-IBM product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to the terms." cta="Read the terms" href="https://www.ibm.com/support/customer/csol/terms/?id=i126-9451&lc=en">gpt-oss-120b</Tooltip> is optimized for high-speed inference and tool calling, designed for multilingual and system-prompt flexibility. | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)              |
| ★ <Tooltip tip="This model is a non-IBM product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to the terms." cta="Read the terms" href="https://www.ibm.com/support/customer/csol/terms/?id=i126-9451&lc=en">**bedrock/openai/gpt-oss-120b-1:0**</Tooltip> | <Tooltip tip="This model is a non-IBM product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to the terms." cta="Read the terms" href="https://www.ibm.com/support/customer/csol/terms/?id=i126-9451&lc=en">GPT-OSS-120b</Tooltip> is available through AWS Bedrock, optimized for high-speed inference and tool calling.                           | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)              |
| ★ **watsonx/openai/gpt-oss-120b**                                                                                                                                                                                                                                                                                                    | GPT-OSS-120b available through watsonx.ai (GovCloud only - available April mid-release)                                                                                                                                                                                                                                                                                                                                      | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)              |
| ★ <Tooltip tip="This model is a non-IBM product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to the terms." cta="Read the terms" href="https://www.ibm.com/support/customer/csol/terms/?id=i126-9451&lc=en">**azure-openai/gpt-5.4**</Tooltip>            | <Tooltip tip="This model is a non-IBM product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to the terms." cta="Read the terms" href="https://www.ibm.com/support/customer/csol/terms/?id=i126-9451&lc=en">GPT-5.4</Tooltip>  is one of OpenAI’s frontier models, built to deliver faster, more reliable results for complex professional work.    | [Microsoft Product Terms](https://ai.azure.com/catalog/models/gpt-5.4) |

<Note>
  **Note:**

  * GPT-5.4 is classified as a Premier Model. It must be explicitly enabled before users can select it. Please see the [Enabling Premier Models](../llm/enable_premier_models.mdx) section to learn more about enabling Premier Models.
  * Usage of Premier Models will incur additional cost (MAUs). Administrators and users should be aware of this before enabling this model.
  * GPT-5.4 is only available on commercial SaaS tenants. It is not available on trial tenants, GovCloud/HIPAA environments, or on-premises deployments.
  * GPT-OSS models require special considerations. For more information, see [Special considerations](../agents/descriptions#special-considerations).
  * If you're migrating from Llama to GPT-OSS-120B, see the [Migration guide](./migrating_to_gpt_oss) for step-by-step instructions and best practices.
  * **GPT-OSS-120b** is a non-IBM product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to the terms. [Read the terms](https://www.ibm.com/support/customer/csol/terms/?id=i126-9451\&lc=en).
  * **GPT-OSS-120b** provided by Groq is not available in AWS GovCloud.
</Note>

## Optimized Models

Models marked with a ★ have undergone extensive evaluation and are optimized for use with watsonx Orchestrate agents. These models appear as preferred when you run the `orchestrate models list` command, and are shown as supported on the Manage Agents page in the UI.

### Using Groq Models Locally

For local development with the Developer Edition, you can use `groq/openai/gpt-oss-120b` by:

1. **Via watsonx Orchestrate proxy** (default): When connected to a SaaS instance, LLM requests are automatically proxied through watsonx Orchestrate without requiring your own Groq API key
2. **With your own API key**: Set `GROQ_API_KEY` in your `.env` file to use your own Groq account

This allows you to test and develop agents locally while leveraging the optimized gpt-oss-120b model.

## Virtual Models

watsonx Orchestrate allows you to register models from additional providers as virtual models. These models can be added via the ADK and will be automatically made available to users of the platform. **\[2]**

### Important Considerations

* **Tool calling required**: Only models that support tool calling will work with watsonx Orchestrate agents
* **Compatibility not guaranteed**: Virtual models are not validated for compatibility with watsonx Orchestrate, unlike the optimized models listed above
* **Testing required**: Thoroughly test virtual models before production use. See [Understanding Virtual Models](./managing_llm#understanding-virtual-models) for detailed compatibility considerations
* **Additional costs**: Using virtual models may incur additional costs from the upstream provider

For detailed information about virtual model compatibility, custom endpoints, and authentication options, see [Managing custom LLMs](./managing_llm).

### virtual-policies

In addition to adding models, you can configure complex routing rules for your LLMs. Use virtual-policies to establish pseudo-LLM names that can **load balance** traffic between models or establish **fallback policies** for when a provider is experiencing an outage.

<Note>
  **\[1]** **In the Developer Edition of watsonx Orchestrate**, when a `WO_INSTANCE` and `WO_API_KEY` are provided in the user's `.env` file, if that instance is a SaaS instance, all LLM requests will be proxied through watsonx Orchestrate without the user needing an additional watsonx.ai entitlement.

  If the instance is CPD and a `WO_INSTANCE` and `WO_API_KEY` are provided, only models deployed via IFM will be available via the `watsonx/` prefix.

  If neither of the above apply, either provide a `WATSONX_SPACE_ID` and `WATSONX_APIKEY` to a watsonx.ai account hosted in `us-south`, or add a virtual model running within your local Developer Edition server.

  **\[2]** virtual-models and virtual-policies added to a SaaS or CPD instance of watsonx Orchestrate will not automatically be available within the Developer Edition. They will need to be manually added by the user.
</Note>

## Next steps

<CardGroup>
  <Card title="Migrating to GPT-OSS-120B" href="./migrating_to_gpt_oss" icon="arrow-right-arrow-left">
    Step-by-step guide for migrating agents from Llama to GPT-OSS-120B with optimization tips.
  </Card>

  <Card title="Managing virtual models" href="./managing_llm#cli-reference" icon="terminal">
    Learn how to manage virtual models by using the ADK's CLI.
  </Card>

  <Card title="Model policies" href="./model_policies" icon="building-shield">
    Learn how to create and manage model policies for complex LLM routing and fallback.
  </Card>

  <Card title="Examples with supported providers" href="./managing_llm#examples-using-the-supported-providers" icon="notebook">
    See how to add OpenAI, Azure, AWS Bedrock, Ollama, and many more models from different providers.
  </Card>

  <Card title="Integrate the Developer Edition with SaaS" href="./watsonx_models" icon="gears">
    Learn how to integrate the watsonx Orchestrate Developer Edition with your watsonx Orchestrate SaaS tenant for LLM inferencing.
  </Card>
</CardGroup>