> ## Documentation Index
> Fetch the complete documentation index at: https://developer.watson-orchestrate.ibm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Configuring model policies

Model policies allow for the coordination of multiple models to accomplish tasks like load-balancing and fallback.

## Adding model policies

```bash BASH theme={null}
orchestrate models policy add --name <model_name> --model <provider1>/<model_id1> --model <provider2>/<model_id2> --strategy <strategy_type> --strategy-on-code 500 --retry-on-code 503 --retry-attempts 3
```

**Flags**:

* `--name` (`-n`): The name of the policy you want to add.
* `--description` (`-d`): An optional description to appear a long side the policy in the list view.
* `--display-name`: An optional display name for the policy in the UI
* `--strategy` (`-s`): The policy mode you want to use.
  * `loadbalance`: These models operate together by distributing the load of requests between them, following the distribution of weight values. By default, both weight values are attributed as `1`, so the loads are evenly balanced between the models. If you want to customize the weight values, see [Importing model policies](#importing-model-policies).
  * `fallback`: If one of the models is unavailable, the agent will try to use the other one as a fallback alternative.
  * `single`: Uses only one model, but allows for `--retry-on-code` and `--retry-attempts`.
* `--strategy-on-code`: A list of HTTP error codes which triggers the strategy. Used for `fallback` strategy.
* `--retry-on-code`: A list of HTTP error codes for which the model should retry the request.
* `--retry-attempts`: How many attempts it should make before stopping.

## Importing model policies

```bash BASH theme={null}
orchestrate models policy import --file my_spec.yaml
```

Where the `my_spec.yaml` file follows this structure:

<Tabs>
  <Tab title="Load balancing">
    ```yaml [my_spec.yaml] theme={null}
    spec_version: v1
    kind: model
    name: anygem
    description: Balances requests between 2 Gemini models
    display_name: Any Gem
    policy:
      strategy:
        mode: loadbalance
      retry:
        attempts: 1
        on_status_codes: [503]
      targets:
        - model_name: virtual-model/google/gemini-2.0-flash
          weight: 0.75   # Weights must be greater than 0 and less than or equal to 1
        - model_name: virtual-model/google/gemini-2.0-flash-lite
          weight: 0.25
    ```
  </Tab>

  <Tab title="Fallback">
    ```yaml [my_spec.yaml] theme={null}
    spec_version: v1
    kind: model
    name: firstgem
    description: Use the first gemeni model that doesn't return 503
    display_name: First Gem
    policy:
      strategy:
        mode: fallback
      retry:
        attempts: 1
        on_status_codes: [503]
      targets:
        - model_name: virtual-model/google/gemini-2.0-flash
        - model_name: virtual-model/google/gemini-2.0-flash-lite
    ```
  </Tab>

  <Tab title="Single">
    ```yaml [my_spec.yaml] theme={null}
        spec_version: v1
        kind: model
        name: retrygem
        description: Gemeni model that retries up to 3 times on 503
        display_name: Retry Gem
        policy:
          strategy:
            mode: single
          retry:
            attempts: 3
            on_status_codes: [503]
          targets:
            - model_name: virtual-model/google/gemini-2.0-flash
    ```
  </Tab>
</Tabs>

**Flags**:

* `--file` (`-f`): File path of the spec file containing the model policy configuration.

## Update model policy

Use either the [`add`](#adding-model-policies) or [`import`](#importing-model-policies) commands with the name of the model policy that you want to update to update the model policy.

## Removing model policies

```bash BASH theme={null}
orchestrate models policy remove -n <name of policy>
```

**Flags**:

* `--name` (`-n`): The name of the model policy that you want to remove.