> ## Documentation Index
> Fetch the complete documentation index at: https://developer.watson-orchestrate.ibm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Tool response structure and annotations

> Python tools can return structured responses with rich content, widgets, and metadata.

The `ToolResult()` and dictionary response formats support text content with audience targeting, structured data for downstream processing, widget integration to render interactive UI components, and metadata support for custom extensions and context information.

* **[`ToolResult`](#toolresult-class) class**\
  Returns tool responses by using a Python class with type-safe parameters and MCP compliance.

* **[Dictionary format](#dictionary-format)**\
  Returns tool responses as a dictionary structure with explicit control over all response fields.

## ToolResult class

The `ToolResult` class provides structured tool responses following the MCP specification. It allows tools to return content blocks, structured data, and widgets in a standardized format.

##### Import

Import the `ToolResult` class to create structured responses in your Python tools:

```python theme={null}
from ibm_watsonx_orchestrate.run import ToolResult
```

##### Constructor parameters

<ParamField path="content" type="List[Union[ContentBlock, str]]" required>
  Content blocks or strings to display to the user and send to the large language model (LLM). Strings are automatically converted to [`TextContent`](#textcontent) blocks.

  The following example shows two ways to specify content by using simple strings or explicit TextContent blocks:

  ```python theme={null}
  # Using strings (automatically converted to TextContent)
  content=["Operation completed", "Status: Active"]

  # Using TextContent blocks explicitly
  from ibm_watsonx_orchestrate.run import TextContent
  content=[TextContent(text="Operation completed")]
  ```

  For details on content structure and audience targeting, see [Content blocks and annotations](#content-blocks-and-annotations).
</ParamField>

<ParamField path="structuredContent" type="Dict[str, Any]">
  Structured data for programmatic use. This data is available for downstream processing but not directly displayed to users.

  The following example demonstrates how to include structured data in a ToolResult:

  ```python theme={null}
  structuredContent={
      "user_id": 123,
      "name": "John Doe",
      "balance": 5000.00
  }
  ```
</ParamField>

<ParamField path="widget" type="Any">
  Form widget to render in the UI. Supports form widgets created by using the `FormWidget` class and its input components.

  The following example demonstrates how to include a form widget in a ToolResult response:

  ```python theme={null}
  from ibm_watsonx_orchestrate.run import ToolResult
  from ibm_watsonx_orchestrate.run.forms import FormWidget, TextInput

  @tool()
  def show_registration_form():
      form = FormWidget(
          title="User Registration",
          inputs=[
              TextInput(
                  name="email",
                  title="Email Address",
                  required=True
              )
          ]
      )
      
      return ToolResult(
          content=["Please complete your registration"],
          widget=form
      )
  ```

  For widget implementations by using `FormWidget()`, see [Widget integration](widget_integration).
</ParamField>

<ParamField path="meta" type="Dict[str, Any]">
  Additional metadata for custom extensions or context. This field is serialized as `_meta` in the response.

  <Note>
    The `ToolResult` class does not support explicitly setting the `_meta` parameter and widget contracts as described for dictionary format. To implement widgets with `ToolResult`, use [`widget`](widget_integration) with the `FormWidget` class.
  </Note>

  The following example shows how to add custom metadata using the meta parameter:

  ```python theme={null}
  # Custom metadata
  meta={
      "priority": "high",
      "department": "finance"
  }
  ```
</ParamField>

##### Example

The following example demonstrates a tool that retrieves account information and returns a comprehensive response with multiple content blocks, structured data, a form widget for updating the budget, and custom metadata:

```python [expandable] theme={null}
from ibm_watsonx_orchestrate.run import ToolResult, TextContent
from ibm_watsonx_orchestrate.run.forms import FormWidget, NumberInput

@tool()
def check_account_status(user_id: str):
    # Fetch account data
    account_data = fetch_account(user_id)
    
    # Create form for next action
    form = FormWidget(
        title="Update Budget",
        inputs=[
            NumberInput(
                name="amount",
                title="New Budget Amount",
                default_value=account_data["budget"]
            )
        ]
    )
    
    return ToolResult(
        content=[
            TextContent(
                text=f"Account Status: {account_data['status']}"
            ),
            TextContent(
                text=f"Current Budget: ${account_data['budget']}"
            )
        ],
        structuredContent=account_data,
        widget=form,
        meta={"account_type": "premium"}
    )
```

## Dictionary format

The dictionary format provides direct control over the response structure, and requires the following top-level fields:

```python theme={null}
{
    "content": [...],           # Array of content blocks
    "structuredContent": {...}, # Structured data (optional)
    "_meta": {...}              # Metadata and extensions
}
```

##### Dictionary fields

<ParamField path="content" type="Array" required>
  Content blocks to display to the user and send to the large language model (LLM).

  The following example demonstrates a simple tool that returns content blocks with user audience targeting:

  ```python [expandable] theme={null}
  @tool()
  def get_dog_breeds():
      return {
          "content": [
              {
                  "type": "text",
                  "annotations": {
                      "audience": ["user"]
                  },
                  "text": "Dog breeds: affenpinscher, african, airedale, akita, appenzeller"
              }
          ]
      }
  ```

  For details on content structure and audience targeting, see [Content blocks and annotations](#content-blocks-and-annotations).
</ParamField>

<ParamField path="structuredContent" type="Dict[str, Any]">
  Structured data for programmatic use. This data is available for downstream processing but not directly displayed to users.

  The following example demonstrates how to return structured data alongside user-facing content:

  ```python [expandable] theme={null}
  @tool()
  def get_account_info(account_id: str):
      account_data = {
          "account_id": account_id,
          "balance": 1500.00,
          "status": "active"
      }
      
      return {
          "content": [
              {
                  "type": "text",
                  "annotations": {
                      "audience": ["user"]
                  },
                  "text": f"Account {account_id} retrieved successfully"
              }
          ],
          "structuredContent": account_data
      }
  ```
</ParamField>

<ParamField path="_meta" type="Dict[str, Any]">
  Additional metadata for custom extensions or context information. Use this field to pass custom data that doesn't fit into standard response fields.

  The following example demonstrates how to add custom metadata to a tool response:

  ```python Custom metadata example theme={null}
  @tool()
  def get_account_info(account_id: str):
      return {
          "content": [
              {
                  "type": "text",
                  "annotations": {
                      "audience": ["user"]
                  },
                  "text": f"Account {account_id} information retrieved"
              }
          ],
          "_meta": {
              "priority": "high",
              "department": "finance",
              "request_id": "REQ-12345",
              "timestamp": "2024-01-15T10:30:00Z"
          },
          "structuredContent": None
      }
  ```

  For widget implementations by using `_meta`, see [Widget integration](widget_integration).
</ParamField>

## Content blocks and annotations

Content blocks follow the MCP specification and support audience targeting through annotations. This controls how content is processed by the agent and displayed to users.

### Text content blocks

Text content blocks represent textual information with optional audience targeting. Use text content to return messages, status updates, or any string-based data from your tool, and to control who sees the content by using the `audience` annotation.

<CodeGroup>
  The following examples demonstrate different ways to create text content blocks with audience targeting:

  ```python ToolResult class theme={null}
  from ibm_watsonx_orchestrate.run import TextContent, Annotations, Role

  # Simple text
  TextContent(text="Operation completed")

  # Text with user audience
  TextContent(
      text="Your account has been updated",
      annotations=Annotations(audience=[Role.USER])
  )

  # Text with assistant audience
  TextContent(
      text="Account data: {...}",
      annotations=Annotations(audience=[Role.ASSISTANT])
  )
  ```

  ```python Dictionary format theme={null}
  # Simple text
  {
      "type": "text",
      "text": "Operation completed"
  }

  # Text with user audience
  {
      "type": "text",
      "annotations": {
          "audience": ["user"]
      },
      "text": "Your account has been updated"
  }

  # Text with assistant audience
  {
      "type": "text",
      "annotations": {
          "audience": ["assistant"]
      },
      "text": "Account data: {...}"
  }
  ```
</CodeGroup>

### Annotations and audience

The `audience` annotation is set inside content blocks to control content visibility and processing behavior.

<ParamField path="audience" type="List[Enum[Role]] or List[str]">
  Specifies who must see and process the content. The possible values varies according to the response format you implement for the tool result.

  <Tabs>
    <Tab title="ToolResult class">
      Possible values:

      * `Role.USER`
      * `Role.ASSISTANT`
      * `[Role.USER, Role.ASSISTANT]`

      The `Role` enum provides type-safe constants for specifying content audience when using the [`ToolResult` class](#toolresult-class):

      The following example shows how to use the Role enum for type-safe audience specification:

      ```python theme={null}
      from ibm_watsonx_orchestrate.run import Role

      Role.USER        # Content for end users
      Role.ASSISTANT   # Content for agent/LLM
      ```
    </Tab>

    <Tab title="Dictionary format">
      Possible values:

      * `["user"]`
      * `["assistant"]`
      * `["user", "assistant"]`
    </Tab>
  </Tabs>

  The audience affects agent reasoning and message display behavior. For more information, see [Audience behavior](#audience-behavior).
</ParamField>

### Audience behavior

The `audience` annotation controls how content is processed by the agent and displayed to users. Behavior varies based on whether a widget is present in the response. The following audiences are the audience types that determine content handling:

* **`user`**\
  Content intended for end users only, bypassing agent reasoning.

* **`assistant`**\
  Content for the agent and large language model to process and reason about.

* **`user` and `assistant`**\
  Content for both user and assistant, where the agent processes it before presenting to users.

##### Content with annotations only

When tool responses contain content with annotations but no widget, the following behavior applies based on the audience type:

<Tabs>
  <Tab title="User">
    | Property                 | Behavior                                                        |
    | ------------------------ | --------------------------------------------------------------- |
    | Agent reasoning          | Text is not used for reasoning that would trigger next actions. |
    | Message to agent context | Text is passed as AI message to prevent hallucinations.         |
    | User output              | The exact text displayed as provided.                           |

    For the following code snippet, you might observe the following:

    * User messages are passed through without agent reasoning.
    * The exact text content is preserved and displayed to the user.
    * Context is maintained to prevent hallucinations in follow-up questions.
    * No transformation or summarization occurs.

    <CodeGroup>
      The following examples show how to create content with user audience that bypasses agent reasoning:

      ```python ToolResult class theme={null}
      from ibm_watsonx_orchestrate.run import ToolResult, TextContent, Annotations, Role

      return ToolResult(
          content=[
              TextContent(
                  text="Account status retrieved successfully",
                  annotations=Annotations(
                      audience=[Role.USER]
                  )
              )
          ]
      )
      ```

      ```python Dictionary format theme={null}
      return {
          "content": [
              {
                  "type": "text",
                  "annotations": {
                      "audience": ["user"]
                  },
                  "text": "Account status retrieved successfully"
              }
          ]
      }
      ```
    </CodeGroup>
  </Tab>

  <Tab title="Assistant">
    | Property                 | Behavior                                             |
    | ------------------------ | ---------------------------------------------------- |
    | Agent reasoning          | Text is used for reasoning, triggering next actions. |
    | Message to agent context | The full reasoning chain is passed to context        |
    | User output              | The agent-generated message after summarization.     |

    For the following code snippet, you might observe the following:

    * The agent performs reasoning on the message.
    * It uses the text for reasoning that results in additional tool calls or response modifications.
    * The message undergoes summarization before being shown to the user.
    * The user sees the agent's processed output rather than the raw content.

    <CodeGroup>
      The following examples demonstrate content with assistant audience that triggers agent reasoning:

      ```python ToolResult class theme={null}
      return ToolResult(
          content=[
              TextContent(
                  text="Account data: status=active, balance=5000",
                  annotations=Annotations(
                      audience=[Role.ASSISTANT]
                  )
              )
          ]
      )
      ```

      ```python Dictionary format theme={null}
      return {
          "content": [
              {
                  "type": "text",
                  "annotations": {
                      "audience": ["assistant"]
                  },
                  "text": "Account data: status=active, balance=5000"
              }
          ]
      }
      ```
    </CodeGroup>
  </Tab>

  <Tab title="Both user and assistant">
    | Property                 | Behavior                                                                |
    | ------------------------ | ----------------------------------------------------------------------- |
    | Agent reasoning          | The text is used for reasoning.                                         |
    | Message to agent context | The full reasoning chain is passed to context.                          |
    | User output              | The agent-generated message. It might differ from the original message. |

    For the following code snippet, you might observe the following:

    * The agent performs reasoning despite user audience inclusion.
    * The original content text might or might not be displayed since the final message is generated by the agent and may involve multiple tool calls.
    * The user sees the agent's synthesized response.

    <CodeGroup>
      The following examples show content with both user and assistant audiences where agent reasoning occurs:

      ```python ToolResult class theme={null}
      return ToolResult(
          content=[
              TextContent(
                  text="Account status retrieved",
                  annotations=Annotations(
                      audience=[Role.USER, Role.ASSISTANT]
                  )
              )
          ]
      )
      ```

      ```python Dictionary format theme={null}
      return {
          "content": [
              {
                  "type": "text",
                  "annotations": {
                      "audience": ["user", "assistant"]
                  },
                  "text": "Account status retrieved"
              }
          ]
      }
      ```
    </CodeGroup>
  </Tab>
</Tabs>

##### Content with annotations and widget metadata

When tool responses contain both content with annotations and a widget, widget-specific behavior is activated.

<Warning>
  When a widget is present in the tool response either through the `ToolResult` class or dictionary format, the agent runtime does not send the response for agent reasoning. The tool response is considered final and passed directly back to the user.
</Warning>

<Tabs>
  <Tab title="User">
    | Property                 | Behavior                                                    |
    | ------------------------ | ----------------------------------------------------------- |
    | Agent reasoning          | The text is not used for reasoning.                         |
    | Message to agent context | The message is passed to context to prevent hallucinations. |
    | User output              | The exact text and widget are rendered.                     |

    For the following code snippet, you might observe the following:

    * No agent reasoning occurs.
    * The user message is preserved exactly as provided.
    * The widget data is rendered alongside the content.

    <CodeGroup>
      The following examples demonstrate user audience content with a widget where no agent reasoning occurs:

      ```python ToolResult class theme={null}
      from ibm_watsonx_orchestrate.run import ToolResult, TextContent, Annotations, Role
      from ibm_watsonx_orchestrate.run.forms import FormWidget, TextInput

          return ToolResult(
              content=[
                  TextContent(
                      text="Account status retrieved successfully",
                      annotations=Annotations(
                          audience=[Role.USER]
                      )
                  )
              ],
              widget=FormWidget(
                  title="Update Account",
                  inputs=[
                      TextInput(
                          name="email",
                          title="Email"
                      )
                  ]
              )
          )
      ```

      ```python Dictionary format theme={null}
      return {
          "content": [
              {
                  "type": "text",
                  "annotations": {
                      "audience": ["user"]
                  },
                  "text": "Account status retrieved successfully"
              }
          ],
          "_meta": {
              "com.ibm.orchestrate/widget": {
                  "response_type": "forms",
                  "json_schema": {
                      "title": "Update Account",
                      "type": "object",
                      "properties": {
                          "email": {"type": "string", "title": "Email"}
                      }
                  },
                  "ui_schema": {
                      "email": {"ui:widget": "TextWidget"}
                  },
                  "form_data": {"email": ""}
              }
          }
      }
      ```
    </CodeGroup>
  </Tab>

  <Tab title="Assistant">
    | Property                 | Behavior                                         |
    | ------------------------ | ------------------------------------------------ |
    | Agent reasoning          | The text is not used for reasoning.              |
    | Message to agent context | The message is passed to context for continuity. |
    | User output              | Shows the widget only and empty text content.    |

    For the following code snippet, you might observe the following:

    * No agent reasoning is performed.
    * The content text is empty and not displayed to the user.
    * Only the widget is shown to the user.

    <CodeGroup>
      The following examples show assistant audience content with a widget where the text is hidden and only the widget is displayed:

      ```python ToolResult class theme={null}
      return ToolResult(
          content=[
              TextContent(
                  text="Internal: Account data processed",
                  annotations=Annotations(
                      audience=[Role.ASSISTANT]
                  )
              )
          ],
          widget=FormWidget(
              title="Account Form",
              inputs=[
                  TextInput(
                      name="account_id",
                      title="Account ID"
                  )
              ]
          )
      )
      ```

      ```python Dictionary format theme={null}
      return {
          "content": [
              {
                  "type": "text",
                  "annotations": {
                      "audience": ["assistant"]
                  },
                  "text": "Internal: Account data processed"
              }
          ],
          "_meta": {
              "com.ibm.orchestrate/widget": {
                  "response_type": "forms",
                  "json_schema": {
                      "title": "Account Form",
                      "type": "object",
                      "properties": {
                          "account_id": {"type": "string", "title": "Account ID"}
                      }
                  },
                  "ui_schema": {
                      "account_id": {"ui:widget": "TextWidget"}
                  },
                  "form_data": {"account_id": ""}
              }
          }
      }
      ```
    </CodeGroup>
  </Tab>

  <Tab title="Both user and assistant">
    | Property                 | Behavior                                 |
    | ------------------------ | ---------------------------------------- |
    | Agent reasoning          | Text is not used for reasoning.          |
    | Message to agent context | The full message is passed to context.   |
    | User output              | The exact text and widget are displayed. |

    For the following code snippet, you might observe the following:

    * No agent reasoning occurs.
    * The message text is displayed exactly as provided to the user.
    * The widget is rendered alongside the text.

    <CodeGroup>
      The following examples demonstrate content with both audiences and a widget where the text and widget are both displayed:

      ```python ToolResult class theme={null}
      return ToolResult(
          content=[
              TextContent(
                  text="Please update your account information",
                  annotations=Annotations(
                      audience=[Role.USER, Role.ASSISTANT]
                  )
              )
          ],
          widget=FormWidget(
              title="Update Account",
              inputs=[
                  TextInput(
                      name="email",
                      title="Email"
                  )
              ]
          )
      )
      ```

      ```python Dictionary format theme={null}
      return {
          "content": [
              {
                  "type": "text",
                  "annotations": {
                      "audience": ["user", "assistant"]
                  },
                  "text": "Please update your account information"
              }
          ],
          "_meta": {
              "com.ibm.orchestrate/widget": {
                  "response_type": "forms",
                  "json_schema": {
                      "title": "Update Account",
                      "type": "object",
                      "properties": {
                          "email": {"type": "string", "title": "Email"}
                      }
                  },
                  "ui_schema": {
                      "email": {"ui:widget": "TextWidget"}
                  },
                  "form_data": {"email": ""}
              }
          }
      }
      ```
    </CodeGroup>
  </Tab>
</Tabs>

## Widget integration

For detailed information about integrating interactive widgets and forms into your tool responses, including form widgets, custom widgets, event handling, and the `FormWidget`, `ToolEvent`, and `MessageEvent` classes, see [Widget integration](widget_integration).