> ## Documentation Index
> Fetch the complete documentation index at: https://developer.watson-orchestrate.ibm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Managing voice configuration

Use the ADK to configure voice features for your agents.

<Note>
  **Note:**
  To use voice in watsonx Orchestrate Developer Edition, enable the voice feature by adding the `--with-voice` flag to the `orchestrate server start` command. For more information, see [Installing watsonx Orchestrate Developer Edition: watsonx Orchestrate server](../developer_edition/wxOde_setup#installing-watsonx-orchestrate-developer-edition%3A-watsonx-orchestrate-server).
</Note>

## Creating voice configurations

To create a voice configuration, first create a YAML file that defines your voice settings. This file includes the voice name, speech-to-text and text-to-speech provider settings, the primary language, and optional configurations for advanced settings.

### Supported Providers

Speech-to-Text (STT) Providers:

* `watson_stt`: IBM Watson Speech to Text - [Docs](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-large-speech-languages)
* `deepgram_stt`: Deepgram Speech to Text - [Docs](https://developers.deepgram.com/docs/stt/getting-started)

Text-to-Speech (TTS) Providers:

* `watson_tts`: IBM Watson Text to Speech - [Docs](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices#language-voices-natural)
* `deepgram_tts`: Deepgram Text to Speech - [Docs](https://developers.deepgram.com/reference/text-to-speech/speak-streaming)
* `elevenlabs_tts`: ElevenLabs Text to Speech - [Docs](https://elevenlabs.io/docs/overview/capabilities/text-to-speech)

<Note>
  **Note:**
  Please refer to the official documentations of each providers for more information on models and parameters.
</Note>

### Configuration Examples

#### Watson STT/TTS Configuration

**Basic Example:**

```yaml YAML theme={null}
name: watson_voice_config
speech_to_text:
  provider: watson_stt
  watson_stt_config:
    api_url: "watson_stt_url"
    api_key: "your_watson_stt_api_key"
    model: "en-US"
text_to_speech:
  provider: watson_tts
  watson_tts_config:
    api_url: "watson_tts_url"
    api_key: "your_watson_tts_api_key"
    voice: "en-US_AllisonV3Voice"
language: "en-US"
```

**Advanced Example:**

```yaml YAML theme={null}
name: watson_voice_config
llm_aggregation_timeout_seconds: 0.8
speech_to_text:
  provider: watson_stt
  watson_stt_config:
    api_url: "https://api.us-south.speech-to-text.watson.cloud.ibm.com/instances/your-instance-id"
    api_key: "your_watson_stt_api_key"
    model: "en-US_Telephony"
    background_audio_suppression: 0.5
    language_customization_id: null
    inactivity_timeout: 30
    profanity_filter: true
    smart_formatting: true
    speaker_labels: false
    redaction: false
    low_latency: true
    learning_opt_out: false
    watson_metadata: null
    smart_formatting_version: null
    customization_weight: null
    character_insertion_bias: null
    end_of_phrase_silence_time: 0.8
text_to_speech:
  provider: watson_tts
  watson_tts_config:
    api_url: "https://api.us-south.text-to-speech.watson.cloud.ibm.com/instances/your-instance-id"
    api_key: "your_watson_tts_api_key"
    voice: "en-US_AllisonV3Voice"
    language: "en-US"
    rate_percentage: 0
    pitch_percentage: 0
    customization_id: null
    meta_id: null
    learning_opt_out: false
language: "en-US"
```

**Watson STT Configuration Parameters:**

<Expandable title="detailed parameter descriptions">
  <ResponseField name="api_url" type="string" required>
    Watson Speech to Text service URL
  </ResponseField>

  <ResponseField name="api_key" type="string" required>
    Watson Speech to Text API key
  </ResponseField>

  <ResponseField name="model" type="string" required>
    Speech recognition model (e.g., "en-US\_Telephony")
  </ResponseField>

  <ResponseField name="background_audio_suppression" type="float">
    Background noise suppression level (0.0-1.0)
  </ResponseField>

  <ResponseField name="language_customization_id" type="string">
    Language customization ID for custom language models
  </ResponseField>

  <ResponseField name="inactivity_timeout" type="integer">
    Seconds of inactivity before stopping (-1 for no timeout, default: 30)
  </ResponseField>

  <ResponseField name="profanity_filter" type="boolean">
    Filter profanity in transcript
  </ResponseField>

  <ResponseField name="smart_formatting" type="boolean">
    Enable smart formatting (converts dates, currency, etc. to readable format)
  </ResponseField>

  <ResponseField name="speaker_labels" type="boolean">
    Enable speaker labels (beta feature)
  </ResponseField>

  <ResponseField name="redaction" type="boolean">
    Redact numbers with three or more consecutive digits
  </ResponseField>

  <ResponseField name="low_latency" type="boolean">
    Enable low latency mode. Available only for some next-generation models. See [Watson STT Models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng).
  </ResponseField>

  <ResponseField name="learning_opt_out" type="boolean">
    Opt out of data collection for learning purposes
  </ResponseField>

  <ResponseField name="watson_metadata" type="string">
    Value for x-watson-metadata header (max 512 characters)
  </ResponseField>

  <ResponseField name="smart_formatting_version" type="integer">
    Version of smart formatting to use
  </ResponseField>

  <ResponseField name="customization_weight" type="float">
    Weight for custom language model (0.0 to 1.0, default: 0.5)
  </ResponseField>

  <ResponseField name="character_insertion_bias" type="float">
    Bias for character insertion (-1.0 to 1.0, default: 0.0)
  </ResponseField>

  <ResponseField name="end_of_phrase_silence_time" type="float">
    Seconds of silence indicating end of phrase (0.0-120.0)
  </ResponseField>
</Expandable>

**Watson TTS Configuration Parameters:**

<Expandable title="detailed parameter descriptions">
  <ResponseField name="api_url" type="string" required>
    Watson Text to Speech service URL
  </ResponseField>

  <ResponseField name="api_key" type="string" required>
    Watson Text to Speech API key
  </ResponseField>

  <ResponseField name="voice" type="string" required>
    Voice model (e.g., "en-US\_AllisonV3Voice")
  </ResponseField>

  <ResponseField name="rate_percentage" type="integer">
    Speech rate adjustment percentage (default: 0)
  </ResponseField>

  <ResponseField name="pitch_percentage" type="integer">
    Speech pitch adjustment percentage (default: 0)
  </ResponseField>

  <ResponseField name="language" type="string">
    Language code for the voice (e.g., "en-US")
  </ResponseField>

  <ResponseField name="customization_id" type="string">
    Custom ID for the Watson TTS service
  </ResponseField>

  <ResponseField name="meta_id" type="string">
    Meta ID for the Watson TTS service
  </ResponseField>

  <ResponseField name="learning_opt_out" type="boolean">
    Opt out of data collection for learning purposes
  </ResponseField>
</Expandable>

#### Deepgram STT/TTS Configuration

**Basic Example:**

```yaml YAML theme={null}
name: deepgram_voice_config
speech_to_text:
  provider: deepgram_stt
  deepgram_stt_config:
    api_url: "wss://api.deepgram.com/v1/listen"
    api_key: "your_deepgram_api_key"
    model: "nova-2"
text_to_speech:
  provider: deepgram_tts
  deepgram_tts_config:
    api_key: "your_deepgram_api_key"
```

**Advanced Example (nova-3 with keyterm):**

```yaml YAML theme={null}
name: deepgram_voice_config
llm_aggregation_timeout_seconds: 0.8
speech_to_text:
  provider: deepgram_stt
  deepgram_stt_config:
    api_url: "wss://api.deepgram.com/v1/listen"
    api_key: "your_deepgram_api_key"
    model: "nova-3"
    keyterm: ["help", "search", "Mr. Smith"]  # Only supported for nova-3 models
    mip_opt_out: false
    channels: 1
    diarize: true
    dictation: false
    endpointing: 10
    extra: null
    interim_results: true
    keywords: null  # Only supported for nova-2 models
    language: "en-US"
    multichannel: false
    numerals: true
    profanity_filter: false
    punctuate: true
    redact: null
    replace: ["apple:orange", "orange:apple"]
    search: ["action item", "follow up"]
    smart_format: true
    tag: ["customer_service", "sales_call"]
    utterance_end_ms: 1000
    vad_events: true
    version: "latest"
    # v2/listen endpoint parameters
    eager_eot_threshold: null
    eot_threshold: null
    eot_timeout_ms: null
text_to_speech:
  provider: deepgram_tts
  deepgram_tts_config:
    api_key: "your_deepgram_api_key"
    api_url: "https://api.deepgram.com/v1/speak"
    language: "en"
    model: "aura-2-thalia-en"
    mip_opt_out: false
language: "en-US"
```

**Deepgram STT Configuration Parameters:**

<Expandable title="detailed parameter descriptions">
  <ResponseField name="api_url" type="string" required>
    Deepgram WebSocket API URL (e.g., "wss\://api.deepgram.com/v1/listen")
  </ResponseField>

  <ResponseField name="api_key" type="string" required>
    Deepgram API key
  </ResponseField>

  <ResponseField name="model" type="string" required>
    Speech recognition model (e.g., "nova-2", "nova-3"). Only `nova-2` and `nova-3` models (and their variations like `nova-2-finance`, `nova-3-medical`) are officially supported.
  </ResponseField>

  <ResponseField name="language" type="string">
    Language code (e.g., "en-US")
  </ResponseField>

  <ResponseField name="language_hint" type="string">
    Language hint for Flux multilingual model to improve transcription accuracy. See [Flux Multilingual & Language Prompting](https://developers.deepgram.com/docs/flux/language-prompting) for supported languages and usage. **Do not set this parameter when using the flux-general-en model.**
  </ResponseField>

  <ResponseField name="numerals" type="boolean">
    Convert numbers to numerals
  </ResponseField>

  <ResponseField name="keyterm" type="array">
    List of keywords to boost recognition accuracy (e.g., \["help", "support", "Mr. Smith"]). **Only supported for nova-3 models and their variations.** Using this parameter with nova-2 models will result in a validation error.
  </ResponseField>

  <ResponseField name="keywords" type="array">
    List of keywords to detect with optional boost/suppression values (e.g., \["snuffleupagus:5", "monster", "cookie:-2"]). **Only supported for nova-2 models and their variations.** Using this parameter with nova-3 models will result in a validation error.
  </ResponseField>

  <ResponseField name="mip_opt_out" type="boolean">
    Opt out of model improvement program
  </ResponseField>

  <ResponseField name="channels" type="integer">
    Number of audio channels
  </ResponseField>

  <ResponseField name="diarize" type="boolean">
    Enable speaker diarization
  </ResponseField>

  <ResponseField name="dictation" type="boolean">
    Enable dictation mode
  </ResponseField>

  <ResponseField name="endpointing" type="integer">
    Endpointing silence duration in seconds
  </ResponseField>

  <ResponseField name="extra" type="array">
    Extra parameters to pass to Deepgram (e.g., \["custom\_param1:value1"])
  </ResponseField>

  <ResponseField name="interim_results" type="boolean">
    Enable interim results
  </ResponseField>

  <ResponseField name="multichannel" type="boolean">
    Transcribe each audio channel independently
  </ResponseField>

  <ResponseField name="profanity_filter" type="boolean">
    Filter profanity
  </ResponseField>

  <ResponseField name="punctuate" type="boolean">
    Add punctuation and capitalization
  </ResponseField>

  <ResponseField name="redact" type="array">
    Redact sensitive information (e.g., \["pci", "pii"])
  </ResponseField>

  <ResponseField name="replace" type="array">
    Replace specified terms (e.g., \["apple:orange", "orange:apple"])
  </ResponseField>

  <ResponseField name="search" type="array">
    Search for specific terms (e.g., \["action item", "follow up"])
  </ResponseField>

  <ResponseField name="smart_format" type="boolean">
    Apply smart formatting to the transcript
  </ResponseField>

  <ResponseField name="tag" type="array">
    Tags for the request (e.g., \["customer\_service", "sales\_call"])
  </ResponseField>

  <ResponseField name="utterance_end_ms" type="integer">
    How long Deepgram will wait to send UtteranceEnd message after word has been transcribed
  </ResponseField>

  <ResponseField name="vad_events" type="boolean">
    Enable Deepgram's voice activity detection events
  </ResponseField>

  <ResponseField name="version" type="string">
    API version (e.g., "latest")
  </ResponseField>

  <ResponseField name="eager_eot_threshold" type="float">
    Only for v2 endpoint. End-of-turn confidence required to fire an eager EOT event (0.3-0.9)
  </ResponseField>

  <ResponseField name="eot_threshold" type="float">
    Only for v2 endpoint. End-of-turn confidence required to finish a turn (0.5-0.9)
  </ResponseField>

  <ResponseField name="eot_timeout_ms" type="integer">
    Only for v2 endpoint. Time (ms) after speech to finish turn regardless of EOT confidence
  </ResponseField>
</Expandable>

**Deepgram TTS Configuration Parameters:**

<Expandable title="detailed parameter descriptions">
  <ResponseField name="api_key" type="string" required>
    Deepgram API key
  </ResponseField>

  <ResponseField name="api_url" type="string">
    Deepgram API URL
  </ResponseField>

  <ResponseField name="language" type="string">
    Language code (e.g., "en")
  </ResponseField>

  <ResponseField name="model" type="string">
    Voice model (e.g., "aura-2-thalia-en")
  </ResponseField>

  <ResponseField name="mip_opt_out" type="boolean">
    Opt out of model improvement program
  </ResponseField>
</Expandable>

#### ElevenLabs TTS Configuration

**Basic Example:**

```yaml YAML theme={null}
name: elevenlabs_tts_config
speech_to_text:
  provider: watson_stt
  watson_stt_config:
    api_url: "https://api.us-south.speech-to-text.watson.cloud.ibm.com/instances/your-instance-id"
    api_key: "your_watson_stt_api_key"
    model: "en-US_Telephony"
text_to_speech:
  provider: elevenlabs_tts
  elevenlabs_tts_config:
    api_key: "your_elevenlabs_api_key"
    model_id: "eleven_turbo_v2_5"
    voice_id: "CwhRBWXzGAHq8TQ4Fs17"
```

**Advanced Example:**

```yaml YAML theme={null}
name: elevenlabs_tts_config
llm_aggregation_timeout_seconds: 0.8
speech_to_text:
  provider: watson_stt
  watson_stt_config:
    api_url: "example-stt-url"
    api_key: "example-stt-key"
    model: "example-model"
text_to_speech:
  provider: elevenlabs_tts
  elevenlabs_tts_config:
    api_url: "https://api.elevenlabs.io/v1/text-to-speech"
    api_key: "your_elevenlabs_api_key"
    model_id: "eleven_turbo_v2_5"
    voice_id: "CwhRBWXzGAHq8TQ4Fs17"
    language_code: "en"
    apply_text_normalization: "auto"
    optimize_streaming_latency: 3
    apply_language_text_normalization: false
    pronunciation_dictionary_locators: null
    seed: 42
    voice_settings:
      speed: 1.0
      stability: 0.5
      style: 0.0
      similarity_boost: 0.75
      use_speaker_boost: true
language: "en-US"
```

**ElevenLabs TTS Configuration Parameters:**

<Expandable title="detailed parameter descriptions">
  <ResponseField name="api_url" type="string">
    ElevenLabs API URL (e.g., "[https://api.elevenlabs.io/v1/text-to-speech](https://api.elevenlabs.io/v1/text-to-speech)")
  </ResponseField>

  <ResponseField name="api_key" type="string" required>
    ElevenLabs API key
  </ResponseField>

  <ResponseField name="model_id" type="string" required>
    TTS model ID (e.g., "eleven\_turbo\_v2\_5")
  </ResponseField>

  <ResponseField name="voice_id" type="string" required>
    Voice ID from ElevenLabs (e.g., "CwhRBWXzGAHq8TQ4Fs17")
  </ResponseField>

  <ResponseField name="language_code" type="string">
    Language code (e.g., "en", "ja" for Japanese)
  </ResponseField>

  <ResponseField name="apply_text_normalization" type="string">
    Text normalization mode ("auto", "on", "off")
  </ResponseField>

  <ResponseField name="optimize_streaming_latency" type="integer">
    Optimize streaming latency (0-4)
  </ResponseField>

  <ResponseField name="apply_language_text_normalization" type="boolean">
    Apply language-specific text normalization. **Currently only supported for Japanese (`ja`).** Using this parameter with other languages will result in a validation error.
  </ResponseField>

  <ResponseField name="pronunciation_dictionary_locators" type="array">
    List of pronunciation dictionary locators with `pronunciation_dictionary_id` and `version_id`
  </ResponseField>

  <ResponseField name="seed" type="integer">
    Seed for deterministic audio generation
  </ResponseField>

  <ResponseField name="voice_settings" type="object">
    Advanced voice customization settings
  </ResponseField>

  <ResponseField name="voice_settings.speed" type="float">
    Speech speed (default: 1.0)
  </ResponseField>

  <ResponseField name="voice_settings.stability" type="float">
    Voice stability - how stable the voice is and the randomness between each generation (0.0-1.0, default: 0.5)
  </ResponseField>

  <ResponseField name="voice_settings.style" type="float">
    Style exaggeration - the higher the value, the more computational resources are used (0.0-1.0, default: 0.0)
  </ResponseField>

  <ResponseField name="voice_settings.similarity_boost" type="float">
    Similarity boost - how closely the AI should adhere to the original voice (0.0-1.0, default: 0.75)
  </ResponseField>

  <ResponseField name="voice_settings.use_speaker_boost" type="boolean">
    Enable speaker boost (default: true)
  </ResponseField>
</Expandable>

### Advanced Voice Settings

Voice configurations support advanced features for enhanced call handling:

#### Voice Activity Detection (VAD)

Automatically detect when the user is speaking:

```yaml YAML theme={null}
vad:
  enabled: true
  provider: "silero_vad"
  silero_vad_config:
    confidence: 0.7          # Speech detection confidence threshold (0.0-1.0)
    start_seconds: 0.2       # Time before transitioning to SPEAKING state
    stop_seconds: 0.8        # Time before transitioning to QUIET state
    min_volume: 0.6          # Minimum audio volume threshold (0.0-1.0)
```

#### DTMF Input (Dual-Tone Multi-Frequency)

Enable keypad input during voice calls:

```yaml YAML theme={null}
dtmf_input:
  inter_digit_timeout_ms: 2500  # Time to wait for next digit (ms)
  termination_key: "#"          # Key to signal end of input
  maximum_count: 10             # Maximum number of digits
  ignore_speech: true           # Disable speech recognition during DTMF
```

#### User Idle Handler

Handle situations when the user stops responding:

```yaml YAML theme={null}
user_idle_handler:
  enabled: true
  idle_timeout: 7                                      # Timeout in seconds
  idle_max_reprompts: 2                                # Max reprompt attempts
  idle_timeout_message: "Are you still there?"         # Message to play on idle
  idle_hangup_message: ""                              # Message to play before hanging up
```

#### Agent Idle Handler

Provide feedback when the agent is processing:

```yaml YAML theme={null}
agent_idle_handler:
  typing_enabled: true                                 # Enable typing sound
  typing_duration_seconds: 5                           # Typing sound duration
  audio_clip_id: "guitar_1"                            # Hold Audio
  hold_audio_seconds: 15                               # Hold audio duration
  pre_hold_message: "Please wait a moment..."          # Message before hold audio
  hold_message: "Still processing your request..."     # Periodic hold message
```

### Complete Advanced Example

```yaml YAML theme={null}
name: advanced_voice_config
speech_to_text:
  provider: deepgram_stt
  deepgram_stt_config:
    api_url: "wss://api.deepgram.com/v1/listen"
    api_key: "your_deepgram_api_key"
    model: "nova-2"
    language: "en-US"
    numerals: true
    keywords: ["urgent", "important", "priority"]  # nova-2 specific
    mip_opt_out: false
text_to_speech:
  provider: deepgram_tts
  deepgram_tts_config:
    api_key: "your_deepgram_api_key"
    api_url: null
    language: "en"
    model: "aura-2-thalia-en"
    mip_opt_out: false
language: "en-US"

# Advanced Settings
vad:
  enabled: true
  provider: "silero_vad"
  silero_vad_config:
    confidence: 0.7
    start_seconds: 0.2
    stop_seconds: 0.8
    min_volume: 0.6

dtmf_input:
  inter_digit_timeout_ms: 2500
  termination_key: "#"
  maximum_count: 10
  ignore_speech: true

user_idle_handler:
  enabled: true
  idle_timeout: 7
  idle_max_reprompts: 2
  idle_timeout_message: "Are you still there?"
  idle_hangup_message: ""

agent_idle_handler:
  typing_enabled: true
  typing_duration_seconds: 5
  audio_clip_id: "guitar_1"
  hold_audio_seconds: 15
  pre_hold_message: "Please wait a moment..."
  hold_message: "Still processing your request..."
```

***

## Importing voice configurations

After creating your YAML file, import it using the following command:

```bash BASH theme={null}
orchestrate voice-configs import --file <path-to-your-voice-file>
```

<Expandable title="command flags">
  <ResponseField name="--file (-f)" type="string" required>
    Path to the YAML file with the voice configuration.
  </ResponseField>
</Expandable>

<Note>
  **Note:**
  The import command creates a new voice configuration or updates an existing one based on the name. If a voice configuration with the same name exists, the command updates it using the new configuration.
</Note>

***

## Listing voice configurations

To list all available voice configurations:

```bash BASH theme={null}
orchestrate voice-configs list
```

<Expandable title="command flags">
  <ResponseField name="--verbose (-v)">
    Show full details of all voice configurations in JSON format.
  </ResponseField>
</Expandable>

***

## Getting voice configuration details

To retrieve details of a specific voice configuration:

```bash BASH theme={null}
orchestrate voice-configs get --name <config_name>
```

<Expandable title="command flags">
  <ResponseField name="--id (-i)" type="string">
    Voice configuration ID to retrieve. Either ID or name is required.
  </ResponseField>

  <ResponseField name="--name (-n)" type="string">
    Voice configuration name to retrieve. Either ID or name is required.
  </ResponseField>
</Expandable>

***

## Exporting voice configurations

To export a voice configuration to a YAML file:

```bash BASH theme={null}
orchestrate voice-configs export --name <config_name> --output <output_path>
```

<Expandable title="command flags">
  <ResponseField name="--id (-i)" type="string">
    Voice configuration ID to export. Either ID or name is required.
  </ResponseField>

  <ResponseField name="--name (-n)" type="string">
    Voice configuration name to export. Either ID or name is required.
  </ResponseField>

  <ResponseField name="--output (-o)" type="string" required>
    Path where the YAML file should be saved.
  </ResponseField>
</Expandable>

***

## Removing voice configurations

To remove a voice configuration by name or ID:

```bash BASH theme={null}
orchestrate voice-configs remove --name <config_name>
# OR
orchestrate voice-configs remove --id <config_id>
```

<Expandable title="command flags">
  <ResponseField name="--id (-i)" type="string">
    Voice configuration ID to remove. Either ID or name is required.
  </ResponseField>

  <ResponseField name="--name (-n)" type="string">
    Voice configuration name to remove. Either ID or name is required.
  </ResponseField>
</Expandable>

<Note>
  **Note:**
  If both `--id` and `--name` are provided, the ID takes precedence.
</Note>

<Warning>
  **Important:**
  When you remove a voice configuration, agents using this configuration will no longer have voice capabilities. Make sure you want to remove the configuration before you proceed.
</Warning>