Observability refers to the ability to monitor and understand the internal state of an LLM application, including its inputs and outputs. watsonx Orchestrate provides observability through a native integration with an open-source project known as Langfuse. To enable IBM’s native observability framework, start the Orchestrate server with the --with-ibm-telemetry or -i flag, then go to https://localhost:8765 in your browser.

​
Enabling Langfuse on SaaS

You can configure your own hosted Langfuse instance for observability. You can configure it directly in the CLI: Flags:
ArgumentDescription
--url / -uThe URL of the Langfuse instance (required if not specified in --config-file).
--api-keyThe Langfuse API key (required if not specified in --config-file).
--health-uriThe Health URI of the Langfuse instance (required if not specified in --config-file).
--config-fileA config file for the Langfuse integration (can be fetched using orchestrate settings).
--config-jsonA config JSON object for the Langfuse integration, this object should contain your Langfuse public_key.
BASH
orchestrate settings observability langfuse configure \
--url "https://cloud.langfuse.com/api/public/otel" \
--api-key "sk-lf-0000-0000-0000-0000-0000" \
--health-uri "https://cloud.langfuse.com" \
--config-json '{"public_key": "pk-lf-0000-0000-0000-0000-0000"}'
Or using an external file in YAML format:
YAML
spec_version: v1
kind: langfuse
project_id: default
api_key: sk-lf-00000-00000-00000-00000-00000
url: https://cloud.langfuse.com/api/public/otel
host_health_uri: https://cloud.langfuse.com
config_json: 
public_key: pk-lf-00000-00000-00000-00000-00000
mask_pii: true
BASH
orchestrate settings observability langfuse configure --config-file=path_to_file.yml