Monitoring your LLMs with Langfuse - IBM watsonx Orchestrate ADK

Observability refers to the ability to monitor and understand the internal state of an LLM application, including its inputs and outputs. watsonx Orchestrate provides observability through a native integration with an open-source project known as Langfuse. To enable IBM’s native observability framework, start watsonx Orchestrate Developer Edition server with the --with-ibm-telemetry or -i flag, then go to https://localhost:8765 in your browser.

Enabling Langfuse on SaaS

AvailabilityThis experience is currently only available on AWS SaaS Commercial and IBM Cloud Trial, Essential and Standard (non-isolated) instances.

You can configure your own hosted Langfuse instance for observability. You can configure it directly in the CLI:

BASH

orchestrate settings observability langfuse configure \
--url "https://cloud.langfuse.com/api/public/otel" \
--api-key "sk-lf-0000-0000-0000-0000-0000" \
--health-uri "https://cloud.langfuse.com" \
--config-json '{"public_key": "pk-lf-0000-0000-0000-0000-0000"}'

Show command flags

--url / -u

string

The URL of the Langfuse instance (required if not specified in --config).

--health-uri

string

The Health URI of the Langfuse instance (required if not specified in --config).

--project-id / -p

string

The Langfuse project ID (required if not specified in --config).

--api-key

string

The Langfuse API key (required if not specified in --config).

--config

string

A config file for the Langfuse integration (can be fetched using orchestrate settings).

--config-json

string

A config JSON object for the Langfuse integration, this object should contain your Langfuse public_key.

Or using an external file in YAML format:

YAML

spec_version: v1
kind: langfuse
project_id: default
api_key: sk-lf-00000-00000-00000-00000-00000
url: https://cloud.langfuse.com/api/public/otel
host_health_uri: https://cloud.langfuse.com
config_json: 
   public_key: pk-lf-00000-00000-00000-00000-00000
mask_pii: true

BASH

orchestrate settings observability langfuse configure --config-file=path_to_file.yml

Getting configuration settings

To retrieve details of your current Langfuse configuration, run the following command:

BASH

orchestrate settings observability langfuse get --output <output file path>

Show command flags

--output / -o

string

Specify a file to store the results. Use one of these extensions: .yml, .yaml, or .json.
When you set this flag, the command writes the Langfuse configuration to the file. Without this flag, the command prints the configuration to standard output.

Removing configuration settings

To remove your current Langfuse configuration, run the following command:

BASH

orchestrate settings observability langfuse remove

​Enabling Langfuse on SaaS

​Getting configuration settings

​Removing configuration settings

Enabling Langfuse on SaaS

Getting configuration settings

Removing configuration settings