New in 1.11.0
quick-eval command provides a fast, reference-less evaluation of your agents and tools.
Note:For now, you can use only Python tools.
evaluate command, it does not require ground truth datasets. Instead, it runs a lightweight check to identify common issues such as schema mismatches and hallucinations in tool calls.
config.yaml
Flags
Flags
Comma-separated list of test files or directories containing ground truth datasets. Required when not using a configuration file.
Understanding the Summary Metrics Table
At the end of the evaluation, you will see a summary similar to the following one:
Metrics explained
Quick Evaluation Summary MetricsIf the value is equal to 1.0 or
True, the table omits the result.
