Guidelines: performance, context, and when to use them

Guidelines add dynamic, context-aware behavior to agents through conditional rules. Before using them, it is important to understand how they work internally — because the way they are evaluated has direct consequences on latency, cost, and reliability.

How guidelines work

Every user message or tool result triggers a two-phase process: Phase 1 — Selection (limited context) The agent runs a separate LLM call to evaluate which guidelines apply to the current conversation. At this stage, the LLM only has access to:

Conversation history (last 20 messages)
The configured guideline conditions

It does not have access to tool definitions, context variables, collaborator definitions, or agent instructions. Phase 2 — Execution (full context) The guidelines matched in Phase 1 are injected into the agent’s system prompt, placed after the agent instructions with explicit override language. The agent then reasons with full context: instructions, tools, collaborators, context variables, and conversation history. Selected guidelines take higher priority than regular instructions at this stage.

User message or tool result
        ↓
[Phase 1] Match guidelines — conversation history only
        ↓
[Phase 2] Agent reasoning — full context + matched guidelines injected
        ↓
Response or tool call

Guidelines are re-evaluated before every user message and before every tool result. They are not re-evaluated after the agent’s own responses.

Performance cost

Because guidelines add a dedicated LLM call at every turn, they have a measurable impact on latency and cost:

Impact	Details
~40% more LLM calls	Every user message and tool result triggers an extra inference call for guideline matching
+0.5–3 seconds per turn	Added latency per turn, measured at Phase 1 evaluation. Can be significantly higher with poorly scoped guidelines
Compounding risk	Extra inference + weaker context + duplicated reasoning + potential classification errors compound together

Why latency can spike: When a guideline condition references something Phase 1 cannot see (a tool name, a context variable, an instruction), the LLM has no reliable grounding signal. It compensates by expanding its reasoning — generating hypotheses, exploring possibilities, and attempting to resolve ambiguity — resulting in longer, less efficient reasoning traces. Example with 5 user messages and 3 tool calls:

8 LLM calls for guideline matching
8 LLM calls for agent reasoning
16 total LLM calls vs. 8 without guidelines

The decision fragmentation risk

Guidelines create a two-phase decision pipeline that can introduce mismatches between classification and reasoning:

Phase 1 classifies with weak signals (conversation only)
          ↓
Phase 2 reasons with strong signals (full context)

There is no reconciliation mechanism between the two phases. A classification error in Phase 1 biases Phase 2 incorrectly. Example failure scenario:

User says: “I need this done ASAP”
Phase 1 matches the “urgent” guideline
Phase 2 sees the full context and determines this is a routine documentation request
Result: mismatched priority handling and wasted resources

When to use guidelines

Guidelines are most reliable when the condition is directly observable in the conversation text — no system knowledge required, no inference needed. Good use cases:

Scenario	Example condition
Off-topic guardrails	`"user asks about weather, sports, or politics"`
Explicit urgency	`"user mentions 'urgent' or 'emergency'"`
Sentiment-based handling	`"user expresses frustration or dissatisfaction"`
Compliance requirements	`"user requests a regulated action"`

Poor use cases:

Scenario	Why it fails
Routing to a collaborator	Phase 1 has no knowledge of collaborators; short-circuits natural reasoning
Tool-based decisions	Phase 1 has no tool definitions; forces guessing, causes misclassification
Complex multi-step sequences	Guidelines are condition→action pairs; use Agentic Workflows instead
Context variable conditions	Phase 1 has no access to context variables; condition cannot be evaluated

Using guidelines for routing or tool-based logic is a common anti-pattern. It short-circuits the agent’s natural reasoning, masks underlying issues in instructions or collaborator definitions, and creates a maintenance burden. If routing is not working as expected, the root cause is almost always unclear agent instructions or collaborator descriptions — fix those directly.

Do’s and don’ts

✅ Do: Use for off-topic guardrails

guidelines:
  - condition: "user asks about weather, sports, or politics"
    action: "politely redirect to product-related questions"

Why it works: The signal is directly observable in the conversation text. No system knowledge is required to evaluate the condition.

✅ Do: Use for explicit urgency detection

guidelines:
  - condition: "user mentions 'urgent' or 'emergency'"
    action: "prioritize the request immediately"

Why it works: The condition is specific, unambiguous, and detectable from message content alone.

❌ Don't: Use for routing logic

# Avoid this pattern
guidelines:
  - condition: "user asks about billing"
    action: "route to billing_specialist collaborator"

Why it fails: Phase 1 has no knowledge of collaborators. This short-circuits the agent’s natural decision-making and hides the real issue — likely unclear instructions or a poorly described collaborator.Better approach: Write clear agent instructions that describe when to delegate, and ensure collaborator descriptions accurately reflect their capabilities.

❌ Don't: Use for tool-based conditions

# Avoid this pattern
guidelines:
  - condition: "user needs the account_lookup tool"
    action: "use account_lookup"

Why it fails: Phase 1 has no access to tool definitions. The LLM must guess what the tool is, causing expanded reasoning paths and unreliable matching.Better approach: The agent already has full tool context in Phase 2. Use instructions to guide tool selection.

❌ Don't: Use for context variable conditions

# Avoid this pattern
guidelines:
  - condition: "if user_id is 'premium_tier'"
    action: "provide premium support options"

Why it fails: Context variables are not available in Phase 1. The condition cannot be evaluated, leading to ambiguous or incorrect matching.Better approach: Handle tier-based logic in agent instructions, where context variables are fully resolved.

❌ Don't: Use for complex multi-step sequences

# Avoid this pattern
guidelines:
  - condition: "user wants to make a purchase"
    action: "check inventory, verify payment, calculate shipping, apply discounts, confirm order"

Why it fails: Guidelines are designed for simple condition→action pairs. Multi-step sequences belong in Agentic Workflows, which provide proper state management, error handling, and flow control.

Guidelines vs. instructions

Both instructions and guidelines influence agent behavior, but they operate differently:

	Instructions	Guidelines
Evaluation	Once per turn, when building the system prompt	Twice per turn: Phase 1 (selection) + Phase 2 (execution)
Context at evaluation	Full context always available	Phase 1 has limited context; Phase 2 has full context
Presence in prompt	Always present	Conditionally injected based on Phase 1 matching
Behavior	Static throughout the conversation	Dynamic — different guidelines can match at different turns
Priority	Baseline	Matched guidelines override instructions in Phase 2
Performance cost	None (already part of agent reasoning)	~40% more LLM calls

Rule of thumb: If the behavior is constant and does not depend on what was just said in the conversation, put it in instructions. Guidelines are for behavior that genuinely needs to activate or deactivate based on observable conversation signals.

Checklist before adding a guideline

Before adding a new guideline, verify:

The condition is directly observable in conversation text — no tool, variable, or instruction knowledge required
The benefit of the dynamic behavior clearly justifies the ~40% LLM call overhead
The behavior truly needs to be conditional (not just always-on instructions)
The risk of a Phase 1 classification error and its downstream impact is acceptable
You have ruled out fixing the underlying issue in instructions or collaborator definitions instead

Foundational architecture considerations Tooling and scalability considerations Building agents

Release Notes

Get Started

Build

Deploy

Analyze

Developer experience

Legal notices

Guidelines: performance, context, and when to use them

How guidelines work

Performance cost

The decision fragmentation risk

When to use guidelines

Do’s and don’ts

Guidelines vs. instructions

Checklist before adding a guideline

​How guidelines work

​Performance cost

​The decision fragmentation risk

​When to use guidelines

​Do’s and don’ts

​Guidelines vs. instructions

​Checklist before adding a guideline

​Related topics

How guidelines work

Performance cost

The decision fragmentation risk

When to use guidelines

Do’s and don’ts

Guidelines vs. instructions

Checklist before adding a guideline

Related topics