Skip to main content
The following table covers the most common knowledge and document processing anti-patterns that affect agent response quality and accuracy.
⭐️ Most common anti-patterns

​
What it is

  • Assuming whatever RAG retrieves is inherently correct
  • Treating vector stores, CMS, Confluence, SharePoint as curated sources of truth
  • No validation of knowledge quality before or after retrieval

​
Why it fails

  • Enterprise content is rarely clean (outdated policies, conflicting versions, stale information)
  • RAG retrieves semantically similar text, not verified truth (similar ≠ correct)
  • Agents hallucinate within retrieved content when chunks have gaps
  • RAG can’t distinguish policy versioning unless explicitly encoded
  • Agents can’t detect contradictions between conflicting chunks
  • All content treated equally (high-risk policy = informal wiki page)

​
What to do instead: Knowledge quality is a first-class concern

  • Knowledge verification pipelines: Validate content for correctness, consistency, recency, conflicts, versioning before indexing
  • Metadata-driven retrieval: Add version, effective date, document type, risk classification, authoritative owner, region/jurisdiction
  • Policy-first structuring: Extract canonical definitions, rules, conditions, exceptions, thresholds, regulatory constraints; expose through structured APIs
  • Retrieval confidence + cross-checking: Let agents query authoritative sources, compare results, escalate contradictions, ask follow-up questions
  • Human-in-the-loop for high-risk domains: Require review for compliance/legal/medical content; provide “verified content only” modes

​
What it is

  • Assuming semantic search can magically infer structure, correctness, and intent from messy content
  • Believing RAG compensates for poor knowledge architecture
  • No cleanup of thousands of pages across Confluence, PDFs, SharePoint, wikis, legacy docs

​
Why it fails

  • RAG amplifies disorganization, doesn’t correct it
  • RAG returns whatever is “closest,” not whatever is “true”
  • Disorganized sources produce conflicting chunks that agents merge incorrectly
  • Semantic retrieval has no notion of versioning or authority
  • Large, messy spaces generate retrieval flooding (noise vs. signal)
  • RAG cannot infer structure that doesn’t exist
  • Garbage in → amplified garbage out (with confident-sounding errors)

​
What to do instead: Prepare knowledge for agents, not humans

  • Identify knowledge types: Definitions, rules/constraints, procedures, exceptions, facts, disclosures, reference material, narrative context (RAG treats these identically; agents do not)
  • Restructure into agent-friendly formats: Long PDFs → rule tables/schemas; paragraphs → JSON rules; buried instructions → deterministic workflow steps; mixed definitions → canonical glossary tool; conflicting versions → versioned hierarchy
  • Use RAG only for right categories: Excellent for concept retrieval, domain context, semantic interpretation, large knowledge spaces; terrible for factual truth, procedural rules, policy enforcement, compliance, exactness
  • Make knowledge quality a responsibility: Introduce metadata, ownership, validation, versioning, proper indexing, lifecycle governance

​
What it is

  • Returning excessive knowledge passages that get sent to the LLM on every conversation turn
  • Knowledge passages, once retrieved, are included in context for every subsequent turn
  • Teams return 10-15 passages when 2-3 would suffice (“just in case” over-retrieval)

​
Why it fails

  • Compounding token costs: 20K tokens × 5 turns = 100K tokens of redundant context
  • Context exhaustion and attention dilution: Large retrievals consume context budget and weaken focus
  • Per-turn cost multiplication: Every token retrieved is sent repeatedly throughout conversation

​
What to do instead: Precision retrieval with token awareness

  • Precision over recall: Return top 3-5 passages, not 10-15; use higher similarity thresholds
  • Progressive retrieval: Start with minimal, high-confidence passages; retrieve more only when needed
  • Monitor utilization: Track tokens per turn, retrieval utilization, redundancy rate, cost per conversation
  • Principle: Knowledge retrieval is not a one-time cost—it’s a per-turn cost. Optimize for precision, not coverage.

​
What it is

  • Using default chunk sizes (512 or 1024 tokens) without understanding document content or use case
  • Treating chunking as one-time configuration rather than content-aware strategy
  • Assuming single chunk size works for FAQs, legal documents, technical manuals, and product catalogs

​
Why it fails

  • Different content has different natural boundaries: FAQs have Q&A pairs, legal docs have clauses, manuals have procedures
  • Default sizes are arbitrary: Not chosen for your content—may split critical information or group irrelevant content
  • Size mismatches create problems: Too large = “needle in haystack”; too small = broken answers requiring synthesis
  • Content structure ignored: Tables, lists, code blocks, hierarchical sections need different strategies than prose

​
What to do instead: Content-aware chunking strategy

  • Match strategy to content type: FAQs (100-300 tokens per Q&A pair), Legal/policy (300-800 tokens per clause), Technical manuals (200-500 tokens per procedure), Product catalogs (50-200 tokens per product)
  • Preserve semantic boundaries: Don’t split mid-paragraph or mid-concept
  • Use overlapping chunks: Provide context continuity at boundaries (10-20% overlap)
  • Test with real queries: Measure retrieval quality, not just similarity scores
  • Monitor multi-chunk needs: Track how often agents need multiple chunks to answer questions

​
What it is

  • Using agent knowledge for questions requiring access to entire document or corpus
  • Expecting RAG to handle counting, aggregation, ordering, or document structure operations

​
Why it fails

  • RAG retrieves top matching chunks (5-10), not complete documents
  • Counting, aggregation, ordering require full dataset access
  • Document structure (rows, pages, sections) is lost in chunking
  • File type and metadata aren’t preserved in vector embeddings
  • Agent tries to retrieve rows matching query words rather than operating on original data representation

​
What to do instead: Use databases and APIs, not RAG for everything

  • Counting/aggregation: Store in database (SQL, NoSQL), query directly (e.g., SELECT COUNT(*) FROM products WHERE company='ACME' for exact counts)
  • Document structure: Use format-specific APIs/parsers (pandas/openpyxl for spreadsheets, PyPDF2/pdfplumber for PDFs)
  • Metadata filtering: Store metadata in queryable fields (file type, date, author), filter with standard database queries
  • Whole-document analysis: Use long-context models (Claude 3.5 with 200K context) or traditional document processing pipelines
  • Principle: RAG excels at semantic search over unstructured content; use database/API tools built for structured queries, counting, aggregation, format-specific operations

​
What it is

  • Treating all document formats equally in knowledge ingestion
  • No format-specific processing strategies

​
Why it fails

  • XLSX files lose row context: Each cell indexed separately, breaking tabular relationships
  • PDFs with complex layouts: Tables, multi-column text, embedded images get mangled
  • Scanned documents: OCR errors introduce noise
  • Presentations: Slide context and visual hierarchy are lost

​
What to do instead: Format-aware processing

  • Tabular data: CSV preferred over XLSX (each row’s data preserved in same context)
  • Structured data: Extract to JSON or database format before indexing
  • PDFs: Use layout-aware parsers that preserve table structure
  • Presentations: Extract speaker notes and slide content separately with metadata
  • Scanned documents: Apply OCR with confidence scoring and manual review for critical content

​
What it is

  • Conflating data extraction (getting data as-is) with document processing (analyzing, calculating, transforming)
  • Cramming extraction AND processing into single Document Extractor node
  • Assuming Document Extractor handles both pulling data AND performing operations on it

​
Why it fails

  • Document Extractor is prompt-engineered for extraction only (data as it appears on page)
  • Verbs in entity definitions (“summarize,” “calculate”) interfere with extraction-focused prompt
  • Mixing extraction with transformation confuses the model (extract verbatim or perform operations?)
  • Multi-step logic doesn’t fit extraction paradigm (no conditionals, aggregations, or multi-pass analysis)

​
What to do instead: Decompose document tasks

  • Step 1 - Extract with Document Extractor: Pull raw data exactly as it appears
  • Step 2 - Process with appropriate tools: Calculations (calculator/code), summarization (separate LLM call), validation (business logic tools), comparison (dedicated tools), aggregation (data processing/SQL)
  • Step 3 - Orchestrate with Agentic Workflow: Coordinate multi-step process (extract → validate → calculate → summarize → return)
  • Each step uses right tool for the job, orchestrated by workflow layer (more reliable, easier to debug, more maintainable, better edge case handling)

​
What it is

  • Using minimal, vague, or ambiguous field definitions in Document Extractor
  • Spending minimal effort on field specs and assuming system will figure it out
  • Examples: amount (which amount?), date (which date?), address (which address?), “the price” (which price?)

​
Why it fails

  • Field names/descriptions become part of the extraction prompt (vague inputs → vague outputs)
  • Complex documents have multiple similar items that need disambiguation (invoices have 7+ different “amounts”)
  • Semantic ambiguity causes wrong extractions (“date” could mean 5+ different dates on one document)
  • Works for simple documents but fails at scale (receipt vs. contract/medical record/financial statement)

​
What to do instead: Explicit, unambiguous field definitions

  • Use specific field names: invoice_total_amount not amount; invoice_date not date; billing_address not address; customer_legal_name not name
  • Provide detailed descriptions: “The final total amount due including all line items, taxes, and fees, typically found at the bottom of the invoice”
  • Specify location hints: “The payment due date, usually labeled as ‘Due Date’ or ‘Payment Due By’ near the top right of the invoice”
  • Disambiguate similar fields: Explicitly state which of multiple similar items you want (unit_price vs. extended_price)
  • Include format expectations: “The date the invoice was issued, in MM/DD/YYYY format, typically labeled ‘Invoice Date’ near the invoice number”
  • Specify data types and constraints: “List of all items purchased. Each item should include: item_description (string), quantity (number), unit_price (decimal), line_total (decimal)”
  • Treat field definitions as prompt engineering: Quality of extraction directly tied to quality of field specifications
Foundational architecture considerations Tooling and scalability considerations watsonx Orchestrate skills