Skip to main content

​
Summary

​
Core Principles

  • Apply GitOps principles using Git branches for versioning instead of manual version directories.
  • Replace Argo CD with Jenkins multi-branch pipeline as the orchestrator.
  • Use wxO CLI to deploy agents, flows, tools, and knowledge-bases via orchestrate commands.
  • Agent YAML files are hand-written (not CLI-generated) and stored in Git.
  • Knowledge-bases are tenant-scoped (not environment-scoped) and affect both Draft and Live environments simultaneously.
  • Maintain secret-free Git via connections.template.yaml + secrets manager substitution at deploy time.
  • Achieve traceable promotions via Git merges, safe rollbacks via Git reverts, and consistent deployments across environments without direct cluster access.

​
Git-Based Versioning

  • Git branches represent environment state (dev, qa, preprod, prod)
  • Git tags mark deployments (v1.2.3-dev, v1.2.3-qa, etc.)
  • Git merges promote code between environments
  • No manual version management: Git handles all versioning automatically
  • Natural evolution: Agent structures can change significantly between environments

​
Two-Tenant Architecture

dev-qa-tenant for non-production workloads
  • Draft environment → Dev (develop branch)
  • Live environment → QA (qa-<version> branches)

​
Key Benefits

Complete Isolation

Production infrastructure is completely separated from development/testing

Security

Different credentials, RBAC, and network policies per tenant

Compliance

Separate audit trails for production vs non-production

Risk Mitigation

Issues in Dev/QA cannot impact PreProd/Prod

Flexibility

Each tenant can have different configurations, quotas, and policies

Simplified Versioning

Git branches and tags replace manual version directories

Natural Evolution

Code structure can evolve naturally through branches

Easy Comparison

git diff shows exactly what will be promoted

​
Promotion Path (Git Branch Merging)

develop → dev-qa-tenant Draft
  ↓ (PR: develop → qa-<version>)
qa-<version> → dev-qa-tenant Live
  ↓ (PR: qa-<version> → staging-<version>) ← CROSS-TENANT BOUNDARY
staging-<version> → prod-tenant Draft
  ↓ (PR: staging-<version> → main)
main → prod-tenant Live (PRODUCTION)

Git Tags:
v1.2.3-dev → v1.2.3-qa → v1.2.3-staging → v1.2.3

​
Critical Cross-Tenant Transition

The qa-<version> → staging-<version> merge is the critical boundary where:
  • Git merge switches from dev-qa-tenant to prod-tenant
  • Pipeline switches authentication and API endpoints
  • Config files (tenant.yaml, values.yaml) may have merge conflicts (keep staging’s config)
  • Additional approval gates are enforced
  • Production-grade validation begins

​
Implementation Checklist

  • Provision two wxO tenants (dev-qa-tenant, prod-tenant)
  • Set up Git repository with protected branches (develop, qa-, staging-, main)
  • Configure branch protection rules and approval requirements
  • Set up tenant-specific secrets in secrets manager
  • Configure Jenkins multi-branch pipeline
  • Configure Jenkins with tenant-aware credentials
  • Create tenant-specific connection templates
  • Implement approval gates for cross-tenant promotions (qa-* → staging-*)
  • Set up Git tagging automation in Jenkins
  • Set up monitoring and alerting per tenant
  • Document merge conflict resolution strategy
  • Document runbooks for each tenant
  • Train team on versioned branch workflow (qa-<version>, staging-<version>)
  • Train team on rollback procedures (git revert, git reset)