The phrase “AI-powered” has become meaningless. Every SaaS vendor slaps it on their product page, and what you actually get is a search bar with a language model behind it. Real agentic AI — systems that can break down a goal, devise a multi-step plan, call external tools, evaluate results, and self-correct — is a fundamentally different engineering problem.
After 18 months of building and observing agentic systems in financial services and insurance contexts, here are the patterns that actually hold up in production.
What Makes a System “Agentic”
A system is agentic when it meets three criteria simultaneously:
- Goal-directedness — it receives an objective, not a script.
- Tool use — it can invoke external APIs, databases, or sub-agents to gather information or cause side-effects.
- Iterative reasoning — it evaluates intermediate results and revises its approach before returning a final answer.
A RAG pipeline with a chatbot UI is not agentic. A system that receives “prepare the quarterly compliance summary for Ontario regulations” and proceeds to query three internal systems, cross-reference them against a regulatory document store, flag discrepancies, and draft a structured report — that’s agentic.
The Orchestrator / Worker Pattern
The most durable pattern I’ve seen in production is a two-tier architecture:
Orchestrator layer — A large reasoning model (GPT-4o, Claude Sonnet, or equivalent) that owns the goal, maintains state, and decides which tools to invoke next. It does not do work itself; it delegates.
Worker layer — Smaller, faster, cheaper models or deterministic functions that execute individual tasks: web search, SQL queries, document parsing, form submissions. Workers are stateless. They receive a task and return a structured result.
The orchestrator maintains a “scratchpad” (in practice, the conversation context window or an external state store for long-running tasks) that accumulates evidence before synthesising a final response.
┌─────────────────────────────────────────────┐
│ Orchestrator │
│ Goal → Plan → [Tool calls] → Synthesis │
└────────────┬────────────────────┬────────────┘
│ │
┌───────▼──────┐ ┌────────▼──────┐
│ Worker: DB │ │ Worker: Docs │
└──────────────┘ └───────────────┘
Guard-rails That Actually Work
Every enterprise AI conversation eventually comes around to the risk question. Here’s what I’ve found to be practically effective:
1. Structured output contracts
Force the orchestrator to return JSON matching a schema before any downstream system acts on its output. Use something like Zod (TypeScript) or Pydantic (Python) to validate at the boundary. A model that can’t satisfy the schema triggers a retry or a fallback to a human queue — not a silent failure.
2. Confirmation checkpoints for irreversible actions
Anything that writes data, sends an email, or charges money should require an explicit human confirmation step unless the confidence score exceeds a threshold you’ve calibrated with your risk team. Build this as a first-class node in your agent graph, not an afterthought.
3. Audit log every decision
Log the full reasoning trace — not just the final answer. When something goes wrong (and it will), you need to reconstruct what the agent believed at each step. This is also what satisfies your compliance team. Store traces in append-only storage.
4. Scope-limited tool permissions
Each worker should have the minimum permissions to do its job. The document-parsing worker should not have write access to your CRM. Treat agent tool permissions like you’d treat service account permissions in any other microservices architecture.
Latency and Cost Realities
Agentic pipelines are slow and expensive compared to a single LLM call. A three-tool chain with an orchestrator and two workers might take 8–15 seconds and cost 10–20x what a simple completion does. This is fine for async, background tasks. It is not fine for real-time customer-facing interactions.
Design your system so the agentic layer runs in the background and surfaces results when ready, rather than holding the user interface hostage to a synchronous chain. Progress indicators, streaming partial results, and asynchronous job patterns are your friends.
Where to Start
If you’re building your first agentic system in an enterprise context, I’d recommend this sequencing:
- Pick a single internal workflow that is currently manual, repetitive, and low-stakes. Regulatory change monitoring is a good candidate.
- Define the goal in plain language. Can you articulate it in one sentence? If not, the scope is too broad.
- Enumerate the tools the agent would need. If it’s more than five, scope down.
- Build with maximum observability from day one — logging, tracing, and human review queues.
- Deploy to a small internal user group before any external exposure.
The technology is real, the productivity gains are real, and the failure modes are also very real. Build accordingly.