← Back to Blog
AIAgentsArchitectureProductionMCP

The Agentic Stack in 2026: What Actually Powers Production AI Agents

Daniel Casale · May 5, 2026 · 10 min read

Demos Are Easy. Production Is a Stack.

In 2026 you can stand up an impressive AI agent demo in a weekend. Pick a model, give it tools, throw a prompt at it, watch it do something useful. The demo will work. It will keep working as long as you are the only person using it and you do not care what it does on bad inputs.

The gap between that demo and an agent you trust to run unsupervised on real customer data is enormous. It is also predictable. Every production agent we have shipped uses roughly the same stack, because the same problems show up every time once real users get involved.

Here is what is actually in the stack, and what each layer is doing for you.

Layer 1: The Model

This is the part everyone gets right. Pick a frontier model from Anthropic, OpenAI, or via Bedrock. Pick a smaller model for cheap calls. Done.

Two real choices live in this layer.

Frontier vs cheap. Frontier models (Claude Opus, GPT-5) for reasoning-heavy steps. Cheap models (Haiku, GPT-5 mini, Llama 3.3) for classification, extraction, and simple transforms. A well-architected agent uses both, routed by step type, not a single model for everything.

Provider strategy. OpenAI direct, Bedrock, or both behind an abstraction. We covered the tradeoffs in AWS Bedrock vs OpenAI API: Which Should Your Startup Use in 2026?. Most production builds we ship end up with a thin model abstraction so the routing decision is changeable.

The model layer is the easiest to get right and the layer founders spend the most time obsessing over. Move on.

Layer 2: The Tool Surface (Where MCP Lives)

Your agent does not do anything useful without tools. Tools are the actions the model can take: read a database, send an email, fetch a webpage, update a record, schedule a job.

The choice is how you expose those tools. Three options, in increasing order of leverage.

Hand-rolled function calling. Each tool is a function in your codebase, registered with the model SDK. Works fine for a small fixed set of tools. Becomes painful past 20 tools or when you want to share tools across multiple agents.

An internal tool registry. A central catalog where tools register themselves, the agent loads what it is allowed to use at runtime, and permissions are scoped per agent role. Better. Most teams reinvent this badly.

Model Context Protocol (MCP). Standardized tool servers that any agent can call. Tools are decoupled from the agent process and can be reused across multiple agents and frameworks. We use MCP for almost everything now. We covered the practical details in MCP Servers: A Practical Guide for Teams Getting Started.

The single biggest mistake we see in production agents is tools designed for the model rather than against the model. Tool descriptions need to be ruthless about what the tool does, what it does not do, and what the inputs and outputs look like. The model is reading those descriptions to decide whether to call the tool. Ambiguity in the description shows up as bad behavior in the trace.

Layer 3: Orchestration

This is where most teams either over-engineer or under-engineer. There are exactly three useful patterns.

Single-pass with tools. The agent gets a goal, calls tools in a loop until done, returns. Works for tasks that complete in 1 to 20 tool calls. Most production agents we ship are this.

Multi-agent with explicit handoffs. A coordinator agent dispatches to specialist agents, each with a focused tool set and prompt. Works when the task naturally decomposes into clearly different specialties. Overkill for most use cases.

Workflows with model steps. A directed graph of steps, where some steps are model calls and some are deterministic code. Use this when the steps are knowable in advance and you mostly want the model for specific judgment calls. Way easier to debug than free-form agents.

We use single-pass for most agent work, workflows for anything with strict ordering or compliance requirements, and multi-agent rarely. If you are reaching for multi-agent, ask whether you actually have multiple agents or just one agent with multiple tool sets.

Layer 4: Memory

Memory is the part that separates "demo agent" from "agent your users actually like."

Three kinds of memory matter.

Conversation memory. What the user and agent said earlier in this session. Trivially handled by the model context window for short sessions. Needs summarization or vector retrieval for long sessions.

Long-term memory. What the agent has learned about this user, project, or domain across sessions. "Pat prefers dry humor in responses." "This client's tax season is March." Stored in a database, retrieved at the start of every session, kept fresh by an explicit memory write step.

Working memory. What the agent has discovered during this task that it needs to keep coherent across tool calls. Often handled inline in the conversation, but for complex multi-step tasks we externalize it to a scratchpad.

The failure mode here is treating memory as a vector database problem. Vector retrieval is one tool in the memory toolkit. Most useful memory is structured, indexed by user or project, and retrieved by exact match, not similarity. A relational database with a memory schema beats a pure vector store for nine out of ten production agent use cases.

Layer 5: Guardrails

Guardrails are not a feature you add at the end. They are how the agent earns the right to take actions on real systems.

Production guardrails we always include:

We covered the safety side in more depth in AI Agents vs Chatbots: Why the Difference Matters for Your Business.

Layer 6: Observability

You will not understand what your agent is doing until you can see it. Observability for agents looks different from regular application observability.

Per-step traces. Every model call, every tool call, every input and output. Stored long enough that you can replay a bad session a week later when the user complains.

Token and cost accounting. Per session, per user, per agent. You need this for billing decisions, for catching runaway loops, and for understanding which features are economically viable.

Outcome metrics. Did the agent finish the task? Did the user retry? Did the agent escalate to a human? These tell you whether the agent is actually working, separate from whether the model is responsive.

Eval harness. A growing set of test cases that run on every prompt change. The single biggest source of agent regression is "we tweaked the prompt and broke three things we did not test." A real eval suite catches this.

We use Langfuse and OpenTelemetry for traces, custom dashboards in CloudWatch for outcomes, and per-feature eval suites that run in CI on every change to a prompt or tool.

Layer 7: The Boring Stuff That Matters

The layers above get the attention. The layers below decide whether your agent is actually deployable.

Authentication and authorization. Who is the user, what can they ask the agent to do, and how do you enforce that across every tool? OAuth2 for user identity, IAM and tenant scoping for tool authorization.

Secrets management. API keys for tools, model provider credentials, customer integrations. Secrets Manager, never environment files in production.

Deployment and versioning. Prompts are code. Tool definitions are code. They need version control, code review, staging environments, and rollback paths. We have seen too many teams treat prompts as configuration data and discover three months later that nobody knows when "the bad prompt" got pushed.

Cost monitoring. Per-feature, per-tenant. We pipe model spend into the same cost dashboard as the rest of the AWS bill. Surprises are the enemy of margin.

For the AWS side specifically, AWS Cost Optimization: 7 Quick Wins That Save Thousands covers the patterns we use to keep the infra layer in check while the model spend grows.

What the Stack Looks Like in Practice

A representative production agent we have shipped:

Total infrastructure cost beyond model spend: under $400 per month for the first 100 active users. Model spend: roughly $0.20 to $1.50 per active session depending on task complexity.

Where Founders Get Stuck

Three failure patterns dominate.

Demo-driven development. The demo works. The team ships. The first real user does something the demo did not handle, the agent loops, and the team spends three weeks on a bandaid. The fix is to invest in evals before you invest in features.

One-tool-per-feature thinking. Every new feature gets its own bespoke tool. The tool surface explodes. The model gets confused. The fix is to design tools for composition: small, sharp, reusable across features.

Skipping memory. The agent works for one-off tasks but feels stupid in a long-running relationship with the user. Memory is what makes the agent feel like it has a continuous identity. Without it, every session starts from zero.

Where to Start

If you are just starting on agents, build one single-pass agent with three tools, no memory, no multi-agent. Get the trace, eval, and guardrail loops working at that scale. Then add capability.

If you are scaling an existing agent, the highest-leverage investments are usually evals and observability. Most teams over-invest in orchestration frameworks and under-invest in seeing what is happening.

If you are deciding whether to build agents at all, start with Why Your Business Needs AI Agents in 2026 and AI Agents vs Chatbots to clarify what you actually need.

When you are ready to design or rebuild your stack, get in touch. Our AI agent deployment service is structured around the layers above, with fixed pricing and concrete deliverables at each layer.

Further reading

Want to Talk About Your Project?

We write about what we do every day. If any of this resonates, let's chat.

Book a call See pricing