Agent Orchestration Frameworks Compared

Klarna spent that period telling the world its AI assistant had replaced a large share of its full-time customer service agents. Not long after, it was rehiring humans. That reversal is the cleanest public case study of a truth most AI teams learn the hard way: systems don't fail because the model is dumb. They fail at the orchestration layer, the wiring that decides what happens after the model responds, what happens when a step breaks, and who (or what) picks up the pieces.

On GAIA, that gap appears in benchmark data as well. According to the Princeton HAL benchmark, Claude Opus 4 scores around 57.6% inside one orchestration scaffold and roughly 64.9% inside another, a 7-point spread driven at least in part by differences in compute tier and scaffold. A week spent fixing how an agent routes, retries, and remembers can move accuracy further than the jump between most frontier model releases. MIT NANDA looked at over 300 enterprise AI deployments and found that only about 5% made it from pilot to production. The model wasn't the problem in most of those cases. The layer sitting above it was.

That's what this piece is actually about: how the major agent orchestration frameworks handle the things that matter once a demo turns into a product, namely state, coordination, failure recovery, and tool reliability.

The four properties that separate a demo-ready framework from a production-ready one

Every framework picks an orchestration model, choosing among a graph, a set of roles, a conversational loop, or a hierarchical tree. That choice isn't cosmetic. It shapes how hard debugging gets, how tightly you can control what runs next, and how gracefully the system recovers when one agent in the chain trips.

State and memory come right after. Long workflows need state that survives past a single crashed session. If a five-step process dies on step four, it should pick back up at step four, not restart from scratch and rebill for work already done.

That "rebill" part isn't abstract. At current LLM pricing, a 12-step workflow that dies on step 11 and restarts from zero burns 11 steps' worth of tokens twice. Multiplying that across enterprise call volume makes it appear on a finance dashboard instead of staying a rounding error.

Then there's error handling. Production orchestration has to answer, ahead of time, what happens when a sub-agent fails, hangs, or hands back garbage. Retries, fallback paths, a human stepping in, all of that needs to be designed into the system from day one. Bolting it on after the first outage is how outages become a pattern.

LangGraph and LangChain: the graph-based reference point for stateful, complex workflows

LangGraph treats a multi-agent workflow as a directed graph. Nodes are agents or functions, edges decide what fires next, and conditional edges look at the current state before routing. It sounds like a small design choice. It sounds like a small design choice, but it isn't a small design choice.

Graph structure buys three things that matter in production. First, real loops, so control can go back to an LLM node after a tool call instead of dead-ending. Second, deterministic routing, which matters a lot in regulated workflows where "the model decided" isn't an acceptable audit trail. Third, debugging that's actually pinpointable, because each node is a checkpoint you can inspect on its own.

Checkpointing is where LangGraph separates itself from frameworks that treat state as an afterthought. It saves a snapshot of the graph's state at every step, organized by thread. That gets you fault-tolerant resume, conversation memory that survives a restart, time-travel debugging, and human-in-the-loop pauses, all without extra plumbing.

LangSmith pairs with it as the observability layer, offering debugging tools that give a clearer picture of what an agent actually did and why, a capability LangChain highlights as central to the platform.

CrewAI: where role-based orchestration helps and where it breaks

CrewAI organizes work around roles. Each agent gets a role, a goal, a backstory, and a set of tools, and crews execute either sequentially, where each task's output feeds the next, or hierarchically, where a manager agent delegates to specialists underneath it.

The appeal is obvious the moment a team tries to describe a workflow out loud. Thinking in terms of "a researcher agent hands off to a writer agent" is a lot more natural than thinking in state machines and edges. That's a real advantage for prototyping speed, not a marketing line.

Content pipelines fit this model well: research, then write, then edit, then publish. Lead qualification workflows fit it too. Anywhere the work already maps to distinct job titles, CrewAI's structure feels less like a framework and more like an org chart.

The tension is visible at scale. The same role-and-backstory scaffolding that aids prototyping has been noted as a source of elevated token overhead on simple, repetitive tasks. The same role-and-backstory scaffolding that makes prototyping intuitive adds overhead on every single call, and that overhead compounds fast once volume climbs.

Microsoft Agent Framework 1.0: what the consolidation of AutoGen and Semantic Kernel means in practice

Microsoft Agent Framework hit 1.0 general availability on April 3, 2026, folding Semantic Kernel and AutoGen into one production SDK and runtime. Python and.NET both shipped their 1.0 runtimes at the same time, which matters for teams that had been stuck choosing between two Microsoft agent stacks with overlapping but incompatible futures.

The practical changes: declarative YAML for agent configuration, which makes deployments version-controllable the way infrastructure-as-code already is, plus migration assistants for teams coming from either Semantic Kernel or AutoGen.

The supervisor pattern, one agent triaging and routing to specialists, dominates customer-facing agent design for a simple reason: it maps cleanly onto how support already works. Triage, resolution, handoff. Microsoft Agent Framework defaults to it. So does OpenAI's Agents SDK. So does Rasa's orchestration layer. When three separate vendors converge on the same default, that's not coincidence, it's a signal about what actually works in front of customers.

On durable execution, Microsoft Agent Framework is catching up to LangGraph through Azure AI Foundry. That's a gap that's narrowing, not one that's closed.

OpenAI Agents SDK, Google ADK, and LlamaIndex: three narrower but coherent options

OpenAI Agents SDK keeps its abstraction minimal on purpose. It's built for tightly scoped assistants and clean handoffs between agents, defaulting to that same supervisor pattern. Tracing ships built in, not bolted on later, so observability is part of the starting package rather than a separate integration project. The tradeoff is coupling: it's tied closely to OpenAI's model ecosystem, so teams wanting to mix providers or swap models freely will hit friction fast.

Google ADK (Agent Development Kit) is opinionated and batteries-included, with built-in debugging UIs and an Apache 2.0 license. For teams already living in GCP, it's an end-to-end runtime that doesn't require assembling a stack from separate parts. That same completeness is the tradeoff for teams outside that ecosystem: the parts that accelerate GCP-native teams add friction and lock-in for everyone else.

LlamaIndex Workflows takes an event-driven approach built for document-heavy pipelines: ingestion, chunking, retrieval, and synthesis all treated as first-class steps rather than glued-on preprocessing. That's a strong fit for teams building document-centric multi-agent systems. It's a weaker fit for conversational or task-routing agents, where the event-driven model adds conceptual overhead without a clear payoff.

Across all three, the real production question isn't which orchestration style looks cleanest in a demo. It's whether the durable execution and observability story holds up against the four properties covered earlier: orchestration model, state, error handling, and tool integration.

Mastra for TypeScript teams and Strands Agents (AWS) for cloud-native Python workloads

Mastra is the clear pick for TypeScript teams that want more out of the box. It ships workflows, memory, and a Studio development environment together, which cuts down the assembly work that a lot of other frameworks leave to the team. The core framework is open source, though enterprise features live in separate ee/ directories under a proprietary Mastra Enterprise License. Check that distinction against internal governance rules before committing, not after.

Strands Agents, from AWS, is an open-source framework with deep AWS integration, aimed at teams already committed to that cloud who want agents that plug into AWS tooling without friction. It sits in the same category as Microsoft Agent Framework on Azure and Google ADK on GCP: cloud-native frameworks that trade some portability for a much smoother fit with the infrastructure a team has already built.

The pattern across all of them is the same. If the infrastructure is already locked into one cloud, that provider's framework cuts integration friction more than a "best" framework picked off a leaderboard ever will. Runtime fit is its own selection axis, separate from raw capability. A framework built for Python teams doing document-heavy retrieval work might be the wrong choice for a.NET shop or a TypeScript team shipping production agents, no matter how it scores on a benchmark. Evaluating against the actual stack in place matters more than evaluating against a chart.

The production failure mode every framework gets wrong: state persistence and autonomous resumption

One test cuts through the marketing faster than any benchmark: build the same approval workflow in multiple frameworks, kill the process while it's waiting on a human, and record what comes back.

Every framework held onto its state. Autonomous resumption, something noticing the process died and restarting it without manual intervention, is a capability the frameworks largely leave to the implementer.

That distinction, between persistence and resumption, gets flattened constantly, but they're two different capabilities. Persistence means the data survives. Resumption means something notices the process died and restarts it without a person manually intervening. Frameworks have gotten good at the first one and largely left the second to whoever's implementing them.

At enterprise volume, that gap is a line item, not a footnote. A workflow that crashes mid-run and has to restart from zero pays for its completed steps twice. At enterprise conversation volumes, that's real money leaking out of a design gap nobody flagged in the demo.

So teams build external resumption logic themselves, layered on top of framework primitives, using tools like Temporal, Redis, or custom checkpointing systems. The external resumption logic layered on top of framework primitives, using tools like Temporal, Redis, or custom checkpointing systems, works. It's also undifferentiated infrastructure work that every team ends up rebuilding independently, because the frameworks haven't absorbed it yet.

Orchestration patterns, live web data, and tool-use reliability

Agent-grade access to the live web breaks down into four separate layers: stable fetching, structured observation, persistent sessions, and tool integration. Most tools marketed as "AI scraping" bundle all four together, which is convenient until something breaks and there's no way to tell which layer failed.

The model side of extraction is largely solved at this point. Getting a clean, usable page in front of the model is not solved. Dynamic content, JavaScript rendering, anti-bot defenses, and session management all sit below the model layer, and none of them care how good the LLM is. If the page doesn't load right, the model never gets a fair shot.

Output format is an orchestration decision. Raw HTML, packed with navigation bars, layout markup, and script tags, forces every downstream step to do cleanup work it shouldn't have to do. Structured Markdown or JSON skips that step. Token-efficient cleaning can cut input token usage by 30 to 50% compared to feeding an agent raw HTML, a meaningful number once that ingestion happens thousands of times a day.

Graph-based orchestration gains the most from clean, structured tool output. When a LangGraph node has a defined input schema, predictable output from a tool maps straight onto that schema without extra logic. Unpredictable output does the opposite: every edge case turns into a new conditional edge, and those edges pile up into a maintenance problem nobody budgeted for. Clean input isn't a nicety at that point, it's what keeps the graph from growing branches it was never designed to carry.