Building production agents

From demo to product

A demo agent is a while-loop with an API key. A production agent must be reliable (consistent results, 10.9 Evaluating agents), observable (you can see what it did and why), bounded (cost, time, blast radius), safe (10.10 Agent safety and security), resumable (crashes and approvals do not lose work) and maintainable (prompt, tool and model changes are tested). Each requirement maps to a specific engineering practice below.

Control flow as an explicit graph

Write the agent's control flow as a state machine: nodes are steps (an LLM call, a tool execution, a human review), edges are transitions (conditional on the state), and a typed state object flows along the edges. LangGraph is the canonical implementation, but the idea is framework-independent and even the "autonomous agent" fits: a node that calls the model, a node that runs tools, and a conditional edge back or to the end.

# Shape of a graph-based agent (framework-agnostic sketch)
State = dict  # {"messages": [...], "plan": [...], "budget_left": int, "approved": bool}

def call_model(s: State) -> State: ...        # appends an assistant turn (may contain tool calls)
def run_tools(s: State) -> State: ...         # executes tool calls, appends results
def needs_approval(s: State) -> bool: ...     # e.g. an outward-facing tool was requested
def route(s: State) -> str:
    if s["budget_left"] <= 0:          return "end"
    if last_turn_has_tool_calls(s):    return "approve" if needs_approval(s) else "tools"
    return "end"

graph = Graph(state=State)
graph.add_node("model", call_model); graph.add_node("tools", run_tools); graph.add_node("approve", wait_for_human)
graph.add_conditional_edges("model", route, {"tools": "tools", "approve": "approve", "end": END})
graph.add_edge("tools", "model"); graph.add_edge("approve", "tools")
app = graph.compile(checkpointer=DurableStore())    # every node transition is persisted

Benefits: you can unit-test nodes, visualize the flow, replay from any state, insert human interrupts as ordinary nodes, and reason about termination. The 12-factor agents principle "own your control flow" says exactly this: do not let a framework's hidden loop decide when your agent stops.

Durable execution

Agents run for minutes to days and wait on humans and external systems. Treat the run as a durable workflow:

• Checkpoint after every step (the state object plus the message history) in a store keyed by a thread/run id. A crash resumes from the last checkpoint instead of restarting a 40-minute run.
• Pause and resume — a human approval that arrives three hours later should resume the same run. Workflow engines (Temporal-style durable execution) and graph checkpointers both provide this; "launch/pause/resume with simple APIs" and "contact humans with tool calls" are two more of the 12 factors.
• Idempotent tools — because retries happen, a tool called twice with the same arguments must not create two orders. Use idempotency keys.
• Separate the brain from the hands — run the model loop in one process and tool execution in isolated, disposable sandboxes; this is the architecture Anthropic described for scaling managed agents in 2026 and it gives you isolation, resumability and horizontal scaling at once (10.8 Coding agents and computer use).

Harness design for long-running work

Anthropic's Effective harnesses for long-running agents (November 2025) describes a pattern that generalizes beyond coding: an initializer phase establishes the environment and writes structured artifacts — a feature or task list as data, a progress file, conventions; then bounded sessions each start with a clean context (10.4 Memory and context engineering), read the artifacts, do one unit of work, verify it, commit (git or your equivalent), update the artifacts and stop. Verification gates (tests, an evaluator agent, schema checks) sit between units. The 2026 follow-up on harness design separates planning, generation and evaluation roles. The point is that state lives in artifacts, not in the context, so any session — or any human — can pick up the work.

Observability

Log a trace for every run: each prompt, tool call with arguments and result, latency, token counts and cost, model and prompt versions, and the final outcome. Tools: LangSmith, Langfuse, Arize Phoenix, Braintrust, or OpenTelemetry's generative-AI conventions with your existing stack. Traces are not only for debugging: they are the raw material for your eval harness (10.9 Evaluating agents) — sample failures, label them, add them as tasks. Add metrics dashboards: success rate, pass^k on the regression suite, cost per successful task, p50/p95 latency, tool-error rate, approval rate, injection-detection hits.

Reliability and cost engineering

Measure cost per successful task, not cost per call; an agent that costs twice as much per run but succeeds twice as often is cheaper.

The 12-factor agents principles

Dex Horthy's 12-factor agents (2025) distills what production teams learned, in the spirit of the twelve-factor app: natural language to tool calls; own your prompts (they are code, versioned and tested); own your context window (you decide what the model sees); tools are just structured outputs; unify execution state and business state; launch/pause/resume with simple APIs; contact humans with tool calls; own your control flow; compact errors into the context window (so the model can recover, 10.2 Tool use and function calling); small, focused agents (3–10 steps each, composed); trigger from anywhere (chat, email, cron, webhooks); make your agent a stateless reducer (state in, state out — which is what the graph above is).

Release engineering

• Version prompts, tool schemas and model ids together; a model upgrade is a release.
• Regression evals in CI on every change; capability evals before major releases (10.9 Evaluating agents).
• Canary and rollback like any service; shadow-run a new agent against production traffic before switching.
• Human review sampling of production trajectories; feed labeled failures back into the eval set.
• Runbooks for incidents: how to pause all agents, revoke credentials, replay a run.

Where this shows up later

• Your capstone (10.12 Project: build a research agent with tools, an MCP server and an eval harness) is built on this skeleton: a graph, checkpoints, traces, budgets, approvals and a regression suite.
• System-design interviews (11.4 ML and LLM system design and interviews) increasingly ask for exactly this architecture; the frontier view in 11.6 Frontier 2026: where the field is heading covers where long-running and managed agents are heading.