Chain, route, fan out, check: pick the pattern by where the uncertainty is

Most agent projects are a fixed sequence with one uncertain step. Where that uncertainty sits tells you whether to route, fan out, check, or let a loop pick its own steps.

Request Router Fixed chain Orchestrator Workers Evaluator Response by input type steps known up front splits the job in parallel retry or pass
A request splits at the router into a fixed chain or an orchestrator with workers, and both lines meet at the evaluator.
On this page
  1. Ask where the uncertainty sits
  2. One page to pick from
  3. A hundred documents at Selwood Packaging
  4. Fifty lines with typed hand-offs
  5. What the two bundles are owed

On the whiteboard for a contract-intake tool, the first box usually says “Agent.” Documents arrive on the left, a loop with tools sits in the middle, and structured terms come out on the right.

I’d erase that box and ask a duller question first: which step of this job can’t be known in advance. Most of the time the answer is one step. Everything around it is a sequence you could write down today, and a sequence you can write down should be ordinary code.

“Agentic” went mainstream last year, and it now covers everything from two prompts in a row to a model running a ReAct loop until it decides it’s done. Those are different machines with different bills. The cheaper ones have been buildable since function calling shipped in 2023, and they cover more ground than they get credit for.

Ask where the uncertainty sits

A plain chain is the baseline. Step one’s output feeds step two, and the order never changes. Start there and add structure only where something is unknown at design time.

If you don’t know what kind of input you’re getting, add a router. One classification call picks a branch, and each branch is its own chain with its own prompt.

If you don’t know how much work there is, fan out. An orchestrator call splits the job into pieces, workers handle the pieces in parallel, and a last step merges them. The orchestrator decides the count at run time. That’s the only freedom it gets.

If you don’t know whether the output is good enough, add a check. An evaluator reads the result against the source and either passes it or sends it back with specific complaints, for a bounded number of tries.

That leaves one case. When you can’t know how many steps the job takes, or which ones, nothing above fits, and a loop where the model chooses its next action is the right tool. I treat it as the last option because it’s the only pattern whose path you can’t read off the diagram before it runs, and because per-step errors multiply over a path that long.

One page to pick from

Question to ask Pattern Cost in calls and latency Failure it introduces
Are the steps and their order known? Fixed chain One call per model step, latencies add up An early mistake carried through every later step
Does the right prompt depend on the kind of input? Router One extra small call before the branch A misroute that the wrong branch answers with full confidence
Does the amount of work vary per request? Orchestrator and workers Two calls plus one per piece, as slow as the slowest worker Pieces with gaps or overlaps that don’t merge cleanly
Can the output be wrong in a way a second read would catch? Evaluator with retry One extra call per attempt, a whole second pass on a retry A checker that approves everything, or retries that never settle
Is the number of steps unknowable until the work starts? Open loop Whatever the step budget allows, and slow Errors that compound, and runs that wander

A hundred documents at Selwood Packaging

Selwood Packaging is fictional, and I picked its numbers. Its legal operations inbox receives NDAs, order forms and amendments, and someone retypes the parties, dates and terms into the contract system. Take 100 documents from that inbox as a sample.

A router that labels the document type, followed by a three-step chain, gets 94 of them right. The chain picks the relevant clauses, extracts the terms, and normalizes dates and party names. Only the middle step calls a model.

Add an evaluator that compares each extracted field with the source text and 4 more come through. Picture an amendment where the first pass took the original contract’s effective date. The evaluator names the field, the second attempt carries that complaint in its prompt, and it passes.

The last 2 are bundles: one PDF holding a master agreement, two order forms and an amendment that changes one of them. How many documents are inside, and which overrides which, can’t be known until something reads the file. That’s the open-ended case, and it’s 2 percent of the inbox.

Count the calls. Classification runs on all 100, extraction and checking on 98, a second pass on 4. That’s 100 + 196 + 8 = 304 model calls, about three per document, and I can point to each one on the diagram. Starting with the loop would have sent all 100 documents down the least predictable path to serve 2.

Fifty lines with typed hand-offs

Every hand-off below is a typed object. Since Structured Outputs arrived last August, a call can be held to a JSON Schema, and a forced tool call gets you much the same on other providers. Each step returns an object whose form the next step can trust.

from typing import Literal, TypeVar
from pydantic import BaseModel

T = TypeVar("T", bound=BaseModel)

class Route(BaseModel):
    doc_type: Literal["nda", "order_form", "amendment", "bundle"]

class Terms(BaseModel):
    parties: list[str]
    effective_date: str
    term_months: int | None
    governing_law: str

class Verdict(BaseModel):
    passed: bool
    problems: list[str]

def ask(prompt: str, text: str, out: type[T]) -> T:
    """One model call that must return `out`. Wire to your provider."""
    raise NotImplementedError

def pick_clauses(doc_type: str, text: str) -> str:
    return text  # plain code: keep only the sections this type needs

def normalize(terms: Terms) -> Terms:
    return terms  # plain code: ISO dates, canonical party names

def chain(doc_type: str, text: str, feedback: str = "") -> Terms:
    clauses = pick_clauses(doc_type, text)
    terms = ask(f"Extract the {doc_type} terms. {feedback}", clauses, Terms)
    return normalize(terms)

def evaluate(text: str, terms: Terms) -> Verdict:
    prompt = "Check every field against the source. List what is wrong."
    return ask(prompt, text + "\n\n" + terms.model_dump_json(), Verdict)

def intake(text: str, max_tries: int = 2) -> Terms | None:
    route = ask("Classify this document.", text, Route)
    if route.doc_type == "bundle":
        return None  # not this path: open loop or a person
    feedback = ""
    for _ in range(max_tries):
        terms = chain(route.doc_type, text, feedback)
        verdict = evaluate(text, terms)
        if verdict.passed:
            return terms
        feedback = "Fix these: " + "; ".join(verdict.problems)
    return None  # out of tries: human queue

ask is the only function that touches a provider, so it’s the only place for network retries and cost counting. There’s no framework here. If you want the graph drawn and checkpointed, LangGraph is one way to do it: these functions become nodes and the if becomes a conditional edge. The control flow doesn’t change.

What the two bundles are owed

For the bundles, a loop with tools is reasonable. Split the PDF, read a section, look up the master agreement, decide what to read next. The step count depends on what’s inside, and no chain can be drawn for that ahead of time.

I’d still fence it. It gets a step budget, it returns the same Terms type, and it goes through the same evaluator on the way out. The evaluator doesn’t care what produced the object it’s reading.

Before building even that, I’d want a real count. If bundles are 2 in 100, a person can handle them for a quarter while the other 98 run through the chain. If they turn out to be 20 in 100, the sample was wrong, and I’d rather learn that before writing a loop.