On the whiteboard for a contract-intake tool, the first box usually says “Agent.” Documents arrive on the left, a loop with tools sits in the middle, and structured terms come out on the right.
I’d erase that box and ask a duller question first: which step of this job can’t be known in advance. Most of the time the answer is one step. Everything around it is a sequence you could write down today, and a sequence you can write down should be ordinary code.
“Agentic” went mainstream last year, and it now covers everything from two prompts in a row to a model running a ReAct loop until it decides it’s done. Those are different machines with different bills. The cheaper ones have been buildable since function calling shipped in 2023, and they cover more ground than they get credit for.
Ask where the uncertainty sits
A plain chain is the baseline. Step one’s output feeds step two, and the order never changes. Start there and add structure only where something is unknown at design time.
If you don’t know what kind of input you’re getting, add a router. One classification call picks a branch, and each branch is its own chain with its own prompt.
If you don’t know how much work there is, fan out. An orchestrator call splits the job into pieces, workers handle the pieces in parallel, and a last step merges them. The orchestrator decides the count at run time. That’s the only freedom it gets.
If you don’t know whether the output is good enough, add a check. An evaluator reads the result against the source and either passes it or sends it back with specific complaints, for a bounded number of tries.
That leaves one case. When you can’t know how many steps the job takes, or which ones, nothing above fits, and a loop where the model chooses its next action is the right tool. I treat it as the last option because it’s the only pattern whose path you can’t read off the diagram before it runs, and because per-step errors multiply over a path that long.
One page to pick from
| Question to ask | Pattern | Cost in calls and latency | Failure it introduces |
|---|---|---|---|
| Are the steps and their order known? | Fixed chain | One call per model step, latencies add up | An early mistake carried through every later step |
| Does the right prompt depend on the kind of input? | Router | One extra small call before the branch | A misroute that the wrong branch answers with full confidence |
| Does the amount of work vary per request? | Orchestrator and workers | Two calls plus one per piece, as slow as the slowest worker | Pieces with gaps or overlaps that don’t merge cleanly |
| Can the output be wrong in a way a second read would catch? | Evaluator with retry | One extra call per attempt, a whole second pass on a retry | A checker that approves everything, or retries that never settle |
| Is the number of steps unknowable until the work starts? | Open loop | Whatever the step budget allows, and slow | Errors that compound, and runs that wander |
A hundred documents at Selwood Packaging
Selwood Packaging is fictional, and I picked its numbers. Its legal operations inbox receives NDAs, order forms and amendments, and someone retypes the parties, dates and terms into the contract system. Take 100 documents from that inbox as a sample.
A router that labels the document type, followed by a three-step chain, gets 94 of them right. The chain picks the relevant clauses, extracts the terms, and normalizes dates and party names. Only the middle step calls a model.
Add an evaluator that compares each extracted field with the source text and 4 more come through. Picture an amendment where the first pass took the original contract’s effective date. The evaluator names the field, the second attempt carries that complaint in its prompt, and it passes.
The last 2 are bundles: one PDF holding a master agreement, two order forms and an amendment that changes one of them. How many documents are inside, and which overrides which, can’t be known until something reads the file. That’s the open-ended case, and it’s 2 percent of the inbox.
Count the calls. Classification runs on all 100, extraction and checking on 98, a second pass on 4. That’s 100 + 196 + 8 = 304 model calls, about three per document, and I can point to each one on the diagram. Starting with the loop would have sent all 100 documents down the least predictable path to serve 2.
Fifty lines with typed hand-offs
Every hand-off below is a typed object. Since Structured Outputs arrived last August, a call can be held to a JSON Schema, and a forced tool call gets you much the same on other providers. Each step returns an object whose form the next step can trust.
from typing import Literal, TypeVar
from pydantic import BaseModel
T = TypeVar("T", bound=BaseModel)
class Route(BaseModel):
doc_type: Literal["nda", "order_form", "amendment", "bundle"]
class Terms(BaseModel):
parties: list[str]
effective_date: str
term_months: int | None
governing_law: str
class Verdict(BaseModel):
passed: bool
problems: list[str]
def ask(prompt: str, text: str, out: type[T]) -> T:
"""One model call that must return `out`. Wire to your provider."""
raise NotImplementedError
def pick_clauses(doc_type: str, text: str) -> str:
return text # plain code: keep only the sections this type needs
def normalize(terms: Terms) -> Terms:
return terms # plain code: ISO dates, canonical party names
def chain(doc_type: str, text: str, feedback: str = "") -> Terms:
clauses = pick_clauses(doc_type, text)
terms = ask(f"Extract the {doc_type} terms. {feedback}", clauses, Terms)
return normalize(terms)
def evaluate(text: str, terms: Terms) -> Verdict:
prompt = "Check every field against the source. List what is wrong."
return ask(prompt, text + "\n\n" + terms.model_dump_json(), Verdict)
def intake(text: str, max_tries: int = 2) -> Terms | None:
route = ask("Classify this document.", text, Route)
if route.doc_type == "bundle":
return None # not this path: open loop or a person
feedback = ""
for _ in range(max_tries):
terms = chain(route.doc_type, text, feedback)
verdict = evaluate(text, terms)
if verdict.passed:
return terms
feedback = "Fix these: " + "; ".join(verdict.problems)
return None # out of tries: human queue
ask is the only function that touches a provider, so it’s the only place for network retries and cost counting. There’s no framework here. If you want the graph drawn and checkpointed, LangGraph is one way to do it: these functions become nodes and the if becomes a conditional edge. The control flow doesn’t change.
What the two bundles are owed
For the bundles, a loop with tools is reasonable. Split the PDF, read a section, look up the master agreement, decide what to read next. The step count depends on what’s inside, and no chain can be drawn for that ahead of time.
I’d still fence it. It gets a step budget, it returns the same Terms type, and it goes through the same evaluator on the way out. The evaluator doesn’t care what produced the object it’s reading.
Before building even that, I’d want a real count. If bundles are 2 in 100, a person can handle them for a quarter while the other 98 run through the chain. If they turn out to be 20 in 100, the sample was wrong, and I’d rather learn that before writing a loop.