The agent thought it was dev. So did the person who approved it.

An invented incident where an agent resets a production database, taken apart link by link, and a command gate that only runs destructive verbs on databases marked disposable.

Agent shell Verb classifier Target resolver Disposable database Refusal Reviewed pipeline Production Backups reads the marker the reset runs no marker found two approvers separate identity
A destructive command runs only where the database carries a disposable marker; otherwise it is refused and sent to review.
On this page
  1. Everyone in the chain believed the same wrong thing
  2. Ask the database what it is
  3. A gate between the agent and the shell
  4. What the database refuses on its own
  5. Five links, five controls
  6. That afternoon, with the layers in place

The confirmation prompt named DATABASE_URL, and the developer typed y.

Run `migrate reset`? This drops and recreates every table in DATABASE_URL. [y/N]

Of course they did. They’d asked the agent to fix a failing migration test, resetting the schema is the normal first move, and on that laptop DATABASE_URL had meant the local database for a year.

It meant production that afternoon.

Larchmere, the appointment-scheduling startup in this story, is invented, along with its numbers, so I’d have a clean failure chain to take apart. Two days earlier the developer had exported a production connection string to debug a slow query, in a terminal that was still open. The agent’s shell inherited it. The reset took 41 tables and 3.2 million rows in under a minute. The newest dump was 19 hours old, the restore ran for 6 hours, and 19 hours of bookings were rebuilt by hand from confirmation emails.

Everyone in the chain believed the same wrong thing

Public write-ups of incidents like this, appearing since mid-2025, mostly blame the same thing: the model ignored an instruction. The repo’s CLAUDE.md or AGENTS.md said never to touch production, and the agent touched it.

I don’t buy that reading. At Larchmere the agent followed the instruction as far as it could. It was told not to run destructive commands against production, and as far as it knew, it didn’t. An instruction file is advice to a reader who has to work out for itself which environment it’s in. It can’t check anything.

The confirmation prompt fails for the same reason. A prompt asks a person to catch what the agent missed, and here the person shared the agent’s picture of the world. Two parties with one assumption make one check, and it had already failed.

So I’d stop asking the agent, or the developer, what the target is. Neither of them knows. The target does.

Ask the database what it is

Every database that may be wiped carries one row saying so. Production carries nothing. Before a destructive command runs, the runner connects with the same connection string the command will use and reads the row. No row, no reset.

The direction of the default is the whole design. A “this is production” flag fails open: forget to set it, or restore a copy without it, and the database looks safe to destroy. A disposable marker fails closed. A staging database restored from a production dump has no marker until someone decides it should.

-- Run on every database that may be wiped. Production never gets this schema.
create schema if not exists ops;

create table ops.environment_marker (
  only_row   boolean primary key default true check (only_row),
  purpose    text not null check (purpose = 'disposable'),
  owner      text not null,
  created_at timestamptz not null default now()
);

insert into ops.environment_marker (purpose, owner)
values ('disposable', 'dev-platform');

Keep the ops schema out of anything promoted upward, for example with pg_dump --exclude-schema=ops. A marker that travels into production is the one way this turns against you.

A gate between the agent and the shell

The agent’s shell runs every command through this. A verb classifier decides whether the command is destructive and lets harmless ones through. A target resolver then asks the database behind the connection string what it is.

#!/usr/bin/env python3
"""Usage: gate.py <command> [args...]
Runs the command, unless it is destructive and its target can't prove it is disposable."""
import os
import re
import subprocess
import sys

import psycopg

ALLOWED_HOSTS = {"localhost", "127.0.0.1", "db.dev.internal"}

ASK_TARGET = [  # destructive, and there is a database to ask
    r"\bdrop\s+(database|schema|table)\b",
    r"\btruncate\b",
    r"\bdelete\s+from\s+[\w.\"]+\s*(;|$)",  # delete with no where clause
    r"\bdropdb\b",
    r"\bdb:(drop|reset)\b",
    r"\bmigrate(:fresh|:reset|\s+reset)\b",
]
NOTHING_TO_ASK = [  # destructive, and no marker to read: never from this shell
    r"\bvolume\s+(rm|remove|prune)\b",
    r"\bdelete-volume\b",
    r"\bdisks\s+delete\b",
]


def first_match(patterns, text):
    return next((p for p in patterns if re.search(p, text, re.I | re.M)), None)


def refuse(reason):
    print(f"gate: refused. {reason}", file=sys.stderr)
    sys.exit(3)


def require_disposable(dsn):
    try:
        with psycopg.connect(dsn, connect_timeout=5) as conn:
            host = conn.info.host
            row = conn.execute("select purpose from ops.environment_marker").fetchone()
    except psycopg.Error as exc:
        refuse(f"could not read a marker from the target ({type(exc).__name__})")
    if row is None or row[0] != "disposable":
        refuse(f"{host} does not say it is disposable")
    if host not in ALLOWED_HOSTS:
        refuse(f"{host} has a marker but is not on the allow-list")


def main():
    argv = sys.argv[1:]
    if not argv:
        refuse("no command given")
    text = " ".join(argv)
    if first_match(NOTHING_TO_ASK, text):
        refuse("this target can't carry a marker; send it through the pipeline")
    if first_match(ASK_TARGET, text):
        dsn = os.environ.get("DATABASE_URL")
        if not dsn:
            refuse("destructive command and no DATABASE_URL to check")
        require_disposable(dsn)
    sys.exit(subprocess.run(argv).returncode)


if __name__ == "__main__":
    main()

The refusal goes to stderr with a reason, and that’s deliberate. An agent that reads “could not read a marker from the target” stops and reports it. An agent that gets a bare failure tries another way in.

The script has holes. It reads DATABASE_URL, so a command carrying its own URL as an argument needs that parsed too. It matches text, so psql -f cleanup.sql walks past it. And it only sees the shell. An MCP server that wraps a cloud or database API is a second road to the same infrastructure, and nothing typed in a terminal passes along it. The same marker check has to live inside that server, or its credential has to be too weak to matter.

That makes the gate the outermost layer and the cheapest. The layers behind it don’t depend on pattern matching.

What the database refuses on its own

On shared databases that aren’t disposable, the agent gets a role that can’t do the damage whatever command arrives.

create role agent_rw login noinherit;

grant connect on database app to agent_rw;
grant usage on schema public to agent_rw;
grant select, insert, update, delete
  on all tables in schema public to agent_rw;

alter default privileges for role app_owner in schema public
  grant select, insert, update, delete on tables to agent_rw;

In Postgres a table can be dropped only by its owner, the owner of its schema, or a superuser. This role is none of those, and truncate is a separate privilege this role never receives. Schema changes go through the migration pipeline, where a second person approves the diff. On Postgres 14 and older, also run revoke create on schema public from public, or the role can still add tables of its own.

Production gets no agent role at all. It sits in its own cloud account and accepts connections only from the pipeline’s network, so a stray connection string on a laptop is a string that times out.

The row-level delete is still there, and so is every mistake I haven’t thought of. For those there’s point-in-time recovery, with the backups and the right to restore held by an identity the agent can’t assume, and the developer can’t either. If the credential that can delete the data can also delete the backups, you have one copy.

Link in the chain Control that breaks it
A production connection string sits in a developer’s shell for two days Secrets brokered per session and expiring, as in the blast radius post
The laptop can open a connection to production Production in a separate account, reachable only from the pipeline’s network
Agent and developer both assume the target is dev The gate asks the target for a disposable marker and refuses without one
The credential in use is allowed to drop tables An agent role with no ownership and no truncate; schema changes need a second approver
The newest backup is 19 hours old Point-in-time recovery, owned by a separate identity

Any one of these would have changed the afternoon. I’d still build all five, because each one covers a case where another is misconfigured.

That afternoon, with the layers in place

The agent asks for the reset. The gate matches migrate reset, connects with the inherited string, and production has no ops schema. It refuses before anything runs and prints why. The agent tells the developer the shell is pointed at something that isn’t disposable, which is the first time anyone in the session learns it.

Suppose the gate is missing. The connection never opens, because the laptop isn’t on production’s allow-list. Suppose that rule is wrong too, and the reset runs. Recovery replays the write-ahead log to the second before the drop, and Larchmere is down for the length of a restore with almost nothing to rebuild by hand.

The developer still types y. It just doesn’t decide anything anymore.