Building an approval gate a language model can't bypass

Every demo of an autonomous business agent ends the same way: the agent does something irreversible, confidently, and slightly wrong. It refunds the wrong order, edits a live listing, emails a customer in a tone you'd never use. Impressive right up until it costs money.

The fix isn't a smarter model. It's putting the boundary in the right place — and making it a property of the system the model cannot route around, not a sentence in a prompt asking it to behave.

This is how we built that in oproom.

The boundary: read and draft are free, writes are gated

Reading is safe. The agent can pull every order, ticket, payout file, and stock level, and reason from it — the worst case is that it reads something wrong, which you catch in the draft. Drafting is safe too: a drafted purchase order is a proposal, a drafted reply is text. Nothing has happened to the outside world.

The danger is concentrated in one place: the external write. The PO submitted to a vendor. The refund issued. The inventory adjusted in Zoho. The dispute filed on a marketplace. So that's the only place the gate lives — not "ask before anything," which makes the agent useless, but read and draft freely, ask before you write.

The mechanism

We build on the AI SDK's tool model rather than hand-rolling our own. Every mutating tool is marked needsApproval. When the model calls one, the SDK pauses that tool call in an approval-requested state instead of executing it. The call is suspended, not fired.

The chat surfaces that paused call as an inline approval card showing the five things you need to decide in thirty seconds: what the call is, why, the data freshness behind it, the risk, and the rescue path if it goes wrong. Only your tap releases it.

On approve, execution doesn't go straight from the model to the connector. It routes through a single chokepoint — dispatchProposedDelta — which runs the write through the same authenticated connector path the read tools use, then writes the outcome to the ledger. One door, server-side, that every write must pass through.

The invariant we hold: there is no model-reachable path to an external write without a human click. Not "not normally." Structurally none.

Unattended runs can't even queue a write

There's a subtle failure mode in approval gates: a scheduled, nobody's-watching run where the agent queues a write that then sits forever waiting for an approval that isn't coming. We close it at the agent layer.

Under scheduled execution, every needsApproval tool is stripped from the model's toolset via the SDK's activeTools. The model literally cannot select a write tool on an unattended turn — so it can't stall on one. Unattended turns are read-only by construction, not by convention.

The ledger makes it auditable

Every step is appended to an operating ledger that is append-only — no updates, no deletes, enforced at the database layer. The approval request, the grant or deny, and the underlying connector call with pre- and post-state snapshots all land there. A compensating action is a new entry, never a quiet edit. So "did the agent change something, and did I approve it?" is always answerable from the record.

The gate is rung one, not the ceiling

Here's the objection I get next: "So I'm tapping approve forever? You've built a dashboard with an extra step."

No. A gate that never moves is a tax on your attention, and attention is the thing we're supposed to be giving back. The point of the gate isn't to make you click — it's to make every click evidence. So the gate is the first rung of a ladder, not the top of one.

Each action-type accumulates a track record: how many times you approved it cleanly, how often you denied, how often you had to reverse it after the fact. When a low-risk, reversible action — say, restocking a known SKU from a known vendor inside a price you've already blessed — clears an objective bar (a long clean streak, near-zero denials and reversals, and a replay of your last real decisions that matches what you actually chose ≥99% of the time), the agent doesn't just start doing it. It proposes the promotion as one more approval card, with the track record and the replay attached. You ratify demonstrated trust instead of guessing at it.

After that, that one narrow action fires without a tap — inside its envelope, with the founder reachable, every execution logged as I did this with one-tap undo. Approval load falls because the agent earned it on the record, not because we loosened the rule.

Trust you can lose is the only kind worth granting

Promotion is slow and evidence-gated. Demotion is fast and automatic. If a promoted action starts going wrong — reversals tick up, you deny it after the notify, related Cases reopen — the grant is revoked back to the gate on its own, and you see why. That asymmetry is the whole point: granting autonomy is only safe if it's losable without a meeting.

Why this earns access, not blocks it

Founders don't withhold their Shopify keys because they doubt the model. They withhold them because they can't see the boundary. Show a boundary you can verify — reads free, writes on a tap, proven writes earned off the tap, every decision logged immutably and reversible — and handing over access becomes a reasonable thing to do.

The approval gate isn't the limitation that makes the agent less capable. It's the floor that makes a capable agent safe to plug into a live business — and the rung your agent climbs off, one proven action at a time.

See the approval model →