Why we feed our agent TOON, not JSON

An operating agent reads a lot of tables. Orders, line items, payout rows, inventory levels, ticket queues — almost every tool result is a list of objects with the same shape. And the default way to hand that to a model, JSON, is quietly the worst format for it.

JSON pays the key tax on every row

Here's 200 order lines in JSON. Every single row repeats "order_id", "sku", "qty", "status" — the keys cost as many tokens as the values, multiplied by every row:

[
  { "order_id": "1042", "sku": "K6", "qty": 2, "status": "unfulfilled" },
  { "order_id": "1043", "sku": "V60-02", "qty": 1, "status": "unfulfilled" }
]

On a model priced and context-limited per token, that repetition is pure overhead. It costs money on every call, and it eats the context budget you'd rather spend on reasoning.

TOON declares the shape once

TOON (Token-Oriented Object Notation) is built for exactly this: uniform arrays of objects. It declares the field names once, then streams the rows:

[2]{order_id,sku,qty,status}:
  1042,K6,2,unfulfilled
  1043,V60-02,1,unfulfilled

Same information, a fraction of the tokens — and the savings grow with the number of rows, which is exactly where our tool results are heaviest. Every chat-tool return in oproom routes through a single toToon helper wrapping the TOON encoder, so tabular payloads reach the model in this dense form without each tool having to think about it.

But the durable record stays JSON

Here's the line we drew, and it's the interesting part: TOON is for the model; the operating ledger stays JSON.

The ledger is the append-only record a human inspects and searches — the answer to "what did the agent do and why." Optimizing that for tokens would be optimizing the wrong thing. There, readability and queryability win, so those rows are written as plain JSON. We keep a toJsonSafe path for exactly the surfaces humans touch: input previews and the durable ledger.

So the split is by audience, not by laziness. Ephemeral, model-facing payloads optimize for tokens. Durable, human-facing records optimize for the human. The same data crosses that line in two different costumes, and each side gets the format that's actually right for it.

It's a small piece of plumbing. But on an agent whose whole job is reading systems all day, "how many tokens does a table cost" is a number that shows up on the bill — and in how much room the model has left to think.

See how the morning loop reads your stack →