You've Been Optimising the Cheapest Part of Your Bill

Everybody edits the prompt.

It's the part you can see, the part you wrote, the part that feels like the lever. So when the bill arrives, that's where the attention goes: trim the system prompt, tighten the instructions, drop the examples.

Meanwhile a single tool call just handed your model 10,000 tokens of JSON, and it did that on every loop iteration, all day.

What changed

headroomlabs-ai/headroom (Apache-2.0, created 7 January 2026, still actively pushed as of 6 August) compresses what goes into the context window from the sources you didn't write: tool outputs, logs, RAG chunks, files, and conversation history.

It ships in four shapes, which is the practical part:

• a library, if you want it in-process
• an HTTP proxy, if you want it out of your code entirely
• an MCP server
• a one-command agent wrap

Note what is absent from that list: any requirement to change a prompt.

And it is reversible. Originals are cached and retrievable, so the model sees the compressed form and the full text is still there if something needs it.

Why it matters

The context window is a cost centre, and most teams are managing the wrong end of it.

Think about where your tokens actually come from in an agent loop:

• The system prompt — written once, maybe a few thousand tokens, read every turn.
• The user's message — small.
• Tool responses — unbounded, machine-generated, verbose by default, and read every turn thereafter as they accumulate in history.

An API returning a full JSON object doesn't know your agent only needed one field. A log tail doesn't know you only wanted the errors. They return everything, because returning everything is correct behaviour for a tool and expensive behaviour for a context window.

So the cheapest place to act is between the tool and the model — after the response exists, before it becomes part of an ever-growing conversation. That is a layer almost nobody has, because the prompt is the part that feels editable and the tool output feels like weather.

The demo, and what it's actually showing

The repo's demonstration: 10,144 tokens reduced to 1,260, with the same

FATAL

log line still findable afterwards.

The second half of that sentence is the whole argument. Compression that loses the FATAL line is not compression, it's deletion — and you find out which one you built at the worst possible moment.

Proof and limits

Be clear about what is measured and by whom.

The reported ratios are 60–95% fewer tokens on JSON and 15–20% on coding agents. Both are the project's own numbers, from its own README, with no independent replication I can find.

There is also a small internal inconsistency worth naming: the repo description says 20% for coding agents while the README says 15–20%. That is not damning — it's the kind of drift any fast-moving repo accumulates — but it does mean the figures should be read as the project's best estimate rather than a measurement you can lift.

The licence and dates are checkable on GitHub and are what they claim to be.

The honest read: the mechanism is obviously sound, and the magnitude is unverified. Which is a reason to measure it yourself, not a reason to ignore it.

What to actually do

You don't need to adopt anything to get the first result.

1. Log the size of every tool response for one day. Not an estimate. The actual token count, per call.
2. Sort descending. The top of that list is your line item, and there is a strong chance it is one endpoint you forgot returns the full object.
3. Ask what the model actually used from it. Usually one field, one status, one error line.
4. Then decide the layer. Sometimes the fix is upstream — request fewer fields from the API. Sometimes the tool is not yours to change, and that is exactly where a compression layer earns its place.
5. If you add one, verify reversibility before you trust it. Compress a response containing something critical, then go retrieve the original. Do it once, deliberately, before production does it for you.

Four questions, honestly answered

What is confirmed. The project exists under Apache-2.0, was created 7 January 2026, is actively maintained, and ships as a library, proxy, MCP server and agent wrap with a documented retrieval path for originals.

What is still unknown. Whether the compression ratios hold on anyone else's workload, what latency the extra hop adds, and how the compressor behaves on inputs its heuristics weren't tuned for.

Why it is worth watching. It names a layer of the stack most teams don't have and will need. Agent loops get longer, tool ecosystems get bigger, and the pressure on this layer only goes one direction.

How it fits the larger ecosystem. It arrives the same week DeepSeek said API prices are going up. If token cost stops falling on its own, controlling how many tokens you spend stops being an optimisation and becomes the plan.

The part that stays true

There is one honest objection to any layer between your tools and your model: it is one more thing that can fail. That is true, and it is the trade you're making.

What makes the trade defensible here is reversibility. Compression you can't undo is data loss with a nicer name.

Do you know what your biggest tool response cost you today? Go find the largest single thing your agent read. It is almost never the prompt.

---

Sources

• headroomlabs-ai/headroom