Token Mizer, Treasury Management for Your AI Agents
Part 2 of 2: Token Optimization and the Token Mizer System
Apr 28, 2026
If Part 1 was about understanding what tokens are and why they matter, this is the part where we do something about it. Practical strategies for cutting token spend in agentic systems, an honest look at where each one actually pays off, and an introduction to the system we built to keep all of it from quietly drifting back into the red.
The Meter Is Still Running
Part 1 ended with an uncomfortable observation: most production agent systems can’t tell you, with any precision, where their tokens go. The bill arrives, the bill is paid, and a few people on the team make educated guesses about what changed.
This paper is about closing that gap.
What follows is a working set of optimization strategies for token-driven systems. None of them are exotic. Most of them are unglamorous. They are the agent equivalent of insulating your house and turning off the lights when you leave the room — boring, repetitive, deeply effective, and consistently neglected because nobody is going to write a press release about a 23% reduction in system prompt size.
The reason to do them anyway is that the alternative — letting agent costs grow with usage — is a business model only a hyperscaler should love.
After the strategies, we’ll get to Token Mizer: the system we built to make sure these strategies actually stick in production rather than quietly eroding the moment nobody is looking. But the strategies come first, because Token Mizer doesn’t replace any of them. It enforces them.
Before You Optimize Anything: Decide What “Good Enough” Means
The most common token optimization mistake isn’t technical. It’s that nobody has decided, in advance, what a good answer is allowed to cost.
Without a target, every prompt grows. Every example feels load-bearing. Every “just in case” instruction stays in the system prompt forever because nobody can prove it isn’t doing anything.
Pick a number. For each agent or task type, decide roughly what a reasonable token budget looks like — input and output, separately. It doesn’t need to be precise. It needs to exist. Once you have a budget, overruns become bugs you can investigate, instead of mysterious line items on a monthly invoice.
This sounds bureaucratic. In practice, it’s the difference between an agent system that costs what you expected and one that doesn’t.
The rest of the strategies in this paper are how you defend that budget once you’ve set it.
Strategy 1: Prompt Engineering — Shorter Is Almost Always Better
System prompts have a tendency to grow like garages. Stuff goes in. Nothing comes out. Six months later you have a 4,000-token system prompt and nobody on the team can confidently explain what half of it does.
A few patterns we’ve seen consistently pay off:
Cut the backstory. Your agent does not need a paragraph explaining its role, its values, its commitment to excellence, and the company mission. A few crisp sentences about scope and constraints almost always outperform a wall of identity-shaping prose. Models in 2026 are good. They don’t need to be motivated.
One strong example beats five mediocre ones. Few-shot examples are powerful, but they’re also expensive — every example is sent on every call. If you find yourself adding a fourth example to handle an edge case, you’re often better off either tightening the instructions or moving that case to a separate, specialized agent.
Move stable content to system prompts; keep volatile content in user prompts. This sounds like a stylistic preference. It isn’t. It’s the precondition for prompt caching, which we’ll get to. The cleaner the line between “this never changes” and “this changes every call,” the more of your bill you can eliminate later.
Audit the cargo cult. A startling fraction of production system prompts contain instructions copy-pasted from a blog post in 2023, addressing a model behavior that was patched eighteen months ago. “Do not hallucinate.” “Think step by step.” “Take a deep breath.” Some of these still help. Many do not. Most have never been tested. Re-read your prompts as if you were paying for every word, because you are.
The reward for doing this work is rarely glamorous: a 30% smaller system prompt, an unchanged output quality, a 30% lower per-call cost on the system-prompt portion of every call you make from now until you change it again. Multiplied across thousands of calls a day, that line item becomes real money.
Strategy 2: Context Management — Stop Sending the Whole Conversation
Long-running agent workflows accumulate context. Every turn, the conversation grows. Every turn, the entire conversation gets sent again. By turn fifteen, you’re paying to re-process the same opening message you already paid to process fourteen times.
There is no clean solution. There are several useful ones.
Sliding windows. Keep only the last N turns. Simple, predictable, occasionally lossy. Works well for tasks where deep history doesn’t matter — most coding sessions, most chat-style interactions, most short-horizon tool use.
Rolling summarization. Periodically replace older conversation turns with a compressed summary. Costs a small amount to generate the summary; saves a much larger amount on every subsequent call. The trick is generating summaries that are actually useful — a bad summary is worse than a missing turn, because the model trusts it.
Selective retrieval over history. Treat the conversation history itself as a knowledge base. When the agent needs context, retrieve only the relevant turns. Adds complexity, but in long workflows it can cut context size by an order of magnitude.
Part 1 mentioned the “lost in the middle” problem — the tendency of models to underweight content buried inside a long context. In a context-management strategy, that’s actually a feature, not a bug. If the model isn’t really paying attention to material in the middle of a 200,000-token context, you weren’t getting much for those tokens anyway. Send less. The output will often improve, not degrade.
Strategy 3: Prompt Caching — The Single Biggest Lever
Most providers now offer some form of prompt caching: the ability to mark a stable prefix of your prompt as cacheable, so that repeated calls don’t re-charge for content that hasn’t changed. The economics are extraordinary. Cached input tokens typically cost a fraction — often 10% — of uncached input tokens.
For an agent making a thousand calls a day with a 2,000-token system prompt and 3,000 tokens of tool definitions, caching that 5,000-token prefix can cut input costs on the cached portion by 80% or more. On high-frequency, stable-prompt agents, prompt caching is usually the single highest-leverage optimization available.
So why doesn’t everyone use it?
Three reasons, in our experience.
They don’t know which prompts are cacheable. Caching only helps if the cached prefix is actually stable. If you’re injecting the current timestamp at position 12, nothing after position 12 caches. Most teams don’t have visibility into how their prompts are constructed in production, and so don’t know which ones are cache-friendly and which are quietly invalidating themselves on every call.
They don’t know which prompts are repeated. The economics of caching depend on hit rate. A perfectly cacheable prompt that’s only sent twice an hour saves almost nothing. The same prompt sent every two seconds is gold. Without telemetry, this is invisible.
They don’t know how much it would save. Caching feels like an engineering project. Without numbers attached, it’s hard to prioritize against everything else on the roadmap.
This is the most common pattern across all of these strategies, and it’s worth saying out loud: the strategy is simple. The visibility is the hard part. We’ll come back to this.
Strategy 4: Retrieval Tuning — RAG vs. Shoving Documents at the Model
Retrieval-augmented generation has become a default architectural pattern for agent systems. It is also a common source of token waste, because the difference between “good RAG” and “documents arriving in the context window” is mostly a matter of whether anyone has tuned it.
The mechanics matter. Chunk size affects what comes back. Embedding quality affects whether what comes back is relevant. Reranking — a second pass that reorders retrieved results by a more careful model — affects whether the most useful chunks land in positions the model will actually attend to. And top-K — the number of retrieved chunks you actually inject into the model’s context on each query — directly determines how much you pay per query, because every retrieved chunk is just more input tokens.
A retrieval system that returns ten documents per query, of which seven are irrelevant, is paying for seven documents of waste — and worse, it’s actively confusing the model. Reducing top-K from ten to four, after investing in a reranker, frequently improves both cost and answer quality at the same time. That is rare in this field. Take the win.
The unsexy version of retrieval tuning is just measuring it. What’s the precision of your retrieval at K=5? How often does the answer the model produces actually use the retrieved content? When it doesn’t, what fraction of the context window did the irrelevant retrievals consume? These questions are answerable. Most teams don’t answer them.
Strategy 5: Model Selection — Not Every Task Needs the Smartest Model
Frontier models are expensive because they’re hard to train and powerful when you need them. They are not always what you need.
Anthropic’s commercial pricing is a useful baseline. Sonnet 4.6 runs $3 per million input tokens and $15 per million output tokens. Haiku 4.5 runs $0.80 input and $4 output — roughly four times cheaper across the board. For tasks Haiku can handle well, every Sonnet call is paying a premium for capability you aren’t using.
The question is which tasks Haiku can handle well. Our rough heuristics:
Use the cheaper model for structured extraction (pulling fields out of a known format), classification, summarization of short inputs, well-specified rewrites, formatting conversions, simple tool routing, and most “decide which of these N options” decisions.
Use the frontier model for open-ended reasoning, long-horizon planning, ambiguous specifications, anything where the cost of a wrong answer dwarfs the cost of an extra dollar in tokens, and tasks that require the model to push back on its own first instinct.
Mixed pipelines are usually the right answer. A common pattern: use a cheap model to triage incoming work, then route only the genuinely hard cases to a frontier model. Done well, this can cut spend by half without measurably affecting quality. Done badly, you discover that your triage model is wrong about which cases are hard. The way to do it well is to measure.
Strategy 6: Open-Source and Self-Hosted Models — The Quiet Revolution
The strategy that gets the least attention in the optimization conversation is also, by a wide margin, the most economically dramatic: don’t pay per token at all.
Open-source models have closed an enormous amount of ground in the last eighteen months. The frontier-vs-open-source gap is no longer a chasm; for many production tasks it’s a step. And the moment your agents are running against open models — whether you host them yourself or rent them from a provider that charges by the month rather than by the token — the per-call unit cost drops to a small fraction of any commercial frontier API price. For some workloads, it rounds to zero.
This is not theoretical. The Token Mizer proof-of-concept ran for two weeks against a mix of locally-served and cloud-hosted open models — Qwen and Gemma running locally on our own hardware, GLM and MiniMax served through a flat-rate Ollama Cloud subscription. Total LLM spend across the entire dogfooding period: about $20, the cost of one monthly subscription, across thousands of calls. We have never hit a usage limit. We computed equivalent commercial pricing for comparison purposes in the report, but the actual bill is the same whether we make a hundred calls a week or a thousand.
There are real tradeoffs. They deserve to be named honestly.
You give up some capability. Open models are excellent at a widening list of tasks but still trail the frontier on the hardest ones — long-horizon agentic reasoning, fresh-knowledge tasks, certain tool-use patterns. Pick your tasks deliberately.
You take on operational burden, unless you don’t. Truly self-hosting means GPUs, serving infrastructure, model updates, capacity planning, and on-call responsibility. For a small team, this is non-trivial. There are two cleaner middle paths. The first is a per-token hosted open-model provider — Together, Fireworks, Groq, and others run open models on managed infrastructure at substantially lower per-token prices than the frontier labs, with none of the ops burden. The second, and often the right starting point for small teams, is a flat-rate subscription service — Ollama Cloud is one example — where a fixed monthly fee buys you generous usage of a curated set of open models with no per-call accounting at all. Our own pipeline uses both: locally-served models for the workloads where we want full control, and a flat-rate cloud subscription for everything else. The combined ops burden is small; the bill is predictable and tiny.
Latency is generally worse, sometimes substantially. Open models — whether self-hosted or run on hosted open-model infrastructure — typically have higher inference latency than the frontier ones. Frontier labs have invested enormous engineering effort in serving optimizations (speculative decoding, custom kernels, dedicated hardware) that the open ecosystem is catching up to but hasn’t fully matched at the top end. The practical consequence: every inefficiency in your prompts, your context management, and your retry logic costs you more wall-clock time on open infrastructure than it would on a hosted commercial API. Sloppy prompts that are merely expensive on the frontier become slow on open models, and on the longest contexts can also become less accurate, because attention behavior on long inputs tends to be more brittle outside the frontier baseline. Long-context calls still degrade output quality. Runaway retries still cascade latency. Engineering discipline isn’t optional on open infrastructure. It’s load-bearing. Moving to open models changes the billing calculus; it doesn’t change the engineering one — and on open infrastructure, it raises the stakes. The reasons for prompt and context discipline shift from “cost” to “performance,” and they tighten rather than loosen. A team that switches to open source without tightening up that discipline often discovers that the cost win is real but the user experience got noticeably worse — and in some cases, so did the answers.
You lose access to the latest features. Prompt caching, fine-grained tool use, native streaming patterns, vision capabilities — the frontier labs ship new capabilities first. If your workflow depends on them, an open model may not be a drop-in replacement.
You have to actually evaluate quality. This is the one that catches teams. Switching to a cheaper model is only a savings if the cheaper model still does the job. The way to know is to build a small evaluation set of representative tasks for each agent and run it against candidate models periodically. Not glamorous. Not optional. The cost discipline of moving to open models only works if the quality discipline keeps up.
The reason to take this seriously is the same reason any infrastructure choice deserves serious thought: at scale, the unit economics dominate. A team spending $50,000 a month on commercial inference for tasks that a hosted open model could handle is leaving roughly $40,000 a month on the table. That is enough to fund a person whose entire job is maintaining the open-model stack, with margin to spare.
The strategy isn’t “switch everything to open source.” It’s “for every agent, ask whether it actually needs to be running on the most expensive option.” Most agents don’t.
Strategy 7: Architecture Patterns That Don’t Hemorrhage Tokens
The biggest token wins are often architectural rather than prompt-level.
Tool minimalism. Every tool definition you give an agent is sent in every call, used or not. An agent with twenty tools is paying for twenty tool schemas on every turn. Audit which tools each agent actually uses. Remove the rest. If different sub-tasks need different tools, give the agent a “load tools for this task” mechanism rather than dumping the whole catalog into the system prompt.
Designate canonical state. A common anti-pattern in multi-agent systems: every agent re-reads the entire conversation, the full task spec, and the latest project state on every turn. The supervisor reads everything the workers produce. The workers read everything the supervisor coordinates. The number of tokens consumed scales with the square of the team size. Designate one place where state lives, give agents a way to read only the slice they need, and accept that not every agent needs to know everything.
Batch when you can. If your workflow involves applying the same operation to a hundred items, you almost never want a hundred separate API calls. Batch them — either as a single call that processes all items at once, or via a provider’s batch API where the unit price is itself discounted. The token savings are often dwarfed by the latency savings.
Stream and stop early. For long-form generation tasks where you can detect a bad output quickly, streaming lets you cut off generation as soon as it’s clear the model is going off the rails. You only pay for the tokens you actually accept. This requires a guard — a small check that runs against partial output — but for tasks with a measurable failure mode, the savings on retries alone usually pay for the engineering.
Limit retries. Speaking of retries: every retry is a full token round-trip. Without explicit caps, agents in error states can burn through hundreds of dollars in minutes. Set retry limits. Make them aggressive. Treat exceeded limits as alerts, not as silent failures.
Idle Is Not Free
There is a category of token spend that almost nobody accounts for explicitly, and that almost nobody knows to look for until they see it in their own data: the traffic your agents generate when they aren’t doing any actual work.
Modern agent systems are constantly talking. Heartbeats — small periodic check-ins that keep an agent connected, healthy, and ready to receive a task — are the most common example. There’s also status polling against shared state, watchdog calls that verify other agents are alive, periodic “any new work?” requests against task queues, self-tests that confirm tools are still reachable, and re-handshakes after any kind of transient disconnect. None of this is producing user-visible output. All of it is consuming tokens.
A single heartbeat is small. The system prompt is sent. A tool list might be sent. The model returns a one-or-two-token acknowledgement. Cost per call rounds to almost nothing.
But the multiplier effect from Part 1 applies in full force, and the cadence numbers are unforgiving. An agent heartbeating once every thirty seconds makes 2,880 calls per day. A team of ten agents on that cadence makes nearly 30,000 calls per day, every day, whether anyone is using the system or not. Multiply by the system prompt and the tool definitions that ride on each call, and the “negligible” per-heartbeat cost becomes a real line item in your monthly bill — paid in full whether your team is in the middle of its busiest week or asleep at the keyboard.
The fix is mostly cultural rather than technical. Audit your ambient traffic explicitly. Decide what each kind of idle traffic is actually for, and how often you really need it. Lengthen heartbeat intervals where you can. Have agents back off their own polling when nothing is happening. Distinguish between “the system is ready” and “the system is being used” — those have very different cost profiles, and they should be measured separately.
For grounding, here’s the cadence our own team landed on after looking at the data. Most of our agents heartbeat once per hour. The single exception is System Doctor — the agent whose job is to watch for runaway behavior and other infrastructure problems — which heartbeats every thirty minutes, because slow detection there has a much higher cost than doubling its heartbeat rate. Beyond the schedule, every new task submission also fires a one-shot heartbeat against the agent that’s about to receive it, which keeps the system responsive to actual work without us having to pay for high-frequency polling to get that responsiveness.
The combination has held up well in practice. We’ve considered going further — eliminating the scheduled heartbeats and living on pure event-driven ones — but probably won’t. Most of our agents now run against open models — some served locally, others through a flat-rate cloud subscription — where the marginal cost of a heartbeat rounds to zero, so the financial pressure to cut them is gone. There is still some operational pressure: even cost-free heartbeats compete with real work for finite GPU capacity, which is its own kind of cadence discipline. But within those bounds, we’ve left the schedule alone.
The point isn’t the specific number. It’s that we picked a number, on purpose, after looking at the data. The choice you don’t make is the one that defaults to whatever your framework ships with, and the framework defaults rarely have your bill in mind.
Ambient traffic is one of the easier wins in token optimization, in part because almost nobody is looking for it. If you have telemetry, the first question to ask is: what fraction of last week’s calls happened on days when no actual user work was done? The answer is often not zero. It is often not small. We have a story about this from our own pipeline, coming up.
Fortunately, you have an agent team to help you with most of this. We’ll come back to how that works near the end of the paper.
Why Strategies Aren’t Enough
If the strategies above are obvious — and most of them are — the natural question is why agent systems still hemorrhage tokens.
The answer is that production drifts.
The 800-token system prompt that someone carefully tuned in March is 4,200 tokens by September. Nobody added 3,400 tokens on purpose. Three different engineers added a “small” instruction here, a clarifying example there, a guard against a one-off bug nobody can quite remember anymore. None of those changes were wrong in isolation. The aggregate is.
The retrieval system that returned a clean four documents per query at launch is returning eleven by year-end, because someone widened the top-K to fix a missed-context bug and never narrowed it back. The Haiku-tier classifier that handled 80% of incoming tasks is now handling 50% because a new task type came in that Haiku can’t quite get right, and the routing logic conservatively escalated everything ambiguous to Sonnet. The prompt caching that was working perfectly stopped working when someone moved the timestamp injection from the user prompt to the system prompt.
None of these problems announce themselves. They show up as a slow, unexplained rise in monthly spend that the team rationalizes — usage is up, the workload is harder this quarter, the new model must be more expensive than the old one. By the time anyone investigates, the drift has compounded.
The strategies in this paper are necessary. They are not sufficient. Without ongoing discipline against drift — and ongoing visibility into how each strategy is performing in production — every one of them will quietly erode.
The most useful illustration of this I can offer is one we lived through ourselves.
How Drift Broke Our Team
A few months into running the agent team, I started noticing that error rates were creeping up and output quality was creeping down. Nothing dramatic — no broken pipelines, no failed runs — just a general sense that the team was running less crisply than it had a few weeks earlier. Hard to put a finger on. Easy to rationalize.
I asked the General Manager agent to design a methodology for an agent health and configuration deep dive: a structured audit pass that could be run against any agent on the team, examining its configuration, its memory, its prompt structure, and its operational state for evidence of drift. Once the methodology was solid, I had the GM run it against itself first.
It found seven distinct drift errors in its own configuration. The most damaging were stale entries in its memory.md — facts that had been true at one point and were no longer accurate, but were still being treated as ground truth in every decision the GM made. The GM had been quietly making decisions on the basis of obsolete information for weeks, and was the last to know.
I had the GM fix its own issues, then run the same audit against every other agent on the team, one at a time. Every single agent had meaningful issues. Three of them had drifted so far that they could no longer reliably perform their roles.
None of the individual changes that produced this state were dramatic. Each agent’s drift was small in isolation — a stale memory entry here, a contradicting instruction there, a prompt addition that conflicted with an earlier one. Aggregated across the whole team, the result was the slow, hard-to-diagnose decline I’d been noticing in our outputs. The team had broken in slow motion, and none of the individual breaks had been visible while they were happening.
After reviewing every audit, I had the GM apply fixes across the team. Things started running noticeably better, immediately.
The discipline we put in place after that experience is now permanent. The full agent health and configuration deep dive runs against every agent on the first of every month. The System Doctor agent runs the most critical subset of it every thirty minutes — and the reason its heartbeat cadence is short, mentioned earlier in this paper, is exactly this. We are not going through that again.
A monthly audit and a thirty-minute critical-path check catch a lot. They don’t catch everything. They catch configuration drift — the kind that lives in agent definitions, memories, and prompts. They don’t catch operational drift — the kind that lives in the live traffic stream: prompts that grow inside tools, retrieval that quietly returns more documents than it used to, heartbeat patterns nobody set out to create, redundant calls being fired over and over because something downstream changed silently. Configuration audits look at the agent. Operational drift shows up only in what the agent does.
For that, you need a different kind of observability — the kind that watches every call as it happens.
Token Mizer
Before any of that, a story about why this exists.
The first time I stood up an AI agent team — well before any of the architecture in this paper had been designed — I gave them a single task: build me a personalized Python training program. The team was running on Anthropic’s commercial APIs at the time, a mix of Sonnet and Haiku. They went to work. Three days later, when I went to check on the team’s progress, I discovered they had burned through more than $500 worth of tokens in a single hard sprint.
Nothing had gone wrong, exactly. They had completed the task. The output was useful. But $500 in three days, on what I had imagined as a modest internal project, was not the bill I had been mentally preparing for. And worse than the number was what I didn’t have around it: no per-agent visibility into who had spent what, no notion of which agent was hitting the API hardest, no way to set a ceiling, no alert when the burn rate jumped. The first time I saw the cost was when the meter had already run.
That was the moment Token Mizer became inevitable. The shape of the system changed over many months of design and prototyping, but the original requirement has never moved: I am never going to be surprised by my agent team’s bill again.
Token Mizer is, at its core, three things stacked on top of each other.
First, a proxy. Your agents call the proxy instead of calling the LLM provider directly. The proxy makes the upstream call on their behalf, captures everything that happened — input tokens, output tokens, cost, latency, model, provider, prompt content, agent identity — and returns the response. From the agent’s perspective, it’s a one-line config change: change the base URL, add an x-agent-id header on the way out, done. No agent code needs to know that Token Mizer exists.
Second, a telemetry store. Every call the proxy handles becomes a row in a database. Over time, that database becomes a complete ledger of who spent what, on which model, for which task, with which prompt. This is the artifact that almost no production agent system has today. Without it, every question about token spend is a guess. With it, those questions become queries.
Third, an intelligence layer. Sitting on top of the telemetry store, a set of components do the work that nobody on a busy team has time to do manually:
A baseline learner watches each agent over time and builds a statistical profile of normal behavior — typical input size, typical output size, typical call rate, typical latency. After enough samples, it knows what “normal” looks like for each agent on each model.
An anomaly detector runs continuously against the telemetry stream and flags calls that deviate sharply from those baselines. When the agent that normally sends 500-token prompts suddenly sends 8,000, somebody finds out within minutes, not at the end of the billing cycle.
A prompt auditor looks for specific failure modes — prompt bloat, redundant identical calls being sent over and over, prompts ballooning past sensible thresholds — and writes findings into the same store.
An optimization planner compares observed token patterns across models and agents, and writes recommendations: this agent could move from Sonnet to Haiku and save an estimated 73% per call; this agent’s prompts have grown enough that compression would recover an estimated 15%.
And enforcement — for the cases where alerting isn’t enough. Every agent has a budget. Every agent can be killed. If something goes truly wrong and an agent enters a runaway loop, the proxy can refuse the next call and return a clear error within a second of the kill switch being thrown.
The architecture, in pictures:
┌─────────────────────────────────────────────────────────────┐
│ Your Agent Team │
└──────────────────────────────┬──────────────────────────────┘
│ x-agent-id header
▼
┌─────────────────────────────────────────────────────────────┐
│ Token Mizer Proxy │
│ Captures every call. Enforces budgets and kill switches. │
│ Routes to upstream LLM provider. │
└──────────────────────────────┬──────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Telemetry Store │
│ The agent ledger: every call, every cost, every prompt. │
└──────────────────────────────┬──────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Intelligence Layer │
│ Baselines · Anomaly Detection · Prompt Audit · Plans │
└─────────────────────────────────────────────────────────────┘
The intentional design choice underneath all of this: capture telemetry before routing the call. If the upstream provider fails, the call is still recorded. There is no path through the system in which a token gets spent and Token Mizer doesn’t know about it.
What It Looks Like in Practice: The Dogfooding Story
We built Token Mizer for ourselves before we built it for anyone else. The first production users were Gr4Ig’s own agent team — a CFO agent, an Engineer agent, a Reviewer agent, a General Manager — running real internal work against a mix of locally-served and cloud-hosted open models, all routed through the same flat-rate Ollama Cloud subscription described above.
The first 48 hours of running Token Mizer against ourselves produced exactly the kind of finding we hoped it would, on exactly the team that should have known better.
The CFO agent’s typical call sent about 500 input tokens — usually a brief financial question, a small amount of context, a request for analysis. Across twenty calls in the first day, the baseline learner settled on a mean of 493 input tokens with a tight distribution. Then a single call came in carrying 8,000 input tokens — sixteen times the baseline.
The anomaly detector noticed within the next scheduler cycle. Two findings were written: one for the input-token spike (z-score of 4.2 against a 2.5 threshold), one for the abnormally large output it produced. A separate finding flagged the CFO’s call rate over the same window: 7.5 calls per hour against a learned baseline of 0.5 — a 14× elevation. The prompt auditor picked up the same 8,000-token prompt and tagged it for review with a recommendation to compress it, with an estimated 15% per-call saving.
What was the underlying cause? An accidentally-pasted document attached to a routine question. A small mistake with no visible symptom. The agent kept working. The output was probably fine. Without instrumentation, this would have been invisible — a small bump in the cost graph that nobody notices because it doesn’t break anything.
That’s the bug we built Token Mizer to find. Not the dramatic failure. The quiet 16× call that doesn’t.
What happened over the two weeks that followed is just as instructive — and more humbling.
The telemetry store has now logged more than 2,700 calls across the same fortnight. Most of that volume isn’t real workload. For ten of the intervening days I was away from the keyboard and the agent team had nothing active to work on — what it had was heartbeats. Token Mizer logged every one of them. Roughly two thousand heartbeat calls, going to and from agents that were doing no actual work, accumulating exactly the kind of ambient cost the section above warned about. The unintended baseline turned out to be useful — it’s hard to capture ten clean days of idle traffic on purpose — but it wasn’t a planned experiment. It was a vacation, and Token Mizer kept the lights on while I was gone.
The original anomaly findings from the working window are still in the store, alongside the recommendation to compress the CFO prompt. Two agent baselines are established. By every measure we set out for the proof-of-concept, the system is working as designed.
The continuous log also revealed something we didn’t go looking for.
Of those 2,700-plus calls, only a few dozen were properly tagged with an agent identifier. The overwhelming majority — close to 2,700 of them — were arriving at the proxy with agent_id=unknown. Even the heartbeats had the gap. There was, in other words, a large part of our own agent traffic that we had not yet wired up to identify itself, and we didn’t know about it until Token Mizer started showing it to us.
This is exactly the failure mode the system is designed to expose. Without per-agent attribution, there is no per-agent accountability — no way to ask which workload is producing which spend, no way to apply a budget, no way to catch a runaway. We had been operating an agent team with a partial ledger and we hadn’t realized.
The fix has since shipped. Going forward, every call — including the heartbeats — carries an agent ID. It’s a fitting case study. A token visibility tool, in the hands of the team that built it, surfaced the team’s own visibility gap, and the team closed it. The proof that the loop works is that the loop closed on us first.
The Honest State
Token Mizer today is a working proof-of-concept, not a finished product. The architecture is in place. The intelligence layer produces real findings against real traffic. The enforcement mechanisms — kill switches and budgets — work as specified. The proxy adds a few milliseconds of latency overhead in normal operation, well below the threshold where it would degrade an agent workflow.
What it does not yet have, in its current form, is the surface area you’d expect from a product:
There is no dashboard. Findings live in the database; reading them today means a SQL query.
There is no multi-tenancy. The current deployment assumes one team, one instance, one database.
There is no high-availability story. A single proxy process is a single point of failure, and the database is a SQLite file. For a team running production-critical agent traffic, that’s not yet adequate.
There is no authentication beyond a header allowlist. For the internal use case it was built for, this is fine. For anyone else, it isn’t.
These are all known. They’re the things we’ve intentionally not built because we don’t need them for our own use case — and they’re worth naming so that anyone building their own version of this knows what to add when they leave the prototype phase.
Make the Agents Do It
Reading the list of strategies earlier in this paper, you may have been doing the same calculation I once did: this is a lot. Auditing prompt sizes per agent, monitoring cache hit rates, retuning retrieval, evaluating model selection, watching ambient traffic, designing per-agent budgets — done seriously, this is a job. Maybe several jobs. The number of teams who have done all of this consistently, in production, over time is not large. The number of solo founders who have done it is approximately zero.
I am one of the people who would not have managed this on my own. I don’t have the working memory to track nine agents’ configurations across seven different optimization dimensions, and I don’t have the patience to do it manually month after month. If executing this discipline depended on my willingness to grind through it, the discipline would not exist.
My approach is to make the agents do it.
Every operational piece of this — every audit, every recurring check, every drift sweep, every optimization recommendation review — is owned by an AI agent, with me providing oversight and direction rather than doing the work. The General Manager agent has been my partner on every strategy in this paper. The agent health and configuration deep dives, the heartbeat cadence reviews, the periodic re-evaluations of model selection per role — GM-designed, GM-executed, GM-proposed fixes that I review and approve. The monthly cadence is on the GM’s calendar, not mine.
The pattern works because it’s the right shape for what these systems are good at. Reading configurations, comparing them against a checklist, summarizing findings, proposing fixes — this is exactly the work language models are competent at and humans find tedious. Use the asymmetry. Make the agents do the parts they’re better at than you are, and reserve your own attention for the calls only you can make: should we actually downgrade this agent’s model, what’s the right policy when an enforcement trigger fires, is this finding a real problem or a false positive.
This is also why Token Mizer matters in the form it takes. The intelligence layer isn’t a dashboard you have to remember to check; it’s a set of components that work continuously and surface what needs your attention. The human stays in the loop — somebody still has to validate a model downgrade or sign off on an enforcement action — but the fetch-look-think-decide loop shrinks dramatically. You’re reading findings, not generating them.
The version of this paper I’d hate to write is the one where every strategy ends with “and you, dear reader, must now do this manually, forever.” That’s not the version that will work. The version that will work is the one where the discipline is real, the strategies are honored, and the operational labor is mostly carried by the agents you’re already paying for.
What Comes Next
Token Mizer is not a product I plan to release. There won’t be an open-source repo. There won’t be a hosted version you can sign up for. The system exists, it works, it runs our own pipeline every day — but the code stays in-house.
What I do plan to keep doing is the rest of it: the research, the building, the dogfooding, and the writing. Gr4Ig is a research and innovation practice, not a product company. We break things, we learn, and we share what we learn. The most useful thing I can offer to other people building agent systems isn’t another tool to evaluate against the seventeen they’re already drowning in. It’s the thinking, the patterns, the lessons, and the honest case studies that help you build your own version, suited to your own constraints. That’s what this paper is. That’s what the next series will be. That’s what the series after that will be.
If that’s useful to you, the simplest way to get more of it is to subscribe. The papers will keep coming as long as the work keeps producing things worth writing about, which so far is most of the time. For teams that want help applying any of this to their own pipeline — direct collaboration on architecture, audits, or implementation — Gr4Ig also takes on a small number of advisory engagements. Reach out if that fits.
In the meantime, if you’re running an agent team and the contents of this paper feel uncomfortably familiar — system prompts you can’t fully account for, monthly bills that don’t quite reconcile to your usage, a creeping suspicion that some of your agents are spending more than they need to — you don’t need to wait for a tool. The strategies in this paper are the actual work. Token Mizer is the kind of thing you build when you’ve already done that work and you want to make sure it stays done. Anyone reading this paper has the materials to build their own.
If tokens are agent currency, Token Mizer is the treasury management system. The point of treasury management is not that it stops you from spending. The point is that the spending is on purpose.
This is Part 2 of a two-part series on token usage in agentic AI systems. For the conceptual foundation — what tokens are, why they’re agent currency, and why the multi-agent shift changes everything — see Part 1: Tokens, The Currency Your AI Agents Are Spending Right Now.
Coming Up Next: Meet the Team
Throughout this paper I’ve referred to “the agent team” without really introducing them. The next series from Gr4Ig fixes that.
It’s an in-depth look at the actual agent team running our pipeline — nine permanent agents, each with a defined role, a specific configuration, and a deliberately chosen model. You’ve already met some of them in this paper: the CFO whose 8,000-token bloat opened the dogfooding case study, the System Doctor whose every-thirty-minute heartbeat earned its own callout, the Engineer, Reviewer, and General Manager who appeared alongside them. The rest get their proper introduction in the next series — who each agent is, what each one does, which model we picked for each role and why, what we got wrong on the first try, and how a nine-agent team coordinates without burning everyone’s context window.
If you’ve been wondering what an actual production agent team looks like — not the architecture diagram, but the team chart — the next series is for you.
Subscribe so you don’t miss it.
About the Author
Greg Cooper is Founder, CTO and Head of R&D at Gr4Ig, LLC (gr4ig.com), an AI research and innovation company focused on agentic intelligence infrastructure. He brings thirty years of experience in critical systems design and engineering leadership. Correspondence: greg@gr4ig.com
About Gr4Ig
Gr4Ig is a research and innovation practice exploring AI-native infrastructure for autonomous agent teams. We don’t ship products. We build, we run, we break, we learn, and we share what we learn. Token Mizer is one of the systems we built and run for ourselves. Gr4Ig Brain is the memory layer. The Machine is the vision. Follow along at gr4ig.com.
AI Assistance Transparency
This paper was produced with the assistance of the Gr4Ig AI agent team, a multi-agent system leveraging a variety of large language models. All ideas, arguments, frameworks, and conclusions represent the author’s own intellectual work. AI assistance was employed for tasks including research synthesis, structural refinement, and prose editing. The author takes full responsibility for the accuracy, integrity, and originality of this work.
A note specific to this paper: during its drafting, the Gr4Ig agent team was itself running under Token Mizer telemetry. The numbers in the dogfooding case study are real. The untagged calls were ours. The fix that retired them has since shipped.