I have been telling people for a while that long AI chats are a tax. Every message re-sends the whole conversation, and the harnesses we use every day add their own instructions and tools on top. I had a feeling that my Brainstem cost pennies because it keeps chats short and puts the useful work into
agent.py files, while Claude Code and Copilot CLI kept growing the chat. A feeling is not a number, so I measured it.

What I ran

Five small business tasks with real rules: tiered sales commission with clawbacks, overtime pay, reorder points, late fees, and invoices paid twice. Each task was packaged three ways from one Python implementation: an Agent Skill written in plain words, the same skill with a bundled script, and a RAPP
agent.py. Claude Code, GitHub Copilot CLI and a Brainstem answered the same questions with the same model, Claude Haiku 5.5. Three runs of every setup, every answer graded to the cent.

What held up

The host charges before you ask. To answer “reply with the number 7”, a fresh Claude Code sends the model 15,176 tokens and a fresh Copilot CLI sends 23,014. A Brainstem sends 2,805. My own Claude Code and Copilot, with everything I have installed over the months, send 32,295 and 33,719. My own Brainstem, with 22 agents loaded, sends 8,716.

Every question pays that charge again. The same 15 questions cost 135k tokens in a Brainstem, every answer right. Claude Code’s best setup used 725k, about 5.4 times as many, also with every answer right.

Weight is what makes them slow. This one surprised me. On fresh installs, with the rules in code, all three answered in about four seconds. My own Copilot, with the same agent.py tools and nothing else running, took 56 seconds per answer and 820k tokens for five questions. The Brainstem did the same five in 35k tokens.

A robot strains under an overloaded backpack of folders, cables and tools while a light robot walks easily ahead.

Lock it in, then clear the chat. Once the rules live in code, history is mostly dead weight. A new chat for every question cut the Brainstem’s total from 135k to 90k tokens, still 45 of 45 right.

Call /chat, not a wrapper. Routing the same Brainstem through Copilot CLI and an MCP plugin cost about 7 times as many tokens per answer, and Copilot often skipped the Brainstem altogether. On my own Copilot install it skipped it every time. The MCP plugin is for hosts that cannot make an HTTP call. Everything else should call /chat.

What did not hold up

  • Spend does not grow exponentially. It grows with the square of the chat’s length. A chat twice as long costs about four times as much. Bad enough, but I was wrong about the word.
  • agent.py is not magic inside Claude Code. There, a skill with a bundled script was cheaper than agent.py tools, because Claude Code hides MCP tools until the model searches for them. It called my agent tools on 13 of 45 questions and improvised the rest. The same files inside a Brainstem were the cheapest setup of all. The host is what saves the tokens, not the file format.
  • Plain-word rules were not the accuracy disaster I expected. My first wording of one rule was ambiguous about rounding, and the models filled the gap their own way. Once I fixed the wording, skills written only in words got 116 of 120 answers right. They were still the most expensive setup on both hosts, and their 4 misses were real arithmetic slips. Rules written as code were right every time the code ran.
  • I did not test the “lobotomy”. Published studies (Chroma’s Context Rot, Lost in the Middle, RULER) report that accuracy drops as context grows. My 15-question chats were too short to show it.

The cost of ownership

Tokens are the sticker price. Keeping a setup working is the rest. In one evening of measuring:

  • Claude Code silently ignored an MCP config written in Copilot’s format. No error, the tools just were not there, and my first “agent.py in Claude Code” numbers measured nothing.
  • Claude Code rewrites $0, $1 and friends inside a SKILL.md as skill arguments, so dollar amounts in your rules can turn into file paths.
  • Copilot’s usage file is a running total for the whole session, which I first read as per-question.
  • The Brainstem’s only failure was one upstream connection reset in 30 questions.

Driving one question from a script took 13 lines of code for the Brainstem (one HTTP request), 29 lines and 8 flags for Claude Code, and 54 lines and 9 flags for Copilot CLI. And the Claude Code session I used to run all of this is itself a long chat: by the end of the measurements it had made 261 model calls and re-sent 81.7 million tokens of context, about 313k per call.

What I do now

  1. Measure your host’s idle cost with one trivial question. That number is charged on every call.
  2. Put repeated work in code: an agent.py, or a skill with a script.
  3. Keep chats short. Finish a job, keep the answer, start fresh.
  4. Unload what you do not use. Every plugin, server and skill rides along on every call.
  5. Call a Brainstem’s /chat directly. Use MCP only where HTTP is not an option.

The full white paper, with every chart and the mistakes I caught along the way, is
The Context Tax. The bench is open source at
github.com/kody-w/token-bench, and it measures your own setup, as you actually use it, in one command.