We counted the same bytes under every frontier tokenizer, using each vendor's own counting endpoints, and cross-checked the counts against real paid requests. Below are the numbers and what they do to the prices on the rate cards.
A model's bill is two numbers multiplied together:
The pricing page shows the second number and treats the first as a constant. It is not a constant. It depends on the model's tokenizer, and tokenizers differ a lot between vendors. Two models can list the same "$5.00 / 1M input tokens" and produce meaningfully different bills for the same paragraph, because one of them turns that paragraph into more tokens. Since nobody publishes tokens-per-content numbers, we measured them.
We took 16 real fixtures: English prose, an HTML page, JavaScript, Python, TypeScript and Rust files, JSON tool schemas and tool results, Chinese chat and prose, symbol-heavy text, and our own agent system prompt. Each fixture was counted, byte for byte, with every model's production tokenizer:
GPT's o200k serves as the 1.00x reference throughout, mainly because it has been frozen and publicly documented for over two years, while Claude's tokenizer is the one that changed. DeepSeek and GLM are left out of the tables entirely: we only have rough characters-divided-by-four estimates for them, not real tokenizer counts, and this post is about measured numbers.
Claude Opus 4.6 and Opus 4.8 have the same $5.00 / $25.00 list price. What changed between them is the tokenizer. Sonnet 4.6 and Opus 4.6 use the old one; Sonnet 5, Opus 4.8, and Fable 5 use the new one. The table counts the same bytes with both, on Anthropic's own endpoint:
Weight those rows the way a real agent request is composed, which is mostly English system prompt, tool schemas, code, and JSON, and the new tokenizer comes out around +32% per request. The Chinese row barely moved, so the inflation is concentrated in English and code.
Sonnet 5 launched at $2.00 / $10.00, down from Sonnet 4.6's $3.00 / $15.00, which looked like a price cut. That is an intro price, and it ends August 31, 2026. While it lasts, the lower rate slightly more than covers the extra tokens, so Sonnet 5 works out a little cheaper than 4.6 for the same code. From September 1 the price returns to $3.00 / $15.00, the extra tokens remain, and the same work will cost about a third more than it did on Sonnet 4.6 at the same list price.
count_tokens is a prediction, so we also sent real paid requests with max_tokens: 1 and read usage.input_tokens , which is what invoices are based on. For the same content, Opus 4.6 billed 2,541 input tokens and Opus 4.8 billed 3,191 , each matching its predicted count exactly. We ran the same check on Fable 5, the most expensive model in the lineup, and it billed 3,191 as well, identical to Opus 4.8. So Fable uses the same new tokenizer and there is no extra per-token markup hidden behind its higher list price. The whole verification cost about $0.08.
The cross-vendor table uses GPT's o200k as the 1.00x reference. Every cell is that model's token count for the identical file divided by GPT's, so 1.20x means 20% more tokens than GPT. Claude's new and old tokenizers are shown side by side:
The code rows sit well above the prose rows: TypeScript at 1.73x , Rust at 1.58x, JavaScript at 1.52x, Python at 1.50x, against 1.40x for English prose. Code is most of what a coding agent processes, so for that workload the 1.50-1.73x band is the relevant one.
Why is TypeScript the worst case? Because o200k is unusually efficient on it: about 4.24 characters per token, which looks like the result of training on a lot of web JavaScript and TypeScript, where camelCase identifiers and JSX patterns compress into single tokens. On Rust its efficiency drops to about 3.51 characters per token. Claude's tokenizer is roughly equally dense on both languages, so the gap is widest exactly where GPT is strongest.
Chinese behaves differently. Claude sits around 1.45-1.55x above GPT with both the old and the new tokenizer (435 vs 433 tokens against GPT's 300 on the prose fixture), so this is a long-standing property of the Claude family on CJK text, not something the new tokenizer introduced. Gemini is actually more efficient than GPT here, at 256 tokens. Which tokenizer costs you more depends on what you write.
Multiply the list price by the measured divergence and you get an effective price for processing the same work. Divergence here is the blended multiplier for a typical English coding request, normalized to GPT's o200k :
A few rows are worth a second look. Opus 4.6 and 4.8 share a list price but differ by about 32% in effective price. GPT-5.5 and GPT-5.6 Sol share the tokenizer, so their identical list prices really are identical in effect. Gemini 3 Flash runs a slightly heavier tokenizer than GPT and still remains the cheapest option by a wide margin.
For an independent data point: Ploy published a production migration to GPT-5.6 Sol：https://ploy.ai/blog/migrating-a-production-ai-agent-to-gpt-5-6 this week and reported 1.70M input tokens against Claude Opus 4.8's 2.60M for the same builds, about 35% fewer. That is a whole-task bill rather than a tokenizer probe, so it also folds in model verbosity, but it points the same way.
Everything above measures one thing: how many input tokens identical bytes become. A full agent task adds more variables on top, and they are big ones. How many output and thinking tokens does the model spend to reach the same result? How much context does the harness load per step? How often does it call tools or spawn subagents? How does the provider price cache reads and writes?
Two consequences are worth spelling out. First, cache traffic is billed per token too, so a tokenizer that produces 32% more tokens also makes every cache write and every cache read about 32% more expensive, and on long agent sessions cache reads are most of the bill. Second, whole-task costs can diverge far more than 1.73x in either direction once verbosity and thinking are folded in. When people report that one model "uses 2-4x the tokens" of another on agent work, that can be true for their setup even though the pure input tokenization gap in our fixtures never exceeded 1.73x. The two numbers measure different layers.
By content type, the input-side range we measured for Claude's new tokenizer against GPT's o200k is: prose, HTML, and JSON at 1.36-1.42x; code at 1.50-1.73x with TypeScript on top; Chinese and symbol-heavy text at 1.44-1.53x. We put TypeScript in the title because it is both the top of the range and the thing a coding agent processes all day, not because the whole world is 1.73x.
We also measured the per-task layer directly, in a follow-up: the same nine models given one identical drawing task, one shot each, with every attempt priced from the providers' own usage numbers. The same drawing ranged from $0.004 to $0.80 depending on model and reasoning effort, and maximum effort made three models fail to finish at all. That experiment is here: The Pelican Benchmark Is Saturated. We Made 9 AI Models Draw a MacBook Pro Instead：/blog/macbook-svg-benchmark.
None of this makes one model universally right. GPT-5.x is the token-lean choice on English and code, Gemini 3 Flash is remarkably cheap in effect, and Claude models earn their place on quality even when they cost more tokens to run. Just make sure the price you compare is the one you actually pay, after the tokenizer.
Sources: Anthropic pricing (anthropic.com/pricing：https://www.anthropic.com/pricing), OpenAI pricing (platform.openai.com/docs/pricing：https://platform.openai.com/docs/pricing), Google Gemini API pricing (ai.google.dev：https://ai.google.dev/gemini-api/docs/pricing), xAI pricing (docs.x.ai：https://docs.x.ai). Token counts come from Anthropic's count_tokens endpoint, OpenAI's o200k_base (verified against live API usage), and the Google and xAI count endpoints. No text was generated to produce these counts.
The measurements are ours; the prose was drafted with AI assistance and edited by a human. Updated 2026-07-14 with a TL;DR, tighter wording, and a section on what input tokenization does not capture, based on reader feedback.
Playcode keeps every one of these models one click apart, so you can run the same prompt on two of them and compare the result that matters, the app it builds, instead of arguing about a sticker. Try it at playcode.io：https://playcode.io.
The follow-up: same models, one unforgiving drawing task, real per-task costs.
A long-form comparison of the three big AI families on real coding tasks.
What each of the three new GPT-5.6 tiers costs and when to reach for it.
The best UI and design quality we have tested, and when the premium is worth it.
We'd love to hear from you! Chat with us or send us an email.
© 2026 Playcode. Headquartered in Europe.
Scan to open this article