Technology’s giants measure the AI revolution in tokens. Tokens are a billing unit, not a unit of intelligence, and the difference is costing companies a fortune.
At its annual developer conference in May, one of the world’s largest technology companies told the audience that its systems now process more than 3.2 quadrillion tokens a month. Two years earlier the figure was under ten trillion. A year earlier it was 480 trillion. The chart went up and to the right, the audience applauded, and the number travelled around the world as proof that artificial intelligence had arrived.
It proves something, but not that. A token is a fragment of text, roughly three-quarters of a word, that a model reads or writes. It is how AI is metered and billed, much as electricity is sold by the kilowatt-hour. Nobody would measure the progress of civilisation by how much electricity it burns, yet that is exactly what the industry now does with AI. A token processed is not a problem solved, a decision improved or a customer served. Very often it is the opposite: a sign that a machine took a long road to a short answer, or answered a question nobody needed asked.
I have spent more than a quarter of a century building and governing AI systems, and I remain a believer in what they can do. But I have watched a perverse metric take hold in boardrooms and engineering teams alike. Call it token-maxxing: the belief that more consumption means more intelligence, more adoption and more value. It is wrong on all three counts, and the bill is arriving.
I. Token-maxxing: how a billing unit became a vanity metric
The phrase began as a joke among software engineers. It is now a management practice.
At one of the largest social-media companies, an employee built an internal leaderboard that ranked some 85,000 staff by how many tokens they consumed, complete with titles such as “Token Legend”. The company reportedly burned through 60 trillion tokens in a single month. At list prices that would be worth close to $900m; even with heavy discounts, the bill plausibly ran past $100m. The leaderboard came down only after it leaked and was mocked online.
It was not an isolated case. At a large software maker, a token leaderboard has run since January, and senior executives who rarely write code sit near the top. One newer engineer admitted to burning tokens simply to avoid looking disengaged: asking an AI to explain documentation that a faster internal tool could answer, prototyping features with no intention of shipping them, and handing tasks to agents that would have been quicker by hand. At a big enterprise-software firm, staff can see their own spending against a minimum monthly target, and their colleagues’ spending too. Engineers there reportedly aim to land just above the average, and spin up side projects nobody will use to get there.
Anyone who has managed people will recognise the pattern. Software engineering spent decades learning that measuring programmers by lines of code rewards verbosity, not value. Economists have a name for it, after a British central banker: when a measure becomes a target, it ceases to be a good measure. Token consumption is lines of code reborn, only now each line comes with an invoice.
The better-run firms have noticed. One e-commerce company quietly renamed its token leaderboard a “usage dashboard” and installed circuit breakers that cut off access when an individual’s daily spend spikes. The breakers caught runaway agents and infrastructure bugs that would otherwise have quietly run up the meter. Its leaders now ask a different question: not who spends the most, but whose spending produces the most. That is the right question, and it is remarkable how rarely it is asked.
II. Tokens are not intelligence
The deeper problem is not that employees game a metric. It is that the industry has quietly encouraged everyone to confuse volume with thought.
Much of the recent explosion in token counts comes from “reasoning” models, which write out long internal chains of deliberation before they answer. Sometimes this genuinely helps, especially on hard mathematical or coding problems. But researchers who have looked closely find a great deal of padding. One widely cited study asked reasoning models to add two and three. They used nearly 2,000% more tokens than conventional models to arrive at the same answer, circling back to check, re-derive and second-guess a sum a child could do. On simple problems, the authors found, less than half of the extra reasoning contributed anything to getting the right answer. Trimming the redundant deliberation cut output by nearly half while leaving accuracy almost unchanged.
Other work is less comforting still. A team of safety researchers documented what they call inverse scaling in test-time compute: on some tasks, letting models think for longer made their answers worse, as they talked themselves into distractions or spurious patterns. Researchers at a large hardware company found that reasoning models suffered a complete collapse in accuracy once puzzles passed a certain complexity, and that, oddly, they began to reason less just as problems became harder. Whatever you make of the debate these papers provoked, they demolish the simple story that more tokens buy more intelligence.
The workplace evidence points the same way. In a careful randomised trial, experienced open-source developers working on their own codebases took about 19% longer to finish tasks when allowed to use AI tools. Before starting, they expected to be 24% faster. Afterwards, they still believed they had been 20% faster. The tokens flowed, the developers felt productive, and the work took longer. A separate survey of American office workers found that 40% had received AI-generated “workslop” in the previous month: polished-looking output that pushes the real effort onto whoever receives it. Each instance took nearly two hours to untangle, costing a 10,000-person company more than $9m a year.
None of this means AI is useless. It means token consumption measures activity, not achievement. A model that answers in 200 tokens what another answers in 20,000 is not a hundred times less intelligent. In most businesses it is a hundred times cheaper and, frequently, just as right.
There is a strategic corollary too. Researchers writing in a leading management journal this week argued that, as AI gives every competitor access to the same analytical tools and the same data, distinctive strategy will increasingly depend on what they call “sticky knowledge”: insights that are difficult to articulate, transfer or separate from their context. Consider what that means for token-maxxing. Tokens are the most fungible input in business. Your rival can buy exactly the same ones, from the same suppliers, at the same price. A firm that measures its AI ambition by consumption is competing on the one dimension where it cannot be different. The advantage lies elsewhere: in the judgment of people who know a customer, a market or a regulator in ways no model has been trained on, and in the decision to point the machine at the right problem in the first place. More tokens make everyone’s analysis look alike. Sticky knowledge is what makes yours worth having.
III. Follow the incentives
It is tempting to see a conspiracy here: a cabal in Silicon Valley inflating a meaningless number to justify the largest capital-spending spree in corporate history. The truth is less cinematic and more durable. There is no need for a conspiracy when the incentives line up this neatly.
Start with the business model. AI providers sell tokens. Every redundant step a reasoning model takes, every agent that loops, retries or re-reads its own context, is revenue. A supplier paid by volume has little reason to make its product more frugal, just as a taxi meter has no interest in the shortest route. The most efficient answer is the least profitable one.
Then consider the capital. The largest technology firms are spending sums approaching $400bn a year on data centres. Investors want evidence that demand will fill them. Tokens processed is the most flattering metric available: it grows exponentially, it is hard to audit from outside, and it conflates very different things. A quadrillion tokens can include a paying customer’s legal analysis, a free user’s idle chat, an engineer’s leaderboard-chasing and machine-generated summaries that appear whether or not anyone asked for them. Nobody outside these firms knows the mix, and the firms have little reason to publish it.
The numbers are also less smooth than the keynote slides suggest. One venture investor who tracked the published figures noted that monthly growth in one giant’s token volume slowed by more than half in the second half of last year, whether because of capacity limits, rationing for free users or genuine efficiency gains. Any of those would be a reasonable business outcome. None fits the narrative of limitless, exponential demand.
Finally, there is the money that goes round in circles. Last year the dominant chipmaker said it would invest up to $100bn in the leading AI lab, which in turn buys enormous quantities of its chips, often through cloud providers in which the chipmaker also holds stakes. Analysts at respected brokerages described a whiff of circular financing and bubble-like behaviour. Students of the dot-com era will remember telecom-equipment makers that lent their customers the money to buy their own kit, booking the sales as growth until the customers failed. I do not claim that history is repeating exactly. I do claim that when suppliers fund their own demand and then cite that demand as proof of the market, a board should ask hard questions before treating token growth as validation.
The vendor market has the same disease. One leading research firm estimates that of the thousands of companies selling “agentic AI”, only around 130 offer anything genuinely agentic; the rest are rebadged chatbots and automation tools, a practice it calls agent-washing. The same firm expects more than 40% of agentic AI projects to be cancelled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls.
Incentives this strong rarely correct themselves, which is why the argument is migrating from boardrooms to legislatures. This week a distinguished legal scholar and one of the pioneers of deep learning wrote, in a leading weekly, that AI scientists have warned for years that they are building technologies they do not know how to control, and that America’s Congress has largely ignored them. Their case is that lawmakers can reduce serious risks without surrendering their country’s technological lead. I would add a corollary from the economics above: a market that rewards volume will not, on its own, reward restraint. When the meter runs on tokens, safety testing, verification and frugality all look like lost revenue. That is precisely why the guardrails have to come from outside the meter, from regulators, auditors and customers who pay for outcomes rather than consumption.
IV. The nine costs nobody budgeted
If tokens overstate the benefits of AI, business cases understate its costs. Most AI proposals I see price the model, the licence and perhaps the integration work. A checklist circulating among practitioners this year lists nine further costs that almost none of them include:
| Hidden cost | When it shows up | Who ends up owning it |
|---|---|---|
| Hallucination remediation | A chatbot states a wrong policy or price | Customer operations |
| Retraining and revalidation | Seasons, product ranges or data drift change the model | Data and machine learning |
| Human review labour | Oversight is required before an AI decision stands | Business operations |
| Evaluation and red-teaming | Before launch, and after every model or prompt change | Engineering and risk |
| Monitoring and drift detection | Performance decays after go-live | Machine-learning platform |
| Explainability and documentation | A customer, auditor or regulator asks “why?” | Product and compliance |
| Audit and certification | Customers or regulators want independent assurance | Risk and audit |
| Incident response | An output harms a customer or breaks a rule | Legal and security |
| Inference and token overruns | Agents loop, retry or scale without a spending cap | Finance |
Three things stand out. First, these are not one-off costs. Most recur with every model update, every regulatory change and every new use. Second, the last line is where token-maxxing meets the balance sheet: an agent with no spending cap is a meter with no off switch. Third, look at the right-hand column. Nearly every cost is owned by people: in operations, risk, compliance, legal and finance. The pitch for AI often promises to take humans out of the loop. The bill quietly puts them back in, usually in more senior and expensive roles.
This helps explain one of the most striking findings of the past year. A study from a leading American university’s AI initiative concluded that about 95% of corporate generative-AI pilots produced no measurable financial return. The authors blamed a “learning gap” between the tools and the workflows they were meant to improve, not the quality of the models. Put differently, firms bought the tokens and skipped the work. The pilots generated enormous volumes of output and almost no value.
The checklist ends with a line every chief financial officer should pin above the desk: every line is a decision someone makes later, or a bill someone receives later.
V. What boards should measure instead
The answer to token-maxxing is not token austerity. Some problems deserve a model that thinks for a long time, and a firm that starves its people of good tools will lose to one that does not. The answer is to stop treating consumption as a proxy for value, and to measure what tokens are for. Five measures would transform most AI programmes I have seen.
- Cost per resolved outcome. Not cost per query or per token, but cost per claim settled, ticket closed, contract reviewed or line of code that survives review. This is the only number that lets a board compare AI with the alternative, including doing nothing.
- Tokens per outcome. Track the ratio over time. If it rises while quality stays flat, the organisation is paying for verbosity. Routing easy questions to small, cheap models and keeping the expensive reasoning models for genuinely hard ones routinely cuts bills by large multiples without hurting results.
- Rework rate. How often does AI output have to be corrected, redone or explained by a human downstream? This is where workslop hides, and it is usually invisible because the cost lands on a different team’s budget.
- Total cost of ownership. Every business case should carry the nine hidden lines above, with a named owner for each. If no one will put their name to the cost of incident response or human review, the project is not ready.
- Hard spending limits. Every agent should run with a budget, a circuit breaker and an alert, set by someone accountable for the spend. An agent that can spend without limit is not autonomous. It is unsupervised.
Above all, boards should stop rewarding the wrong behaviour. Retire the leaderboards. Do not set minimum consumption targets for staff. Ask managers what their teams’ AI use has changed, not how much of it there was. And when a supplier or an executive presents a chart of rising token volumes, ask the only question that matters: what did all that thinking actually produce?
The industry has given us a remarkable technology and a misleading way to measure it. A token is to intelligence what a kilowatt-hour is to a factory: an input, a cost, occasionally a sign of activity, never by itself a measure of output. Nobody judges a manufacturer by how much electricity it burns. We should stop judging AI by how many tokens it consumes, before the bill teaches us the lesson more expensively.
Leave a Reply