Ecosystem

Tokenmaxxing: Amazon Calls Token Usage a 'Deeply Flawed' Measure of AI Adoption

2 min read AI-generated

AWS vice president Marc Brooker names the replacement metric: feature velocity, how many launches land and how many changes reach production. Amazon says there are no token caps on staff.

Featured image for "Tokenmaxxing: Amazon Calls Token Usage a 'Deeply Flawed' Measure of AI Adoption"

On October 27, Amazon posted an internal memo to its own website that had been circulating in-house since September. It rewrites what a software development engineer and what a manager is supposed to be in the age of AI. CNBC read it.

What Amazon considers flawed

Token usage, the adoption rate of AI tools, and the volume of code changes or reviews are, in the memo’s own words, “deeply flawed” ways of measuring teams or organizations. The reasoning is dry: numbers like these drive cost and complexity.

They are not meant to disappear entirely. Token usage and generated code should be weighed “with care and judgment” alongside other data. The sentence the rest hangs on: “We believe that it is best to measure, and be measured by, long-term outcomes.”

What gets measured instead

Marc Brooker, a vice president and distinguished engineer at AWS, told CNBC what Amazon moved to: feature velocity. He describes it as how many launches you manage and how many successful changes reach production. His frame for it is return on investment: if a problem matters to customers or to the business, you put tools, time, tokens and human effort into it.

The interesting part is where he draws the line. What he wants to avoid is wasted effort, tokens spent on problems that do not matter. At the same time he admits you often cannot know that in advance, which is why some appetite for experimentation is needed. That is the opposite of a cap. An Amazon spokesperson confirmed there are no token caps on employees.

The most remarkable sentence sits off to the side

The memo also states plainly that AI does not help everywhere: “If your work does not benefit from current AI tooling, that is a finding, not a failing.” And an engineer who can explain where AI does not yet help their work is showing exactly the judgment being asked for. Coming from a company that sells these tools, that is an unusual line.

The metric produces the behavior it tries to measure

Amazon supplies the proof itself. In May it shut down an internal leaderboard after discovering employees were burning tokens to climb it. I wrote about the turn toward ROI a while back, when it was still industry mood. Now it is a role description inside a company with hundreds of thousands of staff. If someone on your team reports Claude Code adoption in tokens, there is now a document to point at.

Sources

AmazonTokenmaxxingAI CostsEngineering