Between March and August, Infracost’s token usage grew 4.9x while our AI bill grew 2.4x. In this 30-minute session, co-founder and CTO Alistair Scott walks through what we did about it. The obvious fixes barely helped: model routing saved 11% and compression under 1%. The bigger savings came from an internal agent that reads the code behind every model call and opens pull requests to fix waste. It has cut our product AI spend 45% so far.
What you’ll learn
Why model routing and compression did so little for our workloads
What we set up before changing anything: a tokenomics dashboard, an AI gateway, and evals
How the agent uses code, prompts, git history, and pricing data to decide what to change, then ships fixes as pull requests
The prompt caching fix that cut one feature’s cost 60%
How the agent caught a model reading the same file 74 times per run and replaced it with deterministic code
It’s an expanded version of the talk Alistair gave at Tokenomicon Amsterdam in September, for engineering and FinOps teams whose AI bill is growing faster than they’d like.
