Tokens per dollar is the number of large language model (LLM) tokens a given amount of spend buys, calculated as total tokens divided by total cost, and used to compare the cost efficiency of models, providers, and hosting approaches. Tokens per dollar measures text volume rather than completed work, so a higher tokens per dollar figure does not reliably mean a lower cost per request or a lower cost per task. Tokens per dollar is the inverse of cost per token, so a higher tokens per dollar figure corresponds to a lower cost per token for the same model and the same token direction. Tokens per dollar applies to third-party LLM API billing, where the figure derives from a published rate, and to self-hosted models, where the figure is the tokens produced divided by infrastructure cost.
How the Figure Is Computed | What It Measures | What It Hides |
|---|---|---|
Published input rate only | Tokens of prompt and context a dollar buys at the provider's listed input rate | Output tokens, which are typically billed higher per token and often dominate the bill for generative features |
Published output rate only | Tokens of generated response a dollar buys at the provider's listed output rate | Input and context cost, which grows with retrieval, conversation history, and system prompts |
Blended across a real workload | Tokens a dollar bought in practice, reflecting the team's own input to output mix and cache hit rate | The mix itself, so the figure moves with workload shape, not efficiency, and cannot be compared across teams |
Self-hosted infrastructure basis | Tokens produced over a period divided by the cost of the GPU capacity provisioned to serve them | Idle capacity, since low GPU utilization lowers tokens per dollar with no change in model, pricing, or request volume |
Billed token basis | Every token the provider charged for across the period | Retries, truncated responses, safety passes, and billed reasoning tokens, none of which reach the user |
Understanding Tokens Per Dollar as an LLM Cost Metric
Tokens per dollar and cost per token are the same measurement read in opposite directions. Tokens per dollar divides a token count by the cost of those tokens, so a higher figure indicates greater cost efficiency. Cost per token divides cost by token count, so a lower figure indicates the same thing. A comparison that mixes the two conventions will rank the options backwards.
A tokens per dollar figure can be published or measured. A published figure comes from a provider's rate card and describes a single rate in isolation. A measured figure comes from a team's own spend and token counts over a period, and reflects the input to output mix, the cache hit rate, and any retried requests in that workload. The two figures rarely match.
Tokens per dollar is underspecified until it names which token direction it counts. A large language model (LLM) API carries separate published rates for input tokens and output tokens, and output tokens are typically billed at a higher rate per token. A figure quoted on the input rate alone overstates the tokens a dollar buys for a generative feature, because the response that feature returns is billed at the output rate.
Tokens per dollar figures from two providers are not directly comparable at face value. The same text resolves to different token counts under different tokenizers, so two correctly calculated figures can describe different quantities of underlying text. Tokens per dollar therefore compares billing units, and the amount of usable output behind each unit varies by provider.
Tokens per dollar is also distinct from tokens per second. Tokens per dollar is denominated in spend, while tokens per second measures generation throughput over time. The two figures can move in opposite directions, since batching requests to raise tokens per dollar lengthens the wait for any single response.
Tokens per dollar is calculated two different ways depending on where inference runs. For a third-party LLM API, tokens per dollar derives from the provider's published per-token rate, or from measured spend and token counts over a period. For a self-hosted model running on GPU instances, tokens per dollar derives from the tokens produced in a period divided by the cost of the infrastructure provisioned in that period. Both calculations carry the same name and answer different questions.
Why Tokens Per Dollar Is a Poor Proxy for Value Per Dollar
Tokens per dollar can be calculated correctly and still point to the wrong selection. The figure counts billed text volume, while a model is selected to complete work.
Tokens measure volume, not completed work. Tokens per dollar counts text produced. Two models that answer the same question at different lengths produce different token counts for identical delivered value.
Verbosity inflates the figure. A model that answers at greater length consumes more output tokens per request. Cost per request rises while the tokens per dollar figure holds steady or looks favorable.
Accuracy changes the unit that matters. A response that is wrong, incomplete, or truncated still consumes billed tokens and produces a retry. Retried requests bill twice for one delivered answer, so a model with more tokens per dollar and a lower first-pass success rate can cost more per completed task.
Reasoning tokens are billed but not delivered. Where a provider bills the intermediate reasoning tokens a model generates, those tokens are typically charged at the output rate and raise spend without lengthening the visible response. Whether reasoning tokens appear separately in usage reporting varies by provider.
Downstream engineering time sits outside the metric. Time spent correcting or re-prompting a cheaper model is real cost that no tokens per dollar figure captures.
The comparison unit should be the completed unit of work. Cost per request, cost per task, and cost per completed outcome are the figures tokens per dollar should be checked against, not replaced by.
Tokens per dollar can therefore improve while the economics of the feature get worse.
How Tokens Per Dollar Affects AI Unit Economics
Tokens per dollar connects to unit economics, the FinOps dimension concerned with cost per unit of output, not total spend. For an AI feature, the relevant units are cost per request, cost per task, and cost per user. Selecting a model on tokens per dollar alone optimizes the wrong ratio, so total spend per completed request can rise while the tokens per dollar figure improves.
Cost per request is the per-token rate applied to the token count of a typical request. Tokens per dollar therefore constrains cost per request only while the token count per request stays constant. Once response length or retry rate changes, tokens per dollar and cost per request move independently.
Tokens per dollar carries no information about total spend, because a ratio does not describe volume. A team can improve tokens per dollar and still watch the AI bill grow, since request volume sets the total while tokens per dollar sets only the rate. A budget forecast therefore needs a projected request volume alongside the ratio.
A meaningful blended tokens per dollar figure requires spend and token counts separated by model and by feature over the same window. Where a single LLM API account serves several features, a fleet-wide tokens per dollar figure mostly measures model mix. A shift in traffic toward a cheaper model tier will move that figure even when no individual feature became more efficient.
Cost allocation depends on the same per-feature token counts. The data that makes tokens per dollar meaningful is the data that makes AI spend attributable to a team or a product, so a team that cannot compute the metric per feature also cannot trace a rising AI bill to its source.
Tokens per dollar works as a screening metric that narrows a candidate set, not as a decision metric that picks from it. A shortlist built on tokens per dollar still has to be tested on cost per completed task before a selection holds.
Comparing Models and Hosting Options With Tokens Per Dollar
A tokens per dollar comparison holds only when both sides are calculated on the same basis.
Normalize before comparing. Compare on the same token direction, the same prompt set, the same output length cap, and the same input to output ratio. Two figures computed on different workloads are not a comparison.
Measure on a representative prompt set, not a published figure. Because tokenizers differ, the reliable test is total spend for the same set of real requests run through each candidate.
Pair tokens per dollar with a first-pass success rate. The share of requests that succeed without a retry converts tokens per dollar into something closer to cost per completed task. Either figure alone is misleading.
Tie the review trigger to a workload change, not the calendar. A shift in response length, a larger retrieval context, or a drop in cache hit rate all move a measured tokens per dollar figure with no provider price change.
Self-hosted inference makes tokens per dollar a different calculation. The figure becomes the tokens produced divided by the cost of the provisioned infrastructure, which means GPU utilization, batching, and idle capacity move the figure directly, since no published rate is involved. Idle GPU capacity lowers tokens per dollar with no change in model or request volume, because cost keeps accruing while token output does not grow.
For that self-hosted case, tools like Infracost estimate the cost of infrastructure defined in Terraform, including GPU instances, and post the estimate as a pull request comment before those resources are provisioned. That gives a team the cost side of a self-hosted tokens per dollar calculation ahead of deployment, before the first invoice arrives. The estimate covers provisioned infrastructure defined in code, not the usage-based token charges billed by a third-party LLM API provider, so token spend has to be measured separately.
Related Concepts
Cost Per Token: The per-token rate that tokens per dollar inverts, and the figure most published pricing pages quote instead.
Token Economics: The cost behavior created by per-token LLM API billing, within which tokens per dollar is one derived ratio.
Energy Per Token: The same model output denominated in energy, not dollars, which moves with hardware efficiency rather than pricing.
AI Cost Optimization: The broader practice of reducing AI spend, within which tokens per dollar is a screening metric and not a target to maximize.
LLM Observability: The practice that supplies the per-model, per-feature token and spend data a measured tokens per dollar figure depends on.
Frequently Asked Questions (FAQs)
What is tokens per dollar?
Tokens per dollar is the number of large language model (LLM) tokens a given amount of spend buys, calculated as total tokens divided by total cost. Tokens per dollar is used to compare the cost efficiency of models, providers, and hosting approaches. Tokens per dollar measures billed text volume rather than completed work.
How is tokens per dollar calculated?
Tokens per dollar is calculated by dividing a token count by the cost of those tokens over the same period. For a third-party LLM API, tokens per dollar uses the provider's published per-token rate for a stated token direction, or measured spend and token counts from a real workload. For a self-hosted model, tokens per dollar uses the tokens produced divided by the cost of the infrastructure serving them.
What is the difference between tokens per dollar and cost per token?
Tokens per dollar and cost per token are reciprocals of the same two numbers. Cost per token divides cost by token count and reads lower-is-better, while tokens per dollar divides token count by cost and reads higher-is-better. A higher tokens per dollar figure corresponds to a lower cost per token for the same model and the same token direction.
Does a model with more tokens per dollar always cost less to run?
A model with more tokens per dollar does not always cost less to run. Tokens per dollar counts billed tokens, not completed tasks, so a model that answers at greater length, retries more often, or generates billed reasoning tokens can cost more per completed request. Comparing tokens per dollar alongside a first-pass success rate gives a more reliable picture than tokens per dollar alone.
Why can tokens per dollar not be compared directly between providers?
Tokens per dollar cannot be compared directly between providers because the same text resolves to different token counts under different tokenizers. Two tokens per dollar figures can therefore describe different quantities of underlying text even when both are calculated correctly. Running the same prompt set through each candidate and comparing total spend avoids the problem.
How is tokens per dollar calculated for a self-hosted model?
Tokens per dollar for a self-hosted model is calculated by dividing the tokens produced over a period by the cost of the infrastructure provisioned to serve them in that period. Because the cost side is provisioned capacity instead of a billed per-token rate, tokens per dollar moves with GPU utilization and batching. Idle GPU capacity lowers tokens per dollar with no change in model or request volume.
Prevent Cloud Budget
Overruns Earlier
Download the whitepaper to see how teams shift FinOps left and add cost guardrails in pull requests.