Cached Token Pricing

Cached Token Pricing is the set of rates a large language model (LLM) API provider charges for input tokens served from a prompt cache. Those rates typically pair a discount for cached reads with a separate charge to create or store the cache entry, and they apply to input tokens only, leaving output token billing unchanged. Prompt caching pricing affects unit economics because it changes cost per request rather than applying a flat discount, so the effect on any workload depends on how often a cached prompt prefix is actually reused. Where a provider charges a premium to create a cache entry, a workload with low reuse can cost more with prompt caching enabled than without it.

Understanding Cached Token Pricing and Prompt Cache Billing

A prompt cache stores the processed form of a prompt prefix so that repeat requests do not reprocess those tokens from scratch. Cached Token Pricing is the commercial expression of that mechanism, since the provider charges one rate to place a prefix in the cache and a lower rate to reuse it. The underlying structure is the key-value (KV) cache a model builds while processing input, not a stored copy of the response.

Cached Token Pricing is a rate structure rather than a discount. Two or more rates replace a single input token rate, and only the cache read rate sits below the standard input price. Treating prompt caching pricing as a uniform percentage off input cost is the most common modeling error, because that framing ignores the charge incurred to create the cache entry.

Two conditions determine whether a request is billed at the cache read rate. The prompt prefix must match what was stored, exactly and from the start of the request. The cache entry must also still be within its lifetime, since cache entries expire.

Cache lifetime functions as a cost variable rather than a technical detail. Published cache lifetimes commonly range from a few minutes to about an hour, and they vary by provider and configuration. A workload that reuses a prefix every few seconds may pay to create that cache entry once, while a workload that reuses the same prefix hourly may pay to create it repeatedly.

Providers also set a minimum cacheable prefix length, commonly around a thousand tokens, below which no caching applies at all. A prompt that falls under that threshold bills entirely at the standard input rate no matter how often it repeats.

Cached Token Pricing leaves two things untouched. Output tokens bill at the standard output rate regardless of cache state, and the inference itself still runs on every request.

Cache Write, Cache Read, and Cache Storage Charges

Each billed state under prompt caching pricing behaves differently, and those differences drive the cost math.

• Cache write tokens: charged when a prompt prefix is written into the cache. Several providers price cache writes above the standard input rate. The charge recurs whenever a cache entry expires and is rebuilt, which is the variable teams most often overlook.

• Cache read tokens: charged each time a stored prompt prefix is reused within its lifetime, at a rate well below the standard input rate. Cache reads are the only state that produces savings.

• Standard input tokens: what a request costs with no cache involved. Standard input rates are the baseline any caching comparison should be measured against.

• Cache storage: billed by time on some providers for holding an explicit cache entry, charged per token per hour. Cache storage accrues whether or not the entry is ever read, so an unused cache entry is a running meter.

• Output tokens: unaffected by cache state. Prompt caching pricing never discounts generation.

Prompt caching pricing is distinct from two mechanisms it is often conflated with. Semantic caching stores a previous response and can skip the model call entirely, while prompt caching still runs inference and still bills output tokens. Batch API pricing lowers rates in exchange for delayed processing, whereas prompt caching pricing trades a charge to create the cache entry for a lower rate on reuse.

Which of these states a workload actually incurs varies by provider, which is why prompt caching pricing cannot be modeled from a single assumption. Anthropic bills cache writes above its standard input rate and cache reads far below it, with no separate storage charge. Google bills cache creation for Gemini models at the standard input rate and adds an hourly storage charge for explicit context caches, though not for implicit caching. OpenAI applied no cache write charge on earlier model generations and introduced one on later generations, so two models from the same provider can carry different caching economics.

How Cached Token Pricing Changes Cost Per Request

Cached Token Pricing connects directly to unit economics, the practice of measuring cost per unit of output rather than total spend alone. Prompt caching pricing moves cost per request, and the direction of that movement depends on how many times a cached prompt prefix is reused, not on how many requests a workload sends in total.

Cached Token Pricing is a rate optimization lever rather than a usage optimization one, since it changes the price paid per input token instead of reducing the number of tokens a workload sends. That distinction matters because the rate a team actually realizes under prompt caching pricing depends on a usage pattern, namely reuse, which sits outside the FinOps team's direct control.

The cause and effect runs in both directions. A workload that writes a cache entry and then reads it many times spreads the write charge across many discounted reads, which lowers average cost per request. A workload that writes a cache entry and reads it once or twice pays that write charge without recovering it, which raises average cost per request.

The second case is the one teams miss. Enabling prompt caching pricing on a low-reuse workload increases cost per request rather than reducing it.

Total AI spend is the wrong metric for evaluating a caching decision. Total spend can fall while cost per request rises, if request volume dropped for unrelated reasons. Total spend can also rise while cost per request improves, if a feature grew. Only the per-request view isolates the effect of the caching decision itself.

Cached Token Pricing also complicates forecasting. A forecast built on a single blended input token rate drifts once prompt caching is enabled, because the effective input token rate then depends on cache hit rate. Cache hit rate depends on product usage patterns, which shift as adoption changes, so a rate that held last quarter may not hold next quarter.

Evaluating any of this requires cost visibility at the token level. Provider APIs report cached and uncached token counts separately in response metadata, which is what makes a measured cache hit rate possible. A team reviewing only invoice totals cannot separate a cache write charge from a standard input charge, and therefore cannot tell whether prompt caching pricing is helping or hurting.

Evaluating Whether Prompt Caching Pricing Pays Off

Break-even under prompt caching pricing is a reuse question. The charge to create a cache entry is recovered only after enough cache reads have occurred at the discounted rate to offset it. Below that reuse count, prompt caching pricing increases the cost of the workload.

Four inputs determine where the break-even point sits:

• The size of the cacheable prompt prefix, since both the charge and the discount scale with it

• The number of cache reads per cache entry before that entry expires

• The provider's cache write and cache read rates, plus any cache storage charge

• The cache lifetime, which caps how long an entry remains reusable

The second input is the one teams assume rather than measure. A projected saving built on an assumed cache hit rate is an estimate, not a result.

Several workload shapes rarely recover the charge. Long prompt prefixes reused only once per session leave too few reads to amortize against. Low-traffic features where requests arrive further apart than the cache lifetime rebuild the cache entry each time. Prefixes that change frequently invalidate the cache before reuse accumulates.

The practical step for a FinOps team is to request two measured figures before accepting a projected saving: the cache hit rate and the number of cache reads per cache entry. Both should be re-checked after any shift in product usage, since a caching decision that paid off at one traffic pattern may not pay off at another.

Prompt caching pricing sits at the LLM API layer, which is separate from the infrastructure layer a team provisions itself. Tools like Infracost estimate the cost of cloud infrastructure defined in Terraform before it is deployed, surfacing the cost difference in a CI/CD pull request. That covers resources a team provisions directly, such as the compute behind a self-hosted model, rather than a third-party provider's token billing.

Related Concepts

Cost Per Token: the per-token rate structure that Cached Token Pricing modifies for input tokens served from a prompt cache.

Token Economics: the broader cost behavior of per-token LLM API billing, of which Cached Token Pricing is one rate mechanism.

AI Cost Optimization: the practice of reducing AI spend, within which prompt caching pricing is one lever whose payoff depends on reuse rate.

LLM Observability: the measurement layer that produces the cache hit rate and per-request token counts needed to evaluate Cached Token Pricing.

AI Cost Governance: the policy layer that determines whether caching decisions are reviewed against measured reuse rather than assumed savings.

Frequently Asked Questions (FAQs)

What is cached token pricing?

Cached Token Pricing is the set of rates a large language model (LLM) API provider charges for input tokens served from a prompt cache. Cached Token Pricing typically combines a discounted rate for reused tokens with a separate charge for creating or storing the cache entry. Cached Token Pricing applies to input tokens only and does not change output token billing.

Do cache writes cost more than regular input tokens?

On several providers, cache writes are billed above the standard input token rate, while on others there is no separate cache write charge at all. The structure varies by provider and by model, so cache write pricing should be confirmed against the provider's current pricing page. Where a write premium applies, prompt caching pricing produces savings only once enough cache reads offset that premium.

Does prompt caching reduce output token costs?

Prompt caching pricing does not reduce output token costs. Cached Token Pricing applies only to input tokens, and output tokens bill at the provider's standard output rate whether or not a request hit the prompt cache. Workloads dominated by long generated responses see little benefit from prompt caching pricing.

How many times does a prompt need to be reused for prompt caching pricing to save money?

Under prompt caching pricing, a cached prompt prefix must be read enough times at the discounted rate to offset the charge for creating the cache entry. The exact break-even count depends on the provider's cache write and cache read rates, any cache storage charge, and the cache lifetime. Teams should calculate that break-even point from measured reuse rather than assuming that enabling prompt caching produces savings.

What is the difference between prompt caching and semantic caching?

Prompt caching stores a processed prompt prefix so repeat requests skip reprocessing those input tokens, and it still runs inference and still bills output tokens. Semantic caching stores a previous response and returns it for a similar query, which can skip the model call entirely. Cached Token Pricing describes the billing structure for prompt caching rather than for semantic caching.

Can prompt caching pricing increase costs?

Prompt caching pricing can increase costs when reuse is low. If a workload pays a premium to create a cache entry and then reads that entry only once or twice before it expires, the write premium is never recovered and cost per request rises. Prompt caching pricing reduces cost only when a cached prompt prefix is read enough times to offset the cost of creating it.