Markup on Token Cost

Markup on Token Cost is the margin applied on top of the underlying per-token cost of a large language model (LLM) API when that AI capability is resold to customers, expressed as a percentage or multiple of the provider's charge. A cost markup determines the gross margin earned on every AI request, so it sets whether an AI feature is profitable per unit of output, not merely affordable in aggregate for the current month. Because the cost base is usage-based and set by a third-party provider, a cost markup fixed at launch can be invalidated by a provider price change in either direction. Markup on Token Cost applies to AI capability resold to paying customers, unlike internal chargeback, where a platform team recovers cost from other teams at or near zero margin.

Understanding Markup on Token Cost

Markup on Token Cost and gross margin describe the same two numbers using different arithmetic. Markup is price minus cost, divided by cost. Gross margin is price minus cost, divided by price.

The distinction matters because the two produce different figures from identical inputs. A cost markup of 100 percent corresponds to a gross margin of 50 percent. Finance discussions of AI feature profitability usually refer to gross margin, while pricing discussions inside engineering more often refer to markup.

A per-token cost base behaves differently from a fixed infrastructure cost base when a price is being set. A virtual machine or managed database carries a monthly cost that stays largely constant regardless of how heavily any single customer uses the feature it supports. The cost of serving one customer through an LLM API instead moves with that customer's request volume, prompt length, and response length.

This means the cost side of the margin calculation is not a constant. Two customers paying the same subscription price can generate different token costs in the same month. A cost markup applied to an average cost per request will overstate margin on the heaviest users and understate it on the lightest.

Markup on Token Cost is distinct from cost per token. Cost per token is the measured rate an LLM API provider charges for input tokens and output tokens, which is an input to the pricing decision. Markup on Token Cost is the commercial decision applied on top of that measured rate.

Markup on Token Cost also differs from internal chargeback for AI usage. Reselling AI capability to external customers is a margin decision, where the markup produces profit above cost of goods sold. Recovering AI platform cost from internal teams is a cost recovery decision, typically set at or near zero markup. Markup on Token Cost refers specifically to the external resale case, where the objective is profit and not cost recovery.

What Sits in the Cost Base Under a Cost Markup

A cost markup is only as accurate as the cost base it is applied to. Applying Markup on Token Cost to an LLM API provider's per-token charge alone omits several cost components that a customer request actually consumes.

Cost components that belong under a cost markup include:

• Input and output token charges: Output tokens are typically priced higher per token than input tokens, and both counts move with prompt and response length.

• Cached or reused prompt segments: Some LLM API providers offer reduced rate billing for repeated context across requests, though availability and terms vary by provider.

• Embedding and retrieval cost: Retrieval-augmented generation consumes embedding generation and vector store capacity before a model call is made.

• Retries, failed calls, and safety passes: These consume billable tokens without producing a response the customer receives.

• Self-hosted or provisioned compute: Teams that host a model on their own GPU instances carry infrastructure cost instead of per-token API charges.

• Unbilled usage: Free tier, trial, and internal evaluation traffic lands on the same cost base and carries no revenue against it.

Each component has to be attributable to the feature or customer being priced, since a cost that cannot be allocated cannot be built into a cost markup with any confidence.

How Cost Markup Affects AI Unit Economics

Markup on Token Cost connects directly to unit economics, the practice of measuring cost and margin per unit of output instead of in aggregate. For AI features, the relevant units are cost per request, cost per user, and cost per feature. A cost markup sets the gross margin at each of those levels, which is where an AI feature is either sustainable at scale or not.

A cost markup set against an underestimated cost base produces a feature that loses money on every additional request. The loss grows with adoption instead of shrinking, because each new request carries the same negative margin. Volume improves the economics of fixed-cost infrastructure and worsens the economics of a mispriced usage-based feature.

Reviewing total AI spend does not surface this problem. A monthly AI bill that fits the budget can still contain unprofitable customers, because heavy users consume more tokens per dollar of revenue than light users under a fixed price structure. Aggregate spend hides the distribution that determines whether a cost markup is holding.

Setting a defensible cost markup requires token consumption attributable to the feature or customer being priced. Where a single LLM API account serves several features, per-feature token tracking is a precondition for pricing rather than a reporting convenience. Without it, cost allocation across features rests on estimates, and a cost markup built on those estimates inherits their error.

Governance determines who owns the cost markup and what triggers a review of it. A markup with no defined revision trigger is a single decision that ages against a provider free to change its rates. Naming an owner and a trigger converts Markup on Token Cost from a launch assumption into a maintained number.

Setting a Cost Markup That Survives Provider Repricing

Pricing against a fully loaded cost of serving, rather than an LLM API provider's per-token rate alone, is the change that most often turns an apparent margin into a real one. Every cost component that a request consumes belongs in cost of goods sold before any cost markup is applied.

Deciding the repricing posture in advance and recording it removes the need to decide under pressure. Two questions need answers before either event occurs: whether a provider price cut is passed through to customers or retained as margin, and whether a price increase is absorbed or repriced. A decision made after a rate change tends to damage either margin or customer trust.

The markup structure should match the repricing exposure the business can carry. Usage-linked structures transfer provider volatility to the customer and hold margin percentage steady. Fixed price structures protect the customer experience and keep the volatility internal, where it lands on margin instead.

Review triggers work better tied to events than to the calendar. A provider pricing announcement or a measured shift in tokens per request signals that the cost base has moved. A quarterly review cadence will always lag a rate change that happens mid-quarter.

Consumption controls bound the worst case alongside the markup. Output token limits cap the cost of any single response, and per-account quotas limit how far one unprofitable account can run before the next pricing review.

For the self-hosted portion of the cost base, tools like Infracost estimate the cost of infrastructure defined in Terraform, including GPU instances, in a pull request before those resources are provisioned. That estimate covers infrastructure a team provisions itself, not usage-based charges from a third-party LLM API provider. It therefore quantifies one component of the cost base under a cost markup, and token pricing has to be measured separately.

Related Concepts

Cost Per Token: The measured per-token charge from an LLM API provider that forms the cost base a cost markup is applied to.

Token Economics: The usage-driven cost behavior of per-token LLM API pricing, which makes a fixed cost markup difficult to hold over time.

AI Cost Governance: The policy and accountability layer that determines who owns a cost markup and what triggers a revision of it.

AI FinOps: The broader practice of managing AI spend, within which Markup on Token Cost is the revenue-side decision instead of a cost reduction tactic.

Opportunity Cost: The value forgone by committing budget to one AI feature rather than another, which bears on whether a thin cost markup is worth defending.

Frequently Asked Questions (FAQs)

What is markup on token cost?

Markup on Token Cost is the margin applied on top of the underlying per-token cost of a large language model (LLM) API when that AI capability is resold to customers. Markup on Token Cost is expressed as a percentage or multiple of the provider's charge. Markup on Token Cost determines the gross margin earned on each AI request.

What is the difference between markup and gross margin on AI features?

Markup on Token Cost is calculated as price minus cost divided by cost, while gross margin is calculated as price minus cost divided by price. The two figures differ from identical inputs, so a cost markup of 100 percent corresponds to a gross margin of 50 percent. Finance discussions of AI feature profitability usually refer to gross margin rather than to markup.

How do you calculate markup on token cost?

Markup on Token Cost is calculated by subtracting the fully loaded cost of serving a request from the price charged for it, then dividing that result by the cost. The cost figure should include input and output token charges, embedding and retrieval cost, retries, and any self-hosted compute, rather than the provider's per-token rate alone. A cost markup calculated against an incomplete cost base overstates margin.

What determines an appropriate markup on LLM API costs?

An appropriate Markup on Token Cost is determined by the fully loaded cost of serving, the pricing structure chosen, and how much provider repricing exposure the business can absorb. Markup on Token Cost also depends on whether costs can be allocated to the feature or customer being priced, since an unallocated cost cannot be recovered reliably. No single markup figure applies across AI features, because cost bases and pricing structures differ.

What happens to markup on token cost when an LLM provider lowers its prices?

When an LLM API provider lowers its prices, the effect on Markup on Token Cost depends on the pricing structure in use. Under a fixed price per unit of output, the cost markup and the gross margin widen automatically, since the customer price is unchanged. Under a structure that tracks measured token cost, the markup percentage holds and the customer's bill falls instead.

Is markup on token cost the same as internal chargeback for AI usage?

Markup on Token Cost is not the same as internal chargeback for AI usage. Markup on Token Cost applies a margin when AI capability is resold to external customers, while internal chargeback recovers AI platform cost from other teams inside the same organization, typically at or near zero markup. Both mechanisms allocate token cost, but only Markup on Token Cost is a pricing decision intended to produce profit.