Glossary

Spot and preemptible capacity refers to spare cloud compute capacity that a provider sells at a steep discount and can reclaim with little advance notice when that capacity is needed elsewhere. This pricing model gives platform teams one of the largest discount levers in cloud cost optimization, but only for workloads whose retry, checkpointing, and fallback costs don't erode the savings. AWS calls this Spot Instances, Google Cloud calls it Spot VMs (the current version of what it originally launched as Preemptible VMs), and Azure calls it Spot Virtual Machines: the same discount-for-availability trade under three different names, with different constraints on each provider.

Cloud billing data is the set of line item usage and cost records a cloud provider emits for an account, describing what was consumed, how it was metered, and what it was charged. Cloud billing data is the record layer every cost report, allocation model, and chargeback figure is derived from, so its granularity, lag, and identifier quality set a ceiling on what any downstream tool can show. An invoice is a settled financial document covering a billing period, while cloud billing data is the underlying line item record set, more detailed and still changing while the period is open. Cloud billing data records what was charged for metered consumption, not whether a resource was used efficiently, who inside the organization requested it, or what business outcome it supported.

Markup on Token Cost is the margin applied on top of the underlying per-token cost of a large language model (LLM) API when that AI capability is resold to customers, expressed as a percentage or multiple of the provider's charge. A cost markup determines the gross margin earned on every AI request, so it sets whether an AI feature is profitable per unit of output, not merely affordable in aggregate for the current month. Because the cost base is usage-based and set by a third-party provider, a cost markup fixed at launch can be invalidated by a provider price change in either direction. Markup on Token Cost applies to AI capability resold to paying customers, unlike internal chargeback, where a platform team recovers cost from other teams at or near zero margin.

COGS Composition is the breakdown of a software or AI product's cost of goods sold into its underlying direct-cost categories, typically compute, storage, data egress, and model inference, that together determine gross margin. As cloud infrastructure and AI model inference increasingly sit inside COGS rather than operating expenses (opex), COGS Composition directly shapes gross margin and unit economics, giving engineering leaders a direct stake in a number finance also watches closely. COGS Composition differs from total cloud spend, which reports a single aggregate figure with no breakdown by cost category, and the categories that make it up vary by company and product architecture.

Multi-Cloud Cost Optimization is the practice of tracking, managing, and reducing cloud spend across two or more public cloud providers, such as Amazon Web Services, Google Cloud, and Microsoft Azure, used by the same organization. It matters in a FinOps context because each provider bills, prices, and tags resources differently, which fragments spend by default and makes optimization impossible until that spend is normalized into one view. Running multiple providers does not automatically lower cost: splitting spend across providers can reduce the discount a provider grants for committed usage, and add tooling overhead that a single-provider setup avoids.

Cost per request is the fully loaded cost of serving a single call to an AI-powered feature, covering model inference, retrieval, tool calls, and the supporting infrastructure the call consumes. Cost per request connects an AI feature to unit economics by converting a variable per-token rate into a per-call figure that can be compared against the value one call produces. The term also describes per-invocation pricing for API gateways and serverless functions, but cost per request for an AI feature varies from call to call with token consumption. Any cost per request figure depends on how a request is defined, since one user-facing action can trigger several model calls in a multi-step feature.

Distributed tracing is a method of following a single request as it moves across multiple services, recording the sequence, duration, and metadata of each step so engineers can see exactly where time and resources were spent. In a microservices or serverless architecture, a request that touches a dozen services produces no single log or metric that shows its full path, and distributed tracing is what reconstructs it. The same span-level data that already shows which service made a request slow can also show which service made it expensive, once spans carry cost-relevant metadata correlated against actual infrastructure spend. Distributed tracing is an observability technique on its own; it becomes a cost visibility mechanism only when that correlation step is added deliberately.

Databricks Cost Optimization refers to the practices and configuration choices used to reduce compute, storage, and licensing spend on the Databricks Data Intelligence Platform, primarily by controlling how clusters are sized, scaled, and billed in Databricks Units (DBUs). Because Databricks charges by usage rather than a fixed footprint, the same job can cost very different amounts depending on the cluster type, the instance pricing chosen, and how long compute keeps running after the work finishes. This entry covers the compute and storage levers a platform engineer controls directly, such as cluster type, autoscaling, instance pricing, and file layout, rather than general data engineering or Apache Spark tuning practices.

Tokens per dollar is the number of large language model (LLM) tokens a given amount of spend buys, calculated as total tokens divided by total cost, and used to compare the cost efficiency of models, providers, and hosting approaches. Tokens per dollar measures text volume rather than completed work, so a higher tokens per dollar figure does not reliably mean a lower cost per request or a lower cost per task. Tokens per dollar is the inverse of cost per token, so a higher tokens per dollar figure corresponds to a lower cost per token for the same model and the same token direction. Tokens per dollar applies to third-party LLM API billing, where the figure derives from a published rate, and to self-hosted models, where the figure is the tokens produced divided by infrastructure cost.