Glossary Categories
Observability
Observability is the ability to understand a system's internal state from the data it emits externally, including metrics, logs, traces, and profiles. The same telemetry also records capacity consumed and requests served, so observability can attribute cloud spend to a specific service or code path, which a monthly bill cannot. Observability is broader than monitoring, which watches a fixed set of signals for conditions defined ahead of time. Observability describes systems that are already running, which is what separates observability from pre-deployment cost estimation.
AI FinOps
AI FinOps is the practice of applying the FinOps operating model, including its cross-functional collaboration, phases, and capabilities, to artificial intelligence and machine learning (AI/ML) spend. AI spend is created by decisions distributed across engineering, product, and finance, which leaves traditional cloud cost governance without a clear owner for the fastest-moving part of the bill. FinOps for AI covers both the infrastructure a team provisions for model training and the usage-billed inference services it consumes, and it describes the operating model rather than the cost data itself. The FinOps Foundation recognizes FinOps for AI as a formal Technology Category within the FinOps Framework, alongside public cloud, SaaS, data center, and data cloud platforms.
AI Cost Optimization
AI Cost Optimization is the set of techniques used to reduce waste and improve efficiency in the cost of training, fine-tuning, and running artificial intelligence and machine learning workloads. AI Cost Optimization maps to the optimization dimension of FinOps practice, serving as the applied, technical counterpart to the visibility, allocation, and governance work that AI Cost Management coordinates. The techniques span training-side compute decisions, such as instance type and capacity selection, and inference-side usage decisions, such as model routing and prompt design. AI Cost Optimization is one pillar within the broader AI Cost Management practice, not a synonym for it.
Cached Token Pricing
Cached Token Pricing is the set of rates a large language model (LLM) API provider charges for input tokens served from a prompt cache. Those rates typically pair a discount for cached reads with a separate charge to create or store the cache entry, and they apply to input tokens only, leaving output token billing unchanged. Prompt caching pricing affects unit economics because it changes cost per request rather than applying a flat discount, so the effect on any workload depends on how often a cached prompt prefix is actually reused. Where a provider charges a premium to create a cache entry, a workload with low reuse can cost more with prompt caching enabled than without it.
Energy Per Token
Energy per token is the amount of electricity consumed to process a single token through a large language model (LLM), typically expressed in joules or watt-hours per token. Energy per token is the physical cost basis underneath per-token pricing, which makes energy per token the constraint on how far the cost of serving a token can fall over the long run. As used in cloud cost management, energy per token refers to inference, the recurring energy of serving a deployed model, rather than the one-time energy of training it. Energy per token is also not a fixed property of a model, varying with hardware generation, batch size, and whether a token is an input token processed during prefill or an output token generated during decode.
Opportunity Cost
Opportunity cost is the value of the next-best alternative given up when a resource such as cloud budget, engineering time, or commitment spend is allocated to one option instead of another. In cloud cost management, opportunity cost matters because optimization is a comparison between options rather than the reduction of a single number, and the value of the rejected option is the missing half of that comparison. Opportunity cost is an implicit cost, so it never appears on a cloud invoice or in a billing report. In cloud environments, the resources being traded off include compute capacity, engineering time, and the flexibility given up by committing to a discounted rate over a fixed term.
GPU Utilization
GPU utilization is the percentage of time over a sampling period during which one or more kernels were executing on a graphics processing unit (GPU), as reported by NVIDIA's nvidia-smi command line tool and by NVIDIA Data Center GPU Manager (DCGM). Low GPU utilization is one of the most expensive forms of cloud waste, because a GPU instance bills at its full hourly rate whether or not the GPU is doing work. The metric measures time spent executing, not how much of the GPU's compute capacity that execution used. A workload can therefore report high GPU utilization while using a small share of the compute available on the device.
Terraform Cost Estimation
Terraform Cost Estimation is the practice of calculating the projected cloud cost of infrastructure defined in Terraform configuration before that infrastructure is provisioned, by mapping each declared resource to cloud provider pricing data. Terraform cost estimation connects to cost visibility by moving cost information to the point where infrastructure code is written and reviewed, so a cost increase appears in a pull request rather than on the following month's invoice. A Terraform cost estimate is a projection built from declared resource attributes and assumed usage, not a billed amount. Usage-driven services such as AWS Lambda and Amazon S3 require usage assumptions before a monthly figure can be produced.
Cost Per Token
Cost per token is the unit price charged for processing a single token of text through a large language model (LLM), typically quoted per million tokens and priced separately for input and output. Cost per token is the base rate behind every downstream AI cost metric, including cost per request, cost per user, and cost per feature. The published list price for a model and the effective rate an organization actually pays are different numbers, and a forecast built on the wrong one will understate AI spend. Cost per token applies to third-party LLM API billing, where the rate is charged directly, and to self-hosted models, where the rate is derived from infrastructure cost rather than billed by a vendor.
GPU Utilization
GPU utilization is the percentage of time over a sampling period during which one or more kernels were executing on a graphics processing unit (GPU), as reported by NVIDIA's nvidia-smi command line tool and by NVIDIA Data Center GPU Manager (DCGM). Low GPU utilization is one of the most expensive forms of cloud waste, because a GPU instance bills at its full hourly rate whether or not the GPU is doing work. The metric measures time spent executing, not how much of the GPU's compute capacity that execution used. A workload can therefore report high GPU utilization while using a small share of the compute available on the device.
Energy Per Token
Energy per token is the amount of electricity consumed to process a single token through a large language model (LLM), typically expressed in joules or watt-hours per token. Energy per token is the physical cost basis underneath per-token pricing, which makes energy per token the constraint on how far the cost of serving a token can fall over the long run. As used in cloud cost management, energy per token refers to inference, the recurring energy of serving a deployed model, rather than the one-time energy of training it. Energy per token is also not a fixed property of a model, varying with hardware generation, batch size, and whether a token is an input token processed during prefill or an output token generated during decode.
AI FinOps
AI FinOps is the practice of applying the FinOps operating model, including its cross-functional collaboration, phases, and capabilities, to artificial intelligence and machine learning (AI/ML) spend. AI spend is created by decisions distributed across engineering, product, and finance, which leaves traditional cloud cost governance without a clear owner for the fastest-moving part of the bill. FinOps for AI covers both the infrastructure a team provisions for model training and the usage-billed inference services it consumes, and it describes the operating model rather than the cost data itself. The FinOps Foundation recognizes FinOps for AI as a formal Technology Category within the FinOps Framework, alongside public cloud, SaaS, data center, and data cloud platforms.
Cost Per Token
Cost per token is the unit price charged for processing a single token of text through a large language model (LLM), typically quoted per million tokens and priced separately for input and output. Cost per token is the base rate behind every downstream AI cost metric, including cost per request, cost per user, and cost per feature. The published list price for a model and the effective rate an organization actually pays are different numbers, and a forecast built on the wrong one will understate AI spend. Cost per token applies to third-party LLM API billing, where the rate is charged directly, and to self-hosted models, where the rate is derived from infrastructure cost rather than billed by a vendor.
Cached Token Pricing
Cached Token Pricing is the set of rates a large language model (LLM) API provider charges for input tokens served from a prompt cache. Those rates typically pair a discount for cached reads with a separate charge to create or store the cache entry, and they apply to input tokens only, leaving output token billing unchanged. Prompt caching pricing affects unit economics because it changes cost per request rather than applying a flat discount, so the effect on any workload depends on how often a cached prompt prefix is actually reused. Where a provider charges a premium to create a cache entry, a workload with low reuse can cost more with prompt caching enabled than without it.
Observability
Observability is the ability to understand a system's internal state from the data it emits externally, including metrics, logs, traces, and profiles. The same telemetry also records capacity consumed and requests served, so observability can attribute cloud spend to a specific service or code path, which a monthly bill cannot. Observability is broader than monitoring, which watches a fixed set of signals for conditions defined ahead of time. Observability describes systems that are already running, which is what separates observability from pre-deployment cost estimation.
Terraform Cost Estimation
Terraform Cost Estimation is the practice of calculating the projected cloud cost of infrastructure defined in Terraform configuration before that infrastructure is provisioned, by mapping each declared resource to cloud provider pricing data. Terraform cost estimation connects to cost visibility by moving cost information to the point where infrastructure code is written and reviewed, so a cost increase appears in a pull request rather than on the following month's invoice. A Terraform cost estimate is a projection built from declared resource attributes and assumed usage, not a billed amount. Usage-driven services such as AWS Lambda and Amazon S3 require usage assumptions before a monthly figure can be produced.
Opportunity Cost
Opportunity cost is the value of the next-best alternative given up when a resource such as cloud budget, engineering time, or commitment spend is allocated to one option instead of another. In cloud cost management, opportunity cost matters because optimization is a comparison between options rather than the reduction of a single number, and the value of the rejected option is the missing half of that comparison. Opportunity cost is an implicit cost, so it never appears on a cloud invoice or in a billing report. In cloud environments, the resources being traded off include compute capacity, engineering time, and the flexibility given up by committing to a discounted rate over a fixed term.
AI Cost Optimization
AI Cost Optimization is the set of techniques used to reduce waste and improve efficiency in the cost of training, fine-tuning, and running artificial intelligence and machine learning workloads. AI Cost Optimization maps to the optimization dimension of FinOps practice, serving as the applied, technical counterpart to the visibility, allocation, and governance work that AI Cost Management coordinates. The techniques span training-side compute decisions, such as instance type and capacity selection, and inference-side usage decisions, such as model routing and prompt design. AI Cost Optimization is one pillar within the broader AI Cost Management practice, not a synonym for it.
Cached Token Pricing
Cached Token Pricing is the set of rates a large language model (LLM) API provider charges for input tokens served from a prompt cache. Those rates typically pair a discount for cached reads with a separate charge to create or store the cache entry, and they apply to input tokens only, leaving output token billing unchanged. Prompt caching pricing affects unit economics because it changes cost per request rather than applying a flat discount, so the effect on any workload depends on how often a cached prompt prefix is actually reused. Where a provider charges a premium to create a cache entry, a workload with low reuse can cost more with prompt caching enabled than without it.
GPU Utilization
GPU utilization is the percentage of time over a sampling period during which one or more kernels were executing on a graphics processing unit (GPU), as reported by NVIDIA's nvidia-smi command line tool and by NVIDIA Data Center GPU Manager (DCGM). Low GPU utilization is one of the most expensive forms of cloud waste, because a GPU instance bills at its full hourly rate whether or not the GPU is doing work. The metric measures time spent executing, not how much of the GPU's compute capacity that execution used. A workload can therefore report high GPU utilization while using a small share of the compute available on the device.
Observability
Observability is the ability to understand a system's internal state from the data it emits externally, including metrics, logs, traces, and profiles. The same telemetry also records capacity consumed and requests served, so observability can attribute cloud spend to a specific service or code path, which a monthly bill cannot. Observability is broader than monitoring, which watches a fixed set of signals for conditions defined ahead of time. Observability describes systems that are already running, which is what separates observability from pre-deployment cost estimation.
Energy Per Token
Energy per token is the amount of electricity consumed to process a single token through a large language model (LLM), typically expressed in joules or watt-hours per token. Energy per token is the physical cost basis underneath per-token pricing, which makes energy per token the constraint on how far the cost of serving a token can fall over the long run. As used in cloud cost management, energy per token refers to inference, the recurring energy of serving a deployed model, rather than the one-time energy of training it. Energy per token is also not a fixed property of a model, varying with hardware generation, batch size, and whether a token is an input token processed during prefill or an output token generated during decode.
Terraform Cost Estimation
Terraform Cost Estimation is the practice of calculating the projected cloud cost of infrastructure defined in Terraform configuration before that infrastructure is provisioned, by mapping each declared resource to cloud provider pricing data. Terraform cost estimation connects to cost visibility by moving cost information to the point where infrastructure code is written and reviewed, so a cost increase appears in a pull request rather than on the following month's invoice. A Terraform cost estimate is a projection built from declared resource attributes and assumed usage, not a billed amount. Usage-driven services such as AWS Lambda and Amazon S3 require usage assumptions before a monthly figure can be produced.
AI FinOps
AI FinOps is the practice of applying the FinOps operating model, including its cross-functional collaboration, phases, and capabilities, to artificial intelligence and machine learning (AI/ML) spend. AI spend is created by decisions distributed across engineering, product, and finance, which leaves traditional cloud cost governance without a clear owner for the fastest-moving part of the bill. FinOps for AI covers both the infrastructure a team provisions for model training and the usage-billed inference services it consumes, and it describes the operating model rather than the cost data itself. The FinOps Foundation recognizes FinOps for AI as a formal Technology Category within the FinOps Framework, alongside public cloud, SaaS, data center, and data cloud platforms.
Opportunity Cost
Opportunity cost is the value of the next-best alternative given up when a resource such as cloud budget, engineering time, or commitment spend is allocated to one option instead of another. In cloud cost management, opportunity cost matters because optimization is a comparison between options rather than the reduction of a single number, and the value of the rejected option is the missing half of that comparison. Opportunity cost is an implicit cost, so it never appears on a cloud invoice or in a billing report. In cloud environments, the resources being traded off include compute capacity, engineering time, and the flexibility given up by committing to a discounted rate over a fixed term.
Cost Per Token
Cost per token is the unit price charged for processing a single token of text through a large language model (LLM), typically quoted per million tokens and priced separately for input and output. Cost per token is the base rate behind every downstream AI cost metric, including cost per request, cost per user, and cost per feature. The published list price for a model and the effective rate an organization actually pays are different numbers, and a forecast built on the wrong one will understate AI spend. Cost per token applies to third-party LLM API billing, where the rate is charged directly, and to self-hosted models, where the rate is derived from infrastructure cost rather than billed by a vendor.
AI Cost Optimization
AI Cost Optimization is the set of techniques used to reduce waste and improve efficiency in the cost of training, fine-tuning, and running artificial intelligence and machine learning workloads. AI Cost Optimization maps to the optimization dimension of FinOps practice, serving as the applied, technical counterpart to the visibility, allocation, and governance work that AI Cost Management coordinates. The techniques span training-side compute decisions, such as instance type and capacity selection, and inference-side usage decisions, such as model routing and prompt design. AI Cost Optimization is one pillar within the broader AI Cost Management practice, not a synonym for it.