Glossary

Cost per token is the unit price charged for processing a single token of text through a large language model (LLM), typically quoted per million tokens and priced separately for input and output. Cost per token is the base rate behind every downstream AI cost metric, including cost per request, cost per user, and cost per feature. The published list price for a model and the effective rate an organization actually pays are different numbers, and a forecast built on the wrong one will understate AI spend. Cost per token applies to third-party LLM API billing, where the rate is charged directly, and to self-hosted models, where the rate is derived from infrastructure cost rather than billed by a vendor.

FinOps tools are software platforms that give organizations visibility into cloud spending, allocate that spending to the teams and products driving it, and support decisions to optimize or govern how cloud resources get used. Cost visibility is the foundation every FinOps tool provides, and most platforms then layer on additional capabilities such as cost allocation, workload optimization, or spend governance. FinOps tools are the software category that supports the FinOps operating practice, the cross-functional discipline of managing variable cloud spend, rather than being the practice itself.

Harness engineering refers to the practices and workflows used to build, configure, and operate delivery automation on Harness, a software delivery platform, including pipeline design, environment and service configuration, approval-based governance, secrets management, and deployment reliability. Standardized pipelines with approval gates and audit trails give teams a controlled, traceable path to production, which lets delivery speed increase without a matching increase in ungoverned or unaccounted-for cloud spend. The term refers to Harness, the named software delivery platform, not the generic word for a wiring or safety harness. Harness engineering typically spans the build, test, and deployment stages of a pipeline, and the practice becomes especially important for organizations standardizing delivery across many microservices rather than a single application.

AI FinOps is the practice of applying the FinOps operating model, including its cross-functional collaboration, phases, and capabilities, to artificial intelligence and machine learning (AI/ML) spend. AI spend is created by decisions distributed across engineering, product, and finance, which leaves traditional cloud cost governance without a clear owner for the fastest-moving part of the bill. FinOps for AI covers both the infrastructure a team provisions for model training and the usage-billed inference services it consumes, and it describes the operating model rather than the cost data itself. The FinOps Foundation recognizes FinOps for AI as a formal Technology Category within the FinOps Framework, alongside public cloud, SaaS, data center, and data cloud platforms.

AI Cost Optimization is the set of techniques used to reduce waste and improve efficiency in the cost of training, fine-tuning, and running artificial intelligence and machine learning workloads. AI Cost Optimization maps to the optimization dimension of FinOps practice, serving as the applied, technical counterpart to the visibility, allocation, and governance work that AI Cost Management coordinates. The techniques span training-side compute decisions, such as instance type and capacity selection, and inference-side usage decisions, such as model routing and prompt design. AI Cost Optimization is one pillar within the broader AI Cost Management practice, not a synonym for it.

LLM Observability is the practice of monitoring, tracing, and analyzing the behavior of large language model (LLM) applications in production, including prompts, responses, token usage, latency, and output quality. It matters in a FinOps context because large language model API usage is billed per token and per request, so observing that traffic is also how teams see where inference spend is actually going, beyond confirming that an application is working correctly. LLM Observability extends general application observability with signals specific to large language models rather than replacing it, and it applies to live, in-production traffic rather than the offline test sets used in LLM evaluation. It differs most clearly from traditional Application Performance Monitoring (APM), which tracks infrastructure and service-level signals but has no equivalent for token usage, prompt content, or output quality.

Opportunity cost is the value of the next-best alternative given up when a resource such as cloud budget, engineering time, or commitment spend is allocated to one option instead of another. In cloud cost management, opportunity cost matters because optimization is a comparison between options rather than the reduction of a single number, and the value of the rejected option is the missing half of that comparison. Opportunity cost is an implicit cost, so it never appears on a cloud invoice or in a billing report. In cloud environments, the resources being traded off include compute capacity, engineering time, and the flexibility given up by committing to a discounted rate over a fixed term.

AI Cost Governance is the set of policies, approval workflows, and spend guardrails that determine how artificial intelligence (AI) and machine learning (ML) costs get authorized, monitored, and controlled within an organization. It matters in a FinOps context because AI workloads can generate cost exposure faster and less predictably than traditional infrastructure: an engineer can provision GPU capacity or call a metered large language model (LLM) API without the review a traditional infrastructure change typically requires, which is why explicit governance controls matter more for AI spend than for fixed, provisioned cloud resources. AI Cost Governance is one of four pillars of AI Cost Management, alongside cost visibility, allocation, and optimization. It is also distinct from AI Agent Governance, which covers broader accountability and decision-authority questions for autonomous AI agents beyond cost alone.

AI Cost Management is the practice of tracking, allocating, forecasting, and optimizing the costs of building and running AI and machine learning workloads, spanning model training, fine-tuning, and inference. It matters in a FinOps context because AI workloads introduce cost patterns, GPU-based training runs and usage-billed inference APIs, that are harder to see and predict than traditional, largely fixed cloud infrastructure spend, which is why cost visibility has to come before allocation, optimization, or governance can happen. AI Cost Management covers both the infrastructure a team provisions directly, such as GPU compute for training, and the usage-billed services it consumes, such as large language model (LLM) APIs, and it sits alongside general cloud cost management rather than replacing it.