Glossary

AI Cost Governance is the set of policies, approval workflows, and spend guardrails that determine how artificial intelligence (AI) and machine learning (ML) costs get authorized, monitored, and controlled within an organization. It matters in a FinOps context because AI workloads can generate cost exposure faster and less predictably than traditional infrastructure: an engineer can provision GPU capacity or call a metered large language model (LLM) API without the review a traditional infrastructure change typically requires, which is why explicit governance controls matter more for AI spend than for fixed, provisioned cloud resources. AI Cost Governance is one of four pillars of AI Cost Management, alongside cost visibility, allocation, and optimization. It is also distinct from AI Agent Governance, which covers broader accountability and decision-authority questions for autonomous AI agents beyond cost alone.

AI Cost Management is the practice of tracking, allocating, forecasting, and optimizing the costs of building and running AI and machine learning workloads, spanning model training, fine-tuning, and inference. It matters in a FinOps context because AI workloads introduce cost patterns, GPU-based training runs and usage-billed inference APIs, that are harder to see and predict than traditional, largely fixed cloud infrastructure spend, which is why cost visibility has to come before allocation, optimization, or governance can happen. AI Cost Management covers both the infrastructure a team provisions directly, such as GPU compute for training, and the usage-billed services it consumes, such as large language model (LLM) APIs, and it sits alongside general cloud cost management rather than replacing it.

AI workload cost management is the practice of monitoring, allocating, and optimizing cloud infrastructure expenses associated with machine learning (ML) and artificial intelligence operations. This discipline encompasses the costs of training models, running inference workloads, fine-tuning algorithms, and maintaining deployment infrastructure across GPU, TPU, and specialized accelerator hardware. Unlike general cloud cost management, AI workload…

Cost per token is the unit price charged for processing a single token of text through a large language model (LLM), typically quoted per million tokens and priced separately for input and output. Cost per token is the base rate behind every downstream AI cost metric, including cost per request, cost per user, and cost per feature. The published list price for a model and the effective rate an organization actually pays are different numbers, and a forecast built on the wrong one will understate AI spend. Cost per token applies to third-party LLM API billing, where the rate is charged directly, and to self-hosted models, where the rate is derived from infrastructure cost rather than billed by a vendor.

FinOps tools are software platforms that give organizations visibility into cloud spending, allocate that spending to the teams and products driving it, and support decisions to optimize or govern how cloud resources get used. Cost visibility is the foundation every FinOps tool provides, and most platforms then layer on additional capabilities such as cost allocation, workload optimization, or spend governance. FinOps tools are the software category that supports the FinOps operating practice, the cross-functional discipline of managing variable cloud spend, rather than being the practice itself.

Harness engineering refers to the practices and workflows used to build, configure, and operate delivery automation on Harness, a software delivery platform, including pipeline design, environment and service configuration, approval-based governance, secrets management, and deployment reliability. Standardized pipelines with approval gates and audit trails give teams a controlled, traceable path to production, which lets delivery speed increase without a matching increase in ungoverned or unaccounted-for cloud spend. The term refers to Harness, the named software delivery platform, not the generic word for a wiring or safety harness. Harness engineering typically spans the build, test, and deployment stages of a pipeline, and the practice becomes especially important for organizations standardizing delivery across many microservices rather than a single application.

AI Cost Optimization is the set of techniques used to reduce waste and improve efficiency in the cost of training, fine-tuning, and running artificial intelligence and machine learning workloads. AI Cost Optimization maps to the optimization dimension of FinOps practice, serving as the applied, technical counterpart to the visibility, allocation, and governance work that AI Cost Management coordinates. The techniques span training-side compute decisions, such as instance type and capacity selection, and inference-side usage decisions, such as model routing and prompt design. AI Cost Optimization is one pillar within the broader AI Cost Management practice, not a synonym for it.

LLM Observability is the practice of monitoring, tracing, and analyzing the behavior of large language model (LLM) applications in production, including prompts, responses, token usage, latency, and output quality. It matters in a FinOps context because large language model API usage is billed per token and per request, so observing that traffic is also how teams see where inference spend is actually going, beyond confirming that an application is working correctly. LLM Observability extends general application observability with signals specific to large language models rather than replacing it, and it applies to live, in-production traffic rather than the offline test sets used in LLM evaluation. It differs most clearly from traditional Application Performance Monitoring (APM), which tracks infrastructure and service-level signals but has no equivalent for token usage, prompt content, or output quality.

Token Economics refers to the cost structure and spending behavior created by per-token pricing in large language model (LLM) APIs, where usage is billed according to the number of input and output tokens processed in each request. This pricing model makes AI spend hard to forecast with traditional infrastructure budgeting, since token consumption, not fixed infrastructure spend, drives most of the cost variability for teams building on LLM APIs. A team's monthly bill can shift significantly from one month to the next based on how often a feature is used, not on how much infrastructure it provisions.