COGS Composition

COGS Composition is the breakdown of a software or AI product's cost of goods sold into its underlying direct-cost categories, typically compute, storage, data egress, and model inference, that together determine gross margin. As cloud infrastructure and AI model inference increasingly sit inside COGS rather than operating expenses (opex), COGS Composition directly shapes gross margin and unit economics, giving engineering leaders a direct stake in a number finance also watches closely. COGS Composition differs from total cloud spend, which reports a single aggregate figure with no breakdown by cost category, and the categories that make it up vary by company and product architecture.

Understanding COGS Composition in Software and AI Products

COGS Composition describes the specific cost categories that make up the cost of goods sold (COGS) for a software or AI product, rather than treating COGS as a single lump sum. In traditional manufacturing, COGS covers materials and direct labor used to produce a physical good. For a software or AI company, the equivalent direct costs are the cloud and infrastructure expenses required to deliver the product to a paying customer.

Cloud infrastructure and AI model inference increasingly fall into this direct-cost category rather than operating expenses (opex). A cost counts as COGS when it is incurred to deliver the product a customer is already paying for, such as the compute that runs a live feature or the tokens consumed by an AI assistant embedded in the product. Opex instead covers costs that support the business more broadly, including research and development, sales and marketing, and corporate general and administrative (G&A) spend not tied to serving a specific customer.

This distinction matters because COGS Composition determines gross margin, the percentage of revenue left after subtracting the direct cost of delivering the product. A software company with a lean COGS Composition keeps more of each revenue dollar than one where compute, storage, egress, and model inference consume a larger share of that revenue.

COGS Composition is company-specific. A hosted SaaS product with predictable usage patterns often has compute and storage as its two largest categories, while a product built around AI features typically sees model inference grow into the largest or fastest-growing category as usage scales. There is no fixed ratio that applies across every company, only the shared principle that direct product-delivery costs belong in COGS while broader business costs belong in opex.

Core Cost Categories Inside COGS Composition

The categories that typically make up COGS Composition for a software or AI product include:

• Compute: the cost of the servers, containers, and GPU instances that run the product for customers. Compute cost scales with usage volume and the number of instance or GPU hours consumed, and it is usually the largest category for infrastructure-heavy products.

• Storage: the cost of databases, object storage, and backups tied directly to serving the product. Storage cost scales with the volume of data retained and how frequently that data is accessed.

• Data egress: the cost of network transfer when data moves out of the cloud provider to end users or to another service. Egress cost scales with the volume of data served or synced, and it can grow quickly for products that stream media or sync large files.

• Model inference: the per-token or per-request cost of calling a third-party AI model API or running a self-hosted model. Model inference cost scales with request volume and with the length of the prompts and responses involved, and it is the category most responsible for rising COGS Composition in AI-enabled products.

These four categories cover most of what shows up in COGS Composition, though some companies also include CDN, logging, and monitoring infrastructure that exists specifically to serve the product rather than to support internal operations.

How COGS Composition Shapes Gross Margin and FinOps Visibility

COGS Composition connects directly to gross margin, which is calculated as revenue minus COGS. A heavier or more volatile COGS Composition compresses gross margin even when revenue holds steady or grows, because a larger share of each revenue dollar goes toward compute, storage, egress, and model inference rather than staying as margin.

Rising model inference costs are pushing COGS higher for many software companies that have added AI features. This pressure behaves differently from traditional infrastructure cost growth: a fixed virtual machine or database has a cost that stays largely constant regardless of usage, while model inference cost scales directly with how often a feature is used and how long the prompts and responses are. A popular AI feature can become the largest driver of COGS Composition within months of launch.

Cost allocation is what turns COGS Composition from a lagging finance number into something engineering can act on. Attributing compute, storage, egress, and model inference cost to a specific product, feature, or customer requires tracking usage at that level of granularity. Without that allocation, a rising COGS Composition is difficult to trace back to the feature or team responsible for it, which limits accountability.

Governance also plays a role. As usage scales, someone needs to own and approve changes that move COGS Composition, whether that is a new AI feature that adds model inference cost or a product change that increases compute or egress. COGS creep often has no single accountable owner, which is why FinOps and engineering leadership increasingly review COGS Composition together rather than leaving it solely to finance.

Improving Visibility Into COGS Composition

Tagging and allocation practices are the foundation of an accurate COGS Composition. Applying consistent cost allocation tags to compute, storage, egress, and model inference resources makes it possible to trace each cost category back to a specific product, feature, or customer, turning COGS Composition from an estimate into something auditable.

Tracking cost per customer or cost per request alongside total COGS Composition gives engineering and finance a trend line to work from, rather than discovering a margin problem only when the quarterly bill arrives. This unit-level tracking makes it possible to spot a feature whose cost is growing faster than its usage or revenue before it materially affects gross margin.

Reviewing infrastructure and model choices is a direct lever on COGS Composition. Choosing reserved capacity over on-demand pricing for predictable compute workloads, or routing simple AI requests to a lower-cost model tier while reserving flagship models for tasks that need their full capability, both reduce the average cost per unit inside COGS Composition.

Catching COGS-impacting changes before they deploy is more effective than catching them after. Tools like Infracost estimate the cost of infrastructure changes defined in Terraform before they reach production, surfacing changes in pull requests that would move a product's COGS Composition ahead of the deploy rather than after finance notices the bill.

Related Concepts

Token Economics: The cost structure created by per-token pricing in LLM APIs, which is the direct driver behind the model inference line inside COGS Composition for AI-enabled products.

Opportunity Cost: The value given up by directing engineering and cloud budget toward one initiative instead of another, relevant when a company weighs building a capability in-house against buying it in a way that changes COGS Composition.

Capex vs Opex: The broader distinction between capital and operating expenses that COGS Composition sits alongside, since COGS is itself a specific category of operating cost with its own accounting treatment.

Gross and Contribution Margin: The margin metrics that COGS Composition feeds directly, showing why the category breakdown matters beyond a single aggregate COGS figure.

Cloud Cost Allocation: The practice of attributing cloud spend to teams, products, or customers, which is the mechanism that makes COGS Composition traceable rather than a single opaque total.

Frequently Asked Questions (FAQs)

What is COGS Composition?

COGS Composition is the breakdown of a software or AI product's cost of goods sold into its underlying direct-cost categories, typically compute, storage, data egress, and model inference. COGS Composition determines gross margin, since a heavier composition leaves less revenue as margin. Tracking COGS Composition by category, rather than as a single total, shows which cost driver is responsible for margin pressure.

Why does cloud infrastructure spend count as COGS instead of opex?

Cloud infrastructure spend counts as COGS when it is incurred to deliver a product a customer is already paying for, such as the compute or model inference behind a live feature. Operating expenses (opex) instead cover costs that support the business more broadly, including research and development, sales, and corporate G&A. This distinction is what places compute, storage, egress, and model inference inside COGS Composition rather than opex.

What is typically included in COGS Composition for an AI product?

COGS Composition for an AI product typically includes compute for running the application, storage for data and models, data egress for serving responses, and model inference for calling or hosting the AI model itself. Model inference is usually the fastest-growing category in COGS Composition for AI-enabled products, since it scales directly with usage. Some companies also include CDN, logging, and monitoring infrastructure that exists specifically to serve the product.

How does COGS Composition affect gross margin?

COGS Composition affects gross margin because gross margin is calculated as revenue minus COGS. A heavier or more volatile COGS Composition leaves a smaller share of revenue as margin, even when revenue itself is growing. Tracking COGS Composition by category makes it possible to identify which cost driver, such as model inference or compute, is compressing margin the most.

What is the difference between COGS Composition and total cloud spend?

COGS Composition breaks total cloud spend down into specific categories, such as compute, storage, egress, and model inference, while total cloud spend reports a single aggregate figure. COGS Composition is more useful for margin analysis because it shows which category is driving cost growth, while total cloud spend alone does not. A company can hold total cloud spend flat while COGS Composition still shifts toward a more expensive category, such as model inference.

How can engineering teams get better visibility into their COGS Composition?

Engineering teams improve visibility into COGS Composition by applying consistent cost allocation tags to compute, storage, egress, and model inference resources, then tracking cost per customer or cost per request alongside the total. Catching infrastructure changes that would move COGS Composition before they deploy, rather than after the bill arrives, keeps the breakdown current. Reviewing infrastructure and model choices, such as reserved capacity or model tier routing, gives teams a direct lever to manage COGS Composition over time.