OpenAI Service – Consider Using Preferred SKUs

The OpenAI Service preferred SKUs policy checks that every Azure OpenAI model deployment uses a deployment SKU your organization has approved for that workload. The SKU decides where prompts are processed and how the deployment is billed, so the wrong choice creates either a data residency gap or a bill for idle capacity. Apply this policy to every azurerm_cognitive_deployment resource in Terraform, with separate preferred SKUs for production and non-production environments.

Why This Policy Matters

How It Helps Reduce Cloud Costs

An Azure OpenAI deployment SKU sets the billing model for that deployment. Pay-per-token SKUs such as Standard, GlobalStandard, and DataZoneStandard charge only for the tokens a workload processes.

Provisioned SKUs work differently. ProvisionedManaged, GlobalProvisionedManaged, and DataZoneProvisionedManaged bill for deployed provisioned throughput units (PTUs) every hour, whether or not any request arrives.

A provisioned Azure OpenAI deployment is a capacity commitment, not a performance setting. It pays off only when traffic is high and steady enough to keep the reserved throughput busy.

A provisioned deployment created for a development environment, a pilot, or a workload with bursty traffic turns into a fixed hourly charge. The preferred SKUs policy catches that mismatch at the pull request, before the deployment exists.

The SKU also controls data processing location. Global SKUs may process prompts in any Azure region, Data Zone SKUs keep processing inside the US, EU, or Asia Pacific data zone, and Standard keeps processing inside the Azure geography of the resource.

A deployment on a Global SKU can fail a data residency requirement even when its cost is reasonable. Fixing that after launch means recreating the deployment, which interrupts every client that depends on it.

Potential Savings

Azure OpenAI SKU savings come from matching the billing model to the traffic pattern. Moving a low-traffic deployment from a provisioned SKU to a pay-per-token SKU removes the hourly PTU charge entirely and leaves only the cost of the tokens actually used.

The size of that saving depends on the model, the PTU count, and the region. The pattern is consistent: the less traffic a provisioned deployment serves, the larger the share of its bill that pays for idle capacity.

Batch SKUs add a second lever. Microsoft prices GlobalBatch and DataZoneBatch deployments at 50% less than GlobalStandard, with a 24-hour target turnaround. Non-interactive jobs such as document classification or nightly summarization rarely need real-time SKUs.

The OpenAI Service preferred SKUs policy delivers the most value in non-production subscriptions, where provisioned capacity almost never has a legitimate purpose.

Implementation Guide

Infrastructure-as-Code Example (Terraform)

The following examples show an Azure OpenAI deployment on a SKU that breaks two common organizational rules, and the corrected configuration. The rules in this example are: no provisioned SKUs outside production, and EU workloads stay in the EU data zone.

Non-compliant configuration: provisioned SKU in a development account

resource "azurerm_cognitive_account" "openai" {
  name                = "dev-openai-eu"
  location            = "swedencentral"
  resource_group_name = azurerm_resource_group.ai.name
  kind                = "OpenAI"
  sku_name            = "S0"
}

resource "azurerm_cognitive_deployment" "chat" {
  name                 = "chat"
  cognitive_account_id = azurerm_cognitive_account.openai.id

  model {
    format  = "OpenAI"
    name    = var.model_name
    version = var.model_version
  }

  sku {
    name     = "GlobalProvisionedManaged"
    capacity = var.ptu_count
  }
}

resource "azurerm_cognitive_account" "openai" {
  name                = "dev-openai-eu"
  location            = "swedencentral"
  resource_group_name = azurerm_resource_group.ai.name
  kind                = "OpenAI"
  sku_name            = "S0"
}

resource "azurerm_cognitive_deployment" "chat" {
  name                 = "chat"
  cognitive_account_id = azurerm_cognitive_account.openai.id

  model {
    format  = "OpenAI"
    name    = var.model_name
    version = var.model_version
  }

  sku {
    name     = "GlobalProvisionedManaged"
    capacity = var.ptu_count
  }
}

resource "azurerm_cognitive_account" "openai" {
  name                = "dev-openai-eu"
  location            = "swedencentral"
  resource_group_name = azurerm_resource_group.ai.name
  kind                = "OpenAI"
  sku_name            = "S0"
}

resource "azurerm_cognitive_deployment" "chat" {
  name                 = "chat"
  cognitive_account_id = azurerm_cognitive_account.openai.id

  model {
    format  = "OpenAI"
    name    = var.model_name
    version = var.model_version
  }

  sku {
    name     = "GlobalProvisionedManaged"
    capacity = var.ptu_count
  }
}

This deployment bills for every deployed PTU around the clock, in an environment that sees traffic only during working hours. This is a common source of wasted spend. The Global SKU also allows prompts to be processed outside the EU data zone.

Compliant configuration: pay-per-token SKU inside the EU data zone

resource "azurerm_cognitive_deployment" "chat" {
  name                 = "chat"
  cognitive_account_id = azurerm_cognitive_account.openai.id

  model {
    format  = "OpenAI"
    name    = var.model_name
    version = var.model_version
  }

  sku {
    name     = "DataZoneStandard"
    capacity = 10
  }
}

resource "azurerm_cognitive_deployment" "chat" {
  name                 = "chat"
  cognitive_account_id = azurerm_cognitive_account.openai.id

  model {
    format  = "OpenAI"
    name    = var.model_name
    version = var.model_version
  }

  sku {
    name     = "DataZoneStandard"
    capacity = 10
  }
}

resource "azurerm_cognitive_deployment" "chat" {
  name                 = "chat"
  cognitive_account_id = azurerm_cognitive_account.openai.id

  model {
    format  = "OpenAI"
    name    = var.model_name
    version = var.model_version
  }

  sku {
    name     = "DataZoneStandard"
    capacity = 10
  }
}

DataZoneStandard bills per token and keeps processing inside the EU data zone for an account located in an EU region. On pay-per-token SKUs, capacity sets the Tokens-Per-Minute (TPM) limit in units of 1,000, so capacity = 10 allows 10,000 TPM.

Changing the sku block name forces Terraform to destroy and recreate the Azure OpenAI deployment. Plan the change for a maintenance window, because requests to the deployment fail until the new one is ready.

Set var.model_name and var.model_version to a model and version available for the target SKU in your region. Azure OpenAI model and SKU availability differs by region, and az cognitiveservices account list-models lists what an account can deploy.

Step-by-Step Fix Instructions

1. Agree on the preferred Azure OpenAI SKUs per environment and per data residency requirement, and write them down in one shared place.

2. Find every azurerm_cognitive_deployment resource in Terraform and record the model, the sku name, and the environment.

3. For each provisioned deployment, check token usage in Azure Monitor metrics to confirm whether traffic justifies reserved PTUs.

4. Change the sku block name to a preferred SKU, and set capacity to a TPM limit that fits the workload.

5. Run terraform plan and confirm the plan shows the deployment being replaced, not modified in place.

6. Apply the change during a maintenance window and confirm the new SKU on the deployment in the Azure portal or Microsoft Foundry portal.

7. Add Infracost to your CI/CD pipeline so new or changed deployments on a non-preferred SKU get flagged in the pull request.

Best Practices

• Default shared Terraform modules to a pay-per-token SKU. Require a documented usage baseline before a provisioned SKU is allowed.

• Pick Data Zone or Standard SKUs for any workload with a data residency requirement. Reserve Global SKUs for workloads without one.

• Route non-interactive workloads to GlobalBatch or DataZoneBatch when a 24-hour turnaround is acceptable.

• Review provisioned deployments against token usage on a recurring schedule. Steady production traffic is the only good reason to keep them.

• The OpenAI Service preferred SKUs policy does not replace Azure budgets or cost alerts. SKU rules control the billing model, while budgets control total spend.

Tools and Scripts

The OpenAI Service preferred SKUs policy is available in Infracost, including in the free trial. Infracost checks Azure OpenAI deployments in Terraform code against the SKUs your organization treats as preferred.

When Infracost runs in CI/CD, it comments on the pull request when a deployment adds or switches to a non-preferred SKU, before the change merges. Platform engineers see the finding next to the code instead of on next month's invoice or in a compliance review.

Infracost also scans every connected repository and lists existing Azure OpenAI deployments that already violate the policy. FinOps and platform teams use that backlog to burn down SKU violations over time and track progress on a burndown chart.

To list deployments and their current SKU with the Azure CLI:

az cognitiveservices account deployment list \
  --name <account-name> \
  --resource-group <resource-group> \
  --query "[].{name:name, model:properties.model.name, sku:sku.name, capacity:sku.capacity}" \
  --output

az cognitiveservices account deployment list \
  --name <account-name> \
  --resource-group <resource-group> \
  --query "[].{name:name, model:properties.model.name, sku:sku.name, capacity:sku.capacity}" \
  --output

az cognitiveservices account deployment list \
  --name <account-name> \
  --resource-group <resource-group> \
  --query "[].{name:name, model:properties.model.name, sku:sku.name, capacity:sku.capacity}" \
  --output

Examples of Impact

Illustrative example: provisioned capacity in a test environment. A team copies a production Terraform module into a test subscription, including its provisioned SKU. The test deployment serves a few hundred requests a day but bills for every deployed PTU around the clock. Switching it to Standard removes the hourly charge and leaves only per-token cost.

Illustrative example: a residency gap caught before launch. A developer deploys a new model on GlobalStandard for an EU customer workload. Infracost flags the non-preferred SKU in the pull request, and the team switches to DataZoneStandard before any customer data is processed.

(These are illustrative, composite scenarios, not specific customer accounts.)

Considerations and Caveats

• Recreation on change: Changing the Azure OpenAI deployment SKU forces Terraform to destroy and recreate the deployment, which interrupts requests briefly.

• Provisioned pays off at scale: Provisioned SKUs can cost less than pay-per-token for high, steady production traffic. The OpenAI Service preferred SKUs policy targets mismatches, not provisioned SKUs as such.

• Reservations sit outside Terraform: Azure Reservations for PTUs give a term discount, but the azurerm provider cannot purchase them. Track reservation coverage separately.

• Availability varies: Not every model is offered on every SKU in every region. Check availability before standardizing a preferred SKU.

• Data at rest is separate: Microsoft states that data stored at rest stays in the designated Azure geography for all deployment types. The SKU governs where inference processing happens.

Related Policies and Concepts

• OpenAI Service: Consider using preferred Tokens-Per-Minute (TPM) limits: a companion AI-group policy that caps the capacity value on the same azurerm_cognitive_deployment resource.

• OpenAI Service: Consider using latest models: a companion AI-group policy that flags older Azure OpenAI models where newer models offer better cost efficiency.

• Azure Virtual Machines: Consider using a preferred instance type: a Compute-group policy that applies the same preferred-values approach to Azure VM sizes.

• Provisioned Throughput Units (Glossary): defines the PTU billing unit behind every provisioned Azure OpenAI SKU.

• Token Economics (Glossary): explains how per-token pricing and reserved throughput compare, which is the tradeoff this policy enforces.

Frequently Asked Questions (FAQs)

Is the OpenAI Service preferred SKUs policy supported in Infracost?

Yes. The OpenAI Service preferred SKUs policy is available in Infracost, including in the free trial. Infracost flags Azure OpenAI deployments in pull requests when they use a SKU outside your organization's preferred list.

Can the OpenAI Service preferred SKUs policy be customized?

Yes. The OpenAI Service preferred SKUs policy checks deployments against your organization's own requirements, so teams decide which SKUs count as preferred, for example Data Zone SKUs only for EU workloads or no provisioned SKUs outside production.

Does Infracost automatically fix Azure OpenAI SKU violations?

No. Infracost detects and reports Azure OpenAI deployments on a non-preferred SKU. Engineers change the SKU name in Terraform and apply the change themselves.

Which Azure OpenAI SKU should I use for a variable workload?

A pay-per-token Azure OpenAI SKU such as Standard, DataZoneStandard, or GlobalStandard fits variable workloads, because it bills only for tokens processed. Provisioned SKUs fit high, steady traffic that keeps reserved PTUs busy.

Is the OpenAI Service preferred SKUs policy cloud-agnostic?

No. The OpenAI Service preferred SKUs policy targets Azure OpenAI deployments managed with the azurerm Terraform provider. Other AI platforms use different resources and deployment options.

Does changing an Azure OpenAI deployment SKU cause downtime?

Changing the SKU name on an azurerm_cognitive_deployment resource forces Terraform to destroy and recreate the Azure OpenAI deployment. Requests fail until the new deployment is ready, so schedule the change in a maintenance window.

How often should I review Azure OpenAI deployment SKUs?

Infracost checks Azure OpenAI deployment SKUs on every pull request that adds or changes a deployment. For existing provisioned deployments, review token usage against reserved PTUs at least quarterly.