OpenAI Service – Consider Using Latest Models

Oct 9, 2026

Infracost

OpenAI Service – Consider Using Latest Models

Oct 9, 2026

Infracost

The OpenAI Service latest models policy checks that every Azure OpenAI deployment uses a model on your organization's preferred list of current, cost-efficient OpenAI models. Older models often cost more per token than their successors, and every model version eventually retires and stops serving requests. Apply this policy to every Azure OpenAI deployment defined in Terraform, and update the preferred list whenever Microsoft publishes new models or retirement dates.

Attribute	Detail
Cloud Provider	Microsoft Azure (Azure OpenAI)
Resource Type	Azure OpenAI model deployment (`azurerm_cognitive_deployment`)
Terraform Attribute	`name` and `version` in the `model` block
Compliant Value	A model name that matches your preferred list, for example a pattern such as `gpt-5*`
Cost Impact	Lower per-token rates on newer models, and no forced migration when an old model version retires

Why This Policy Matters

How It Helps Reduce Cloud Costs

Azure OpenAI Standard and Global Standard deployments bill for the input and output tokens each request processes. The model name on the deployment sets the per-token rate, so model choice is the biggest single lever on Azure OpenAI spend.

Newer OpenAI models have repeatedly launched at lower per-token prices than the flagship models they replaced. On OpenAI's API, gpt-4 launched at $30 per million input tokens and $60 per million output tokens, while gpt-4o launched at $5 and $15.

A deployment pinned to an old model keeps paying the old model's rate long after a cheaper option exists. Nothing in Azure flags that gap, because the old deployment keeps working until its retirement date.

Older Azure OpenAI models also carry a reliability cost. Microsoft documents an 18-month lifecycle for generally available OpenAI models, followed by retirement. A retired model returns HTTP 410 errors, and the deployment stops serving traffic.

Potential Savings

Savings from the OpenAI Service latest models policy depend on token volume and on the price gap between the old model and its replacement. High-volume workloads, such as support chat, document summarization, or batch classification, see the largest absolute savings.

The price gap between model generations is often several times, not a few percent. Moving a high-volume workload from an older flagship model to a newer, smaller model can remove most of its per-token cost.

Not every workload should move to the cheapest model. This policy reduces cost most when teams also validate output quality on the new model before switching production traffic.

Implementation Guide

Infrastructure-as-Code Example (Terraform)

The following examples show an Azure OpenAI deployment pinned to a deprecated model, and the corrected configuration.

Non-compliant configuration: deprecated model with auto-upgrade disabled

resource "azurerm_cognitive_account" "openai" {
  name                = "prod-openai"
  location            = azurerm_resource_group.ai.location
  resource_group_name = azurerm_resource_group.ai.name
  kind                = "OpenAI"
  sku_name            = "S0"
}

resource "azurerm_cognitive_deployment" "summarizer" {
  name                   = "summarizer"
  cognitive_account_id   = azurerm_cognitive_account.openai.id
  version_upgrade_option = "NoAutoUpgrade"

  model {
    format  = "OpenAI"
    name    = "o3-mini"
    version = "2025-01-31"
  }

  sku {
    name     = "GlobalStandard"
    capacity = 50
  }
}

resource "azurerm_cognitive_account" "openai" {
  name                = "prod-openai"
  location            = azurerm_resource_group.ai.location
  resource_group_name = azurerm_resource_group.ai.name
  kind                = "OpenAI"
  sku_name            = "S0"
}

resource "azurerm_cognitive_deployment" "summarizer" {
  name                   = "summarizer"
  cognitive_account_id   = azurerm_cognitive_account.openai.id
  version_upgrade_option = "NoAutoUpgrade"

  model {
    format  = "OpenAI"
    name    = "o3-mini"
    version = "2025-01-31"
  }

  sku {
    name     = "GlobalStandard"
    capacity = 50
  }
}

resource "azurerm_cognitive_account" "openai" {
  name                = "prod-openai"
  location            = azurerm_resource_group.ai.location
  resource_group_name = azurerm_resource_group.ai.name
  kind                = "OpenAI"
  sku_name            = "S0"
}

resource "azurerm_cognitive_deployment" "summarizer" {
  name                   = "summarizer"
  cognitive_account_id   = azurerm_cognitive_account.openai.id
  version_upgrade_option = "NoAutoUpgrade"

  model {
    format  = "OpenAI"
    name    = "o3-mini"
    version = "2025-01-31"
  }

  sku {
    name     = "GlobalStandard"
    capacity = 50
  }
}

Microsoft lists o3-mini version 2025-01-31 as deprecated, with a retirement date of November 19, 2026. With NoAutoUpgrade, this Azure OpenAI deployment stops working on that date. This is a common source of both wasted spend and unplanned incidents.

Compliant configuration: preferred model with a guardrail in code

variable "summarizer_model" {
  type    = string
  default = "gpt-5-mini"
}

variable "summarizer_model_version" {
  type    = string
  default = "2025-08-07"
}

locals {
  preferred_openai_models = ["^gpt-5-mini$", "^gpt-5$"]
}

# Add this as a new resource block. Keep the existing
# azurerm_cognitive_deployment.summarizer block until traffic moves.
resource "azurerm_cognitive_deployment" "summarizer_v2" {
  name                   = "summarizer-v2"
  cognitive_account_id   = azurerm_cognitive_account.openai.id
  version_upgrade_option = "OnceNewDefaultVersionAvailable"

  model {
    format  = "OpenAI"
    name    = var.summarizer_model
    version = var.summarizer_model_version
  }

  sku {
    name     = "GlobalStandard"
    capacity = 50
  }

  lifecycle {
    precondition {
      condition = anytrue([
        for pattern in local.preferred_openai_models :
        can(regex(pattern, var.summarizer_model))
      ])
      error_message = "summarizer_model is not on the preferred OpenAI model list."
    }
  }
}

variable "summarizer_model" {
  type    = string
  default = "gpt-5-mini"
}

variable "summarizer_model_version" {
  type    = string
  default = "2025-08-07"
}

locals {
  preferred_openai_models = ["^gpt-5-mini$", "^gpt-5$"]
}

# Add this as a new resource block. Keep the existing
# azurerm_cognitive_deployment.summarizer block until traffic moves.
resource "azurerm_cognitive_deployment" "summarizer_v2" {
  name                   = "summarizer-v2"
  cognitive_account_id   = azurerm_cognitive_account.openai.id
  version_upgrade_option = "OnceNewDefaultVersionAvailable"

  model {
    format  = "OpenAI"
    name    = var.summarizer_model
    version = var.summarizer_model_version
  }

  sku {
    name     = "GlobalStandard"
    capacity = 50
  }

  lifecycle {
    precondition {
      condition = anytrue([
        for pattern in local.preferred_openai_models :
        can(regex(pattern, var.summarizer_model))
      ])
      error_message = "summarizer_model is not on the preferred OpenAI model list."
    }
  }
}

variable "summarizer_model" {
  type    = string
  default = "gpt-5-mini"
}

variable "summarizer_model_version" {
  type    = string
  default = "2025-08-07"
}

locals {
  preferred_openai_models = ["^gpt-5-mini$", "^gpt-5$"]
}

# Add this as a new resource block. Keep the existing
# azurerm_cognitive_deployment.summarizer block until traffic moves.
resource "azurerm_cognitive_deployment" "summarizer_v2" {
  name                   = "summarizer-v2"
  cognitive_account_id   = azurerm_cognitive_account.openai.id
  version_upgrade_option = "OnceNewDefaultVersionAvailable"

  model {
    format  = "OpenAI"
    name    = var.summarizer_model
    version = var.summarizer_model_version
  }

  sku {
    name     = "GlobalStandard"
    capacity = 50
  }

  lifecycle {
    precondition {
      condition = anytrue([
        for pattern in local.preferred_openai_models :
        can(regex(pattern, var.summarizer_model))
      ])
      error_message = "summarizer_model is not on the preferred OpenAI model list."
    }
  }
}

The lifecycle precondition makes terraform plan fail when someone sets a model outside the preferred list. The new resource block, summarizer_v2, creates a second deployment named summarizer-v2. The old and new Azure OpenAI deployments run side by side until callers move to the new deployment.

Changing model.name or the deployment name on an existing azurerm_cognitive_deployment forces Terraform to destroy and recreate the deployment. Editing the old block in place causes an outage window, so add a new block instead.

Microsoft retires model versions on a rolling schedule, so treat gpt-5-mini version 2025-08-07 as an example. Confirm current availability with az cognitiveservices account list-models before you apply.

Step-by-Step Fix Instructions

Find every azurerm_cognitive_deployment resource with model.format = "OpenAI" and record its model name, version, and version_upgrade_option.
Compare each model and version against Microsoft's Azure OpenAI model retirement schedule.
Choose a preferred replacement model for each workload, and test output quality against a sample of real prompts.
Add a new azurerm_cognitive_deployment resource block with the preferred model, its own resource address, and a new deployment name. Do not edit the existing block in place.
Run terraform plan and confirm the plan adds the new deployment without destroying the old one.
Point application traffic at the new deployment name, then monitor error rates and token usage.
Remove the old Azure OpenAI deployment from Terraform once traffic has fully moved.
Add Infracost to your CI/CD pipeline so pull requests that add a non-preferred OpenAI model get flagged before merge.

Best Practices

Keep one preferred OpenAI model list for the whole organization instead of separate lists per Terraform module.
Use wildcard patterns for model families, such as gpt-5*, so new versions in a preferred family stay compliant.
Leave version_upgrade_option at OnceNewDefaultVersionAvailable unless a workload needs a pinned version for validation.
Subscribe to Azure Service Health alerts for Azure OpenAI, which carry retirement notices for deployed models.
This policy is not a quality gate. Teams still need prompt evaluations before moving production traffic to a new model.

Tools and Scripts

The OpenAI Service latest models policy is available in Infracost, including in the free trial. Infracost scans Terraform for OpenAI model references and compares each one against the preferred model list your organization defines.

When Infracost runs in CI/CD, it comments on the pull request that adds or changes a non-preferred OpenAI model. Teams choose whether the finding warns the engineer or blocks the merge. Platform engineers fix the model choice before deployment, not after a retirement notice arrives.

Infracost also lists existing Azure OpenAI deployments that already violate the policy across connected repositories. FinOps and platform teams use that list to burn down outdated models over time and track progress toward full compliance.

To list deployments and their current model versions with the Azure CLI:

az cognitiveservices account deployment list \
  --name <account-name> \
  --resource-group <resource-group> \
  --query "[].{name:name, model:properties.model.name, version:properties.model.version, upgrade:properties.versionUpgradeOption}" \
  --output

az cognitiveservices account deployment list \
  --name <account-name> \
  --resource-group <resource-group> \
  --query "[].{name:name, model:properties.model.name, version:properties.model.version, upgrade:properties.versionUpgradeOption}" \
  --output

az cognitiveservices account deployment list \
  --name <account-name> \
  --resource-group <resource-group> \
  --query "[].{name:name, model:properties.model.name, version:properties.model.version, upgrade:properties.versionUpgradeOption}" \
  --output

Examples of Impact

Illustrative example: a summarization service on a legacy flagship model. A team built a ticket summarization service on gpt-4 and never revisited the choice. Moving the service to a smaller model on the preferred list removed most of its per-token cost. Request volume did not change.

Illustrative example: a deprecated model with auto-upgrade disabled. A platform team pinned a reasoning model with NoAutoUpgrade during testing and shipped the same Terraform to production. Infracost flagged the model on the next pull request that touched the module, and the team migrated before the retirement date.

(These are illustrative, composite scenarios, not specific customer accounts.)

Considerations and Caveats

Quality varies by task: A newer or cheaper OpenAI model can score lower on a specific task. Validate with real prompts before switching production traffic.
Model changes recreate the deployment: Changing model.name destroys and recreates an azurerm_cognitive_deployment. Use a side-by-side deployment for zero-downtime migration.
Provisioned deployments differ: Provisioned throughput deployments bill by reserved capacity, not per token, and Microsoft does not auto-upgrade them.
Regional availability: Not every OpenAI model is available in every Azure region or for every SKU. Check availability before adding a model to the preferred list.
The preferred list goes stale: A model that is current today will be deprecated within its lifecycle. This policy is only as accurate as the list behind it.

Related Policies and Concepts

OpenAI Service: Consider using preferred Tokens-Per-Minute (TPM) limits: a companion AI-group policy that caps the TPM capacity of each Azure OpenAI deployment, bounding worst-case token spend.
OpenAI Service: Consider using preferred SKUs: a companion AI-group policy that restricts which deployment SKUs teams can use for each OpenAI model.
Cost Per Token (Glossary): defines the unit rate that model choice sets for every Azure OpenAI request.
Token Economics (Glossary): explains how token volume and per-token pricing combine to drive AI spend.
Tokens Per Dollar (Glossary): the inverse of cost per token, useful for comparing how much work each OpenAI model delivers per dollar.

Frequently Asked Questions (FAQs)

Is the OpenAI Service latest models policy supported in Infracost?

Yes. The OpenAI Service latest models policy is available in Infracost, including in the free trial. Infracost flags Terraform changes that reference an OpenAI model outside your preferred list, so engineers see the finding in the pull request before merge.

Can the OpenAI Service latest models policy be customized?

Yes. Teams define which OpenAI models count as preferred, and wildcard patterns such as gpt-5* match every model in a family. Teams can also choose whether the OpenAI Service latest models policy warns engineers or blocks the pull request.

Does Infracost automatically fix outdated OpenAI model violations?

No. Infracost detects and reports Azure OpenAI deployments that use a model outside the preferred list, and engineers update the model in Terraform. Infracost Cloud also offers AutoFix, which proposes fixes for policy failures as pull requests that engineers review and merge.

Is the OpenAI Service latest models policy cloud-agnostic?

No. The OpenAI Service latest models policy targets OpenAI models deployed through Azure OpenAI and managed with the azurerm Terraform provider. Other AI platforms use different resources, model catalogs, and pricing.

How often should I review the preferred Azure OpenAI model list?

Review the preferred Azure OpenAI model list at least quarterly, and whenever Microsoft announces a new model or a retirement date. Infracost checks every pull request against the current list, so an outdated list is the main way this policy loses value.

What happens to an Azure OpenAI deployment when its model is retired?

When an Azure OpenAI model version is retired, inference requests return an HTTP 410 error and the deployment stops serving traffic. Standard and Global Standard deployments can auto-upgrade before that point, but provisioned deployments must be migrated manually.

Does changing the model on an Azure OpenAI deployment cause downtime?

It can. Changing the model name on an azurerm_cognitive_deployment resource forces Terraform to destroy and recreate the Azure OpenAI deployment. Creating a second deployment, moving traffic, and then removing the old one avoids an outage.