Azure Virtual Machines – Upgrade ND Series Machines

The Azure ND-series upgrade policy flags Azure Virtual Machines set to a retired ND-series size and recommends NCas T4 v3 or NDamsr A100 v4 instead. Microsoft retired the P40-based ND-series on September 6, 2023, so Terraform pinned to an ND size can no longer deploy. Apply this policy to any azurerm_linux_virtual_machine or azurerm_windows_virtual_machine whose size is Standard_ND6s, Standard_ND12s, Standard_ND24s, or Standard_ND24rs.

Why This Policy Matters

How It Helps Reduce Cloud Costs

Azure ND-series virtual machines paired NVIDIA Tesla P40 GPUs with Intel Xeon E5-2690 v4 (Broadwell) processors. Microsoft describes the ND-series as a midrange platform originally designed for AI and deep learning workloads.

Microsoft retired the ND-series on September 6, 2023. Any ND-series VM still in a subscription after that date was deallocated and removed from its host.

An ND-series size left in Terraform is therefore a broken configuration, not an idle cost. The next terraform apply that tries to create or start that VM fails, and teams often respond under pressure.

That pressure is where the cost risk lives. A team that needs a working GPU quickly tends to pick the largest available size rather than the right one.

This policy turns the ND-series migration into a planned sizing decision. The NCas T4 v3 series covers inference and lighter training on NVIDIA T4 GPUs. The NDamsr A100 v4 series covers large-scale training on NVIDIA A100 GPUs with InfiniBand.

Potential Savings

Savings from the Azure ND-series upgrade come from matching each workload to the smallest replacement that meets its GPU memory and throughput needs.

Microsoft's own migration guide maps the smallest ND-series size, Standard_ND6s, to Standard_NC4as_T4_v3 or Standard_NC8as_T4_v3. Those are the two smallest sizes in the NCas T4 v3 series, with 4 and 8 vCPUs respectively.

The T4 is one generation newer than the P40 (Turing versus Pascal). For batch inference, which the ND-series was designed for, a newer GPU generally finishes the same work in fewer billed hours.

Over-sizing erases that gain. Moving an inference workload from Standard_ND6s straight to Standard_ND96amsr_A100_v4 replaces one P40 GPU with eight A100 GPUs, a jump that rarely matches the workload.

Compare current list prices for the specific target sizes and region in the Azure Pricing Calculator before committing. Azure GPU pricing varies by region, operating system, and commitment type.

Implementation Guide

Infrastructure-as-Code Example (Terraform)

The following examples show an Azure Linux virtual machine pinned to a retired ND-series size, and the corrected NCas T4 v3 configuration.

Non-compliant configuration: Azure VM on a retired ND-series size

resource "azurerm_linux_virtual_machine" "inference" {
  name                = "inference-vm-01"
  resource_group_name = azurerm_resource_group.ml.name
  location            = azurerm_resource_group.ml.location
  size                = "Standard_ND6s"
  admin_username      = "azureuser"

  network_interface_ids = [
    azurerm_network_interface.inference.id,
  ]

  admin_ssh_key {
    username   = "azureuser"
    public_key = file("~/.ssh/id_rsa.pub")
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Premium_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "0001-com-ubuntu-server-jammy"
    sku       = "22_04-lts-gen2"
    version   = "latest"
  }
}

resource "azurerm_linux_virtual_machine" "inference" {
  name                = "inference-vm-01"
  resource_group_name = azurerm_resource_group.ml.name
  location            = azurerm_resource_group.ml.location
  size                = "Standard_ND6s"
  admin_username      = "azureuser"

  network_interface_ids = [
    azurerm_network_interface.inference.id,
  ]

  admin_ssh_key {
    username   = "azureuser"
    public_key = file("~/.ssh/id_rsa.pub")
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Premium_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "0001-com-ubuntu-server-jammy"
    sku       = "22_04-lts-gen2"
    version   = "latest"
  }
}

resource "azurerm_linux_virtual_machine" "inference" {
  name                = "inference-vm-01"
  resource_group_name = azurerm_resource_group.ml.name
  location            = azurerm_resource_group.ml.location
  size                = "Standard_ND6s"
  admin_username      = "azureuser"

  network_interface_ids = [
    azurerm_network_interface.inference.id,
  ]

  admin_ssh_key {
    username   = "azureuser"
    public_key = file("~/.ssh/id_rsa.pub")
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Premium_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "0001-com-ubuntu-server-jammy"
    sku       = "22_04-lts-gen2"
    version   = "latest"
  }
}

Standard_ND6s requests a size Azure no longer provides. This configuration cannot be deployed, and every change to this module stays blocked until the size is replaced.

Compliant configuration: Azure VM on NCas T4 v3

resource "azurerm_linux_virtual_machine" "inference" {
  name                = "inference-vm-01"
  resource_group_name = azurerm_resource_group.ml.name
  location            = azurerm_resource_group.ml.location
  size                = "Standard_NC4as_T4_v3"
  admin_username      = "azureuser"

  network_interface_ids = [
    azurerm_network_interface.inference.id,
  ]

  admin_ssh_key {
    username   = "azureuser"
    public_key = file("~/.ssh/id_rsa.pub")
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Premium_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "0001-com-ubuntu-server-jammy"
    sku       = "22_04-lts-gen2"
    version   = "latest"
  }
}

resource "azurerm_linux_virtual_machine" "inference" {
  name                = "inference-vm-01"
  resource_group_name = azurerm_resource_group.ml.name
  location            = azurerm_resource_group.ml.location
  size                = "Standard_NC4as_T4_v3"
  admin_username      = "azureuser"

  network_interface_ids = [
    azurerm_network_interface.inference.id,
  ]

  admin_ssh_key {
    username   = "azureuser"
    public_key = file("~/.ssh/id_rsa.pub")
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Premium_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "0001-com-ubuntu-server-jammy"
    sku       = "22_04-lts-gen2"
    version   = "latest"
  }
}

resource "azurerm_linux_virtual_machine" "inference" {
  name                = "inference-vm-01"
  resource_group_name = azurerm_resource_group.ml.name
  location            = azurerm_resource_group.ml.location
  size                = "Standard_NC4as_T4_v3"
  admin_username      = "azureuser"

  network_interface_ids = [
    azurerm_network_interface.inference.id,
  ]

  admin_ssh_key {
    username   = "azureuser"
    public_key = file("~/.ssh/id_rsa.pub")
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Premium_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "0001-com-ubuntu-server-jammy"
    sku       = "22_04-lts-gen2"
    version   = "latest"
  }
}

The only change is the size attribute. Standard_NC4as_T4_v3 provides 4 vCPUs, 28 GiB of memory, and one NVIDIA T4 GPU with 16 GB of GPU memory.

Use the mapping below, taken from Microsoft's GPU migration guide, as the starting point for each ND-series size.

Step-by-Step Fix Instructions

1. Search Terraform for ND-series sizes in the size attribute of azurerm_linux_virtual_machine and azurerm_windows_virtual_machine, and in vm_size on the legacy azurerm_virtual_machine resource.

2. Check shared modules and variable defaults too. ND sizes often hide in a variable "vm_size" default rather than in the resource block itself.

3. For each match, record the workload type and its peak GPU memory use. The NVIDIA T4 has 16 GB of GPU memory, while the P40 had 24 GB.

4. Choose the replacement from the mapping table. Use NCas T4 v3 for inference and models that fit in 16 GB per GPU. Use Standard_ND96amsr_A100_v4 only for multi-GPU training that needs InfiniBand.

5. Request vCPU quota for the target VM family in the target region before applying. Azure assigns vCPU quota per VM family, so ND-series quota does not cover the new family.

6. Change the size value and run terraform plan. Confirm the plan shows size as an in-place update, not a resource replacement.

7. Apply during a maintenance window. Moving to a different hardware family requires Azure to deallocate and restart the virtual machine.

8. Install the NVIDIA driver and CUDA version the new GPU requires, then validate the workload before decommissioning anything else.

9. Add Infracost to CI/CD so any new ND-series size added in a pull request is flagged before it merges.

Best Practices

• Pin approved GPU sizes in shared Terraform modules with a validation block, so retired sizes fail at terraform plan instead of at deploy time.

• Size replacements from measured GPU memory and utilization, not from the old ND size's GPU count.

• Migrate one environment at a time, starting with development, so driver and CUDA issues surface before production.

• Rebuild custom images with current NVIDIA drivers. Microsoft notes that images built for older GPU series may carry a CUDA runtime and driver version the new series does not support.

• This policy does not cover the NCv3, NC T4 v3, or ND v2 series. Microsoft's ND-series retirement notice explicitly excludes those series.

Tools and Scripts

How does Infracost detect retired ND-series sizes automatically?

The Azure ND-series upgrade policy is available in Infracost, including in the free trial. When Infracost runs in CI/CD, it checks Azure virtual machine resources in each pull request. It flags ND-series sizes in the PR comment, with the file and line to change.

The same Infracost pull request comment shows the estimated monthly cost of the chosen replacement, such as Standard_NC4as_T4_v3. Reviewers see that number before the change merges.

Infracost also scans connected repositories for ND-series sizes already in code. FinOps and platform teams can burn down that backlog and track progress on a burndown chart, separating new issues prevented from existing issues fixed.

Teams can set the policy to warn engineers or to block pull requests that fail it, and can limit it to new resources only.

To find ND-series sizes in a Terraform codebase:

grep -rnE '"Standard_ND(6|12|24|24r)s"' --include="*.tf" --include="*.tfvars"

grep -rnE '"Standard_ND(6|12|24|24r)s"' --include="*.tf" --include="*.tfvars"

grep -rnE '"Standard_ND(6|12|24|24r)s"' --include="*.tf" --include="*.tfvars"

To list Azure virtual machines still assigned an ND-series size with the Azure CLI:

az vm list \
  --query "[?contains(['Standard_ND6s','Standard_ND12s','Standard_ND24s','Standard_ND24rs'], hardwareProfile.vmSize)].{name:name, rg:resourceGroup, size:hardwareProfile.vmSize}" \
  --output

az vm list \
  --query "[?contains(['Standard_ND6s','Standard_ND12s','Standard_ND24s','Standard_ND24rs'], hardwareProfile.vmSize)].{name:name, rg:resourceGroup, size:hardwareProfile.vmSize}" \
  --output

az vm list \
  --query "[?contains(['Standard_ND6s','Standard_ND12s','Standard_ND24s','Standard_ND24rs'], hardwareProfile.vmSize)].{name:name, rg:resourceGroup, size:hardwareProfile.vmSize}" \
  --output

If a virtual machine is managed by Terraform, update size in code rather than resizing it in the portal. Otherwise the next terraform apply tries to set the retired ND size again.

Examples of Impact

Right-sizing the replacement, not the Azure ND-series upgrade itself, is where these scenarios recover cost.

Illustrative example: a batch inference service. A platform team finds Standard_ND6s hardcoded as the default for a shared inference module. The model uses about 10 GB of GPU memory.

Moving to Standard_NC4as_T4_v3 fits that footprint on the smallest NCas T4 v3 size. The alternative, an emergency move to a much larger GPU size, would bill for capacity the model never uses.

Illustrative example: a distributed training cluster. An ML team runs Standard_ND24rs nodes because the training job needs InfiniBand between GPUs. The mapped replacement is Standard_ND96amsr_A100_v4, with eight NVIDIA A100 80 GB GPUs per VM.

The team may be able to consolidate several ND24rs nodes into fewer A100 nodes. Microsoft's migration guide lists moving a clustered workload onto a single 8-GPU VM as one option.

(These are illustrative scenarios, not specific customer accounts.)

Considerations and Caveats

The Azure ND-series upgrade is mandatory, but the choice of replacement size is where teams win or lose money.

• Less GPU memory on T4. The NVIDIA T4 has 16 GB of GPU memory, against 24 GB on the P40. Models that relied on the full 24 GB may need a larger size or a smaller batch.

• InfiniBand only on the A100 target. Only Standard_ND96amsr_A100_v4 keeps InfiniBand. NCas T4 v3 sizes do not support RDMA networking.

• Processor architecture changes. NCas T4 v3 and NDamsr A100 v4 run on AMD EPYC processors, while the ND-series used Intel Xeon. Test any code with CPU-specific optimizations.

• Quota is separate. Each GPU VM family has its own vCPU quota. Request it before the migration window, not during it.

• Provider-specific. This policy targets Azure Virtual Machines through the azurerm provider. AWS and Google Cloud GPU instances use different families and attribute names.

Related Policies and Concepts

• Azure Virtual Machines - consider upgrading DS series machines to Dv5: the same older-to-newer series upgrade applied to general-purpose Azure VMs.

• Azure Virtual Machines - consider upgrading D series machines to Dv5: a Compute-group policy moving original D-series sizes to the newer Dv5 generation.

• Azure Virtual Machines - consider upgrading Ls series machines to Lsv2: the storage-optimized equivalent of moving off an older Azure VM series.

• Azure Virtual Machines - Consider using a preferred instance type: a Compute-group policy that restricts Azure VM sizes to an approved list, which can also flag retired sizes in the pull request.

• Azure Virtual Machines - consider using Azure Hybrid Benefit for Windows VMs: a related Compute-group policy that lowers Azure VM licensing cost without changing the VM size.

Frequently Asked Questions (FAQs)

Is this policy supported in Infracost?

Yes. The Azure ND-series upgrade policy is supported in Infracost and available in the free trial. Infracost flags Azure virtual machine resources using an ND-series size in pull requests, so the team can replace the size before merge.

Can this policy be customized?

Yes. Teams can configure how Infracost enforces the Azure ND-series upgrade policy. For example, a finding can warn engineers or block the pull request, and can trigger on new resources only.

Does Infracost automatically fix violations?

Infracost reports each Azure ND-series size in the pull request with the file and line to change. The choice between NCas T4 v3 and NDamsr A100 v4 depends on GPU memory and networking needs. The team makes and reviews that sizing decision.

Is this policy cloud-agnostic?

No. The Azure ND-series upgrade policy targets Azure Virtual Machines through the azurerm Terraform provider. AWS and Google Cloud GPU instances have their own generations, with different resource and attribute names.

When was the Azure ND-series retired?

Microsoft retired the Azure ND-series on September 6, 2023, after extending the original date by one year. Any Azure ND-series virtual machine remaining after that date was deallocated and removed from its host.

Should I choose NCas T4 v3 or NDamsr A100 v4 to replace an Azure ND-series VM?

Choose NCas T4 v3 for inference and models that fit in 16 GB of GPU memory per GPU. Choose NDamsr A100 v4 to replace an Azure ND-series VM only for large multi-GPU training that needs InfiniBand networking.

How often should I review Azure GPU VM sizes?

Teams running Infracost in CI/CD get an Azure ND-series check on every pull request that touches an Azure virtual machine resource. For existing Azure GPU VMs, a quarterly review against Microsoft's retirement announcements is a reasonable cadence.