EMR - consider upgrading gp2 storage type to gp3

This policy recommends upgrading Amazon EMR cluster storage from the older gp2 EBS volume type to gp3, which costs less per GB and includes a higher baseline performance tier at no extra charge. gp2 ties IOPS to volume size, so smaller EMR instance group volumes can be significantly under-provisioned on performance while still paying a higher per-GB rate. Teams provisioning or resizing EMR master, core, or task instance group storage should default to gp3 unless a specific workload has a documented reason to stay on gp2.

Why This Policy Matters

How It Helps Reduce Cloud Costs

Amazon EBS gp2 volumes tie IOPS to volume size, at a rate of 3 IOPS per GB, with a 100 IOPS floor and a burst ceiling of 3,000 IOPS. A modest 200 GB EMR core node volume on gp2 gets only 600 baseline IOPS.

gp3 decouples performance from capacity. Every gp3 volume includes 3,000 IOPS and 125 MB/s of throughput at no extra cost, regardless of size. Teams can provision up to 16,000 IOPS and 1,000 MB/s if a workload genuinely needs more, paying only for the increment above the baseline.

This makes gp2 a common source of quietly under-provisioned EMR storage. A small or mid-sized core node volume can sit well below what gp3 includes by default, while billing at gp2's higher per-GB rate for the privilege.

AWS prices gp3 at up to 20% less per GB than gp2, and that discount applies uniformly across every volume attached to an EMR master, core, or task instance group.

Potential Savings

Savings scale with the total EBS storage attached across master, core, and task instance groups. The 20% per-GB discount compounds across every volume in every instance group, and it stacks with avoiding any provisioned IOPS add-on charge that a gp2 volume might otherwise need to reach acceptable performance.

Order of magnitude, a cluster storing several terabytes of core node storage on gp2 sees a proportional reduction in monthly EBS spend after switching to gp3, without changing cluster size, instance count, or job configuration. This is a configuration-only saving, not a usage reduction, which is what makes it low-risk to apply broadly.

Implementation Guide

Infrastructure-as-Code Example (Terraform)

The following examples show an EMR cluster provisioned with gp2 storage, and the corrected configuration using gp3.

Non-compliant configuration: EMR instance groups on gp2 storage

resource "aws_emr_cluster" "analytics" {
  name          = "analytics-cluster"
  release_label = "emr-7.9.0"
  applications  = ["Spark"]

  master_instance_group {
    instance_type = "m5.xlarge"

    ebs_config {
      size = 100
      type = "gp2"
    }
  }

  core_instance_group {
    instance_type  = "m5.2xlarge"
    instance_count = 4

    ebs_config {
      size                 = 500
      type                 = "gp2"
      volumes_per_instance = 1
    }
  }
}

resource "aws_emr_cluster" "analytics" {
  name          = "analytics-cluster"
  release_label = "emr-7.9.0"
  applications  = ["Spark"]

  master_instance_group {
    instance_type = "m5.xlarge"

    ebs_config {
      size = 100
      type = "gp2"
    }
  }

  core_instance_group {
    instance_type  = "m5.2xlarge"
    instance_count = 4

    ebs_config {
      size                 = 500
      type                 = "gp2"
      volumes_per_instance = 1
    }
  }
}

resource "aws_emr_cluster" "analytics" {
  name          = "analytics-cluster"
  release_label = "emr-7.9.0"
  applications  = ["Spark"]

  master_instance_group {
    instance_type = "m5.xlarge"

    ebs_config {
      size = 100
      type = "gp2"
    }
  }

  core_instance_group {
    instance_type  = "m5.2xlarge"
    instance_count = 4

    ebs_config {
      size                 = 500
      type                 = "gp2"
      volumes_per_instance = 1
    }
  }
}

This configuration pays gp2 pricing while capping each 500 GB core node volume at roughly 1,500 baseline IOPS, well under what gp3 includes for free at any size. This is a common source of wasted spend on EMR clusters provisioned before gp3 became the default recommendation.

Compliant configuration: EMR instance groups on gp3 storage

resource "aws_emr_cluster" "analytics" {
  name          = "analytics-cluster"
  release_label = "emr-7.9.0"
  applications  = ["Spark"]

  master_instance_group {
    instance_type = "m5.xlarge"

    ebs_config {
      size = 100
      type = "gp3"
    }
  }

  core_instance_group {
    instance_type  = "m5.2xlarge"
    instance_count = 4

    ebs_config {
      size                 = 500
      type                 = "gp3"
      volumes_per_instance = 1
      iops                 = 3000
      throughput           = 125
    }
  }
}

resource "aws_emr_cluster" "analytics" {
  name          = "analytics-cluster"
  release_label = "emr-7.9.0"
  applications  = ["Spark"]

  master_instance_group {
    instance_type = "m5.xlarge"

    ebs_config {
      size = 100
      type = "gp3"
    }
  }

  core_instance_group {
    instance_type  = "m5.2xlarge"
    instance_count = 4

    ebs_config {
      size                 = 500
      type                 = "gp3"
      volumes_per_instance = 1
      iops                 = 3000
      throughput           = 125
    }
  }
}

resource "aws_emr_cluster" "analytics" {
  name          = "analytics-cluster"
  release_label = "emr-7.9.0"
  applications  = ["Spark"]

  master_instance_group {
    instance_type = "m5.xlarge"

    ebs_config {
      size = 100
      type = "gp3"
    }
  }

  core_instance_group {
    instance_type  = "m5.2xlarge"
    instance_count = 4

    ebs_config {
      size                 = 500
      type                 = "gp3"
      volumes_per_instance = 1
      iops                 = 3000
      throughput           = 125
    }
  }
}

Setting type = "gp3" on every ebs_config block lowers the per-GB rate and raises the baseline IOPS and throughput for each volume, with no additional charge over the default gp3 tier. Infracost includes a named EMR gp2-to-gp3 FinOps policy that flags this exact pattern in pull requests, so the team sees the suggestion before the cluster is ever launched.

Step-by-Step Fix Instructions

1. List every aws_emr_cluster resource in Terraform state and check each master_instance_group, core_instance_group, and task_instance_group block for an ebs_config with type = "gp2".

2. Confirm the cluster's release_label supports gp3; EMR releases 6.15.0 and later already default new clusters' root volumes to gp3.

3. Change type to "gp3" on each affected ebs_config block, adding explicit iops and throughput values only if a workload needs more than the included baseline.

4. Run terraform plan and read the output carefully. Changing ebs_config on an existing, already-launched instance group typically forces Terraform to replace the entire cluster, since EMR does not support modifying an instance group's storage configuration after launch.

5. For a running production cluster, treat this as a planned cluster replacement rather than an in-place update: launch a new cluster with gp3 configured, migrate jobs and data over, then terminate the old cluster during a maintenance window.

6. Add Infracost to CI/CD so future Terraform changes are checked against Infracost's built-in EMR gp2-to-gp3 policy before they merge.

Best Practices

• Set type = "gp3" as the default in shared EMR Terraform modules so new clusters never launch on gp2 in the first place.

• Only add explicit iops or throughput values when a workload's actual I/O pattern demands more than the 3,000 IOPS / 125 MB/s baseline; most EMR jobs never need it.

• This policy does not apply to clusters that depend on an EBS volume type other than gp2 or gp3, such as io1 or io2 for latency-sensitive shuffle-heavy workloads.

• Because changing storage type forces cluster replacement, batch this change with other planned cluster upgrades rather than treating it as an isolated hotfix.

Tools and Scripts

This policy check is available in Infracost, including in the free trial. Infracost's FinOps policy library includes a named EMR gp2-to-gp3 check that surfaces this pattern in pull requests, giving the team a chance to switch to gp3 before a cluster is created.

This turns a one-time cleanup into an ongoing check. Infracost enables platform and FinOps teams to track how many gp2-configured resources remain across the organization and burn that number down incrementally, with a clear trend line to report on.

To check the storage configuration of an existing cluster using the AWS CLI:

aws emr describe-cluster --cluster-id <cluster-id> \
  --query "Cluster.InstanceGroups[].EbsBlockDevices[].VolumeSpecification"

aws emr describe-cluster --cluster-id <cluster-id> \
  --query "Cluster.InstanceGroups[].EbsBlockDevices[].VolumeSpecification"

aws emr describe-cluster --cluster-id <cluster-id> \
  --query "Cluster.InstanceGroups[].EbsBlockDevices[].VolumeSpecification"

Examples of Impact

Illustrative example: a data platform team standardizing on gp3. A team maintaining a dozen long-running EMR clusters for batch ETL finds that most core instance groups were provisioned on gp2 years ago, before gp3 was recommended. Rolling gp3 into the shared Terraform module and replacing clusters during their next scheduled maintenance window lowers the per-GB storage rate across every cluster, with no change to job runtime or cluster size.

Illustrative example: catching it before launch. An engineer copies an older EMR module as a starting point for a new cluster and inherits a gp2 ebs_config block without noticing. Infracost's gp2-to-gp3 policy check flags the type = "gp2"attribute in the pull request, and the engineer switches to gp3 before the cluster ever launches, avoiding a configuration that would otherwise persist for the cluster's lifetime.

(These are illustrative, composite scenarios, not specific customer accounts.)

Considerations and Caveats

• Cluster replacement, not resize: Changing type on an existing instance group's ebs_config typically forces Terraform to replace the entire EMR cluster, since EMR does not support modifying an already-launched instance group's storage configuration.

• Root volume behavior varies by release: EMR releases 6.15.0 and higher already default new clusters' root device volume to gp3; this policy is most relevant to storage volumes and to root volumes on older release labels.

• Not every workload benefits equally: Clusters with very small storage footprints see a smaller absolute dollar impact, even though the percentage discount is the same.

• Provider-specific: This policy targets the aws_emr_cluster resource; equivalent services on other clouds are out of scope.

• Instance fleets are a separate case: Clusters configured with instance fleets rather than instance groups have a different ebs_config shape and are outside the scope of this specific policy check.

Related Policies and Concepts

• RDS - consider upgrading gp2 storage type to gp3: the same storage-type discipline applied to Amazon RDS, another Compute-group FinOps policy in Infracost's library.

• PostgreSQL - consider enabling autogrow on non-production projects: a related Compute-group policy focused on not over-provisioning storage ahead of actual need.

• RDS Cluster - consider enabling CloudWatch log exports: a Compute-group policy on operational visibility for the same class of managed data infrastructure.

• "How to Enforce Cloud Cost Policies in Your CI/CD Pipeline": a planned Infracost resource article covering the broader workflow of catching cost policy violations like this one at the pull request stage, before they reach production billing.

Frequently Asked Questions (FAQs)

Is this policy supported in Infracost?

Yes. Infracost's FinOps policy library includes a named EMR gp2-to-gp3 check that flags aws_emr_cluster resources with a gp2 ebs_config block in pull requests, so the team can switch to gp3 before the cluster launches. This check is available in the free trial and all paid plans.

Can this policy be customized?

Yes. Teams can adjust the policy's scope in Infracost, for example limiting the check to specific instance groups or excluding clusters that have a documented reason to remain on gp2.

Does Infracost automatically fix violations?

No. Infracost identifies and reports a gp2 ebs_config block on an aws_emr_cluster resource in a pull request. Changing the attribute to gp3, and planning the resulting cluster replacement, is a decision the team makes and applies manually.

Is this policy cloud-agnostic?

No. This specific policy targets Amazon EMR and the aws_emr_cluster Terraform resource. Other cloud data-processing services have their own storage-tier tradeoffs, but the resource and attribute names differ.

How often should I review EMR storage type compliance?

Teams running Infracost in CI/CD get a check on every pull request that touches an aws_emr_cluster resource. For clusters already running, a quarterly review alongside other planned maintenance is a reasonable cadence, since applying this change requires a cluster replacement rather than an in-place update.

Does switching from gp2 to gp3 require downtime?

For an existing, already-launched EMR cluster, yes in practice. Because EMR does not support modifying an instance group's storage configuration after launch, applying this change means provisioning a new cluster on gp3 and migrating jobs over, rather than resizing the running cluster in place. New clusters can simply be created with gp3 from the start.