Amazon EC2 – Consider Upgrading HS1 Instances to D2

The Amazon EC2 HS1 to D2 upgrade policy flags Amazon EC2 instances running the first-generation hs1.8xlargedense-storage instance type and recommends moving them to the D2 family. A d2.4xlarge has the same 16 vCPUs, more memory, and costs about 40% less per month on demand than an hs1.8xlarge in US East (N. Virginia). Apply this policy to any aws_instance or aws_launch_template whose instance_type is hs1.8xlarge.

Why This Policy Matters

How It Helps Reduce Cloud Costs

Amazon EC2 dense-storage instances pair large amounts of local HDD storage with moderate compute. Teams use them for Hadoop and MapReduce clusters, data warehousing, log processing, and distributed file systems.

HS1 was the first-generation Amazon EC2 high-storage family, and hs1.8xlarge is its only size. It offers 16 vCPUs, 117 GiB of memory, and 24 x 2 TB HDDs for 48 TB of instance storage.

D2 replaced HS1. D2 runs on Intel Xeon E5-2676 v3 (Haswell) processors, supports EC2 Enhanced Networking, and scales up to 36 vCPUs and 244 GiB of memory on the d2.8xlarge.

Running HS1 today means paying a premium for older hardware. This is a common source of wasted spend in long-lived AWS accounts, where a storage cluster was sized years ago and never revisited.

Potential Savings

The Amazon EC2 HS1 to D2 upgrade saves money because a d2.4xlarge delivers equivalent compute for a much lower hourly rate.

Monthly figures assume 730 hours at US East (N. Virginia) on-demand Linux rates. Prices vary by region, operating system, and purchase option.

Moving one hs1.8xlarge to a d2.4xlarge saves about $1,343 per month, or roughly $16,000 per year per instance. A ten-node HS1 cluster moved to d2.4xlarge saves about $160,000 per year at on-demand rates.

The d2.4xlarge saving comes with half the local storage. Workloads that need the full 48 TB per node should move to d2.8xlarge, which costs about 20% more per node but more than doubles compute and memory. That can allow a smaller cluster when compute, not storage, was the limit.

Implementation Guide

Infrastructure-as-Code Example (Terraform)

The following examples show an Amazon EC2 instance on HS1 and the corrected configuration on D2.

Non-compliant configuration: Amazon EC2 instance on hs1.8xlarge

resource "aws_instance" "hdfs_data" {
  ami                    = var.ami_id
  instance_type          = "hs1.8xlarge"
  subnet_id              = aws_subnet.private.id
  vpc_security_group_ids = [aws_security_group.hdfs.id]

  tags = {
    Name = "hdfs-data-01"
  }
}

resource "aws_instance" "hdfs_data" {
  ami                    = var.ami_id
  instance_type          = "hs1.8xlarge"
  subnet_id              = aws_subnet.private.id
  vpc_security_group_ids = [aws_security_group.hdfs.id]

  tags = {
    Name = "hdfs-data-01"
  }
}

resource "aws_instance" "hdfs_data" {
  ami                    = var.ami_id
  instance_type          = "hs1.8xlarge"
  subnet_id              = aws_subnet.private.id
  vpc_security_group_ids = [aws_security_group.hdfs.id]

  tags = {
    Name = "hdfs-data-01"
  }
}

This configuration pays about $3,358 per month for a first-generation instance type with less memory per dollar than its replacement.

Compliant configuration: Amazon EC2 instance on d2.4xlarge

resource "aws_instance" "hdfs_data" {
  ami                    = var.ami_id
  instance_type          = "d2.4xlarge"
  subnet_id              = aws_subnet.private.id
  vpc_security_group_ids = [aws_security_group.hdfs.id]

  tags = {
    Name = "hdfs-data-01"
  }
}

resource "aws_instance" "hdfs_data" {
  ami                    = var.ami_id
  instance_type          = "d2.4xlarge"
  subnet_id              = aws_subnet.private.id
  vpc_security_group_ids = [aws_security_group.hdfs.id]

  tags = {
    Name = "hdfs-data-01"
  }
}

resource "aws_instance" "hdfs_data" {
  ami                    = var.ami_id
  instance_type          = "d2.4xlarge"
  subnet_id              = aws_subnet.private.id
  vpc_security_group_ids = [aws_security_group.hdfs.id]

  tags = {
    Name = "hdfs-data-01"
  }
}

The only change is the instance_type value. Terraform applies it as an update to the existing aws_instance, which stops and starts the instance.

The same fix applies to launch templates used by Auto Scaling groups or EMR-style worker pools:

resource "aws_launch_template" "hdfs_data" {
  name_prefix   = "hdfs-data-"
  image_id      = var.ami_id
  instance_type = "d2.4xlarge" # previously "hs1.8xlarge"
}

resource "aws_launch_template" "hdfs_data" {
  name_prefix   = "hdfs-data-"
  image_id      = var.ami_id
  instance_type = "d2.4xlarge" # previously "hs1.8xlarge"
}

resource "aws_launch_template" "hdfs_data" {
  name_prefix   = "hdfs-data-"
  image_id      = var.ami_id
  instance_type = "d2.4xlarge" # previously "hs1.8xlarge"
}

Step-by-Step Fix Instructions

1. Search Terraform code for instance_type = "hs1.8xlarge" in aws_instance and aws_launch_templateresources.

2. Check shared modules, .tfvars files, and variable defaults. Older instance types often sit in a variable "instance_type" default rather than in the resource block.

3. Measure how much of the 48 TB HS1 instance store the workload actually uses. Pick d2.4xlarge if usage fits in 24 TB per node, or d2.8xlarge if it does not.

4. Copy any data on the HS1 instance store to Amazon S3, Amazon EBS, or another cluster node. Instance store data does not survive a stop and start.

5. Confirm the AMI is an HVM AMI with current drivers. Very old paravirtual (PV) AMIs built for HS1 should be replaced with a current Amazon Linux or Ubuntu HVM AMI.

6. Change the instance_type value and run terraform plan. Review whether the plan shows an in-place update or a replacement.

7. Apply one node at a time for clustered workloads such as HDFS, and let replication recover before moving the next node.

8. Add Infracost to CI/CD so any new hs1.8xlarge instance type is flagged in pull requests before it merges.

Best Practices

• Pin approved instance types in shared Terraform modules with a validation block, so hs1.8xlarge fails at terraform plan.

• Size D2 nodes to measured storage use, not to the old HS1 footprint. Check actual per-node instance store usage before matching the 48 TB of an hs1.8xlarge.

• Treat the upgrade as a rolling cluster change, not a bulk edit. Clustered storage workloads lose a node's local data on every stop and start.

• Review Reserved Instances before migrating. A Reserved Instance purchased for HS1 does not cover D2 usage.

• This policy targets HS1 to D2 only. Newer dense-storage families such as D3 exist and are worth a separate evaluation, with their own instance storage and pricing tradeoffs.

Tools and Scripts

How does Infracost detect Amazon EC2 HS1 instances automatically?

The Amazon EC2 HS1 to D2 upgrade policy is available in Infracost, including in the free trial. When Infracost runs in CI/CD, it checks Amazon EC2 resources in each pull request and flags the hs1.8xlarge instance type in the PR comment, with the file and line to change.

The same Infracost pull request comment shows the estimated monthly cost of the change. Reviewers see what an hs1.8xlarge costs before it merges, not on next month's AWS bill.

Infracost also scans connected repositories for HS1 instance types already in code. FinOps and platform teams can burn down that backlog over time, tracking issues prevented in pull requests separately from existing issues fixed.

Teams can set the policy to warn engineers or block pull requests, and can limit it to new resources only so an unrelated PR does not force a risky storage migration.

To find HS1 instance types in a Terraform codebase:

grep -rn '"hs1\.' --include="*.tf" --include="*.tfvars"

grep -rn '"hs1\.' --include="*.tf" --include="*.tfvars"

grep -rn '"hs1\.' --include="*.tf" --include="*.tfvars"

To list running Amazon EC2 instances on HS1 with the AWS CLI:

aws ec2 describe-instances \
  --filters "Name=instance-type,Values=hs1.8xlarge" \
  --query "Reservations[].Instances[].{id:InstanceId,type:InstanceType,state:State.Name}" \
  --output

aws ec2 describe-instances \
  --filters "Name=instance-type,Values=hs1.8xlarge" \
  --query "Reservations[].Instances[].{id:InstanceId,type:InstanceType,state:State.Name}" \
  --output

aws ec2 describe-instances \
  --filters "Name=instance-type,Values=hs1.8xlarge" \
  --query "Reservations[].Instances[].{id:InstanceId,type:InstanceType,state:State.Name}" \
  --output

If an instance is managed by Terraform, change instance_type in code rather than in the console. Otherwise the next terraform apply sets HS1 again.

Examples of Impact

Savings from the Amazon EC2 HS1 to D2 upgrade scale with cluster size, because each node is a single attribute change.

Illustrative example: an HDFS cluster with headroom. A data platform team runs six hs1.8xlarge data nodes that use about 18 TB of local storage each. Rolling the cluster to d2.4xlarge cuts on-demand compute cost by about 40%, roughly $8,000 per month, while keeping the same vCPU count per node.

Illustrative example: a full-capacity log archive. A team runs four hs1.8xlarge nodes close to the 48 TB limit. A d2.4xlarge cannot hold that data, so the like-for-like option is d2.8xlarge at about 20% more per node. Here the Amazon EC2 HS1 to D2 upgrade buys newer hardware and double the compute, not a lower bill, and the team weighs it against Amazon S3 tiering instead.

(These are illustrative scenarios, not specific customer accounts.)

Considerations and Caveats

The Amazon EC2 HS1 to D2 upgrade is low risk for compute, but instance storage is where migrations go wrong.

• Half the storage on d2.4xlarge. A d2.4xlarge has 24 TB of instance storage, half of an hs1.8xlarge. Check per-node usage before choosing it.

• Instance store data is lost on stop. HS1 and D2 instance store disks are ephemeral. Copy data off before changing the instance type.

• Restart required. Changing instance_type stops and starts the instance. Without an Elastic IP, the public IPv4 address changes.

• Reservations do not move automatically. Standard Reserved Instances for HS1 must run out or be sold on the Reserved Instance Marketplace.

• Provider-specific. This policy targets Amazon EC2 through the aws Terraform provider. Azure and Google Cloud have their own storage-optimized families with different names.

Related Policies and Concepts

The Amazon EC2 HS1 to D2 upgrade is one of several Infracost instance-generation policies, and teams get the most value by enforcing them together.

• Amazon EC2 - consider using latest generation instances for i family instances: the same generation upgrade for SSD-based storage-optimized Amazon EC2 instances.

• Amazon EC2 - consider using latest generation instances for r family instances: a related Compute-group policy that applies the same instance-generation discipline to memory-optimized instances.

• Amazon EC2 - Consider using a preferred instance type: a Compute-group policy that lets teams restrict Amazon EC2 to an approved list of instance types, which can exclude HS1 entirely.

• Amazon EBS - consider upgrading gp2 volumes to gp3: a storage-cost policy for teams that move HS1 instance store data onto Amazon EBS during the migration.

Frequently Asked Questions (FAQs)

Is this policy supported in Infracost?

Yes. The Amazon EC2 HS1 to D2 upgrade policy is supported in Infracost and available in the free trial. Infracost flags Amazon EC2 resources that use the hs1.8xlarge instance type in pull requests, so the team can change the instance type before merge.

Can this policy be customized?

Yes. Teams can configure how Infracost enforces the Amazon EC2 HS1 to D2 upgrade policy. A finding can warn engineers or block the pull request, and the policy can be limited to new resources only.

Does Infracost automatically fix violations?

No. Infracost reports each Amazon EC2 HS1 instance type in the pull request with the file and line to change. Choosing between d2.4xlarge and d2.8xlarge depends on how much instance storage the workload needs, so the team makes that decision.

Is this policy cloud-agnostic?

No. The Amazon EC2 HS1 to D2 upgrade policy targets Amazon EC2 through the aws Terraform provider. Azure and Google Cloud have their own storage-optimized VM families with different resource and attribute names.

Which D2 size should replace an hs1.8xlarge instance?

A d2.4xlarge matches the hs1.8xlarge on vCPU count, adds memory, and costs about 40% less on demand, but has 24 TB of instance storage instead of 48 TB. A d2.8xlarge matches the 48 TB of the hs1.8xlarge and doubles compute and memory, at a higher hourly rate.

Is data on HS1 instance storage preserved when changing to D2?

No. Amazon EC2 HS1 and D2 data on instance store disks does not survive a stop and start. Copy any data on the HS1 instance store to Amazon S3, Amazon EBS, or another node before changing the instance type.

How often should I review Amazon EC2 storage-optimized instance types?

Teams running Infracost in CI/CD get an Amazon EC2 HS1 check on every pull request that touches an EC2 resource. For instances already running, a quarterly review of storage-optimized instance generations is a reasonable cadence for most teams.