Amazon S3 – Consider Deleting or Moving Old Objects to a Cheaper Storage Class

An Amazon S3 lifecycle policy for old objects moves aging S3 objects to a cheaper storage class, or deletes them, at a set age. Without an S3 lifecycle rule, every object stays in S3 Standard and is billed at the S3 Standard rate for as long as it exists. Apply this policy to buckets where data is read less often as it ages, such as logs, backups, exports, and build output.

Policy at a Glance

Why This Policy Matters

How It Helps Reduce Cloud Costs

Amazon S3 bills storage per GB-month, and the rate depends on the storage class. S3 Standard is the default class, and it costs more per GB than every infrequent-access and archive class.

Amazon S3 does not move an object out of S3 Standard on its own. Without a lifecycle rule, a bucket that grows every day keeps every byte at the S3 Standard rate indefinitely.

This is a common source of slow, compounding S3 spend. Log, backup, and export buckets grow quietly, and most of their contents are never read again after the first few weeks.

An S3 lifecycle rule fixes the pattern at the bucket level. Objects transition to S3 Standard-IA, S3 Glacier Instant Retrieval, or S3 Glacier Flexible Retrieval after a set number of days. Objects with no long-term value expire and stop billing entirely.

The tradeoff is access. Colder storage classes charge per-GB retrieval fees and, for S3 Glacier Flexible Retrieval, add a restore delay. This policy pays off for data that is rarely read, not for data in active use.

Potential Savings

In US East (N. Virginia), S3 Standard is listed at $0.023 per GB-month for the first 50 TB. S3 Glacier Flexible Retrieval is listed at $0.0036 per GB-month.

That means 1 TB of data costs about $23 per month in S3 Standard and about $3.60 per month in S3 Glacier Flexible Retrieval. That is an 84% reduction in storage cost for the same data.

Intermediate classes sit between those two points. S3 Standard-IA is listed at $0.0125 per GB-month and S3 Glacier Instant Retrieval at $0.004 per GB-month.

The saving grows with the bucket. A log bucket that adds 1 TB per month and never transitions anything pays for its entire history at the S3 Standard rate. These figures use list prices at the time of writing and exclude request and retrieval charges.

Implementation Guide

Infrastructure-as-Code Example (Terraform)

The following examples show an Amazon S3 bucket with no lifecycle rule, and the corrected configuration.

Non-compliant configuration: no S3 lifecycle rule

resource "aws_s3_bucket" "app_logs" {
  bucket = "acme-app-logs"
}

resource "aws_s3_bucket" "app_logs" {
  bucket = "acme-app-logs"
}

resource "aws_s3_bucket" "app_logs" {
  bucket = "acme-app-logs"
}

Every log object written to this bucket stays in S3 Standard forever. Nothing in this configuration moves older objects to a cheaper class or removes them. This configuration increases storage costs every month without improving access to the data.

Compliant configuration: transition, then expire old objects

resource "aws_s3_bucket" "app_logs" {
  bucket = "acme-app-logs"
}

resource "aws_s3_bucket_lifecycle_configuration" "app_logs" {
  bucket = aws_s3_bucket.app_logs.id

  rule {
    id     = "age-out-logs"
    status = "Enabled"

    filter {
      prefix = "logs/"
    }

    transition {
      days          = 30
      storage_class = "STANDARD_IA"
    }

    transition {
      days          = 90
      storage_class = "GLACIER"
    }

    expiration {
      days = 365
    }
  }
}

resource "aws_s3_bucket" "app_logs" {
  bucket = "acme-app-logs"
}

resource "aws_s3_bucket_lifecycle_configuration" "app_logs" {
  bucket = aws_s3_bucket.app_logs.id

  rule {
    id     = "age-out-logs"
    status = "Enabled"

    filter {
      prefix = "logs/"
    }

    transition {
      days          = 30
      storage_class = "STANDARD_IA"
    }

    transition {
      days          = 90
      storage_class = "GLACIER"
    }

    expiration {
      days = 365
    }
  }
}

resource "aws_s3_bucket" "app_logs" {
  bucket = "acme-app-logs"
}

resource "aws_s3_bucket_lifecycle_configuration" "app_logs" {
  bucket = aws_s3_bucket.app_logs.id

  rule {
    id     = "age-out-logs"
    status = "Enabled"

    filter {
      prefix = "logs/"
    }

    transition {
      days          = 30
      storage_class = "STANDARD_IA"
    }

    transition {
      days          = 90
      storage_class = "GLACIER"
    }

    expiration {
      days = 365
    }
  }
}

This rule moves objects under the logs/ prefix to S3 Standard-IA 30 days after creation. It moves them to S3 Glacier Flexible Retrieval at 90 days and deletes them at 365 days.

In Terraform, storage_class = "GLACIER" refers to S3 Glacier Flexible Retrieval. Use "GLACIER_IR" for S3 Glacier Instant Retrieval or "DEEP_ARCHIVE" for S3 Glacier Deep Archive.

Replace the prefix filter with an empty filter {} block to apply the rule to every object in the bucket. Use tag inside filter to target objects by tag instead.

Step-by-Step Fix Instructions

1. Find every aws_s3_bucket resource in Terraform and check whether a matching aws_s3_bucket_lifecycle_configuration resource exists.

2. For each bucket without a lifecycle rule, ask the bucket owner how the data is read over time and how long it must be kept.

3. Pick a target storage class from that access pattern. Use S3 Standard-IA for occasional reads, S3 Glacier Instant Retrieval for rare reads that still need millisecond access, and S3 Glacier Flexible Retrieval for archives.

4. Add an aws_s3_bucket_lifecycle_configuration resource with one or more transition blocks and, where the data has an end of life, an expiration block.

5. Run terraform plan and confirm the change only adds or updates the lifecycle configuration. It does not modify existing object data.

6. Apply the change. Amazon S3 runs lifecycle actions asynchronously, so existing objects transition or expire over the following days.

7. Add Infracost to CI/CD so any new S3 bucket without a lifecycle rule is flagged in the pull request before merge.

Best Practices

• Make an S3 lifecycle rule a default in shared Terraform modules that create buckets. Let teams override the day counts, not remove the rule.

• Match the transition age to the storage class minimum. S3 Standard-IA bills a 30-day minimum, and S3 Glacier Instant Retrieval and S3 Glacier Flexible Retrieval bill 90 days.

• Add an abort_incomplete_multipart_upload block with days_after_initiation to the same rule, so abandoned uploads are cleaned up too.

• Consider S3 Intelligent-Tiering (storage_class = "INTELLIGENT_TIERING") for buckets with unpredictable access patterns. It moves objects between tiers based on observed access, for a small per-object monitoring fee.

• Not every bucket needs expiration. Buckets that hold audit records or regulated data often need transition only, with deletion handled by a retention process.

Tools and Scripts

How does Infracost detect missing S3 lifecycle rules automatically?

The Amazon S3 lifecycle policy for old objects is available in Infracost, including in the free trial. When Infracost runs in CI/CD, it checks S3 bucket configurations in each pull request. It flags buckets with no lifecycle rule directly in the PR comment, with the file and line to change.

This catches the problem while the fix is a few lines in the same pull request. The bucket never fills with data at the S3 Standard rate. Infracost also scans connected repositories for existing buckets already in violation. FinOps and platform teams can burn down that backlog over time and track progress on a burndown chart in Infracost Cloud.

To check a bucket's current lifecycle rules with the AWS CLI:

aws s3api get-bucket-lifecycle-configuration --bucket

aws s3api get-bucket-lifecycle-configuration --bucket

aws s3api get-bucket-lifecycle-configuration --bucket

A NoSuchLifecycleConfiguration error means the bucket has no lifecycle rule at all.

S3 Storage Lens and S3 Storage Class Analysis show how much data a bucket holds in each storage class. Neither requires listing every object.

Examples of Impact

Illustrative example: application logs. A platform team ships application logs to an S3 bucket at about 1 TB per month, and nobody reads logs older than two weeks. After a year with no lifecycle rule, the bucket holds 12 TB in S3 Standard. Moving logs older than 90 days to S3 Glacier Flexible Retrieval puts most of that history at the lower rate. A 365-day expiration caps the bucket's size.

This pattern is typical of any bucket where automation writes new objects on a schedule.

Illustrative example: database backups. A team keeps nightly database exports for compliance but restores from them only during incident drills. Transitioning exports to S3 Glacier Instant Retrieval after 30 days keeps millisecond access for the rare restore. Storage then costs a fraction of the S3 Standard rate.

(These are illustrative, composite scenarios, not specific customer accounts.)

Considerations and Caveats

• Retrieval is not free. S3 Standard-IA and the S3 Glacier classes charge per-GB retrieval fees, and S3 Glacier Flexible Retrieval restores take minutes to hours. Frequently read data belongs in S3 Standard.

• Minimum storage durations apply. Deleting or transitioning an object before its class minimum still incurs the full minimum charge.

• Small objects may not pay off. By default, Amazon S3 does not transition objects smaller than 128 KB. Each transition is also billed as a request. Buckets of many tiny objects can cost more to transition than they save.

• Versioned buckets need a second rule. In a versioned Amazon S3 bucket, expiration only adds a delete marker. Removing the old data requires a noncurrent_version_expiration rule as well.

• One lifecycle configuration per bucket. aws_s3_bucket_lifecycle_configuration manages the bucket's whole lifecycle configuration, so all rules for one bucket belong in a single resource.

Related Policies and Concepts

• Amazon S3 - consider deleting or moving non-current object versions to a cheaper storage class: the companion Object Storage policy that applies lifecycle rules to noncurrent versions rather than current objects.

• Amazon S3 - consider deleting incomplete multi-part uploads: another Object Storage policy that removes S3 storage that is billed but not visible in normal object listings.

• Azure Storage Accounts - consider using a lifecycle policy for blob storage: the Azure equivalent, using lifecycle management rules to tier or delete blobs by age.

• Amazon ECR - consider using a lifecycle policy: a parallel policy that stops old container images from accumulating storage cost indefinitely.

• Amazon CloudWatch - consider using a retention policy to reduce storage costs: the same age-based retention principle applied to CloudWatch Logs log groups.

Frequently Asked Questions (FAQs)

Is this policy supported in Infracost?

Yes. The Amazon S3 lifecycle policy for old objects is supported in Infracost and available in the free trial. Infracost flags S3 buckets that have no lifecycle rule to transition or expire objects, so the team can add one before the pull request merges.

Can this policy be customized?

Yes. Teams can choose whether Infracost blocks the pull request or only warns the engineer when the Amazon S3 lifecycle policy for old objects fails. Teams can also limit the policy to newly added resources. The transition days, target storage class, and expiration age are chosen by the team in Terraform, not by Infracost.

Does Infracost automatically fix violations?

No. Infracost identifies and reports S3 buckets without a lifecycle rule for old objects. Choosing the storage class, the transition age, and whether to expire objects is a decision the team makes and reviews in its own pull request.

Is this policy cloud-agnostic?

No. This policy targets Amazon S3 through the aws_s3_bucket_lifecycle_configuration Terraform resource. Azure Blob Storage and Google Cloud Storage offer their own lifecycle management rules, with different resource and attribute names.

How often should I review S3 lifecycle rules?

Teams running Infracost in CI/CD get an Amazon S3 lifecycle check on every pull request that changes S3 bucket configuration. For existing buckets, a quarterly review of S3 lifecycle rules against actual access patterns and retention requirements is a reasonable cadence.

Which S3 storage class should old objects move to?

It depends on how often the old Amazon S3 objects are read. S3 Standard-IA suits data read occasionally. S3 Glacier Instant Retrieval suits data read rarely but needed in milliseconds. S3 Glacier Flexible Retrieval suits archives that can wait minutes to hours for a restore.

Does an S3 lifecycle expiration rule delete objects in a versioned bucket?

Not immediately. In a versioned Amazon S3 bucket, an expiration action adds a delete marker and the previous data becomes a noncurrent version. A separate noncurrent version expiration rule is needed to remove that data and stop paying for it.