Amazon S3 – Consider Deleting Incomplete Multi-Part Uploads

An S3 incomplete multipart upload policy adds a lifecycle rule that automatically aborts Amazon S3 multipart uploads that never finished. Amazon S3 keeps every uploaded part of an unfinished multi-part upload and bills it as storage, even though no usable object exists. Apply this rule to every bucket that receives large files, with a 7-day window as the default.

Policy at a Glance

Why This Policy Matters

How It Helps Reduce Cloud Costs

Amazon S3 multipart upload splits a large file into parts that are uploaded separately. Amazon S3 assembles the parts into an object only when the client sends a complete request.

If the client crashes, times out, or loses its network connection, that complete request never arrives. Amazon S3 keeps every part it already received.

AWS documentation is explicit on billing. You are billed for all storage, bandwidth, and requests for a multipart upload and its parts until you complete or stop it.

This is a common source of invisible S3 spend. Incomplete multipart uploads are not objects, so they do not appear in normal object listings. AWS also notes that the parts of an incomplete upload cannot be viewed in the AWS Management Console.

The AWS CLI and the high-level transfer utilities in AWS SDKs switch to multipart upload automatically for large files. The AWS CLI default threshold is 8 MB. Any bucket that receives backups, media, data exports, or build artifacts is exposed.

An S3 lifecycle rule with the AbortIncompleteMultipartUpload action fixes this at the bucket level. Amazon S3 stops each upload that has not completed within the set number of days and deletes its parts.

Potential Savings

Savings from the S3 incomplete multipart upload policy scale with how often uploads fail and how large the files are. Buckets fed by unattended batch jobs, backup tools, and CI pipelines accumulate orphaned parts fastest.

Consider a nightly export job in US East (N. Virginia) that leaves about 50 GB of orphaned parts behind each week. S3 Standard storage is listed at about $0.023 per GB-month for the first 50 TB.

After 12 months with no abort rule, the bucket holds roughly 2.6 TB of parts that will never become objects. That costs about $60 per month, and the figure keeps rising by about $1.15 each week.

With a 7-day abort rule, the same bucket holds at most about one week of orphaned parts at any time. That caps the waste at roughly $1 to $2 per month, however long the job runs.

These figures are illustrative and use list prices at the time of writing. The dollar amount per bucket is often small. The value comes from applying the rule everywhere, since unmanaged parts grow without limit and nobody sees them.

Implementation Guide

Infrastructure-as-Code Example (Terraform)

The following examples show an Amazon S3 bucket with no rule for incomplete multipart uploads, and the corrected configuration.

Non-compliant configuration: no lifecycle rule for incomplete multipart uploads

resource "aws_s3_bucket" "data_exports" {
  bucket = "acme-data-exports"
}

resource "aws_s3_bucket_server_side_encryption_configuration" "data_exports" {
  bucket = aws_s3_bucket.data_exports.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "aws:kms"
    }
  }
}

resource "aws_s3_bucket" "data_exports" {
  bucket = "acme-data-exports"
}

resource "aws_s3_bucket_server_side_encryption_configuration" "data_exports" {
  bucket = aws_s3_bucket.data_exports.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "aws:kms"
    }
  }
}

resource "aws_s3_bucket" "data_exports" {
  bucket = "acme-data-exports"
}

resource "aws_s3_bucket_server_side_encryption_configuration" "data_exports" {
  bucket = aws_s3_bucket.data_exports.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "aws:kms"
    }
  }
}

Every export that fails partway through leaves its uploaded parts in this bucket. Nothing in this configuration ever removes them, so billed storage grows with every failed upload.

This gap is easy to miss, because neither a Terraform plan nor the S3 console shows the orphaned parts.

Compliant configuration: abort incomplete multipart uploads after 7 days

resource "aws_s3_bucket" "data_exports" {
  bucket = "acme-data-exports"
}

resource "aws_s3_bucket_lifecycle_configuration" "data_exports" {
  bucket = aws_s3_bucket.data_exports.id

  rule {
    id     = "abort-incomplete-multipart-uploads"
    status = "Enabled"

    filter {}

    abort_incomplete_multipart_upload {
      days_after_initiation = 7
    }
  }
}

resource "aws_s3_bucket" "data_exports" {
  bucket = "acme-data-exports"
}

resource "aws_s3_bucket_lifecycle_configuration" "data_exports" {
  bucket = aws_s3_bucket.data_exports.id

  rule {
    id     = "abort-incomplete-multipart-uploads"
    status = "Enabled"

    filter {}

    abort_incomplete_multipart_upload {
      days_after_initiation = 7
    }
  }
}

resource "aws_s3_bucket" "data_exports" {
  bucket = "acme-data-exports"
}

resource "aws_s3_bucket_lifecycle_configuration" "data_exports" {
  bucket = aws_s3_bucket.data_exports.id

  rule {
    id     = "abort-incomplete-multipart-uploads"
    status = "Enabled"

    filter {}

    abort_incomplete_multipart_upload {
      days_after_initiation = 7
    }
  }
}

This rule tells Amazon S3 to abort any multipart upload that has not completed 7 days after it started. Amazon S3 then deletes all parts of that upload.

The empty filter {} block applies the rule to every multipart upload in the bucket. Use prefix inside filter to scope the rule to part of the bucket instead.

The rule has no effect on completed uploads or on existing objects. It also applies to incomplete uploads that already exist, so the current backlog is cleaned up after the first run.

Step-by-Step Fix Instructions

1. Find every aws_s3_bucket resource in Terraform and check whether it has a matching aws_s3_bucket_lifecycle_configuration resource.

2. For each lifecycle configuration, check for a rule that contains an abort_incomplete_multipart_upload block.

3. If the bucket already has a lifecycle configuration, add the abort block to an existing rule or add a new rule. Do not create a second lifecycle resource for the same bucket.

4. Set days_after_initiation = 7 unless the bucket receives uploads that legitimately run longer than a week.

5. Run terraform plan and confirm the change only adds or updates the lifecycle configuration.

6. Apply the change. Amazon S3 runs lifecycle actions asynchronously, so expect existing incomplete uploads to be removed over the following days.

7. Add Infracost to CI/CD so any new S3 bucket without an abort rule is flagged in the pull request before merge.

Best Practices

• Make the abort rule a default in every shared Terraform module that creates an S3 bucket. Let teams change the day count, not remove the rule.

• Combine the abort rule with other lifecycle rules in the same aws_s3_bucket_lifecycle_configuration resource, such as noncurrent version expiration.

• Use S3 Storage Lens to find buckets where incomplete multipart upload bytes are already high, and fix those first.

• Make upload clients retry or abort cleanly on failure. The lifecycle rule is a safety net, not a substitute for correct error handling.

• A 7-day window is too short for some workloads. Very large uploads over slow links can legitimately take longer, so size the window to the slowest expected upload.

Tools and Scripts

How does Infracost detect missing S3 incomplete multipart upload rules automatically?

The Amazon S3 incomplete multipart upload policy is available in Infracost, including in the free trial. When Infracost runs in CI/CD, it checks S3 bucket configurations in each pull request. It flags buckets with no rule to abort incomplete multipart uploads, directly in the PR comment.

This catches the gap when the fix is a few lines in the same pull request. Infracost also finds existing buckets already in violation across repositories. FinOps and platform teams can then burn down that backlog over time and measure progress as the count of non-compliant buckets falls.

To check a bucket's current lifecycle rules with the AWS CLI:

aws s3api get-bucket-lifecycle-configuration --bucket

aws s3api get-bucket-lifecycle-configuration --bucket

aws s3api get-bucket-lifecycle-configuration --bucket

To list incomplete multipart uploads in a bucket, with their start time:

aws s3api list-multipart-uploads \
  --bucket acme-data-exports \
  --query 'Uploads[].[Key,UploadId,Initiated]' \
  --output

aws s3api list-multipart-uploads \
  --bucket acme-data-exports \
  --query 'Uploads[].[Key,UploadId,Initiated]' \
  --output

aws s3api list-multipart-uploads \
  --bucket acme-data-exports \
  --query 'Uploads[].[Key,UploadId,Initiated]' \
  --output

Each request returns at most 1,000 uploads, and the AWS CLI paginates automatically. For large accounts, S3 Storage Lens is faster. Its free Cost Efficiency metrics include Incomplete Multipart Upload Storage Bytes and Incomplete Multipart Upload Object Count.

Examples of Impact

Illustrative example: backups over an unreliable link. A platform team ships nightly database backups to Amazon S3 from an on-premises site. Network drops interrupt a few uploads each month, and the backup tool simply retries from scratch.

After a year, the bucket carries hundreds of gigabytes of orphaned parts that never appear in object listings. A 7-day abort rule removes the backlog and keeps it from returning.

Illustrative example: CI artifacts from cancelled jobs. A CI pipeline uploads large build artifacts to Amazon S3. Engineers cancel jobs mid-upload when they push a new commit.

Each cancellation can leave parts behind. Adding the abort rule to the shared bucket module fixes every artifact bucket at once.

(These are illustrative, composite scenarios, not specific customer accounts.)

Considerations and Caveats

• Long uploads need a longer window. Amazon S3 aborts any upload older than days_after_initiation, including one still in progress. Set the window longer than the slowest legitimate upload.

• No early delete charges. AWS states there are no early delete charges for removing incomplete multipart upload parts, regardless of storage class.

• One lifecycle configuration per bucket. aws_s3_bucket_lifecycle_configuration manages the bucket's whole lifecycle configuration, so all rules for one bucket belong in a single resource.

• Lifecycle runs asynchronously. Amazon S3 does not abort uploads the moment they reach the age limit. Removal can take a day or more.

• Storage only. The abort rule stops future storage charges for orphaned parts. It does not refund charges already incurred.

Related Policies and Concepts

• Amazon S3 - consider deleting or moving non-current object versions to a cheaper storage class: a companion Object Storage policy that removes another category of S3 storage hidden from normal object listings.

• Amazon S3 - consider deleting or moving old objects to a cheaper storage class: the Object Storage policy that applies lifecycle rules to current objects, often in the same lifecycle configuration.

• Amazon ECR - consider using a lifecycle policy: a parallel Object Storage policy that stops old container images from accumulating storage cost indefinitely.

• Azure Storage Accounts - consider using a lifecycle policy for blob storage: the Azure equivalent, using lifecycle management rules to tier or delete blobs automatically.

• Amazon CloudWatch - consider using a retention policy to reduce storage costs: the same retention principle applied to CloudWatch Logs log groups.

Frequently Asked Questions (FAQs)

Is this policy supported in Infracost?

Yes. The Amazon S3 incomplete multipart upload policy is supported in Infracost and available in the free trial. Infracost flags S3 buckets with no lifecycle rule to abort incomplete multipart uploads, so the team can add one before the pull request merges.

Can this policy be customized?

Yes. The team chooses the number of days in the days_after_initiation argument of the Amazon S3 lifecycle rule. Seven days is the recommended default, and buckets with very long uploads can use a longer window.

Does Infracost automatically fix violations?

Infracost does not change S3 buckets on its own. Infracost reports the missing abort rule, and the fix is applied through a Terraform pull request that the team reviews and merges.

Is this policy cloud-agnostic?

No. This policy targets Amazon S3 through the aws_s3_bucket_lifecycle_configuration Terraform resource. Other object storage services handle unfinished uploads differently, with different resource and attribute names.

Does aborting incomplete multipart uploads delete any S3 objects?

No. The Amazon S3 abort incomplete multipart upload action only removes parts of uploads that never completed. Completed objects, including objects uploaded with multipart upload, are not affected.

Are there early delete charges when S3 removes incomplete multipart upload parts?

No. AWS states there are no early delete charges for removing incomplete multipart upload parts, regardless of the storage class specified for the upload.

How often should I review S3 incomplete multipart upload rules?

Teams running Infracost in CI/CD get an S3 incomplete multipart upload check on every pull request that changes S3 bucket configuration. For existing buckets, a quarterly review of S3 Storage Lens incomplete multipart upload metrics is a reasonable cadence.