Amazon Cloudwatch – Consider Using a Retention Policy to Reduce Storage Costs

A CloudWatch log retention policy sets how long Amazon CloudWatch Logs keeps log events before deleting them, using the retention_in_days attribute on the aws_cloudwatch_log_group Terraform resource. Without a retention policy, CloudWatch log groups keep every log event indefinitely, so storage cost grows every month for as long as the application runs. Apply this policy to every log group whose logs lose operational value after a known period.

Policy at a Glance

Why This Policy Matters

How It Helps Reduce Cloud Costs

Amazon CloudWatch Logs bills log storage per GB per month, and a log group with no retention setting never deletes anything. Every day of logs is added to the bill permanently.

This is a common source of slow, invisible spend. No single month looks alarming, but the storage line rises every month as long as the log group exists.

A CloudWatch log retention policy turns that growing curve into a flat one. Once the retention window is full, CloudWatch deletes the oldest events as new ones arrive, so stored volume stays roughly constant.

Retention only addresses storage. It does not change CloudWatch Logs ingestion charges, which are billed per GB as logs arrive.

Potential Savings

Consider an application that writes 5 GB of logs per day in US East (N. Virginia), where CloudWatch Logs storage is listed at $0.03 per GB per month. After one year without retention, the log group holds about 1,825 GB, or roughly 1.8 TB.

At that point the log group costs about $55 per month in storage alone, and the figure keeps rising by about $4.50 every month. First-year storage adds up to roughly $330, and year two costs close to $1,000 because it starts from a full year of data.

With a 30-day retention policy, the same log group holds about 150 GB at steady state. That is roughly $4.50 per month, or about $54 per year, regardless of how long the application runs.

These figures are illustrative. CloudWatch Logs measures stored data after compression, so real storage bills are usually lower than raw log volume suggests, but the growth pattern is the same.

Implementation Guide

Infrastructure-as-Code Example (Terraform)

The following examples show a CloudWatch log group with no retention setting, and the corrected configuration.

Non-compliant configuration: no retention set

resource "aws_cloudwatch_log_group" "orders_api" {
  name = "/aws/lambda/orders-api"

  tags = {
    Team = "orders"
  }
}

resource "aws_cloudwatch_log_group" "orders_api" {
  name = "/aws/lambda/orders-api"

  tags = {
    Team = "orders"
  }
}

resource "aws_cloudwatch_log_group" "orders_api" {
  name = "/aws/lambda/orders-api"

  tags = {
    Team = "orders"
  }
}

With retention_in_days omitted, the AWS provider creates the log group with no expiration. Every event this function writes stays in CloudWatch Logs, and on the bill, indefinitely.

Compliant configuration: 30-day retention

resource "aws_cloudwatch_log_group" "orders_api" {
  name              = "/aws/lambda/orders-api"
  retention_in_days = 30

  tags = {
    Team = "orders"
  }
}

resource "aws_cloudwatch_log_group" "orders_api" {
  name              = "/aws/lambda/orders-api"
  retention_in_days = 30

  tags = {
    Team = "orders"
  }
}

resource "aws_cloudwatch_log_group" "orders_api" {
  name              = "/aws/lambda/orders-api"
  retention_in_days = 30

  tags = {
    Team = "orders"
  }
}

Setting retention_in_days = 30 tells CloudWatch Logs to delete events older than 30 days automatically. The change is in-place: Terraform updates the existing log group without replacing it.

retention_in_days accepts only specific values: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1096, 1827, 2192, 2557, 2922, 3288, and 3653. A value of 0 means never expire, which is the same as leaving the attribute unset.

Step-by-Step Fix Instructions

1. Search Terraform configuration for every aws_cloudwatch_log_group resource and note which ones have no retention_in_days value.

2. Find log groups created outside Terraform. Run aws logs describe-log-groups and look for entries with no retentionInDays field.

3. Agree on a retention period per log category with the teams that own the logs and with security or compliance, since some logs have mandated minimums.

4. Add retention_in_days to each aws_cloudwatch_log_group resource using one of the supported values.

5. For log groups that AWS services created automatically, such as /aws/lambda/<function-name>, import them into Terraform with terraform import or declare them before the service writes its first event.

6. Run terraform plan to confirm each change is an in-place update, then apply.

7. Add Infracost to CI/CD so any new aws_cloudwatch_log_group without a retention setting is flagged in the pull request before merge.

Best Practices

• Set retention_in_days as a required input in shared Terraform modules that create log groups, so no team ships a log group without one.

• Use shorter retention, such as 7 or 14 days, for development and test environments where logs are only used for debugging.

• Export logs that must be kept for audits to Amazon S3 with a lifecycle rule, rather than keeping them in CloudWatch Logs storage for years.

• Consider the log_group_class = "INFREQUENT_ACCESS" setting for high-volume logs that are rarely queried, since it lowers the ingestion price.

• Short retention is not right for every log group. Security and audit logs often carry regulatory minimums that override cost goals.

Tools and Scripts

How does Infracost detect missing CloudWatch log retention automatically?

The Amazon CloudWatch log retention policy is available in Infracost, including in the free trial. When Infracost runs in CI/CD, it checks aws_cloudwatch_log_group resources added or changed in a pull request and flags any that have no retention setting, directly in the PR comment.

This catches the problem before the log group exists and starts accumulating data, when the fix is a one-line change in the same pull request. Infracost also scans existing repositories for log groups already in violation, so FinOps and platform teams can burn down the backlog over time and measure progress as the count of non-compliant log groups falls.

To list log groups in the current region that have no retention setting:

aws logs describe-log-groups \
  --query "logGroups[?retentionInDays==null].[logGroupName,storedBytes]" \
  --output

aws logs describe-log-groups \
  --query "logGroups[?retentionInDays==null].[logGroupName,storedBytes]" \
  --output

aws logs describe-log-groups \
  --query "logGroups[?retentionInDays==null].[logGroupName,storedBytes]" \
  --output

Examples of Impact

Illustrative example: Lambda logs left on default. A platform team runs dozens of Lambda functions, each writing to an auto-created /aws/lambda/ log group with no retention. Two years later, much of the stored data comes from functions that were decommissioned long ago. Deleting the log groups of retired functions, and importing the rest into Terraform with a 30-day retention_in_days value, stops the storage line from growing and removes the old data.

This pattern is typical in serverless-heavy AWS accounts, where CloudWatch log groups outlive the Lambda functions that created them.

Illustrative example: debug logging in a test account. A test environment with verbose debug logging accumulates hundreds of GB that nobody reads after a week. Setting a 7-day retention policy on those log groups keeps the debugging value and drops the long tail of storage cost.

(These are illustrative, composite scenarios, not specific customer accounts.)

Considerations and Caveats

• Deletion is permanent. CloudWatch Logs deletes expired log events automatically, and they cannot be recovered. Export anything needed for audits before setting a short window.

• Compliance minimums come first. Frameworks such as PCI DSS or internal security policies may require one year or more of log retention for specific log groups.

• Ingestion is not affected. A CloudWatch log retention policy reduces storage cost only. High ingestion bills require logging less, filtering at the source, or changing the log group class.

• Auto-created log groups are easy to miss. AWS services such as Lambda create log groups on first write, outside Terraform, with no expiration by default.

• Only supported values work. retention_in_days rejects values outside the supported list, so 45 or 100 fails at terraform apply.

Related Policies and Concepts

• Azure Monitor - consider using a retention policy to reduce storage costs: the Azure equivalent of this Logging-group policy, applying the same storage-growth discipline to Log Analytics workspaces.

• RDS Cluster - consider enabling CloudWatch log exports: a related policy that creates new CloudWatch log groups, which need their own retention setting to avoid unbounded storage growth.

• Amazon S3 - consider deleting or moving old objects to a cheaper storage class: the natural destination for CloudWatch logs that must be kept long term, using S3 lifecycle rules instead of CloudWatch storage.

• Amazon ECR - consider using a lifecycle policy: a parallel policy that stops container images from accumulating storage cost indefinitely.

• Google Compute Engine - consider using a retention policy for snapshots: the same retention principle applied to disk snapshots on Google Cloud.

Frequently Asked Questions (FAQs)

Is this policy supported in Infracost?

Yes. The Amazon CloudWatch log retention policy is supported in Infracost and available in the free trial. Infracost flags aws_cloudwatch_log_group resources that have no retention_in_days value, so the team can set one before the pull request merges.

Can this policy be customized?

Yes. Teams can adjust how the Amazon CloudWatch log retention policy is enforced in Infracost, for example whether a finding only warns the engineer or blocks the pull request. The retention period itself is chosen by the team in Terraform, not by Infracost.

Does Infracost automatically fix violations?

No. Infracost identifies and reports aws_cloudwatch_log_group resources without a retention setting. Choosing a retention_in_days value and applying the change is a decision the team makes and reviews in its own pull request.

Is this policy cloud-agnostic?

No. This policy targets Amazon CloudWatch Logs through the aws_cloudwatch_log_group Terraform resource. Azure Monitor and Google Cloud Logging have their own retention settings, with different resource and attribute names.

How often should I review CloudWatch log retention settings?

Teams running Infracost in CI/CD get a CloudWatch log retention check on every pull request that adds or changes an aws_cloudwatch_log_group resource. For existing log groups, a quarterly review of retention against current compliance requirements is a reasonable cadence.

Does setting a CloudWatch retention period reduce log ingestion costs?

No. A CloudWatch Logs retention period only reduces storage cost. Ingestion is billed per GB as logs arrive, so reducing ingestion cost requires logging less data or using the Infrequent Access log class.

What happens to logs when the CloudWatch retention period expires?

Amazon CloudWatch Logs deletes log events older than the retention period automatically. Deleted events cannot be recovered, so logs needed for audits should be exported to Amazon S3 before they expire.