|

AWS ECS – Consider Avoiding Secrets in Container Environment Variables

Storing sensitive information such as database passwords, API keys, and authentication tokens directly in ECS container environment variables creates significant security risks and compliance violations. This FinOps policy helps organizations identify and remediate insecure secret management practices in their ECS infrastructure while reducing potential security breaches.

Why this policy matters

Container environment variables in ECS task definitions are stored in plaintext and lack the security controls necessary for protecting sensitive data. When secrets are exposed through environment variables, they become accessible through multiple attack vectors including shell access, application logs, process listings, and the /proc filesystem. This exposure creates substantial financial and operational risks for organizations.

Implementation Guide

Infrastructure-as-Code Example (Terraform)

# Problematic configuration - secrets in environment variables
resource "aws_ecs_task_definition" "bad_example" {
  family                   = "my-app"
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  cpu                      = 256
  memory                   = 512
  execution_role_arn       = aws_iam_role.execution_role.arn
  task_role_arn           = aws_iam_role.task_role.arn

  container_definitions = jsonencode([
    {
      name  = "my-app"
      image = "my-app:latest"
      environment = [
        {
          name  = "DATABASE_PASSWORD"
          value = "super-secret-password"  # Security risk!
        },
        {
          name  = "API_KEY"
          value = "abc123xyz789"  # Security risk!
        }
      ]
      logConfiguration = {
        logDriver = "awslogs"
        options = {
          awslogs-group         = "/ecs/my-app"
          awslogs-region        = "us-east-1"
          awslogs-stream-prefix = "ecs"
        }
      }
    }
  ])
}

# Secure configuration using AWS Secrets Manager
resource "aws_secretsmanager_secret" "database_password" {
  name        = "my-app/database-password"
  description = "Database password for my-app"
}

resource "aws_secretsmanager_secret_version" "database_password" {
  secret_id     = aws_secretsmanager_secret.database_password.id
  secret_string = "super-secret-password"
}

resource "aws_ssm_parameter" "api_key" {
  name  = "/my-app/api-key"
  type  = "SecureString"
  value = "abc123xyz789"
}

resource "aws_ecs_task_definition" "secure_example" {
  family                   = "my-app"
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  cpu                      = 256
  memory                   = 512
  execution_role_arn       = aws_iam_role.execution_role.arn
  task_role_arn           = aws_iam_role.task_role.arn

  container_definitions = jsonencode([
    {
      name  = "my-app"
      image = "my-app:latest"
      secrets = [
        {
          name      = "DATABASE_PASSWORD"
          valueFrom = aws_secretsmanager_secret.database_password.arn
        },
        {
          name      = "API_KEY"
          valueFrom = aws_ssm_parameter.api_key.arn
        }
      ]
      logConfiguration = {
        logDriver = "awslogs"
        options = {
          awslogs-group         = "/ecs/my-app"
          awslogs-region        = "us-east-1"
          awslogs-stream-prefix = "ecs"
        }
      }
    }
  ])
}

# Required IAM permissions for secrets access
resource "aws_iam_role_policy" "task_secrets_policy" {
  name = "task-secrets-policy"
  role = aws_iam_role.task_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "secretsmanager:GetSecretValue"
        ]
        Resource = aws_secretsmanager_secret.database_password.arn
      },
      {
        Effect = "Allow"
        Action = [
          "ssm:GetParameter"
        ]
        Resource = aws_ssm_parameter.api_key.arn
      }
    ]
  })
}

The secure implementation uses the secrets block instead of the environment block for sensitive data. Infracost automatically detects when ECS task definitions use environment variables that appear to contain secrets and flags them for remediation.

Manual Step-by-Step Instructions

  1. Audit existing task definitions: Review all ECS task definitions to identify environment variables containing sensitive data such as passwords, keys, tokens, or connection strings.
  2. Create secrets in AWS Secrets Manager or SSM Parameter Store:
    • For frequently rotated secrets (database passwords, API tokens): Use AWS Secrets Manager
    • For configuration parameters that change infrequently: Use SSM Parameter Store with SecureString type
  3. Update IAM roles: Ensure the ECS task role has appropriate permissions to access the secrets from Secrets Manager or Parameter Store.
  4. Modify task definitions: Replace environment blocks containing secrets with secrets blocks that reference the ARNs of stored secrets.
  5. Test the deployment: Verify that applications can successfully retrieve secrets and function correctly.
  6. Remove old secrets: Clean up any hardcoded secrets from previous configurations and rotate them if they were previously exposed.

Best Practices

  • Use AWS Secrets Manager for secrets that require automatic rotation
  • Implement least-privilege IAM policies for secret access
  • Enable CloudTrail logging for secret access auditing
  • Use consistent naming conventions for secrets across environments
  • Implement secret scanning in CI/CD pipelines to prevent future violations
  • Regular audit secret access patterns and remove unused permissions

Tools and Scripts

Infracost supports this policy in its FinOps policy engine, including in the free trial. The platform automatically scans ECS task definitions and identifies when environment variables contain potential secrets based on variable names and patterns. Organizations can use Infracost to continuously monitor their infrastructure-as-code for secret management violations and track remediation progress over time.

Considerations and Caveats

While this policy generally applies to all sensitive data, there are some considerations:

  • Performance impact: Retrieving secrets from external services adds minimal latency compared to environment variables, but this should be considered for high-frequency operations
  • Cost implications: AWS Secrets Manager charges per secret per month, while SSM Parameter Store has different pricing tiers
  • Legacy applications: Some older applications may require code changes to support retrieving secrets from external sources
  • Development environments: Consider using different secret management strategies for development versus production environments
  • Network dependencies: Applications must have network access to AWS APIs to retrieve secrets

Frequently Asked Questions (FAQs)

Yes, this policy is available in Infracost, including in the free trial. The platform automatically detects environment variables that appear to contain secrets and provides recommendations for remediation.

The policy identifies environment variables with names containing common secret-related keywords such as “password”, “key”, “token”, “secret”, “credential”, and others, as well as variables with values that match common secret patterns.

Use Secrets Manager for secrets that require automatic rotation or cross-service access. Use SSM Parameter Store for configuration values that change less frequently and when cost optimization is a priority.

Infracost continuously monitors your infrastructure-as-code repositories and can integrate with CI/CD pipelines to prevent new violations from being deployed. This enables FinOps teams to track progress and ensure compliance over time.

Non-sensitive configuration values such as feature flags, application settings, or public endpoints can safely remain as environment variables. The policy specifically targets variables that appear to contain sensitive information.

Yes, the policy can be configured to have different sensitivity levels for development, staging, and production environments, allowing for more flexible implementation based on your organization’s risk tolerance.

Review secret management practices monthly as part of your security audit process, and implement automated scanning in your CI/CD pipeline to catch violations before they reach production.