Data Exports

Data exports are required for Infracost to see individual billing line items, cost optimization recommendations, and S3 usage metrics — complete the manual AWS Console setup first, then follow this guide to create them by hand, or bring your own existing exports instead. This sets up the same exports as the Terraform module and CloudFormation template.

Everything in this guide is performed on the AWS organization's management account, in the us-east-1 region — BCM Data Exports only exists in us-east-1. The data it exports still covers every region you use; only the export configuration and S3 buckets themselves must live in us-east-1.

note

AWS can take up to 24 hours to populate the buckets with the first data exports after you complete this setup.

Prerequisites

Enable Cost Optimization Hub and Compute Optimizer

The Cost Optimization Hub recommendations export requires your AWS Organization's management account to be opted in to both AWS Compute Optimizer and AWS Cost Optimization Hub, org-wide.

1. Open the Compute Optimizer console, go to Preferences, and enable it with Include member accounts of my organization checked.
2. Open Billing and Cost Management > Cost Management Preferences > Cost Optimization Hub tab. Check Enable Cost Optimization Hub and Enable Cost Optimization Hub for all member accounts.

AWS documents that enrollment can take up to 24 hours to take effect, though it is often faster.

Enable trusted access for S3 Storage Lens

1. Sign in to the AWS Console as the management account (or a role with organizations:EnableAWSServiceAccess).
2. Open AWS Organizations > Services, find Amazon S3 Storage Lens, and enable trusted access.

This is required for cross-account visibility — Storage Lens configuration created in the next section won't be able to see member-account buckets without it.

Step 1: Create the export S3 buckets

Create two private S3 buckets in us-east-1 — one for BCM Data Exports, one for S3 Storage Lens exports.

For each bucket:

1. Switch the AWS Console's region selector (top-right) to US East (N. Virginia) us-east-1 — buckets are created in whichever region your console session is in; there's no region field in the Create bucket form itself.
2. Open S3 > Buckets > Create bucket.
3. Bucket name: infracost-bcm-exports-<your-account-id> for the first, infracost-storagelens-exports-<your-account-id> for the second.
4. Under Block Public Access settings, leave all four boxes checked (block all public access).
5. Under Default encryption, leave Amazon S3 managed keys (SSE-S3) selected — or see Optional: encrypt with a KMS key below to use your own key instead.
6. Click Create bucket.

Add a lifecycle rule to each bucket

1. Open the bucket, go to Management > Lifecycle rules > Create lifecycle rule.
2. Rule name: intelligent-tiering. Apply to all objects in the bucket.
3. Under Lifecycle rule actions, check Transition current versions of objects between storage classes, Expire current versions of objects, and Delete expired object delete markers or incomplete multipart uploads.
4. Checking those reveals a configuration section further down the page for each one — fill them in:
   • Transition current versions of objects between storage classes: storage class Intelligent-Tiering, 0 days after object creation.
   • Expire current versions of objects: 365 days after object creation.
   • Delete expired object delete markers or incomplete multipart uploads: select Delete incomplete multipart uploads, 7 days after initiation.
5. Click Create rule.

Add a bucket policy to the Storage Lens exports bucket

Only set a policy on infracost-storagelens-exports-* here. Skip infracost-bcm-exports-* — creating the export in Step 3 makes AWS overwrite whatever policy you set on it, so that one gets its policy fixed up after the export exists instead. The Storage Lens bucket is different: creating the dashboard in Step 4 makes AWS append its own statement to whatever's already there rather than replacing it, so it's safe to set this now.

On the infracost-storagelens-exports-* bucket (Permissions > Bucket policy > Edit), paste, substituting your account ID:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowStorageLensDataExportsWrite",
      "Effect": "Allow",
      "Principal": {
        "Service": "storage-lens.s3.amazonaws.com"
      },
      "Action": ["s3:PutObject", "s3:PutObjectAcl"],
      "Resource": [
        "arn:aws:s3:::infracost-storagelens-exports-111111111111",
        "arn:aws:s3:::infracost-storagelens-exports-111111111111/*"
      ],
      "Condition": {
        "StringEquals": { "aws:SourceAccount": "111111111111" }
      }
    },
    {
      "Sid": "DenyNonSSLRequests",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::infracost-storagelens-exports-111111111111",
        "arn:aws:s3:::infracost-storagelens-exports-111111111111/*"
      ],
      "Condition": {
        "Bool": { "aws:SecureTransport": "false" }
      }
    }
  ]
}

Step 2: Create the BCM Data Exports service-linked role

Creating the exports below requires the AWSServiceRoleForBCMDataExports service-linked role to exist in the management account.

1. Open IAM > Roles > Create role.
2. Select AWS service, then search for and select a service that uses BCM Data Exports — AWS creates the correctly named service-linked role for you automatically when you first create an export through the console (Step 3 below also triggers this), so in practice you can usually skip this and let Step 3 create it implicitly.
3. If you'd rather create it explicitly first, use the AWS CLI once instead, since the console flow for service-linked roles varies by service: aws iam create-service-linked-role --aws-service-name bcm-data-exports.amazonaws.com --no-cli-pager.

caution

If your account has previously used BCM Data Exports (via the console or another tool), this role may already exist — that's fine, both paths above are safe to skip if you see an "already exists" message.

Step 3: Create the FOCUS and Cost Optimization Hub exports

1. Open Billing and Cost Management > Data Exports > Create export.
2. Export type: select Standard data export.
3. Export name: InfracostDailyFocusExport.
4. Under Data table content settings, select the FOCUS with AWS columns table, and set Table content version to FOCUS 1.2.
5. Time granularity: select Daily (defaults to Hourly).
6. Column selection: select the checkbox in the table header to include all columns.
7. Compression type and file format: Parquet – Parquet.
8. File versioning: select Overwrite existing data export file to avoid accumulating a new file on every refresh.
9. Under Data export storage settings, choose This account, click Configure, and select the existing infracost-bcm-exports-* bucket you created in Step 1.

caution

AWS shows a prompt warning that it will overwrite the bucket's policy with one that lets Data Exports deliver to it — accept this, it's expected since we skipped setting a policy on this bucket in Step 1. This happens again when you create the second export below, so don't bother re-adding the deny statement until both exports exist.

10. S3 path prefix: InfracostDailyFocusExport (required by AWS — reuse the export name).
11. Review and create the export.

Now create a second export for Cost Optimization Hub recommendations:

1. Open Billing and Cost Management > Data Exports > Create export.
2. Export type: select Standard data export.
3. Export name: InfracostDailyCostOptimizationExport.
4. Under Data table content settings, select the Cost optimization recommendations table, with Include all recommendations checked and no additional filter.
5. Compression type and file format: Parquet – Parquet.
6. File versioning: select Overwrite existing data export file to avoid accumulating a new file on every refresh.
7. Under Data export storage settings, choose This account, click Configure, and select the same infracost-bcm-exports-* bucket — AWS shows the overwrite-policy prompt again, accept it.
8. S3 path prefix: InfracostDailyCostOptimizationExport (reuse this export's name).
9. Review and create the export.

Now that both exports exist and have finished overwriting the bucket policy, open Permissions > Bucket policy on infracost-bcm-exports-* and append a DenyNonSSLRequests statement to whatever AWS generated, substituting your account ID:

{
  "Sid": "DenyNonSSLRequests",
  "Effect": "Deny",
  "Principal": "*",
  "Action": "s3:*",
  "Resource": [
    "arn:aws:s3:::infracost-bcm-exports-111111111111",
    "arn:aws:s3:::infracost-bcm-exports-111111111111/*"
  ],
  "Condition": {
    "Bool": { "aws:SecureTransport": "false" }
  }
}

Exact settings reference

Setting	FOCUS export	Cost Optimization Hub export
Export type	Standard data export	Standard data export
Table	FOCUS with AWS columns	Cost optimization recommendations
Table content version	FOCUS 1.2	N/A
Time granularity	Daily	N/A
Column selection	All columns	Include all recommendations
Compression type and file format	Parquet - Parquet	Parquet - Parquet
S3 path prefix	InfracostDailyFocusExport	InfracostDailyCostOptimizationExport

Step 4: Create the S3 Storage Lens configuration

1. Open S3 > Storage Lens > Create Storage Lens dashboard.
2. General: enter Dashboard name InfracostStorageLensExport, leave Status as Enabled, then Next.
3. Dashboard scope: under Choose accounts, select Include all accounts in your organization (it defaults to limiting the scope to your signed-in account only). Leave Include Regions and buckets selected, with Include all Regions and Include all buckets checked. Then Next.
4. Storage Lens tier: choose Advanced tier, then under Advanced metric categories check Activity metrics and Cost optimization metrics, then Next. Choosing Advanced tier adds a Metrics aggregation step to this wizard.
5. Metrics aggregation (optional step): leave prefix and Storage Lens group aggregation off — not needed for Infracost — then Next.
6. Metrics export and publishing (optional step): under Metrics export, choose Default metrics report, set the bucket type to a general purpose bucket, and use Browse S3 to select the infracost-storagelens-exports-* bucket you created — output format Parquet. Then Next.
7. Review your settings, then click Submit to create the dashboard.

note

Submitting adds another Allow statement to the bucket's policy automatically (visible as a "Bucket policy successfully created" confirmation) — on top of the one you already set in Step 1. That's expected; the two overlap but don't conflict, and no further action is needed.

note

This wizard has more steps than the other sections here, and is the one most likely to have shifted slightly since this guide was written — double check each screen against what's shown above as you go, and let us know if anything doesn't match.

Data ownership and lifecycle

Your AWS account retains ownership of the data. Infracost is granted solely read-only access to the export buckets — you granted this via the infracost-management-account-readonly policy when you set up the cross-account role, plus the S3 access policy below.

Each data exports bucket has a 365-day object lifecycle policy (configured in Step 1) with intelligent tiering applied to reduce storage costs.

note

The data exports are read periodically by Infracost, which may incur minor AWS query costs (~$0.0004 per 1,000 S3 requests). S3 storage costs also apply.

Step 5: Grant Infracost's role access to the export buckets

Add an inline policy to the infracost-readonly role from the core setup guide, substituting your account ID:

1. Open IAM > Roles > infracost-readonly > Permissions > Add permissions > Create inline policy.
2. Paste, using the JSON editor:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "InfracostS3BucketAccessExports",
      "Effect": "Allow",
      "Action": ["s3:GetBucketLocation", "s3:ListBucket"],
      "Resource": [
        "arn:aws:s3:::infracost-bcm-exports-111111111111",
        "arn:aws:s3:::infracost-storagelens-exports-111111111111"
      ]
    },
    {
      "Sid": "InfracostS3ObjectAccessExports",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:HeadObject"],
      "Resource": [
        "arn:aws:s3:::infracost-bcm-exports-111111111111/*",
        "arn:aws:s3:::infracost-storagelens-exports-111111111111/*"
      ]
    }
  ]
}

3. Policy name: infracost-s3-access. Click Create policy.

Optional: provide custom data exports instead

If you prefer to manage your own data exports, or want to provide additional data to Infracost, you can grant access to your own bucket(s) instead of (or alongside) the ones above — add statements for each extra bucket to the same infracost-s3-access policy, following the same GetBucketLocation/ListBucket/GetObject pattern. Contact support@infracost.io to confirm your use case is supported.

If you are creating the FOCUS billing data export manually via the console (as in Step 3 above) but managing it outside of this guide's naming conventions, use these settings so Infracost can still ingest it:

Manual FOCUS settings

Setting	Value
Export type	Standard data export
Table	FOCUS with AWS columns
Table content version	FOCUS 1.2
Time granularity	Daily
Column selection	All columns
Compression type and file format	Parquet - Parquet

Provide the following to Infracost so it can be added to your integration:

Field	Example
AWS S3 bucket ARN	arn:aws:s3:::my-custom-export

Optional: encrypt export buckets with a KMS key

To use a customer-managed KMS key instead of the default SSE-S3 (AES-256) encryption:

1. When creating (or later editing) each export bucket, under Default encryption select AWS Key Management Service key (SSE-KMS), choose your key, and enable Bucket Keys.
2. Grant the Infracost cross-account role decrypt access — add another inline policy to the infracost-readonly role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "InfracostKMSDecrypt",
      "Effect": "Allow",
      "Action": ["kms:Decrypt", "kms:DescribeKey"],
      "Resource": "arn:aws:kms:us-east-1:111111111111:key/mrk-abc123"
    }
  ]
}

Policy name: infracost-kms-decrypt.

3. Update the KMS key's own key policy to grant two sets of principals access — open KMS > Customer managed keys > your key > Key policy > Edit, and add:

{
  "Statement": [
    {
      "Sid": "AllowExportServicePrincipals",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "bcm-data-exports.amazonaws.com",
          "billingreports.amazonaws.com",
          "storage-lens.s3.amazonaws.com"
        ]
      },
      "Action": [
        "kms:GenerateDataKey",
        "kms:Decrypt"
      ],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "aws:SourceAccount": "111111111111"
        }
      }
    },
    {
      "Sid": "AllowInfracostRoleDecrypt",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111111111111:role/infracost-readonly"
      },
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Resource": "*"
    }
  ]
}

Replace 111111111111 with your management account ID in both policies above.

Removing

1. Delete the two Data Exports (Billing and Cost Management > Data Exports) and the Storage Lens configuration (S3 > S3 Storage Lens > Dashboards).
2. Empty and delete the two S3 buckets.
3. Remove the infracost-s3-access (and infracost-kms-decrypt, if created) inline policies from the infracost-readonly role.
4. The AWSServiceRoleForBCMDataExports service-linked role is shared across a service's features, not owned by any one consumer — leave it in place unless you're certain nothing else in the account uses BCM Data Exports.

Troubleshooting

Export creation fails with a validation error

This typically happens for one of the following reasons:

• Management account is not yet opted in to Compute Optimizer and Cost Optimization Hub — see Prerequisites above, then retry.
• You're not working in the management/payer account.
• IAM billing access is not enabled. If this setting is deactivated, IAM users and roles in the account can't access the Billing and Cost Management console pages, even with administrator access.
  • As the root user, open the account menu in the upper-right corner and select Account.
  • Scroll to IAM user and role access to Billing information, click Edit, and check Activate IAM Access.
• Your IAM user lacks cur:PutReportDefinition or bcm-data-exports:CreateExport permissions.

Storage Lens configuration doesn't show data from member accounts

Confirm trusted access for S3 Storage Lens is enabled (see Prerequisites), and that you selected Include all accounts in your organization in Step 4's Dashboard scope screen.