Most cloud providers enable users to set budget alerts on their actual cloud spend. This is a critical safety net as usage-based resources could incur a lot of cost (e.g. data transfer). There is also another safety net that companies should set up, and that is catching significant cost changes to their infrastructure before going live. For example, finding out how much increasing the RAM for a Lambda function costs before putting the new function into production. Usage estimates can also be considered during cost estimation.

Infracost is an open source tool that can be put into CI/CD pipelines (GitHub, GitLab, CircleCI, Bitbucket and Atlantis…) and will leave a comment with the cloud cost implications of changes to your infrastructure-as-code: "this change to your terraform file will increase your cloud bill by 25% next month".

In some cases, you may only want an Infracost comment when a threshold is reached. For example, if the cost implications of the change are minor (e.g. under 3%), then no comment is needed. We have now added this ability into Infracost - from our CI/CD integration docs, select your CI system, and set the percentage_threshold flag.

We hope Infracost can help your enterprise become more cost-aware when it comes to cloud spend, and maybe we can help reduce that 30% cloud waste! ^[1]

As always, if you have any feedback, add an issue to our GitHub repo, join our community Slack channel, or reach out to me directly on Twitter: @hassankhosseni.

[1] Flexera 2020 State of the Cloud Report

Last week I wrote about giving cloud cost estimates to DevOps teams via pull requests as they make changes to infrastructure components. The hope is to create a “Prius Effect” for cloud costs: it was observed that many Prius drivers would drive more efficiently simply because they were presented with immediate feedback on the Prius dashboard.

In this blog, I'd like to answer a question that a user posed to us:

We need to know when significant budget changes are expected so we can plan for them before the money is spent. Billing alerts help us react to unexpected changes but they still can be a surprise. My DevOps now have costs in pull requests. What about my team leads and managers? They can’t go through every pull request to see what the cost changes are.

When you have a single engineer with access to change infrastructure a simple discussion about significant cost changes will suffice. Once your team grows to multiple engineers or multiple teams a process can really help.

Infracost now has a new infracost report command which generates a table or HTML report from multiple Infracost JSON files. The output shows a breakdown of all the cost information in a straightforward format alongside tags. You can then upload these reports to AWS S3 and share them with management and team leads.

Example command:

This is the output you'd get in HTML format. Notice that the filename and all tags are shown:

This is our first solution to this use-case. I'd love to hear your feedback so we can iterate on it and improve it! This is available now. Please see our Infracost Docs: Report section for usage instructions.

If you have any feedback add an issue to our GitHub repo, join our Community Slack channel, or reach out to me directly on Twitter: @hassankhosseni.

Over the last 10 years we have seen more enterprises give direct cloud access to their developers and DevOps teams. There are many ways this has been achieved, from giving admin logins to the cloud console, to having devs create request tickets to the central IT team, to creating a self-service catalogue in which business units can select what they need from a list of pre-approved resources or environments.

What ever model enterprises have chosen, one common thing happens. The cloud bill becomes less predictable. Someone didn't know what they were launching, they didn't turn it off, they didn't consider that when you create one resource, it will auto-create a bunch of other resources. As we shift left with Infrastructure as Code, and further with Serverless, in which code decisions have a direct impact on the cloud bill, we need a better answer than just policing what developers have access to and doing post-bill analysis.

Myself and my co-founders have been working in the cloud cost space for 10 years, and we now want to address this issue. Our theory is that if we give developers and DevOps engineers a clear picture of costs of their infrastructure before they launch, then they will be more mindful about what they launch. Think of it like The Prius Effect in which it was observed and documented that a large subset of Prius drivers would respond to the data on the dashboard by driving in a manner that decreased fuel consumption, but for cloud costs.

We launched infracost.io, an open source tool which looks at a terraform project and creates a cost estimate for the resources directly in the CLI:

You might say “developers don't care about cloud costs since they don't pay for it”. I would say two things:

1. The traction we have seen with Infracost indicates to us that this assumption might be false and that developers do care. They are just unaware of cost implications due to the complexities of cloud pricing models.
2. If a developer doesn't care about cloud costs, what if we put Infracost in the CI pipeline so the cost implications can also be peer reviewed in the pull request? Will a single person in a team pull the cost efficiency up?

This is what Infracost looks like if you put it into the CI pipeline, a comment summarizes the percentage change in the cloud cost due to the changes in the terraform project, as well as a detailed breakdown:

Our Theory#

We are working on validating this theory: can we get better cloud cost efficiency if we give developers clear visibility into potential cloud costs directly in their workflow with no added effort. I'd like to invite you to join this journey, and let's see if this theory is right or not.

1. Install Infracost - it's free and open source.
2. Write a blog, or a tweet about the impacts you see in the short, medium and long term. Do developers talk more about costs? Do they change their code? Are there comments in peer review process about costs? Do others suggest different resources to be used? What else are you seeing?
3. Share the blogs with me, tag me in tweets etc, and I will collate all the results and write another blog with a summary and link to your results.

Cheers!

Infracost helps developers and DevOps engineers get cost estimates from their IaC (Infrastructure as Code). Here's an example of it running:

The complexity of cloud costs keeps increasing - When we were building PlanForCloud in 2012 AWS had just hit 10,000 different pricing points for their services - now there are over 300,000.

We found existing tools fit in too late in the process and are not aimed at the people in control of the infrastructure. It's difficult to get cost estimations when you are building and deploying your services, which often leads to bill shock and no easy way to track down these costs. So we wanted to build a CLI tool that can plug into your existing development and operations processes and bring cost visibility to the engineers.

Currently Infracost supports AWS and Terraform, but we will add support for more cloud vendors (GCP, Azure) and other IaC tools (Pulumi).

Update 25 Nov 2020 : We have now added initial support for Google cloud.

We also want to go beyond just the baseline costs of infrastructure - data transfer costs and other usage-based costs can often be a significant portion of a cloud bill, and are also the hardest to predict and track down. If you have any ideas about the best way to handle these then please reach out to me on Twitter.

Infracost is open source, you can check it out at https://github.com/infracost/infracost.