Open Standards in FinOps: The Key to Understanding Cloud Cost
Key takeaways:
- The FinOps Foundation unveiled version 1.0 of its FinOps Open Cost and Usage Specification, known as FOCUS.
- AWS joined the steering committee of FOCUS, joining Google and Microsoft.
- The FinOps Foundation, under The Linux Foundation, continues promoting open best practices, education, and standards around FinOps, including the recent second edition of O’reilly’s Cloud FinOps book
Understanding the cloud bill is damn hard
It starts when the CFO or CIO notices the cloud bill, and goes to engineering asking “Hey, have you noticed that last month we spent a hundred thousand and this month we spent a hundred and fifty thousand? Is this going to continue every month? Can you justify this?” With the current cost-conscious mindset, these conversations are becoming ever more frequent.
Then the engineer, who hasn’t been paying much attention to the cost so far, goes to the cloud bill, to try and figure out the numbers. Only to realize that these bills aren’t really comprehensible.
Let’s take Amazon’s AWS, the biggest cloud provider out there, as an example. Amazon generates two kinds of bills to its users. There’s the one page bill, which is very high level, and therefore not really actionable. I mean, knowing you spent a million dollars of EC2 doesn’t tell you much, and it offers no breakdown.
And then the other bill, the cost and usage report, which is an extremely fine grained JSON report, that can easily get to a multi-gigabyte file, of everything that you do in EC2 that costs money, which amounted to that million dollars you saw on the first report.
Amazon is one example, but Azure, GCP and all the other cloud providers out there provide similarly highly fine detailed, complicated bills that aren’t human readable and need to be processed somehow, yet don’t lend themselves to ingestion and visualization by your usual visualization and analytics tool of choice.
Cloud providers offer some tools, such as Amazon Cost Explorer, that can help people drill down and see how much was on EC2, how much on storage etc., get down to instance level, and so forth, but again these tools are not machine readable.
But most importantly, each cloud provider reports the bill using a different terminology and schema. This makes it difficult to compare different cloud providers, and to operate in a multi-cloud environment across several providers. The need for open standards has become evident.
Public cloud bills can get more tricky with all the negotiated discounts, continued usage discounts (as in Google cloud) and credits left from various plans, which can affect the actual pricing. Doing that reconciliation from the list price can be challenging, especially since these discounts could show up later, days or even weeks after you’ve spent the money.
The rise of FinOps: demystifying the cost of your operations
The pain of understanding the cost of your operations has given birth to the new FinOps discipline. The formal definition of FinOps, according to the FinOps foundation, is:
FinOps is an evolving cloud financial management discipline and cultural practice that enables organizations to get maximum business value by helping engineering, finance, technology and business teams to collaborate on data-driven spending decisions.
Moving from datacenters to the cloud introduced a different cost model, shifting from CapEx to OpEx. Instead of investing upfront capital expenditure (CapEx) in buying a bunch of servers, and waiting for them to be racked and eventually deploying your applications a few months later, you buy on demand using operational expenditure (OpEx). And so financially that’s an entirely different model. This new model offers much more flexibility and potential for cost savings, but its dynamic nature also introduces challenges in tracking and attributing the costs over the shared infrastructure.
This also surfaced the disconnect in organizations between engineering and business. With IT costs being the biggest line item in the company’s expenses besides salaries, it becomes crucial to optimize it. Yet the business and finance people themselves cannot do it, without understanding the technicalities of the cloud and the applications running on it, mastered by engineering.
For this reason, FinOps is not just about providing observability into the cost unit, but also about creating the culture of an ongoing conversation about cloud costs, bringing the business and finance side of the house to talk with engineering or with product. This is no trivial task, as these groups inherently think differently: engineering with the agile and fail fast approach, versus finance with a conservative, risk-averse and formal process-oriented approach.
These core challenges drove the establishment and of the FinOps discipline and related open standards, which are integral drivers for compatibility, collaboration and convergence. The FinOps Foundation was then founded to steward this as an open source community endeavor.
The FinOps Foundation: Standardizing on FinOps
The FinOps Foundation was founded back in 2019 to advance best practices, education, and standards around FinOps. Shortly after, in 2020, it merged into The Linux Foundation. Yet the project is not about software code and tools, but rather about principle and guidelines. It covers things such as the personae, maturity model and principles. I particularly found the formalization of the principles very useful for new entrants, so let’s have a quick look at that:
The FinOps Foundation formalized six core principles of FinOps (my notes in parentheses):
- Teams need to collaborate (esp. Business & Finance with Engineering & Product)
- Everyone takes ownership for their cloud usage
- A centralized team drives FinOps
- Reports should be accessible and timely (observability plays a key role here)
- Decisions are driven by business value of cloud (internal benchmarking, trend analysis)
- Take advantage of the variable cost model of the cloud (rightsizing instances, compare pricing between services and resource types)
The FinOps Foundation has an active community with over 3500+ companies involved. It pursues many avenues of FinOps, as can be seen in its active community slack workspace with the many channels dedicated to different clouds, toolsets and working groups.
You can find the slack details and many useful resources on the foundation’s website, conveniently named finops.org. The foundation also organizes the global FinOps X event series, and has published the Cloud FinOps book with O’Reilly, with the second edition released earlier this year.
The FOCUS Project: Open Specification for Cloud Bills
A few weeks ago the FinOps Foundation unveiled version 1.0 of its FinOps Open Cost and Usage Specification, a.k.a. FOCUS™. FOCUS delivers consistency and standardization to cloud cost data, enabling companies to better visualize the value of cloud.
In particular, it aims to address multi-cloud scenarios, where users utilize more than a single cloud provider. FOCUS aims to simplify multi-cloud billing data and improve reporting consistency between multiple vendors. The goal is to remove complexity and overhead from processes such as allocation, chargeback, budgeting, forecasting, and the other FinOps capabilities, by using a vendor-neutral and cross-cloud specification.
FOCUS originated from the FinOps Foundation’s FinOps Open Billing working group, but has spun off to its own separate open source project, sponsored by the FinOps Foundation.
FOCUS version 0.5 was announced in May 2023. In its public github repo, it outlined a common schema for billing datasets, which normalizes key cost and usage dimensions and metrics to simplify ingestion and analysis of cost and usage data with any analytics backend of choice. The schema has since evolved into the current v1.0.
Alongside the v1.0 announcement, the other big news is that AWS joined the steering committee last month. AWS, the largest cloud vendor, is joining Google and Microsoft that were already on the steering committee. Having all the three leading cloud providers on the steering committee and adopting the open specification is a prerequisite to enabling the specification to really become an open standard, as these three run together over 90% of today’s workloads.
The list of contributor companies also includes smaller cloud providers IBM, Oracle and VMWare, as well as major players such as Meta, Goldman Sachs, Capital One, Accenture and Walmart, which can further increase the adoption and popularization of the specification among end-users.
Endnote
As organizations migrate to the cloud, understanding the infrastructure bill becomes more challenging. This drives the need for formalization and standardization of FinOps. The FinOps Foundation and its FOCUS specification offer an open standardization effort to converge the industry.
Organizations that produce or consume billing data will benefit from the increased understanding and trust in billing data due to the clarity that such a standard can provide.
The real test, however, is in implementing the specification in their products and services, and making it their ubiquitous format for reporting and consuming cloud cost data.
In my following post, I look into the world of Kubernetes, microservices and containerized workloads, and the complexity that it adds to our FinOps challenge, and how open standards can help us there.
