If you’re in the business of deploying technology to, well, business, unless you’ve been sheltering in a cave, you’re aware that Microsoft has deepened its partnership with OpenAI by making the latter’s services available via Azure.
Since my focus is on FinOps and the impact of an organization’s technology choices on cloud runtime costs, one of my first thoughts upon learning about the Azure/OpenAI integration was, how much does this cost and what are the cost elements? And a follow-up, what are the metrics for calculating the cost of Azure OpenAI services?
As always, the first place to start is with the Azure Pricing Calculator:
You can find the OpenAI offerings by typing OpenAI in the search products field:
Here’s the default view of the OpenAI pricing calculation widget:
As you can see, the four pricing elements are:
- Region
- Series
- Model
- Tokens
Region aside, all of these terms, in this context, are unique to the world of machine learning and large language models so definitions and clarifications are needed to help you understand cost scenarios.
Series and model refer to the OpenAI model (you can learn more about available models here) and the series of that model you choose as the engine for your project. Here are the options for series:
And here are the options for models:
Obviously, making the appropriate choices requires significant expertise with AI methods so for organizations, the challenge, from a FinOps perspective, will be translating the technical requirements, as interpreted by an organization’s AI or data analytics team, to terms understandable to finance and other stakeholders (i.e., how can these choices be justified from a unit economics point of view).
Here’s a look at some of the other cost generating elements
Once again, we see options that require a high degree of knowledge of machine learning terminology for informed configuration. How many training hours are needed? How many tokens for inference and why? These are some of the key questions.
There’s much more to dive into but this post isn’t a primer on AI or machine learning but rather, the cost factors associated with deploying this technology. For a good overview of how OpenAI technology is deployed via Azure, I suggest the article, What is Azure OpenAI Service?
The good news for organizations that utilize cloud (practically everyone, at this point) is that awareness of FinOps as a practice area is growing. The more challenging news (not exactly ‘bad’ but not necessarily an easy ‘good’) is that as more services are added to cloud platforms the difficulty of building an effective FinOps practice also increases.
The key to success is understanding how elements of a cloud solution generate cost. When it comes to tried and true technologies, such as virtual machines, it’s fairly straightforward: factors such as disk size and type, memory and processor class are well understood. The same can be said of PaaS databases which, although newer than VMs, have cost generating elements that are, although translated to cloud terms, mostly familiar.
Technologies such as AI and machine learning, made available via API, introduce a new type of FinOps challenge and will require unique types of coordination to ensure their use is performing for your organization.
Resources
What is Azure OpenAI Service?
https://learn.microsoft.com/en-us/azure/cognitive-services/openai/overview
Azure OpenAI Service pricing
https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/
Azure OpenAI Service models
https://learn.microsoft.com/en-us/azure/cognitive-services/openai/concepts/models
Learn how to customize a model for your application
Azure FinOps 