In a previous post, I discussed Azure’s AI and ML offerings: their types and purpose and presented a simplified decision workflow.
In this post, I’ll apply these concepts to a typical example, showing the importance of using an informed methodology – resistant to hype – to help you make good choices for your organization.
Understanding Azure Cognitive Services Pricing Metrics
Consider the graph shown below, which illustrates the cost sources of the major Azure Cognitive services:
You can download a copy of this graphic here.
The Athena Fashion Co. Example
Now let’s imagine a firm, a fashion jeans company called Athena Co. Athena’s leaders have decided there’s competitive advantage to adding a speech to text feature to their customer service platform and have selected Azure Cognitive Speech Services (included in the picture shown above) as the preferred engine.
Azure Cognitive Speech Services, is an API offering which Microsoft describes as enabling you to “build voice-enabled apps […] with the Speech SDK. Transcribe speech to text with high accuracy, produce natural-sounding text-to-speech voices, translate spoken audio, and use speaker recognition during conversations. Create custom models tailored to your app with Speech studio.”
Note the pricing tiers of ‘Free’ and ‘S0’. Within the S0 tier there’s a variety of cost metrics which must be understood to give you the ability to forecast spending based on expected usage.
In many, perhaps most, organizations, the level of detailed work required to get a handle on AI/ML costs is missing. There’s an emphasis on technical capabilities but much less on the synergy of technical skill and business outcome understanding that comprises FinOps.
How do you build this skill?
Building a Doctrine: Applying FinOps to Cognitive Services
If you read the previous post, you saw this workflow:
This is a good start for a use-case in which you need to analyze a large, and ever growing amount of data and apply statistical algorithms (for example, a fraud detection solution for a financial institution). In our present scenario however, in which we’re consuming an API to enhance a service, this doesn’t directly apply and needs to be modified:
Returning to our Athena Co. example, we can use the doctrine shown above to create an approach. Athena’s business leadership decided that adding a speech to text function to the company’s customer service platform provides a competitive advantage and the dev team, realizing that attempting to create such a service (instead of consuming it as a utility) would not be a good use of their time, chose an Azure API service to fulfill the requirement.
Having gotten past the ‘do we need it?’ step, the team’s next move was to calculate potential costs based on the selected service’s tier and cost metrics.
The Azure Pricing Calculator is key:
Note how the service’s S0 tier contains several sub-services as distinct cost metric lines. It’s possible, for example, to consume the Speech to Text service while not making use of the Custom Speech Endpoint. Choosing which service(s) to consume should be foundational to the decision doctrine. It’s common for companies to stumble into over consumption via a combination of technologist enthusiasm, lack of granular and consistent cost monitoring and the unfamiliarity of business personnel with the technical elements and oversight opportunities (FinOps).
Finally, it’s critical to make use of the decision aids Microsoft offers. Consider, for example, the article, Choose a pricing tier for Azure Cognitive Search:
Summary
It’s possible to apply FinOps principles to Azure AI/ML services but it requires several steps:
1.) Deciding if an AI/ML service is needed (or fits a desired use-case)
2.) Deciding if the model(s) should be built or consumed as a utility and tailored per requirement
3.) Deciding what tier to use
4.) Forecasting spending based on cost metrics (per selected tier elements)
5.) Monitoring spending and analyzing outcomes from return on investment and cost of goods serviced perspectives
In the next post, I’ll dive deeper into the fifth point, to show how the use of AI/ML services can be tied to business outcomes.
Azure FinOps 