Corporate cloud strategies that yesterday centered on the unglamorous transition of legacy apps from out of the data center, along with some new cloud-native apps, must now incorporate transformative technologies with radically different value propositions — all without breaking the IT budget. Public cloud provider AI managed services such as Amazon Bedrock, Azure AI, and Google Cloud’s Vertex have put large language models at the fingertips of both IT teams and business users.
Rise of the AI cloud
My colleague Mike Gualtieri and I wrote The Rise of The AI Cloud, a report that provides guidance on integrating AI into your cloud strategy and that examines the potential and limitations of running AI workloads in the cloud. Certainly, cloud isn’t always the best infrastructure for AI workloads. Why? See Mike’s comprehensive evaluation of the AI infrastructure market and his related blog. But for many users, the accessibility of cloud AI managed services may be the fastest on-ramp.
Our report on the AI cloud isn’t an evaluation or ranking of any particular cloud provider. Rather, it’s an overview of the key considerations that cloud customers should have as they sort out which cloud offerings fit and which don’t — and how this can change market dynamics. AI workload requirements may align well with your current cloud strategy in some cases; in others, they may collide.
For starters: Does the cloud provider actually have the GPU resources that are needed to meet an organisation’s goals? Is cloud providers’ custom AI-oriented silicon — currently on the market from AWS and Google and announced by Microsoft — good enough, or is NVIDIA the must-have choice? Are NVIDIA-rich upstarts like CoreWeave and Vultr viable alternatives? Or does Oracle Cloud Infrastructure’s SuperCluster have a place in the mix?
AI strategy
The answers to the above questions will differ based on an organisation’s priorities and AI strategy. In some cases, AI capabilities may come built in by providers that leverage hyperscalers as a platform, such as Salesforce Hyperforce or Snowflake Data Cloud. In each case, successful delivery of AI infrastructure requires maximising performance for a range of AI workloads, including data preparation, model training, and inferencing.
The explosion of easy-to-use cloud AI options and the pay-as-you-go options have been alluring to cloud customers, if cloud provider earnings reports are any indication. But just because it’s easy to get going with an AI cloud doesn’t mean that it’s the best choice for AI infrastructure in every case. Cost and complexity can quickly mount as users wrestle with data and optimise inferencing, which can lead to greater spending then anticipated. Our report explains why certain core AI infrastructure capabilities can make cloud an awkward and pricey fit in many scenarios. That’s a big reason why some major cloud providers are backing an open-source effort to optimise Kubernetes for AI/ML workloads.
Whichever approach users take to AI/ML, it’s bound to reshape cloud strategy, either by embracing it fully, using it selectively, or by avoiding cloud AI managed services in favor of on-premises alternatives.
Originally posted in Forrester
