Data from IDC’s AI InfrastructureView study revealed that while artificial intelligence/machine learning (AI/ML) initiatives are steadily gaining traction, 31% of respondents saying they now have AI in production, most enterprises are still in experimentation, evaluation/test, or prototyping phase.
Of the 31% with AI in production, only one-third claim to have reached a mature state of adoption wherein the entire organization benefits from an enterprise-wide AI strategy.
For organizations investing in AI, improving customer satisfaction, automating decision making, and automating repetitive tasks are the top three stated organization-wide benefits.
"IDC research consistently shows that inadequate or lack of purpose-built infrastructure capabilities are often the cause of AI projects failing," said Peter Rutten, research vice president and global research lead on Performance Intensive Computing Solutions.
"With this in mind, IDC set out to probe deeper into the way in which organizations evaluate and invest in infrastructure solutions as part of their AI strategy. Our findings and analysis provide a wealth of data points for vendors and service providers to address the needs of their clients and prospects."
Key findings
Most consequential yet least mature of infrastructure decisions
Organizations have still not reached a level of maturity in their AI infrastructure – this includes initial investments, realizing the benefits and return on investments, and ensuring that the infrastructure scales to meet the needs of the business.
High costs remain the biggest barrier to investments leading many to run their AI projects in shared public cloud environments. Upfront costs are high, leading many to cut corners and thus exacerbate the issue.
People, process, and technology remain the three key areas where challenges lie and where organizations must focus their investments for greater opportunities.
Dealing with data is the biggest hurdle
Businesses lack the time to build, train, and deploy AI models. They say that much of their AI development time is spent just on data preparation alone. Many also lack the expertise or the ability to prepare data.
This is leading to a new market for pre-trained AI models. However, like anything off the shelf, pre-trained models have their limitations, which include model availability and adaptability, infrastructure limitations to run the model, and insufficient internal expertise.
Model sizes are also growing, making it challenging for them to run on general-purpose infrastructure. Organizations do expect that once they have crossed this hurdle, they will shift their efforts to AI inferencing.
AI infrastructure investments follow familiar patterns
"It is clear to us that most organizations have embarked or will imminently embark on their AI journey," said Eric Burgener, research vice president, storage and converged system infrastructure at IDC.
He opined that what is becoming clearer is that gaining consistent, reliable, and compressed time to insights and business outcomes requires investments in purpose-built and right-sized infrastructure.
Businesses are increasing their investments in public cloud infrastructure services, but for many on-premises is and will remain the preferred location. Today, AI training and inferencing, it is divided equally between cloud, on-premises, and edge.
However, many businesses are shifting towards AI data pipelines that span between their datacentre, the cloud, and/or the edge. Edge offers operational continuity where there is no or limited network connectivity.
Security/compliance and cost also play a role. GPU-accelerated compute, host processors with AI-boosting software, and high-density clusters are top requirements for on-premises/edge and cloud-based compute infrastructure for AI training and inferencing.
FPGA-accelerated compute, host processors with AI-boosting software or on-prem GPUs, and HPC-like scale-up systems are the top 3 priorities for on-premises/edge-based compute infrastructure for AI inferencing.
In the cloud, the highest-ranked priorities are GPU acceleration and a host processor with AI-boost, followed by high-density clusters. More AI workloads use block and/or file than objects at this point.
