Enterprises may have to make major adjustments to their IT infrastructure in order to support agentic artificial intelligence (AI) workloads, with most likely needing to do so if they want to deploy AI agents at scale.
Organisations that already are running generative AI (GenAI) today will be able to work with agentic AI with more ease. There will, however, be differences if they want to do so at a larger scale, Soo Mei May, chief of AI global solution specialist at Dell Technologies, said in a video interview with FutureCIO.
And if they do not already have GenAI running, organisations will likely have to make big adjustments if they want to deploy AI agents, said Soo, who is based in Singapore.
Agentic AI systems need large language models (LLMs) and the latter run best on servers powered by GPUs (graphics processing units). Traditional servers cannot support these AI models, so organisations will need to bring in the souped-up servers if they want to roll out AI agents, she said.
They also will need plenty of storage and “tool calling”, she added.
Tool calling allows agentic AI systems to connect and interact with various tools and services, including APIs (application programming interfaces) and databases, to carry out their tasks. Sometimes referred to as function calling, it enables AI agents to autonomously access and communicate with external resources, such as querying databases and retrieving real-time data, to complete a workflow.
Tool calling is essential for AI agents to execute their tasks, Soo said.
And the larger the number of concurrent users that is expected to utilise these agents, the larger the compute power needed, she said.
Multiple LLMs also may be needed to power the different tasks that AI agents perform, including web searches, summarisation, and internal search queries, she noted.
For instance, when given a prompt, the agentic AI system has to check the request against the company’s established guardrails. It then accesses the LLM and plans out the steps needed to fulfil the prompt.
In AI agent orchestration, these steps or tasks are distributed to the respective AI agents to be carried out. One agent, for example, may have to search an internal repository to determine a customer’s last interaction with the company, while another agent has carry out a web search for price comparisons.
After each task is completed, the AI orchestration agent assesses whether it was performed well enough and, if not, instructs the agent to redo the task.
This is done for all tasks that are needed to respond to the user prompt and everything is processed within milliseconds.
Decisions needed to keep cost efficient, optimised
To ensure there is low latency, organisations will need sufficient compute power if they want to run a large-scale agentic AI environment.
They also need to optimise the whole process, Soo said, pointing to human skillsets and expertise that understand their organisation’s workloads, so they can design an infrastructure that is most efficient.
This can further optimise the compute power they need, leading to a more cost-efficient agentic network, she said.
Extra thought also will be required for the storage and memory component, she noted.
Both short- and long-term memory resources are needed to power an agentic AI environment, so businesses will have to assess which and, how much of each, is needed for different tasks, she explained.
Putting everything on “hot” storage, to provide fast access to often-needed data, can be a costly endeavour.
Again, organisations will need the right expertise to decide how this is best optimised, Soo said.
Enterprises that already are running GenAI in their network can leverage this infrastructure to implement agentic AI.
Those that do not, will have to overhaul their IT infrastructure to fit agentic AI into their traditional IT environment, she said.
While they can tap cloud as an alternative, this option is significantly expensive, in particular, if they want to roll out agentic AI across their entire organisation, she noted.
Each AI agent will require tokens to complete its task and will need more, if it has to redo a task. With potentially hundreds of thousands of AI agents running each day in a full-scale agentic AI enterprise environment, cloud costs can skyrocket.
Organisations then will have to decide what they want to run on the cloud and on-premises to better manage their cost, Soo said.
Some 24% in Asia-Pacific already cite unpredictable rising cloud costs as a key challenge in their GenAI strategies, according to IDC research commissioned by Akamai Technologies.
Released last month, the study found that 31% of organisations in Asia-Pacific have moved Gen AI applications into production, while 64% are running tests.
Across Asean, 91% of businesses expect GenAI disruption within 18 months, the report noted. Some 16% in the region have pushed GenAI applications into production, while 84% are in the pilot phase.
Another 96% are adopting public cloud IaaS to support their AI workloads.
By 2027, 80% in the wider Asia-Pacific region will depend on edge services to support their AI workloads, IDC projected.
“GenAI is shifting from experimentation to enterprise-wide deployment [and], as a result, organisations are rethinking how and where their infrastructure operates,” said IDC’s Asia-Pacific research director Daphne Chung. “Edge strategies are no longer theoretical. They’re being actively implemented to meet real-world demands for intelligence, compliance, and scale.”
“AI is only as powerful as the infrastructure it runs on,” said Parimal Pandya, Akamai’s Asia-Pacific senior vice president of sales and managing director. “Asia-Pacific businesses are adopting more distributed, edge-first infrastructure to meet the performance, security, and cost needs of modern AI workloads.”
As it is, 70% of AI-related cloud workloads contain at least one critical vulnerability, compared to 50% of non-AI workloads, notes Tenable’s 2025 Cloud Security Risk report, which looked at AI workloads across Amazon Web Services, Microsoft’s Azure, and Google Cloud Platform.
“Organisations have made real strides in tackling toxic cloud risks, but the rise of AI workloads introduces a fresh wave of complexity,” Ari Eitan, Tenable’s director of cloud security research, said in the report. “AI’s data-intensive nature, combined with persistent misconfigurations and vulnerabilities, demands a new level of diligence.”
Checks and balances just as essential
This includes transforming their cybersecurity strategy to ensure a safe agentic AI architecture, Soo said.
Their security team also will need to understand data science and neural networks, including how LLMs behave and AI workflows, she said.
While most companies would have put in place guardrails and rules for AI, agentic AI is highly autonomous and will need closer scrutiny, she noted.
She suggested that organisations establish a structured workflow, so they know every detail of each workflow, what each agent is tasked to do, and what each agent is only permitted to do. These include having access to and searching only specific LLMs and databases.
“The guardrails you put around each of the agent will be essential, so the agents don’t have full liberty and are controlled, rather than have full autonomy,” she said.
Topmost, it is critical that organisations have a robust data strategy and maintain a “clean” data platform, Soo said.
Agentic AI systems need rules to know how to handle sensitive or proprietary data, and they cannot do so if the organisation itself lacks clarity on the data it has and does not put in place the right rules.
Data should be properly labelled and should be accurate and updated, she said.
“There’s a lot of housekeeping that needs to be done,” she added, noting that this can be challenging for an enterprise-wide agentic AI environment.
A chain of accountability also should be established and there are grey areas here, according to Soo.
Who decides the AI agent’s task is good enough and how many checks should be done before a task is deemed completed, she posed.
Should the person who designed the agentic system be accountable? Should the technical experts be responsible when they may not be most familiar with the business logic behind the workflows?
“So ultimately, who’s responsible in the chain of responsibility?” Soo said. “When agents give the wrong answer, who’s at fault? It’s murky right now.”
“And AI agents run autonomously, so it can be difficult to determine the responsibility and accountability,” she said.
She recommended that companies looking to adopt agentic AI start with a good data management plan.
AI agents today also remain erratic and hallucinate, so they should not yet be trusted to handle highly sensitive data, she added.
