Organisations will not be able to move their artificial intelligence (AI) initiatives into production, if they fail to identify key components during the pilot stage.
“Ask yourself what will scale and what the pilot is supposed to deliver,” said Bouke Hoving, CTO of wholesale banking at ING Group.
The Dutch multinational banking and financial services provider was able to transition more than 90% of its GenAI (generative AI) pilots to production last year.
It appears to buck a trend where organisations struggle to move past their AI pilots.
In fact, 71% remain stuck in the “builder” phase, where they struggle to scale their pilot projects into actual production environments capable of delivery measurable returns on investment (ROI), according to research commissioned by ST Telemedia Global Data Centres. Conducted by Ecosystm, the study surveyed more than 600 enterprises across nine Asian markets, including India, Japan, Singapore, South Korea, and Thailand.
Asked why ING was able to achieve a high AI pilot to production conversion rate, Hoving said the bank zoomed in on a handful of key focus areas and worked to scale these.
It spent time collating feedback and studying the results from its AI pilots, he told FutureCIO in a video interview.
More time was spent on analysing the project’s potential to scale, he said.
If a pilot worked well in one market, ING then would look to replicate that success in other global markets, while keeping in mind local regulations that it had to adhere to, he explained.
The bank would look at how the project could work in the local market, before deciding to scale out to global markets.
“If you don’t have the scalability from that perspective, and don’t have the data available [to power the AI models], there will be big [barriers] in terms of scaling your pilots,” Hoving said.
ING aimed to ensure the pilot can benefit and be scaled to more than 20 markets, or it would not start on the project, he said.
This allowed its scarce resources to be redirected to other more interesting projects, instead of those that had limited potential to scale, he noted.
ING’s engineering team, too, had started tapping Microsoft Copilot and coding agents, leading to faster deployment, he said.
Too much rework means less returns
The bank also would evaluate how much reengineering work was required to scale the AI solution, Hoving said.
Pilots often fail, even if the technology is scalable and data available, because processes require too much reengineering to get any meaningful result, he explained.
“If you don’t have the time or if you have to spend too much time redesigning the process, you’ll typically see the pilot generate only marginal results and it’s not worth scaling it out,” he said.
On the flip side, if the project requires minimal reengineering, then it has the potential to scale well, he noted.
ING’s GenAI projects include a chatbot that has been deployed across most of its global retail markets, automating 65% to 75% of basic customer questions.
ING worked with McKinsey to develop and test the customer-facing chatbot, with the aim to cut customers’ waiting time to get help.
ING’s existing classic chatbot resolved 40% to 45% of customer calls, which clocked 85,000 by phone and online chat every week in the Netherlands.
The bank wanted to improve call resolution and customer experience with the GenAI-powered chatbot.
Guardrails were applied to mitigate risks, including ING-specific guardrails, for example, to prevent providing advice on mortgages and investment products.
The GenAI chatbot was initially tested on 10% of customers in the Netherlands who used the chat function on the bank’s mobile app.
ING and McKinsey then developed a transition plan and scalable model that could be extended to ING’s other global markets.
Agentic to automate more decisions
The bank’s GenAI efforts also established the necessary foundation for the next phase of its AI journey, where agentic AI could be deployed to augment work processes and improve efficiencies, such as loan and mortgage approval.
AI agents, for instance, can be assigned to analyse annual reports of companies that apply for loans, so ING staff can then decide whether to grant the loan, Hoving said.
AI agents also can extract data, check against policies, and generate documents for approval.
There are many opportunities where agentic can facilitate more automation in decision making, but with the necessary safeguards in place, he said.
Humans should still review the results and AI agents must escalate to humans, if fulfilling certain tasks might require them to move outside the guardrails, he noted.
The essential controls are in place, with humans always in the loop, he said.
“There will always be guardrails guiding the agents on what they can and cannot do,” he added.
Hoving noted that escalation to humans might reduce over time, as AI continued to advance and processes optimised, but the need for human oversight would always remain, especially for the FSI (financial services, insurance) sector.
The Singapore government recently urged organisations to be more deliberate in their AI adoption, with human involvement essential for some tasks.
“We cannot charge ahead driven solely by commercial considerations, even as recursive AI systems gain self-reinforcing intelligence, agency, and influence,” said Ong Ye Kung, Singapore’s Minister for Health and Coordinating Minister for Social Policies.
“Otherwise, the machines just seem wiser than their makers,” Ong said. “We must be wiser, more humanistic, and practical. We must decide deliberately where to embrace AI, where to rein it in, and where human judgement and effort must prevail.”
He added that human oversight remained essential in safety-critical systems, while work still should be primarily carried out by humans in sectors where human trust and empathy were paramount.
