In today’s technology-driven landscape, data has become the lifeblood of organisations throughout the Asia Pacific & Japan. According to Dell EMC Global Data Protection Index, organisations in Singapore manage 9.15 petabytes of data in 2018 on average, a 458% increase from 1.64 petabytes in 2016. Organisations are also beginning to tap into this data through machine learning and deep learning, with 34% of business leaders in Singapore monetising data to some degree.
For many of these organisations, it is no longer about the quantity of data but rather the quality and diversity of this data. Without diversity of data, it becomes easy for data bias to become a problem, which will lead to the delivery of sub-par products and possible legal repercussions due to discrimination against individuals or groups.
In this first part we talk to Romain Bottier, HPC & AI senior solution architect, South Asia, Dell Technologies about how businesses can move from merely analysing data into making proper use of it with ML and AI tools.
In Part 2 we look at the importance of security.
How can businesses move from Big Data into AI/ML?
Many, if not most businesses, struggle with the transition from Big Data into AI/ML because they lack the necessary skills and experience needed to extract value beyond big data approaches developed within the last decade. Rather than diving head-first into sophisticated technologies and products, businesses should take a practical approach to the AI/ML journey.
Businesses should start with simple and practical projects which will give them the expertise they need to handle more advanced projects and technologies. This could take the form of adding ML extensions to existing applications before building AI value-added services and complex products based on deep learning. Importantly, the move into AI/ML should also be supported by training and consulting services by a business partner or a full-service solutions vendor.
How are companies currently using AI or ML to analyse data? Can this analysis process be fully automated?
Companies like Mastercard aren’t the only ones diving deep into AI and ML. Governments have also begun to invest in AI, looking to boost their economies through home-grown AI solutions. One such government-funded organisation is AI Singapore, which is currently building a range of cutting-edge AI solutions and products for various industry partners in the fields of health, urban development and finance. AI Singapore recently announced that they had selected Dell Technologies to deliver High-Performance Computing (HPC) infrastructure that is optimised for AI workloads to help drive performance and flexibility for its researchers and to scale up its flagship 100 Experiments (100E) programme.
With the increasing usage of HPC infrastructure, we are also seeing AI being used to analyse data in unprecedented ways. In Australia, the Garvan Medical Institute is currently using extensive big data analytics and deep learning capabilities to analyse genomic data – a type of data that would ordinarily take up to 700 hours to process – at an expedited pace. While complicated processes like these are far from being fully automatable, we are seeing data being analysed much faster than ever before.
Are the traditional methods of tiering data still relevant?
Traditional IT concepts are fast shifting with emerging technologies. For a long time, data tiering was based on how often certain data is accessed with the aim of eventually moving the least-accessed data to cheaper storage mediums.
However, with AI/ML now in play, data is being accessed according to the business relevance of the said data and the actual business problems that companies are trying to solve. From data streaming to in-memory databases to standard NAS, parallel filesystems or even object storage, data is being moved based on business relevance.
Another aspect impacting tiering of data would be data governance and data privacy, where data pre-processed for a specific purpose might not be shareable across business units and lines of business. Data governance implementation, along with new tiering flows, are becoming mandatory for AI projects.
