Most business leaders of today believe the big lie about data: that its value lies in insights.
Millions of careers and billions of market capital are dependent on classical data systems like data warehouses, high-performance query engines, and data visualisation software. In fact, the combined market size of the business intelligence and analytics market worldwide is set to hit SGD$24.5 billion by 2024.
However, there is a common misconception – the value of data lies in actions, and not insights. Most dashboards fail to provide useful insights and quickly become derelict. A usage report of any online business intelligence portal will quickly reveal that 80–90% of all dashboards are rarely if ever accessed.
Meanwhile, the few dashboards that do offer useful insights rarely provide a concrete basis for action, often requiring further multiple laborious analyses.
The limitations of classical data systems
The hard reality is that the capabilities of classical data systems are limited. Most of these classical data systems lack the required intelligence to go beyond providing basic insights, and they end up destroying a big part of the information value of the data they process. While some insights they deliver do have value and are actionable, only a handful result in action.
Business leaders and tech professionals have chosen to accept the big lie about data not because they buy into it, but because it is easier to work with the limitations of classical data systems rather than challenging and overcoming them.
Thankfully, new and upcoming data systems are arriving on the scene and are designed to overcome these existing limitations. These future systems are built with the intelligence to surpass the ‘insights’ of the past and provide business leaders with the recommended next steps that they can take action on.
However, if we are going to understand these future data systems, we need to see how they compare with classical data systems. Classical data systems start with a simple query using Structured Query Language (SQL), and the data is then translated into a chart to aid in comprehension.
But while the analysis can describe the past, it cannot predict how each action might impact the future and does not show the potential return on investment or impact. This results in us having relevant insights, but no clear plan of action.
The fundamental challenge is that it is humanly impossible to look at a billion data points and make sense of them. Instead, we yield to summarising the data to six, sixty or maybe six hundred data points, just enough to be considered a valid sample size.
Compounding the limitations of classical data systems is the constraint on the power of human intelligence. The tools we use in classical data systems to compress data are simply not intelligent enough to retain sufficient information value and produce insights that are truly actionable.
Getting from data to actions with AIÂ
Conversely, new data systems powered by a range of technologies such as machine learning, deep learning, statistical methods, and others are demonstrating the potential to surpass the limitations of human and classical data systems. We increasingly refer to these technologies collectively as Artificial Intelligence (AI).
The crucial difference in approach is that instead of starting with a target insight, new data systems begin with a target action. We use AI to understand each individual customer’s needs, demands, and preferences. We can also leverage AI-powered data systems to make individually targeted communications and deliver more value; one customer at a time.
Beyond extracting every drop of information from the data to develop a detailed and comprehensive understanding, the crux of implementing new data systems is the ability for businesses to harness this understanding to power millions of micro-actions.
