“The reports of my death are greatly exaggerated,” said author Mark Twain in 1897 in response to reading his obituary. In the data storage business, similar claims have been made of hard drives over the years.
But like Twain, a prolific writer, the hard drive continues to prosper, housing nearly 90% of all data in hyperscale cloud data centres, where the vast majority of the world’s exabytes reside. With the dramatic increase in the volume of data being generated, from the cloud and AI, the demand for mass-capacity storage has never been greater, especially in Asia.
Of course, increased demand for storage isn’t something new. What’s new is the speed at which the data needing storage is skyrocketing. Market researcher IDC predicts 291ZB of data will be generated in 2027, up from 129ZB of data in 2023, representing a 2022-2027 CAGR of 22.4%.
According to Statista, the volume of data created, captured, copied and consumed globally has increased 60 times from 2010 to 2022, with rates set to continue accelerating.
Driving this is the deployment of Gen AI and LLM use cases. The best AI models are and will be developed using large volumes of data to train and optimise them and are therefore largely dependent on mass capacity data storage. Market-leading companies will increasingly want to both keep their raw data sets and the results of AI processing.
Think of Tesla and the huge amounts of data that it collects from its millions of electric vehicles on the roads around Asia Pacific which can be used to train AI driving models and mapping.
Social media is another example. Platforms such as Xiaohongshu, Facebook, LinkedIn, and YouKu collect massive volumes of data, analyse it using AI and then serve targeted, customised adverts to users. Booming global advertising revenues of US$679.8 billion in 2023 is a testament to the success of the business model.
Housing all that data
To cater to this increase, more storage capacity is needed and that requires more physical data centres housing denser, higher-capacity storage devices.
Most communications service providers and data centre operators use a mix of hard drives and solid-state drives (SSDs) deployed in a symbiotic manner. They design storage architectures consisting of a carefully optimised mix of storage media that consider the cost, capacity, and performance requirements of the workloads they support.
As data volumes skyrocket, both hard drives and SSDs will continue to be in high demand, with hard drives serving high-capacity workloads, and SSDs meanwhile addressing workloads with higher performance sensitivity.
Hard drives to the fore
To satisfy the forecasted data storage increase, hard drives must get bigger and better in terms of storage capacity. This means the areal density of the drives needs to greatly increase.
Fortunately, the hard drive industry has been hard at work addressing this issue with new technologies such as heat-assisted magnetic recording (HAMR) which allows disk platter density to be scaled exponentially.
First-generation HAMR drives allowed for 3 Terabytes (TB) per platter as compared to 2.4 TB per platter, the highest-capacity conventional perpendicular media recording (PMR) drive. Future generations of HAMR-enabled drives are expected to further boost areal density to 4 or 5 TB per platter. This equates to 40 to 50 TB drives shortly.
By allowing for increased storage in the same form factor, data centres across the region can easily swap out their existing drives when they reach the end of their lifespan and scale the storage capacity of the whole data centre at minimal cost and with little to no disruption.
At multi-petabyte and exabyte scale, replacing a 16TB conventional PMR drive for a 30TB HAMR-enabled drive can yield significant savings in terms of power, and other operating costs.
Indeed, many countries have had a moratorium on data centre deployments because of the impact it has on national power generation capacity. Singapore is a good example which just ended a four-year restriction of data centre new-builds and is allowing construction again.
Sustainability at scale
Innovation in hard drive areal density is a powerful solution to the problem that many data centre operators are experiencing in managing data abundance with limited resources.
Areal density equals greener data centres and more sustainable storage in general. Improved energy efficiency reduces the data centre’s environmental impact and operational costs while improving performance in the long run.
Conclusion
As we stand on the brink of the AI wave, current industry conversations mainly focus on AI’s disruptive potential, the growing demand for AI processors and compute engines, and the need for yet more storage capacity in the cloud.
Sometimes overlooked in this discussion is that cloud storage relies on hard drives. For the full potential of AI to be realised will require a dramatic scaling up of hard disk drive capacity through increased areal density.
