Data Protection Series: Are You Ready for the Zettabyte Apocalypse? (Part 1)

By: In: Data Privacy & Protection On: Jan 27, 2016
Data Protection Series: Are You Ready for the Zettabyte Apocalypse? (Part 1)

Depending on the analyst you read, the next four years will see the addition of between 40 to 60 zettabytes of new data to the digital information burgeon. Each zettabyte equals 1000 exabytes or one billion terabytes. That’s a lot of bits, and they’ll all be seeking a home that is durable and affordable.

Cloud folk (and some large enterprises) are already looking over the options for storing all the zettabytes. Microsoft Azure planners recently provided some insight into their thinking on the problem – and it was telling. Without tape, Redmond says, there will not be sufficient capacity to store all of the digital information that’s coming.

Flash & Disk Capabilities

Flash storage manufacturing is ramping up. The flash makers will generate as much as 500 exabytes of capacity every year from now until 2020, when the “zettabyte apocalypse” is expected. And, despite the woo, the cost of storing data on flash, even for the big guys, will be astronomical.

Hard disk drives, long a stalwart of data storage, will continue to play an important role. However, the experience of prior years—in which each generation of disk drive was twice as capacious as the previous generation (and for approximately half the price) – are behind us, according to analysts. Perpendicular magnetic recording (PMR) appeared on the scene in the 1990s to extend the dynamic capacity growth curve, but we have maxed out that technology. Today, drives get bigger by virtue of adding platters in a helium-filled container (helium reduces the friction as platters spin allowing more platters to be driven by the same size motor). Even that strategy can’t go on indefinitely.

Shingled Magnetic Recording (SMR) and Related Options

Seagate has introduced shingled magnetic recording (SMR), but it is fraught, according to the storage doomsayers. You must know what data is being written to the disk, especially its likelihood of change or update, since rewriting tracks on SMR is disruptive to the data stored on adjacent tracks. For this reason, SMR is regarded as a potentially useful archival disk medium, but not a candidate for general purpose or performance storage.

Other candidates for future disk capacity improvement are heat-assisted magnetic recording (HAMR) and acoustically-assisted magnetic recording, but these are still very much in the testing stages. Additionally, it is unclear how much cost they may add to the disk storage paradigm when they are ready for delivery to market.

Last year, Facebook made some noise when it invested in optical disc, suggesting that archival storage of social networking information might reside on spinning plastic write-once media. The industry (especially Sony) made a lot of hay out of their pre-announcement of 500 GB discs within the next couple of years, followed by a terabyte-sized BluRay disc sometime after 2020. Given the billion-to-one relationship between a zettabyte and a terabyte, nearly all observers have greeted the optical industry’s roadmap with a collective yawn.

Bottom line on disk: Disk storage is no longer growing its capacity at a sufficient rate or with the affordability characteristics that will make it suitable for zettabyte storage, at least according to Microsoft. Moreover, even as innovations come on line to expand capacity, the total manufacturing capability of the disk industry will not be able to keep pace with data growth in any case.

So, we are left with one remaining digital medium to catch the tremendous influx of data that analysts are predicting. That medium is tape. In my next blog, find out why tape is the best way to protect your growing pile of data.

← Data Protection Series: Are You Ready for the Zettabyte Apocalypse? (Part 2) Data Protection Series: How Do New Data Privacy Laws Affect Records Managers? →

Leave A Comment

About the author

Jon Toigo

Jon Toigo is the CEO and managing principal of Toigo Partners International and chairman of the Data Management Institute. He is the author of 15 books, including five on the subject of business continuity planning.