According to a study recently completed by research firm IDC, the amount of data stored by computers and media has reached 161 ExaBytes (161 billion GigaBytes, or 272,340,572,418 CDs).  According to this report, the world’s data will soon surpass our storage capacity for the first time in history.  By the end of this year they expect 255 ExaBytes to exist despite our theoretical global capacity of 246 ExaBytes.

Words cannot describe how vast this amount of binary data is.

If that’s not enough, we can expect the world’s data to increase by a factor of six over the next three years.  As a software developer and database engineer, this is absolutely incredible.  I am curious to know just how much of this information is “unique data”, though.

In the world of peer-to-peer downloading, a single 200 MegaByte episode of some TV program could be found on 50,000 computers.  That single file would be using 1,000,000 MegaBytes of storage across all the machines.  Of course some of these files could be stored on CD or DVD (or any number of other storage mediums), but it is still a massive amount of duplication.  Corporations make backups of their databases on a nightly (at the very least, I hope) basis, and these archives can easily reach tens of TeraBytes (1 million MegaBytes) of storage in the span of anywhere from a week to a year.

Of course this does raise a question … how long until we stop being so wasteful with our storage?

Right now, storage is cheap.  I happened to see a 320 GigaByte hard drive on sale for $99 CDN at a local computer retailer, and recordable DVDs are also very inexpensive and superb for longer-term storage.  Personally I have almost 3 TeraBytes of storage on my network and archived DVDs.  Much of this data is stored in several other places on the planet.

So this makes me wonder about the potential of some centralized storage area that everyone has access to.  Of course, the word that comes to mind is “Google”, but I really hope this is not the company to take over the world’s data.  There are several constraints to having a central globalised data storage system, of course.  Bandwidth would certainly be a problem, as would copyright restrictions.  Lord knows that quite a bit of the 161 ExaBytes of data is pirated material.

For the moment, I don’t think we’ll see such a rise of global centralized storage, though.  But home network storage systems are starting to catch on, and this is a great tool for households and communities to share common data.  Transmitting something over a personal network is a heck of a lot better than downloading the same information to several computers.

By 2010, IDC expects the world’s data to reach a ZetaByte.  I hope to reach 6 TeraBytes before then.