will our data be preserved?

[alocispepraluger102]

JOHN DVORAK'S SECOND OPINION

The end of information?

Commentary: Smart investors know storage technology will never go away

By John C. Dvorak

Last Update: 5:47 PM ET Mar 7, 2007

BERKELEY, Calif. (MarketWatch) -- According to experts, the world generated 161 exabytes of information last year that needed to be stored.

What does that mean? I sure didn't know off the top of my head what an exabyte was, but I knew it was big.

Technically, it is (you're going to love this) 1 quintillion bytes of information. Great; I'm not sure how many zeros are in a quintillion, either. And apparently we are talking about 161 quintillion.

I had to look it up. A kilobyte has three zeros -- it's 1,000 bytes. A megabyte is a million. A gigabyte is 1,000 million or a U.S. billion (nine zeros). A terabyte is a trillion bytes (12 zeros). Then comes the petabyte, which has 15 zeros. Finally, the quintillion-ish exabyte with 18 zeros! That's a lot of zeros.

FYI: Next comes the zettabyte with 21 zeros, and then the yottabyte with 24 zeros, which eventually will become a disk drive for sale at Costco for $150. After that, perhaps the yotta-yottabyte.My advice to investors: Storage technology is never going away.

So how did we find a crisis in our midst regarding the need for all this storage? The Associated Press reports that a study done at the University of California at Berkeley found we were humming along in 2003, creating the need for 3 exabytes per year worldwide, and then the rate jumped to 161 exabytes.

Surely the passage of the Sarbanes-Oxley Act can't be the only reason for this.

This situation concerning the long-term reliability of stored data is the biggest joke in high technology.

The need arises from the emergence of digital storage for videos; larger formats of digital images; the requirement to archive entire corporate databases for years on end; the storage of trillions of e-mails here and there; multiple stores of the entire Internet (or as much as can be found); digitization of film archives and digital distribution of new films. It just goes on and on.

When you start to look into this, it's not the actual storage of all this data and all the backups that's alarming, but the integrity of any of it. (The AP article concludes the same thing.) This is what nobody really wants to discuss.

This situation concerning the long-term reliability of stored data is the biggest joke in high technology.

Does anyone really think the 4.7 gigabytes that you moved to a DVD-R disc will be readable 100 years from now? Or 20 years from now, for that matter? I have CD-ROMs that are a decade old and failing.

For example, it's a well-known fact that there are only a few working 2-inch tape, 4-head Ampex video machines still in operation. These are the monsters that were the workhorses of network television in the 1960s and were phased out in favor of newer technologies.

What few are left get stripped for parts to keep a few others working, in a futile effort to transfer old master tapes to new formats. Time is running out as the defunct technology falls apart.

Because of never-ending changes in technology, all sorts of things will be lost -- from entire movies, photos and electronic publications to corporate records and entire digitized libraries (the ones that threw out the books).

Of course, this loss always has been a problem with media since the first great library burnt down. The difference is that the mean time to failure (a great tech phrase) is not in the hundreds of years anymore. Now it's just a few years.

The irony is that digital information can be easily moved from here to there intact, unlike analog media that cannot be copied perfectly and degrades over time from copying itself.

But once you cannot read the data, you're out of luck. Usually nothing is salvageable.

The implications and consequences of this inevitable loss of information needs to be studied sooner rather than later. End of Story

TRUSTe: Click to Verify

Copyright © 2007 MarketWatch, Inc. All rights reserved.

By using this site, you agree to the Terms of Service and Privacy Policy (updated 4/3/03

http://www.marketwatch.com/news/story/smar...p;dist=printTop

Edited March 8, 2007 by alocispepraluger102

[Sundog]

The implications and consequences of this inevitable loss of information needs to be studied sooner rather than later. End of Story

More and more I spend a large part of my work day wrestling with this very subject. Electronic data is a blessing and a curse.

[alocispepraluger102]

The implications and consequences of this inevitable loss of information needs to be studied sooner rather than later. End of Story

More and more I spend a large part of my work day wrestling with this very subject. Electronic data is a blessing and a curse.

my trusty vinyls, many more than 50 years old, will probably outlast my modern cd-r's. they probably should. the music is better.

as we are a plastic plugged in species, will we leave any signs at all that we were here? i am in no way implying that we/our data are worth preserving.

Edited March 8, 2007 by alocispepraluger102

[md655321]

Data is fine. Remember, data can be transferred. So that cd-r that breaks down will likely be transferred to HD well before it has a chance. Especially as more and more and more people end up with the same data. There will never be a thing as OOP because it always out there somewhere.

The real problem will continue to be wading through the stunning amount of info in the world. That info itself is very very safe. No more libraries in Alexandria burning, because a million people will have the entire contents of that library on their computer. A massive data crash might mean we are only down to 999,999 sources.

[Sundog]

Data is fine. Remember, data can be transferred.

Sure it can be transferred but can you "read it". Migration of the data to a viable format that can be read is always going to be the big question as we move forward.

[alocispepraluger102]

Data is fine. Remember, data can be transferred.

Sure it can be transferred but can you "read it". Migration of the data to a viable format that can be read is always going to be the big question as we move forward.

fine point well stated.

[md655321]

Data is fine. Remember, data can be transferred.

Sure it can be transferred but can you "read it". Migration of the data to a viable format that can be read is always going to be the big question as we move forward.

Sure you can.

1. It is much easier to transfer digital data to a new form than it is to change analog. No more making a dub in real time, but just instantly re-coding some 1 and 0s.

2. Physical machines will no longer be necessary, only software programs. Will software programs be disassembled and sold for parts? Of course not. Even talking hardware, we have already seen the switch from 5 inch floppies, to 3.5 inch floppies, to CDs, to DVDs, to pure HD storage. Have we lost any massive amounts of data? Doesnt seem like it. In fact, because it was all digital it was incredibly easy to do.

We will some data? Almost certainly. But the data that matters will be transferred long before the format becomes obsolete.

[rockefeller center]

Data is fine. Remember, data can be transferred.

Sure it can be transferred but can you "read it". Migration of the data to a viable format that can be read is always going to be the big question as we move forward.

Sure you can.

1. It is much easier to transfer digital data to a new form than it is to change analog. No more making a dub in real time, but just instantly re-coding some 1 and 0s.

In regards to file formats/file format specifications conversion could become problematic. -> one aspect of the Open XML vs OpenDocument discussion.

Edited March 8, 2007 by rockefeller center

[Big Beat Steve]

@md655321:

Maybe (and hopefully likely) so....

BUT, if we regard this from the perspective of the music collector:

What's so convenient about having to store your collected (music) data every couple of years because the platforms that will provide reading (and playing) access are about to become obsolete or because the data storage media themselves are becoming more and more likely to fail?

Compared to the minimum outlay of making sure your turntable stays functional (or, if your old one is beyond repair, getting a new one - which shouldn't be that difficult to do as long as certain subculture DJ's haven't turned away from vinyl for good) and just pulling out your trusty vinyl and enjoying it even FIFTY years later without worrying about any compatibility issues, that's really not very convenient.

Just imagine this, you data whiz kid - a FIFTY-year old analog data storage medium such as vinyl, or an even older HARDCOPY reference book (o.k. I am not talking about pulp printed matter turning brown and brittle after 50 years or so)! How often will you have to have saved, recopied and transferred your digital data in the same time frame (just in order to ensure even the most basic form of usability), and eventually lost maybe a couple of major files just because the storage media became unreadable before you realized this?

Sorry, this may all sound terribly old-fashioned to a lot of you out there, and digital data and their storage media have a LOT of advantages in today's everyday life, but when it comes to digital media as CONVENIENT LONG-TERM storage media that can just be stored by the owner of the data and forgotten, only to be retrieved when he needs to look up or play something (as in the case of a book or vinyl platter), it is not all gold that glitters in the brave new digital world.

Edited March 8, 2007 by Big Beat Steve

[alocispepraluger102]

@md655321:

Maybe (and hopefully likely) so....

BUT, if we regard this from the perspective of the music collector:

What's so convenient about having to store your collected (music) data every couple of years because the platforms that will provide reading (and playing) access are about to become obsolete or because the data storage media themselves are becoming more and more likely to fail?

Compared to the minimum outlay of making sure your turntable stays functional (or, if your old one is beyond repair, getting a new one - which shouldn't be that difficult to do as long as certain subculture DJ's haven't turned away from vinyl for good) and just pulling out your trusty vinyl and enjoying it even FIFTY years later without worrying about any compatibility issues, that's really not very convenient.

Just imagine this, you data whiz kid - a FIFTY-year old analog data storage medium such as vinyl, or an even older HARDCOPY reference book (o.k. I am not talking about pulp printed matter turning brown and brittle after 50 years or so)! How often will you have to have saved, recopied and transferred your digital data in the same time frame (just in order to ensure even the most basic form of usability), and eventually lost maybe a couple of major files just because the storage media became unreadable before you realized this?

Sorry, this may all sound terribly old-fashioned to a lot of you out there, and digital data and their storage media have a LOT of advantages in today's everyday life, but when it comes to digital media as CONVENIENT LONG-TERM storage media that can just be stored by the owner of the data and forgotten, only to be retrieved when he needs to look up or play something (as in the case of a book or vinyl platter), it is not all gold that glitters in the brave new digital world.

beautifully stated. thanks.

[Claude]

It's a good thing when many data will be lost, as long as the essential data are being preserved.

Imagine a sociology student in 100 years having to go through thousands of archived internet blogs and private videos in order to draw a picture of the society at the beginning of the 21st century.

From our grand-grand parent who lived at the beginning of the 20th century, we usually have only a few portrait photos. With our digicams, we could leave a million private photos for our grand-grand children. Will they be interested in such a mass of boring daily life or holiday pics?

We should learn how to make sure to preserve the important data, but also how to get rid of those that do not need to be saved for posterity.

[JSngry]

Imagine a sociology student in 100 years having to go through thousands of archived internet blogs and private videos in order to draw a picture of the society at the beginning of the 21st century.

Well, if they wanted a truly "accurate" picture, that would be what they would have to do.

Impractical, obviously, but it ought to serve as a reminder that "history" is never complete, that it's always going to be an interpretation of what's left behind, not a true representation of things as they really were. Nor should it be anything else.

As long as everybody recognizes that and proceeds accordingly, everything's cool. But that's not alwyas the case, is it...

[alocispepraluger102]

...e.g. historical fiction made for tv,

or a certain npr producer's history of jazz

Edited March 9, 2007 by alocispepraluger102

[David Ayers]

I d n't re ly s e wha he pro l m is.

[Noj]