Alison's take on digital storage

Yay! Alison the Archivist is back, with a post on digital storage: 

First things first. I want to make it clear that digital files can be maintained over the long term. Digital information is not hopelessly ephemeral.

I believe that most of our current misgivings about the longevity of digital data spring from the fact we have historically treated them as short-lived, transient objects. We make copies of files willy-nilly all over our computers. We send them all around the world via email. Digital storage is inexpensive, so when we run out of room, we just buy more—a luxury that we do not have in the physical world. We do not often consider digital files even to be the same type of animal as paper letters or paper photographs.

For many of us here in the Moxieverse, though, we now recognize that we possess digital files that are neither disposable or ephemeral. We are creating unique and indispensable records of our children's youth with our digital cameras. We do not have space to store all of our children's glorious artworks, so we scan them, hoping to find a place for them in the digital realm where none existed in the physical world. What are we to do? How are we to ensure that some of this precious evidence of a time now passed will persist?

First, we should be clear about our timeframe. How long do we want these files to last? You'll note that archivists rarely (if ever) use the words "forever" or "permanent." We are too privy to the effects of time on information. Instead, archivists consider how things might best be kept for the long term, or on an on-going basis.

I make this distinction because it is important for all of us to set our expectations at a reasonable level. What we are trying to do with both our most precious physical and digital records is keep them in the best possible manner so that they will last as long as possible. This is not the same as expecting them to last forever—a quite unreasonable expectation. We will do the best we can, knowing that our descendents will take up the same mantle and make their own decisions about how they wish to proceed.

Enough with the theoretical ramblings.

We all have a pretty good, innate understanding of what paper can and cannot be expected to do. We've used it our entire lives. It has been passed down to us by the past and we have seen how it deteriorates. On the other hand, digital files often seem hidden, tricky—the domain of other people "who know more about such things." Given the vast number of digital files that surround us, now is the time to shed such preconceptions. The moment has come to acquire a better, basic understanding of what digital files need to thrive so that we can move forward without trepidation.

So, how to proceed? Well, unfortunately, the one thing we should not do is to apply this innate understanding of how best to store physical objects to our decisions about storing digital objects. Let me explain.

In order to increase the life expectancy of a piece of paper or a photograph, the most prudent thing to do is to place such an object in a very cold, very dark, humidity-controlled environment. In fact—if you will allow me to exaggerate my point here for effect—to make a photograph last for the longest period of time possible, it would be best if you were to keep it in its cold, dark place without ever looking at it or touching it. Looking at a photograph exposes it to light which degrades the print even just a little, while touching the photograph exposes it to all the dangers inherent in physical manipulation, such as oils from the skin being transferred to the print or any accidental ripping or bending.

This is the exact opposite of the way that we should treat digital objects if we want them to last for the longest period of time possible. We need to open them, look at them on all sides, make sure they are still readable. It is not wise just to dump all of your most precious digital files on CDs or DVDs and put them on a shelf in the same way you might store a box of old photographs. The photographs do well in the dark. Digital objects do best in the "light," and the reason to keep them visible is two-fold.

First, as I mentioned in my original post, the physical medium on which we store our digital files is as subject to degradation as everything else in the physical world...CDs, DVDs, magnetic tapes, hard drives, whatever. Every type of digital storage has a panoply of ways in which it could fail, just like all physical objects. The scariest part of this process is the fact that it is not always patently obvious when digital storage devices are on their way to failure. Like your car, sometimes they give hints of an upcoming breakdown, but other times, they just die in the middle of the road. But when digital devices do fail, they tend to fail utterly.

The second reason that we need to maintain an active connection with our digital data is that both the file formats and the storage formats we use to maintain our digital information periodically become obsolete.** As mentioned by one of the commenters to my original post, if you have all of your files on old iOmega Zip disks but you no longer have an iOmega Zip drive, no matter how perfectly those disks still function, they will you no good if you cannot retrieve your information from them. As for old file formats, did anyone here use PeachText back in the day? Trying to open those files now is a tough prospect. Commodore 64 files with no Commodore 64? If you don't have the hardware or the software to read your data, you don't actually have the information at all.

So, in summary, the longer you leave your digital files alone without looking at them, the greater the risk that the information will not be useable when you go back to find it.

Scared? Well, don't be. Again, just like everything else in life, maintaining your digital records is a question of risk management. Ask yourself how much effort are you willing to put into the process of maintaining your data for the long term. You make these types of decisions already with your physical stuff, you just may not think about it so much because of your innate understanding of the way the physical world works. For digital stuff, we probably have to be a little bit more proactive for the moment, but, hopefully, this will all become old-hat in the future.

OK, I hear you all saying, bring on the practical tips for this balanced approach towards digital recordkeeping!!

As with my last post, I have no suggestions that involve actual brands or products. I'm afraid you will all have to evaluate what's on the market for yourselves. Moreover, you are not required to follow all (or even any) of my suggestions. Pick which ones fit into your lifestyle. Adapt them. Riff on them. Feel free to decide that you think all of this is too much bother and accept the fact that you may lose digital information. Or, feel free to follow them all to the letter, and know that you still might lose some data, just like you know you may lose those precious photos in your attic due to unforeseen circumstances. Whatever you do...pick a system that you will be happy to maintain, and don't just follow someone else's mandates. This is your information we're talking about. You get to decide what to keep and how to keep it. I am hoping to provide you with information that gets you thinking.

1) Controlled redundancy is a very good thing. You should consider storing multiple copies of your digital records on separate storage devices (say, on a designated hard drive at your house, and also a set of CD's stored off-site). By doing this, you will greatly reduce your risk of loss, mainly because you have made two copies—no other fancy reason. Remember, though, keeping your files in two places means that you need to keep your eye on both copies on an ongoing basis.

2) I like to use hard drives for my long-term storage because I don't like to spend hours plopping CD after CD into a drive to check each and every one of them.

3) Whatever type of storage medium you've chosen to use—say a hard drive—you should seriously consider copying your data onto a new hard drive every once in a while, say every three to five years, in order to help ensure that the physical medium is as fresh as possible. The same goes for CDs, DVDs...anything. And, do feel free to use the phrase, "I'm currently refreshing my digital storage media" in conversation. It impresses people.

4) When you do refresh your digital storage media, or indeed at any time that pleases you, take a moment to look at what you have stored there. Does it include everything you want it to include? If not, add or remove what you desire. Do you still have all the hardware and software needed to read and understand all of your stored information? If not...lesson learned. If so, do you foresee having the necessary hardware and software available the *next time* you do this operation? If not, update your choice of storage medium and/or decide which files should now be saved in newer file formats. When you do this, make sure that all of the information you were once able to see and use in the old files can be seen and used in the new ones.

5) Do not delegate the responsibility of storing your files to any online company who does not promise to take care of them until you tell them to stop. This includes Flickr, Snapfish and all those kinds of sites. If and when these guys go out of business, *poof* there goes your data.

Over time, we may find that we don't have to go back and check in with our data so often. But I don't think we'll never be able just to put digital data into "cold storage." I also think it might be clearer now why we are finding that digital files are more expensive to store over the long term. When you wish to ensure that your digital data is usable on an ongoing basis, it is much more labor-intensive to maintain, both mentally and physically, than paper records.

Finally, allow me to mention that there are those who feel that, given the reality of this situation, it might just be best to keep every single one of your digital files. These folks suggest, "Why think about it? Just backup the whole lot. Storage is cheap and we can just search the text of the files to find what we need." This solution may work for periodic computer backups, but it is a risky way to move forward with the information we want to keep for the long term for a variety of reasons. Let me just offer you offer one or two. How do you find a picture with a text search? If you save every digital picture you ever are you going to find the one of Aunt Sara? Finally, if you never take the time to remember what it is that you've stored, you may very well end up with a lot of useless ones-and-zeroes because you no longer have the software to interpret your files. You may certainly choose to follow this path if it pleases you; I just want to point out its risks.

How do you pick what to keep and what to purge? Again, I'm afraid that's up to you. These kinds of decisions are made in the precise same way for both the digital and physical worlds, because, at its heart, you are dealing with the same thing: collecting, expunging, remembering, forgetting, embracing, letting go, thinking about the future, coming to terms with the past. Information  is meaningful.

That's a lot of words, I know, and I haven't even answered all of your questions. I invite all of you to discuss this further in the comments, and I especially invite responses from my fellow archivists...I know they'll have more to say on this topic!


**Those of you who have old slides might have just perked up your ears. Yes, regular-old slides are indeed becoming obsolete in just this same manner. Kodak no longer makes slide projectors and it is becoming harder and harder to find a working machine. If you wish to enjoy looking at your slides, therefore, reformatting them as digital files makes an awful lot of sense. But please, consider keeping the old slides once you have had them scanned. In future, you may wish to have them scanned again when the technology improves. Think of your digital scans as copies of the original, not as a replacement