Assessing digital formats for preservation and use

August 29, 2008 11:52 PM

Sustainability of Digital Formats: a repository of mostly technical information about digital content file formats related to storing images (moving and still), text, sound and websites
posted by Gyan (9 comments total)

This is absolutely fascinating.
posted by Arturus at 12:11 AM on August 30, 2008

Just in time for our next meeting on digital continuity. Excellent.
posted by fallingbadgers at 2:13 AM on August 30, 2008

I'm printing this on archival paper right now.
posted by ZenMasterThis at 2:30 AM on August 30, 2008

For those of you looking to print those images and keep the prints stable, look to Wilhelm Imaging Research.
posted by Muddler at 7:12 AM on August 30, 2008

Fascinating information.
This little footnote was a bit startling (at least to me)

Files in the proprietary native formats for commercial software used by professional graphics designers may include combinations of vector graphics and bit-maps, as well as a layering structure. Such files are expected to be reduced to bit-mapped images before being added to the Library of Congress collection.

It's sad to think that a lot of commercial work might exist in a flattened form when stored for posterity. Personally, I think it would be a great learning tool to be able to access the raw, layered files in order to see how the artists/designers did what they did.
posted by Thorzdad at 7:42 AM on August 30, 2008

Sure, it would be great to have access to the raw, layered files rather than a flattened work, Thorzdad. But would those files be accessible? What program would you use to open them? The problem with proprietary formats is that the software used to open them is not entirely backward-compatible. Try opening a file created in Word for Windows 1.0 with Word 2007. If you're even able to open it, you're going to lose some or all formatting, some or all of the look and feel of that original document. Saving it as a flat file, however -- as a bitmap image -- will preserve the look and the informational content of the file for future generations. Sure, you'll lose some interactivity and, therefore, another part of the essence of that information, but isn't having the content better than having nothing at all? Saving just the original document would be, perhaps, a much poorer decision than anything. And what's the point of saving both the original and a flattened version of the original, if you won't be able to open that original in five or ten years anyhow? I think that's LOC's thought process here, and it does sort of make sense.

Here's a few more articles on digital preservation and preservation of file formats, if that link just wasn't enough (seems impossible, with such a vast site):


Arms, Caroline and Fleischhauer, Carol (2005). "Digital Formats: Factors for Sustainability, Functionality and Quality." Paper presented at IS&T Archiving 2005 Conference, Washington, DC. http://memory.loc.gov/ammem/techdocs/digform/Formats_IST05_paper.pdf

Gladney, H. M. (2003). "Trustworthy 100-Year Digital Objects: Syntax and Semantics--Tension Between Facts and Values" in ERPAePRINTS. http://eprints.erpanet.org/archive/00000027/

Lawrence, Gregory W. et al. (2000). Risk Management of Digital Information: A File Format Investigation. Washington DC: Council on Library & Information Resources. http://www.clir.org/pubs/reports/pub93/pub93.pdf

Rothenberg, Jeff (Jan. 1999). Avoiding Technological Quicksand: Finding a Viable Technical Foundation for Digital Preservation. Washington, DC: Council on Library & Information Resources.http://www.clir.org/pubs/reports/rothenberg/pub77.pdf

Stanescu, Andreas (Nov. 2004). "Assessing the Durability of Formats in a Digital Preservation Environment" D-Lib Magazine 10(11). http://www.dlib.org/dlib/november04/stanescu/11stanescu.html
posted by k8lin at 10:17 AM on August 30, 2008 [2 favorites]

It's somewhat unfortunate that libraries like the LoC aren't saving the proprietary-format files as well, but the reduction to a least-common-denominator, well-documented, and non-proprietary format makes sense. Keeping only the proprietary-format version would create the possibility of an accessibility disaster later on (or at least would make accessing old files expensive and cumbersome).

Particularly when the proprietary versions are much smaller than the bitmaps, I think it would be worth keeping them if space permits (and with storage growing at the rate it's doing now, it seems like that ought not to be that onerous). That way if someone really wants to see the original version, they would have the option of assembling the hardware and software necessary, and opening it up.

But if you only have room for one, a standardized and well-documented format is the obviously better choice for most users down the road.
posted by Kadin2048 at 12:04 PM on August 30, 2008 [1 favorite]

Excellent!
posted by shinyshiny at 4:13 PM on August 30, 2008

****************
DISCLAIMER: I have a dog in this fight ... in fact, Tuesday morning, when I get back on campus, I"m re-distributing the links found here ...

****************

1. Kudos to K8lin for posting the excellent set of additional links. I especially like the report on the "Trustworthy 100-year Digital Objects" ... although, I think that's short-term planning!

2. Kadin2046, proprietary file formats are going to be the digital bane of our existence. While I appreciate the sentiment of the "richness of experience", I have to balance that against the all-too stark reality of not having a budget that allows me to even be sure that the digital repositories I am creating will be migrated in a safe and timely manner, much less proper back-up and restore procedures being observed. Do I do what I can to preserve the intellectual content and forego the interactivity (and I'd like ot see more evidence of those proprietary file formats you want to keep really being smaller! especially at the level and scope we are talking about here!) just because I can't afford to maintain a hardware and software museum? Emulation has yet to completely prove itself as a viable solution on this scale.

I found this passage especially politically-appropriately understated:

This challenge in decision-making will intensify with the proliferation of new media-dependent formats like Super Audio CD (SACD) and DVD-Audio, which offer surround sound and multimedia (DVD-Audio) but which will include technological protection that make preservation reformatting more difficult.

No kidding ... ya think? We have to battle to make preservation copies of our digital materials now, thank you very much ...and it'sonly going to get worse.

Stop thinking in terms of access 5 years from now and start thinking about access 100 or 200 years down the road.

I've said it before: books are new formats in the grand scheme of things and archivists and librarians are just starting to understand how to preserve them. We need to get cracking on audio-visual materials AND the digitally-born things, too ...

This post should be digg'd and whatever else you can think of to get people thinking and looking at the problem. It's really that serious.
posted by aldus_manutius at 8:12 AM on August 31, 2008 [2 favorites]