Digital Domesday Book lasts 15 years not 1000

March 3, 2002 6:20 AM

Digital Domesday Book lasts 15 years not 1000 On the 900th anniversary of the Domesday Book, thousands of people, of all ages were asked to take part in a project to create a digital version. The result was a couple of laserdiscs which could be read on a specially modified BBC Micro. It was quite a success and again there was record of what the world was like in the mid-Eighties. But in the intervening years, technology has moved on and now the discs have become inaccessible without that obsolete technology. So ironically, the original millenium old manuscripts have more usability. In the rush to digitise everything, isn't there a danger that we're going to repeat this mistake over and over again?
posted by feelinglistless (21 comments total)

Gosh, burn it to CD, then in a few years burn it to DVD, then in a few years burn it to the next technology. I think that might take a whole 30 minutes each time.
posted by fleener at 6:39 AM on March 3, 2002

Aren't they overstating this particular problem (retrieving the data from the 1980s Domesday book)? The BBC Micro computer the book is presented on has a serial port. That alone should be sufficient to pipe the data out of the BBC Micro so it can be saved in a modern format, though it might take days.
posted by rcade at 6:44 AM on March 3, 2002

Oddly enough I've actually used the system ... the actual data (photographic, sound, text) aren't in the Beeb Micro format -- the Beeb was the control board -- it simply doesn't have the capacity for data retrieval on this scale. Of course you could have a PC running a Beeb emulator and the correct adaptor, but this would mean that the data would still be used at the same speed as it was then, which would be a waste of resources. What they obviously want to do is transfer the data to a usable format like DVD-Rom -- but the conversion simply isn't that easy.
posted by feelinglistless at 8:32 AM on March 3, 2002

For anyone interested, I've found the Camelion project website.
posted by feelinglistless at 8:36 AM on March 3, 2002

the original Domesday Book ... can be accessed by anyone who can read

Anyone who can read 11th-century French, that is. If no one had kept up with that "obsolete format", the original book would be unreadable too.

'We have got a couple of rather scratchy pairs of discs ...'

If the original Domesday Book had been preserved as carefully as this, it would be long gone. For something to be preserved, someone needs to care about it. With that, technological issues can be resolved as they arrive. Without it, then the collective judgement of society is that it doesn't deserve to be saved.

Archimedes' work was erased and the parchment re-used for a religious text. When people stop caring about historical artifacts, don't blame computers.

the 15-year-old version is unreadable ... 'we are confident we will eventually be able to read all their images, maps and text'

When people say professional journalists do their job better than bloggers, we can always whip out this article as a counterexample.
posted by anewc2 at 8:44 AM on March 3, 2002

ASCII. HTML. GIF. JPEG.

I am convinced that these will all still work in 50 years. Media I'm not so sure of. Maybe the network is the computer after all.

(Tried to buy a VCR lately? There aren't many left to choose from. Most stores around here only carry 4-6 models. How many years before VHS is obsolete?)
posted by joemaller at 9:15 AM on March 3, 2002

People seem to be laughing this off. While the original article may have had some hyperbole, the problem is quite real and the Domesday book is only one example. Of course it would seem simple to just "burn it to CD", but of course you need a system that a) can read from the peripheral device that reads the digital media format, b) can write to another format, or at least c) transmit it to a device which can. Remember that before 1995 or so there wasn't even a standard network protocol! The equipment costs money. The people to run and maintain the equipment cost money. Even the conversion process costs money. In many cases it will come down to a question of preserving something of marginal value to a handful of scholars or buying a bunch of new copies of a videotape that will be checked out 1000 times in the next year. And like many problems, this is one that gets worse the longer it is not addressed.
posted by dhartung at 11:17 AM on March 3, 2002

Remember that before 1995 or so there wasn't even a standard network protocol!

Any computer that has a serial port and/or modem can get data out. Rapidly obsolete digital storage is a problem, but anything that was created on computers after 1980 ought to be relatively inexpensive to transfer to a new format, especially if emulator programmers are working in that area. For example, I can run just about anything that was available for the early '80s TRS Model I on my PC.
posted by rcade at 1:28 PM on March 3, 2002

Great Wired article on this: Saved.
posted by MarkO at 1:48 PM on March 3, 2002

I was once talking to a script writer who had worked for a number of well known British soap operas. She wanted to collect her best work together in a portfolio. Unfortunately she originally work on an Amstrad 8256 word processor, and 3" discs from those are incompatible with anything else.

We decided that the only way for her to retrieve her words were to go out and buy a second hand 8256 with a printer, print everything out, then hire someone to copytype them back into a PC.

That's if you can find one ...
posted by feelinglistless at 1:57 PM on March 3, 2002

I don't know, dhartung... after reading your link ("Digital preservation: a time bomb for Digital Libraries"), it seems like the author is too traditional in her approach to digital storage. Scanning a book, saving it to disk, and archiving the disk is unwise -- archival paper has a ridiculously long shelf life (something like 1000 years), compared to the hardiest digital storage media.

However, the fundamental advantage of digitization is the ability to make infinitely many lossless copies of an original. Instead of saving disks, "digital librarians" should focus their resources on distribution. A sufficiently large network would ensure that several copies of a work exist in whatever medium is popular. A typical hard drive today can hold more information than the ancient library of Alexandria, and a network of such drives, surrounding the globe, is less likely to be destroyed by invasion.
posted by Eamon at 2:04 PM on March 3, 2002

(Tried to buy a VCR lately? There aren't many left to choose from. Most stores around here only carry 4-6 models. How many years before VHS is obsolete?)

VCRs are not at all disappearing. More than twice as many VCRs were sold during over the holidays in 2001 than DVD players. What's different is that they've become a commodity item. You don't have to go to Best Buy or Circuit City and shell out $500; you can grab one at your local pharmacy for $50. (And thus you don't see entire departments devoted to them at electronics stores.) I've even seen them in the checkout lanes at Wal-Mart. That means people are buying them as impulse purchases, just like they would a candy bar.

VCRs will not even start the downtrend towards obsolescence until the prices of DVD recorders (not players, recorders) and blank DVD-Rs drop to roughly the price points that CD burners and CD-Rs are at right now. Even then, it will take years for VCRs to truly start disappearing. (How many audio cassettes and cassette players do you still own? How hard is it to buy audio cassettes of the current top 40 albums?) And if the RIAA manages to get some sort of "DRM" bullshit legislated into every DVD recorder, then the VCR will never go away.

The reason things like Beta VCRs and 8-tracks disappeared so quickly, as well as so many personal computer models of the 70s and 80s, is because they never achieved critical mass in the first place. Something better and cheaper, or at least better-marketed, came along and caused the entire consumer market to switch almost instantly. VHS VCRs, on the other hand, have 95 percent market penetration, and is thus going to be as hard to get rid of as Microsoft OSes. Once you achieve near-total market penetration, mere technological superiority is nowhere near enough to get consumers to move over to your product. If it were, Apple would have destroyed Microsoft 15 years ago.
posted by aaron at 3:19 PM on March 3, 2002

Although Apple is making good progress. Let's give them that ;)
posted by jragon at 4:27 PM on March 3, 2002

I just feel I have to put a link here to the Longnow Project
posted by bregdan at 4:40 PM on March 3, 2002

if you take zero as a positive integer then I'll give them that ;-j
posted by lagado at 5:33 PM on March 3, 2002

(How many audio cassettes and cassette players do you still own? )

I just recently dumped my cassette deck. For the past few years I only used it to route open reel tapes through the dolbyNR processor (My OR deck doesn't have DB) for encoded tapes.

I still have several 'walkmans' around...somewhere...
posted by HTuttle at 8:18 PM on March 3, 2002

This is so stupid. Obsolite technology still exists its just obsolite. Just store the computer with the media. Duh.

It would take a future historian and enginere like a week to build a reader capable of reading the disks in the future even if the hardware broke or something.
posted by delmoi at 10:33 PM on March 3, 2002

We did this on February 17 here, where fleener even made the same opening remark. Here's part of what I said before (to save myself the typing without leaving myself out of this rerun):

"Maybe the best digital permanent storage is the Internet. Keep your data in at least two or three different places, and let server backups and replacements keep refreshing and moving your data. Anything of lasting interest (and everything is interesting to someone, especially if you interlard it with porn*) will likely be copied to other places if you give people free access to it. I suspect that this thread will be somewhere in a thousand years, though there may be no one to read it."
posted by pracowity at 11:05 PM on March 3, 2002

What's the problem with moving this data from the propriatry format they developed just for this project to something else on a desktop PC? Most PCs still have RS-232 serial ports and Maplins used to sell a home electronics project to convert the user port on a BBC to do RS-232.

There are standard wizards for the near endemic MS Access that will create the kind of database they need and conversion tools for this format used in nearly every office in the western world will proliferate for years to come. The worst trouble they will have will be file formats but they have just text and still images. In mid-eighties tech' at best the images will be RGB values with some information on dimensions.

That said, I think this project is worth saving not for the history it contains but for history the project itself is in dreams of people using computers. It had become possible top do it so they did it. Now the network is the computer, the world is pretty much self-documenting and repeating the project seems like it would be a waste of time.
posted by vbfg at 5:59 AM on March 4, 2002

> Gosh, burn it to CD, then in a few years burn it to DVD,
> then in a few years burn it to the next technology. I
> think that might take a whole 30 minutes each time.

So much for the Domesday Book. Now for the Library of Congress...

I'd love to have everything I've got just on electromechanical media copied over to 320 bit mp3s (leave the books out of it, I'd be scanning/OCR-ing/editing until y3k) but the idea of transferring half a wall full of bakelite 78rpm records and two walls full of vinyl 33s makes me real tired. And at the moment I don't even have a functioning turntable.

I've got the Recorded Books' version of The Lord of the Rings on forty-odd cassettes and I am going to go to the trouble of making mp3's of these. Absolutely wonderful performance, I doubt Tolkein himself could have read them better.
posted by jfuller at 9:55 AM on March 4, 2002

Paul Wheatley, the man behind the CAMiLEON emulation project has been in touch and asked me to post his thoughts based upon the comments on the thread:

-----------

As feelinglistless suggested, there really is a danger we will repeat this mistake over and over again. In fact we've een doing it since the dawn of computing and are still doing it today!

Domesday discs on 12" videodisc? Burn them onto CD you fool! How long do you think CDs will last before becoming obsolete? Someone suggested widely used media will not go obsolete. Just like vinyl records then? Or magnetic floppy discs (total market penetration, but they changed in size every couple of years). This is also missing the point somewhat. Who is going to pay to copy this massive amount of data from format to format every few years?

BBC software obsolete? Copy it into HTML, JPEG and MPEG then it will last forever! You think these formats will still be around in the same form and supported by current software in 10 years time? Has the world stopped writing new software? Has Microsoft lost the will to change its software every year so people buy it? Of course not!

Software obsolescency is real and *will* keep happening! Even traditional librarians are beginning to recognise this. Look here's a conference about it http://www.dlmforum2002.org/ This is a real issue!

History shows us the lessons we need to learn. If we do not preserve our digital culture there will be no record of the history of computing in 100 years time. My colleague Dave Holdsworth recently rescued an ICL mainframe machine running the George3 OS (from the 70s) by emulating it. It was on the verge of obsolesence as the experience to do this work had all but disappeared (the documentation was not enough to capture all the "features" or bugs in the original - when this is lost, thats it. Gone forever). I should probably also mention that none of the original machines survive to the present day.

Dave has got hold of paper tape of the software from the first ever business machine - the Leo3 (1960s). A landmark in the history of computing. He found a paper tape reader (just) and has bytestreams preserved in an archival store. But whether there is the experience and documentation to allow someone to write an emulator or not is unclear. Its probably too late...

Preserving the hardware isn't a viable option. Electronics break down. Spare parts don't exist. Knowledge of how to fix them disappears. Computers have a finite lifetime. Technology preservation isn't an option.

Oh and by the way, true to form, the journalist who wrote the Observer article made up all the "quotes". Most of them are wildly innacurate. I have a working Domesday kit here in my office, but how much longer it will run for I'm not sure. We are working to rescue it with emulation... See this for our thoughts on how to do it and more importantly prevent the emulator itself from becoming obsolete:

http://www.rlg.org/preserv/diginews/diginews5-4.html#feature2

Paul Wheatley
UK Project Manager, CAMiLEON
[e-mail]
posted by feelinglistless at 2:26 PM on March 13, 2002