Disk keeps getting cheaper.

May 18, 2006 9:53 AM

Amazon S3, now for the masses. Amazon S3 has been discussed previously, but several user-facing services have appeared in the last few weeks that allow ordinary non-programmer end users to take advantage of it. One of the most useful of these appears to be Jungle Disk, a free front-end (free beer!) that lets you use S3 as a webdav-mounted disk drive. It works on Windows, Mac, and Linux, and there's GPL code available (free speech!) that lets other people develop alternative compatible front-ends.
posted by dmd (29 comments total)

Some errata:
Jungle Disk allows you to encrypt your files (or not, if you prefer) with RC4. It uses its own key-file/directory mapping scheme which is very transparent - the GPL'd code provides all the details.

You communicate directly with S3, not through an intermediate service provider (unlike all the other services existing thus far). You pay Amazon directly, at Amazon's rates.

DO NOT USE filicio.us — they do their security wrong

This is not a self-link. The careful reader may notice lots of posts by me on JD forums; I'm simply a very active participant and early beta user.
posted by dmd at 9:53 AM on May 18, 2006

Wait, wait, wait. What's this about free beer?
posted by antifreez_ at 10:05 AM on May 18, 2006

The hidden cost is in transmission fees. In addition to Amazon's $.15/GB/month charge, there is a transmission charge of $.20/GB, such that anytime a file is transferred, either to or from Amazon, you have to pay that transmission charge.

These cost scheme adds up quickly if you use this service for anything more than archival purposes.
posted by Mr. Six at 10:07 AM on May 18, 2006

Of course, at the same time these services come out, consumer hard drives continue to drop in price and rise in storage capacity. So I'm not sure that this sort of thing really ends up being a great value.
posted by selfnoise at 10:12 AM on May 18, 2006

Yes, that's true. Personally I'm using it for exactly that - extremely easy-to-access archival. I run Jungle Disk, then mount S3 via davfs (I recommend using the stable 0.2.8 version), and then rsync all my stuff to it.

Important note - if you're going to rsync to S3, you must use the --inplace and --size-only options. (inplace, because S3 doesn't support key renaming, and rsync normally uploads a tmp file and then renames it over the real file; size-only because otherwise you'd have to download every file to compute the MD5; the rsync-davfs-jungledisk chain isn't smart enough to work together to request the MD5 from S3 directly.)
posted by dmd at 10:17 AM on May 18, 2006

Another similar service is Elephant Drive, which is free during beta (for unlimited storage, yet).
posted by brool at 10:18 AM on May 18, 2006

As for using hard drives - that kind of misses the point. Hard drives fail, or are stolen, or your house burns down ... there are unlimited failure modes. The purpose of S3 is to be reliable. Within seconds of uploading a byte to S3, that byte is replicated on dozens of physical disks across several continents, and Amazon takes over responsibility for keeping it backed up.
posted by dmd at 10:19 AM on May 18, 2006

Brool, I linked to that in the post. Elephant Drive (a) only works on Windows, (b) won't be free after the beta, (c) uses a proprietary storage format, (d) requires you to send your data through their servers, and (e) depends on Elephant Drive staying in business.
posted by dmd at 10:21 AM on May 18, 2006

The price of uploading 250gb of stuff to a JungleDisk is $50, according to them, plus $37.50 a month to store it. So $87.50 just for the first month, with more charges once you start retrieving it.

Newegg offers 250gb hard drives for $75 + shipping.

Hmmmmmmmmmmmmmmmmmm.
posted by kafziel at 10:22 AM on May 18, 2006

The other side of the coin is that most people just don't need that level of paranoia for their files.
posted by selfnoise at 10:23 AM on May 18, 2006

Another concern with WebDAV is that the protocol offers no provisions user authentication and data transmission security. It's not clear from the Jungle Disk forum questions section how these two important aspects of data security are addressed.
posted by Mr. Six at 10:28 AM on May 18, 2006

Right. The other side of the coin is that I can't really imagine that anyone needs to store 250GB on S3.

Storing 5GB of my critical files, though, only costs a buck to upload, and $9 a year to store.

That seems pretty reasonable to me.
posted by dmd at 10:29 AM on May 18, 2006

Mr. Six - you don't talk webdav to S3, you talk webdav to localhost, where the Jungle Disk program is running.
posted by dmd at 10:30 AM on May 18, 2006

Yeah but kafzliel, as dmd mentioned, your HD might get lost, stolen, etc. This is for ultimate backup, possibly best suited to small business, web development (even of the very small variety) not so much personal mp3 storage or whathaveyou.
posted by cell divide at 10:32 AM on May 18, 2006

I cannot locate a description of the Jungle Disk protocol, or how it handles security. I do see some vague mention of a secure key but no description of how or where that key is used or stored. Is there an RFC somewhere?
posted by Mr. Six at 10:38 AM on May 18, 2006

The GPL specification code is available for download at the bottom of the "Download" page.

direct link

There's a whole bunch of files in that zip, but the only relevant ones are JungleDiskCmd.cs, which defines how files and directories are mapped onto S3 keys, and RC4.cs, which implements the RC4 encryption protocol. Everything else is just Amazon's API code.
posted by dmd at 10:43 AM on May 18, 2006

Cool, thanks.
posted by Mr. Six at 10:44 AM on May 18, 2006

I almost lost my iPhoto library (4 years worth of thousands of images) on a dead powerbook HD, so I'm definitely going to try this out for ~5-10Gb of photos. It should make it easy to move from one computer to the next and I'll always have a backup available.
posted by mathowie at 10:46 AM on May 18, 2006

Is the Amazon data georedundant?
posted by Mean Mr. Bucket at 10:58 AM on May 18, 2006

Is the Amazon data georedundant?

From what I've read, yeah, your stuff should be mirrored in several different physical locations throughout the world (or the US, I forget).
posted by mathowie at 11:01 AM on May 18, 2006

Another option is to get a cheapo dreamhost account. You get 20 GB of space for whatever the current promo rate is.
posted by smackfu at 11:27 AM on May 18, 2006

Dreamhost definitely makes sense if you're planning on doing lots of file transfers. You'd get 24GB of space (averaged in first year, taking into account the 160MB extra you get each week). Your data would be backed up by Dreamhost at one location, and you'd pay $240.

Or, let's assume you're going to upload all 24GB ten times in a year, and download it all ten times. With S3, that would cost $140.

Being a little more reasonable, and saying you're not going to completely replace the entire contents ten times in a year: let's say you're going to upload all 24GB just twice, and download it, oh, say four times. That's $72.

On the other hand, a Dreamhost account comes with all sorts of other stuff -- mysql databases, unlimited email, domains, etc... they're a great company. I host all my domains with them. I'm not sure it makes financial sense to get an account for storage, though...
posted by dmd at 11:41 AM on May 18, 2006

Er, I meant to say, "definitely doesn't make sense unless you're planning to make a lot of file transfers."
posted by dmd at 11:50 AM on May 18, 2006

Another neat feature of Jungle Disk, by the way, is that it's a caching, queuing webdav server - so even if you've just dragged a 100MB file onto your S3 disk, from your point of view that operation is just as fast as copying it to a local disk, and you can use the file right away.

The queue gets written to S3 in the background. If the network connection fails, the queue retries until it succeeds. If JD were to crash somehow (I've never seen this happen), the queue just restarts when JD starts up again.

It's rather nice not having the machine grind to a halt as it might normally do when blocking on a network write that takes several minutes.
posted by dmd at 12:07 PM on May 18, 2006

I almost lost my iPhoto library (4 years worth of thousands of images) on a dead powerbook HD, so I'm definitely going to try this out for ~5-10Gb of photos. It should make it easy to move from one computer to the next and I'll always have a backup available.

I'm surprised that not too many companies have tried to capitalize on this kind of story. A company where I used to work (so this isn't a self-link, right?) called PhotoSite just recently launched something called "PhotoSafe". Basically you upload all of your original photos and it keeps them synchronized, safe, etc.

Someone should write a plugin for iPhoto to allow you to do exactly this.
posted by pkingdesign at 1:51 PM on May 18, 2006

Nice. I'm downloading the netdrive WebDAV tool, hopefully with that and synctoy from MS I can automate my backup of critial files. Very snazzy.
posted by tiamat at 1:51 PM on May 18, 2006

I don't get the point of PhotoSafe. Why not just use Flickr? Hundreds of third-party tools available (already works with iPhoto, for example, as well as almost everything else you can name), cheaper, certainly more reliable ...
posted by dmd at 3:00 PM on May 18, 2006

Places I store stuff:

Connected TLM account
FTP Space (Using one of many to FTP backup programs)
my .mac account

Places I would store stuff.
Gmail
Strongspace

This is simply one more option for offsite backup. I'm a firm believer in offsite backup, and I'm glad this is here, but it is simply one more late arrival to the party.
posted by seanyboy at 12:13 AM on May 19, 2006

Interesting - since all the stuff I need to backup to paranoid levels is plain old text, it looks like this would come in a hell of a lot cheaper than .Mac does. (I'm still not sure why I ever signed up for .Mac, really - slow, worrying reports of unreliability, expensive...)
posted by jack_mo at 3:21 AM on May 19, 2006