yEnc
March 27, 2002 11:01 AM Subscribe
yEnc tightens up Usenet binaries. It is controversial but good. Newsreaders are evolving too. I like this one. If you don't know what this is about, maybe you shouldn't know.
Well, the bloke here has a point.
What yenc does is good, but how it does it is pretty crummy.
posted by i_am_joe's_spleen at 12:22 PM on March 27, 2002
What yenc does is good, but how it does it is pretty crummy.
posted by i_am_joe's_spleen at 12:22 PM on March 27, 2002
I'm all for speeding downloads, but the people who are really pushing for yEnc are the uploaders, not the downloaders. If you've been to Usenet alt.binaries in the last couple years, you know that it's 99% lurkers, 1% posters ratio. The posters like yEnc, because it allows them to upload big binaries faster.
posted by patrickje at 12:39 PM on March 27, 2002
posted by patrickje at 12:39 PM on March 27, 2002
yEnc is, in a word, bullshit.
All high-speed modems use compression at a hardware level, completely invisible to the user. It doesn't matter what your software sends out the port, the modem attempts to compress it; the receiving modem decompresses the data flow before it even hits the UART FIFO buffer.
UUE and MIME64 contain enough redundent/non-random data that it can be compressed. yEnc is, on any binary file, near enough random that the built-in modem compression is more or less ineffective.
Take a 100,000 byte binary. Using UUE, it would be encoded into about 140,000 bytes. But the modems are going to compress it, so they're only transmitting 107,000 bytes (I just tested this using GZip on its worst compression ratio, on a 102,000 byte PDF).
Now, if you compress with yEnc, you get a smaller file, true: about 105,000 bytes. The modems are going to attempt to compress it. They're not going to do any better than getting it down to 103,000 bytes.
So we're really talking a difference of a whopping 5,000 bytes for every 100,000 bytes transferred. Hardly worth the effort.
Now, there may be advantages on the server side. On the other hand, I find it difficult to imagine any ISP with a good Usenet feed storing all that data in uncompressed format. In all likelyhood, they're running hardware-based on-the-fly data compression in their drive controllers.
So once again, yEnc accomplishes nothing in terms of data storage requirements savings.
What we really have happening over on Usenet right now is a bunch of underinformed kids having fun "instigating a revolution." They're getting to feel all sorts of righteousness, engage in wonderful flamefests, and "put it to" everyone that protests. Can't imagine it gets much more entertaining than all that!
But as for being an effective solution to a problem? Nope. yEnc ain't that at all.
posted by five fresh fish at 12:40 PM on March 27, 2002
All high-speed modems use compression at a hardware level, completely invisible to the user. It doesn't matter what your software sends out the port, the modem attempts to compress it; the receiving modem decompresses the data flow before it even hits the UART FIFO buffer.
UUE and MIME64 contain enough redundent/non-random data that it can be compressed. yEnc is, on any binary file, near enough random that the built-in modem compression is more or less ineffective.
Take a 100,000 byte binary. Using UUE, it would be encoded into about 140,000 bytes. But the modems are going to compress it, so they're only transmitting 107,000 bytes (I just tested this using GZip on its worst compression ratio, on a 102,000 byte PDF).
Now, if you compress with yEnc, you get a smaller file, true: about 105,000 bytes. The modems are going to attempt to compress it. They're not going to do any better than getting it down to 103,000 bytes.
So we're really talking a difference of a whopping 5,000 bytes for every 100,000 bytes transferred. Hardly worth the effort.
Now, there may be advantages on the server side. On the other hand, I find it difficult to imagine any ISP with a good Usenet feed storing all that data in uncompressed format. In all likelyhood, they're running hardware-based on-the-fly data compression in their drive controllers.
So once again, yEnc accomplishes nothing in terms of data storage requirements savings.
What we really have happening over on Usenet right now is a bunch of underinformed kids having fun "instigating a revolution." They're getting to feel all sorts of righteousness, engage in wonderful flamefests, and "put it to" everyone that protests. Can't imagine it gets much more entertaining than all that!
But as for being an effective solution to a problem? Nope. yEnc ain't that at all.
posted by five fresh fish at 12:40 PM on March 27, 2002
How many people download usenet binaries over a modem? Broadband offers no compression.
And I seriously doubt that the modem compression offers any where near gzip performance.
posted by kaefer at 12:59 PM on March 27, 2002
And I seriously doubt that the modem compression offers any where near gzip performance.
posted by kaefer at 12:59 PM on March 27, 2002
Except data compression is commonly only used on dial-up modems, not on cable modems or leased line connections.
posted by kerplunk at 1:06 PM on March 27, 2002
posted by kerplunk at 1:06 PM on March 27, 2002
five fresh fish:
I suspect, but cannot at this moment prove, that you are wrong about cable-modem compression. Modems do hardware compression (v.something), but cable-modems are really just smart router/hub units. I've had a packet sniffer on both sides of my cable-modem, and no compression was taking place.
That said, I disagree with yEnc because it is a truly ugly hack, and was obviously invented by a kid with a limited education on compression/decompression algorithms. Not bad as a school project, but not something I'd want becoming a standard.
If you really want to downloadporn perfectly legal binary files quickly, the zip them up first, then post them via MIME or UUENCODE. What's so hard?
The simple fact is that USENET was not designed to carry binary content in an efficient way. It's amazing to me that USENET has become such a haven forporn perfectly legal binary files.
posted by mrmanley at 1:30 PM on March 27, 2002
I suspect, but cannot at this moment prove, that you are wrong about cable-modem compression. Modems do hardware compression (v.something), but cable-modems are really just smart router/hub units. I've had a packet sniffer on both sides of my cable-modem, and no compression was taking place.
That said, I disagree with yEnc because it is a truly ugly hack, and was obviously invented by a kid with a limited education on compression/decompression algorithms. Not bad as a school project, but not something I'd want becoming a standard.
If you really want to download
The simple fact is that USENET was not designed to carry binary content in an efficient way. It's amazing to me that USENET has become such a haven for
posted by mrmanley at 1:30 PM on March 27, 2002
Ah but zipping still isnt as good.
A .jpg (let's say a public domain picture of a sunset, the kind of thing we all like to download), for example, is a compressed format. UUE it and it comes out at 4000 lines, yEnc is and it comes out at 1000. My own ancedotal experience is that the lovely sunset picture does come down much faster on my DSL modem yEnc'd up.
this article does have a wide variety of other valid attacks on yEnc, while accepting that it does improve download/upload speed.
posted by malphigian at 1:37 PM on March 27, 2002
A .jpg (let's say a public domain picture of a sunset, the kind of thing we all like to download), for example, is a compressed format. UUE it and it comes out at 4000 lines, yEnc is and it comes out at 1000. My own ancedotal experience is that the lovely sunset picture does come down much faster on my DSL modem yEnc'd up.
this article does have a wide variety of other valid attacks on yEnc, while accepting that it does improve download/upload speed.
posted by malphigian at 1:37 PM on March 27, 2002
Who cares about cable or DSL modems? You're not likely to be downloading more than about 10Mb from a Usenet posting. At cable speeds, there's bugger-all difference between UUE and yEnc. Certainly not enough time to make it worth fubaring the standards.
Mal, the line count is irrelevent: the two methods use different line lengths.
posted by five fresh fish at 3:01 PM on March 27, 2002
Mal, the line count is irrelevent: the two methods use different line lengths.
posted by five fresh fish at 3:01 PM on March 27, 2002
I regularly download 1gig or more of binaries. Daily. A better format would be nice, but yEnc doesn't seem to be it at this time.
posted by Su at 3:10 PM on March 27, 2002
posted by Su at 3:10 PM on March 27, 2002
Five, your argument is solid bullshit. The number of nntp servers using compressed file systems vanishingly small.
posted by NortonDC at 3:15 PM on March 27, 2002
posted by NortonDC at 3:15 PM on March 27, 2002
Like Woody Allen said of Catskills food: it's terrible, and such small portions. (In other words, even a shitty implementation of this standard was long overdue.) As a lurker, I've been resisting yEnc myself for a number of weeks, but it works and the new apps are interesting. It will almost certainly be good for Usenet.
posted by xowie at 4:25 PM on March 27, 2002
posted by xowie at 4:25 PM on March 27, 2002
UUE it and it comes out at 4000 lines, yEnc is and it comes out at 1000.
Don't forget that the actual line length of a yEnc-encoded file is at least double that of a UUE one.
posted by rmannion at 11:23 PM on March 27, 2002
Don't forget that the actual line length of a yEnc-encoded file is at least double that of a UUE one.
posted by rmannion at 11:23 PM on March 27, 2002
« Older The Next World Order. | Actor Dudley Moore Dies at 66 Newer »
This thread has been archived and is closed to new comments
porn... um, I meanpirated software, er, I mean perfectly legal binary files.posted by malphigian at 11:47 AM on March 27, 2002