Comments on: STOOPID %*@&ING COMPUTER!

STOOPID %*@&ING COMPUTER!

loquacious — Fri, 02 Jan 2009 02:15:11 -0800

Shouting in the datacenter - increased disk latency caused by shouting: Yelling at your computer* may cause an increase in disk latency and a decrease in performance.

By: loquacious

loquacious — Fri, 02 Jan 2009 02:15:20 -0800

*Hitting, slapping, punching and otherwise Fonzi-ing your computer should be considered equally harmfull. Unless, of course, you're trying the old trick of unplugging everying and dropping the chassis on the ground for the purposes of reseating socketed chips.

By: dunkadunc

dunkadunc — Fri, 02 Jan 2009 02:34:09 -0800

This is awesome.

By: dirty lies

dirty lies — Fri, 02 Jan 2009 02:36:40 -0800

This is so weird! loquacious and me were reading reddit at the same time!

By: dirty lies

dirty lies — Fri, 02 Jan 2009 02:38:34 -0800

This is the complete opposite: Using hard drives to make sound. Check the videos.

By: zengargoyle

zengargoyle — Fri, 02 Jan 2009 02:51:04 -0800

No, you need to kick it on that black 'X' we taped on the side. The Magic Switch.

By: louche mustachio

louche mustachio — Fri, 02 Jan 2009 02:53:22 -0800

So I yell at it because it's a piece of goddamn shit fucking stupid trashcan bastard, and thus make it more useless? Sounds like the circle of life to me.

By: surlycat

surlycat — Fri, 02 Jan 2009 02:55:44 -0800

Disk latency goes up if you leave the disk loose in the case too, instead of screwing it down to the rails.

By: dunkadunc

dunkadunc — Fri, 02 Jan 2009 03:09:21 -0800

Disk latency goes up if you leave the disk loose in the case too, instead of screwing it down to the rails. That probably explains the truly crappy performance of the computers I built myself in college.

By: zengargoyle

zengargoyle — Fri, 02 Jan 2009 03:12:54 -0800

No, you just built crappy computers.

By: dunkadunc

dunkadunc — Fri, 02 Jan 2009 03:21:10 -0800

That too.

By: _aa_

_aa_ — Fri, 02 Jan 2009 03:36:50 -0800

Me? I call pish posh. HDDs are sealed. Air tight. This means no sound waves. I suspect the latency spike is caused by the gentleman pressing his face up against the sensitive disk array, but what do I know? The lights are blinking because they're doing work.

By: srboisvert

srboisvert — Fri, 02 Jan 2009 03:39:26 -0800

I am surprised this was on reddit. Where are the other 9 ways to increase your disk array latency?

By: dunkadunc

dunkadunc — Fri, 02 Jan 2009 03:45:30 -0800

The vibration can still go through the case, even if it's sealed.

By: zengargoyle

zengargoyle — Fri, 02 Jan 2009 03:48:06 -0800

It's not sealed.

By: zengargoyle

zengargoyle — Fri, 02 Jan 2009 04:22:02 -0800

Guess I should... Wikipedia it... an HDD today is typically a sealed unit (except for a filtered vent hole to equalize air pressure)The HDD's spindle system relies on air pressure inside the enclosure to support the heads at their proper flying height while the disk rotates. Hard disk drives require a certain range of air pressures in order to operate properly. The connection to the external environment and pressure occurs through a small hole in the enclosure (about 0.5 mm in diameter), usually with a carbon filter on the inside (the breather filter, see below). If the air pressure is too low, then there is not enough lift for the flying head, so the head gets too close to the disk, and there is a risk of head crashes and data loss. Specially manufactured sealed and pressurized disks are needed for reliable high-altitude operation, above about 3,000 m (10,000 feet).

By: special-k

special-k — Fri, 02 Jan 2009 04:32:16 -0800

This is so weird! loquacious and me were reading reddit at the same time! Were you two holding hands too?

By: twoleftfeet

twoleftfeet — Fri, 02 Jan 2009 04:33:17 -0800

I like the way all of the tags are in uppercase. Nicely done!

By: msconduct

msconduct — Fri, 02 Jan 2009 05:18:48 -0800

one time i had some friends of friends visiting & let them stay at my apartment. when they left, they gave me a bottle of wine & a small mexican god-like thing that had a tail with a hole at the end of it. a 'thanks for letting us stay here' gift. i showed it to my future ex & told him, 'if you blow into it's tail, it whistles.' he looked at me & said, 'if you blow up my ass i'd whistle, too.' the video doesn't show them shouting *at* the computers, it shows them shouting *into* the computers. and i'm guessing that if you shouted into my hdd, my latency would increase, too. just a theory.

By: Drastic

Drastic — Fri, 02 Jan 2009 06:22:10 -0800

HDDs are sealed. Air tight. This means no sound waves. This is true, and also why the inside of a submarine is a very quiet place.

By: The Bellman

The Bellman — Fri, 02 Jan 2009 06:45:51 -0800

MetaFilter: The lights are blinking because they're (not) doing work.

By: eriko

eriko — Fri, 02 Jan 2009 06:49:51 -0800

that if you shouted into my hdd, my latency would increase, too. At a certain SPL, it definitely affects the drive -- 500msec latency is *bad*, and there was a huge spike. However, air is a lousy way to carry vibration and couple it to other objects. I can a much easier way for this to be a problem -- the big compressor(s) in the AC unit. I know I can feel it in my feet when they kick in and out, and I'm thinking about my data center and where the big storage arrays are in relation to the AC unit. Answer: Very close. I've also seen cases where disk latency has shot up for no reason whatsoever. What I'm trying to come up with is something that I can use to measure vibration at the rack, tray and drive level, and measure physical latency as well. It may well be that this is a real problem -- not from guys yelling at racks, but from things shaking the floor -- and a rigid floor, with a AC unit and a rack rigidly mounted on it, would be a great way to couple that energy.

By: zengargoyle

zengargoyle — Fri, 02 Jan 2009 07:16:30 -0800

I'm wondering why we stay in the top 100 super-computers list and have scores and scores of servers and a metric butt-load of disks and robot arm tape storage and 'physical latency' is nothing I've heard about. They bitch about everything else..... AC, generators, unscheduled DWP work, backhoes, rats eating fiber.... never have I heard 'physical latency' as a problem. Maybe I'm in the wrong business.

By: Drastic

Drastic — Fri, 02 Jan 2009 07:36:39 -0800

I suspect eriko's on the money there--data centers aren't just noisy places. These data centers, they vibrate. A lot. (Noisy and cold and constantly shaking. I'm glad I don't work in data centers any more.) The latency stuff isn't a problem to hear about for the most part precisely because of those metric butt-loads of disks. Drives fail all the time in those big hot-swappable arrays, and those arrays are used because you can just yank out the failed drive and slap in a new one whenever they start beeping frantically.

By: dirigibleman

dirigibleman — Fri, 02 Jan 2009 07:37:47 -0800

In related news, watching a pot delays boiling indefinitely.

By: ElvisJesus

ElvisJesus — Fri, 02 Jan 2009 08:01:23 -0800

These data centers, they vibrate. Only the Nimbus 2000 Data Centers vibrate

By: eriko

eriko — Fri, 02 Jan 2009 08:03:40 -0800

The latency stuff isn't a problem to hear about for the most part precisely because of those metric butt-loads of disks. Or you just weren't looking for it -- note how short the latency problem happens -- or that you are seeing issues because of vibration, and you're blaming the wrong things. Or it's just so scattered that you've not been able to track it down and written it off as one of those things. The interesting thing is that accelerometers have gotten cheap enough that we can easily get vibration data -- indeed, the first computer use I know of was in notebooks.¹ I'm seriously looking at trying it. The hardware is easy. The question is can I get the data that I need out of the storage arrays quickly enough to be useful. Alas, I'm not running Sun gear here -- thus, the SAN isn't going to have DTraceonboard. I'm going to have to wade into SNMP and see if I can't get the data I need (physical latency, by disk) out, and then correlate that with the accelerometer data. Right now, this is all hand waving. First, I'm going to see if that data is reachable. If it is, expect something in Projects. 1) Apple, IIRC, was the first to use one in a consumer unit but they weren't looking for vibration. On the notebooks, they were looking for a sudden transition to near zero G acceleration. Why? Sitting on a desk, you'd measure a steady 1G acceleration, towards the center of the Earth -- Gravity, of course. The most likely reason for that acceleration to disappear? The notebook is falling, thus, it shows zero G. That means you just dropped the notebook. When the sensor sensed that, it sends a command to the drive telling it to get the heads off the disk platters -- making it far more likely that your data will survive the fall, even if your notebook doesn't. Of course, developer quickly tapped that signal and did fun things with it. Yay, hacking!

By: eye of newt

eye of newt — Fri, 02 Jan 2009 08:12:43 -0800

I could think of several theories: 1. Hoax 2. Vibration affecting disk/disk head dynamics--the vibration may affect the high speed spinning disk, changing the disk/head spacing or maybe some air pressure affect like zengargoyle mentioned 3. Vibration affecting ceramic capacitors (many people don't realize that they respond to vibration) which affects the disk power supplies, causing dropouts. I've never heard of this happening but I've always wondered if it could. 4. Vibration shaking loose the connectors to a poorly mounted disk as surlycat mentioned. If it is real I'd guess at #2. Though they do their best to design disks so the 7200 RPM spinning disks with disk heads floating right above them aren't affected by your typical office vibrations, maybe they never accounted or tested for the frequencies involved with some guy shouting at the disks.

By: Chuckles

Chuckles — Fri, 02 Jan 2009 08:27:52 -0800

It may well be that this is a real problem -- not from guys yelling at racks, but from things shaking the floor -- and a rigid floor, with a AC unit and a rack rigidly mounted on it, would be a great way to couple that energy. Well.. sort of. I mean, it is all about resonances. Those guys have found a frequency of excitation that causes the drive problems -- something a little over 1kHz, probably. It is very unlikely that sufficient 1kHz vibrations could be coupled into a drive array through the floor!

By: StickyCarpet

StickyCarpet — Fri, 02 Jan 2009 08:33:06 -0800

I'm just glad that they're listening, that means there's still a chance I can get through to them.

By: Chuckles

Chuckles — Fri, 02 Jan 2009 08:41:27 -0800

3. Vibration affecting ceramic capacitors (many people don't realize that they respond to vibration) which affects the disk power supplies, causing dropouts. I've never heard of this happening but I've always wondered if it could. Audiophiles have long considered the microphonics in electrolytic capacitors to be of great importance. Of course some of them also think that the material used for volume knobs is of great importance, but in the case of capacitor microphonics it might actually be true.

By: The Whelk

The Whelk — Fri, 02 Jan 2009 09:22:40 -0800

This is why any experienced user knows to increase performance, you speak to your computer in calm, even tones, gently suggesting that failure to perform would lead to reprimand, followed by being thrown out the window.

By: Skorgu

Skorgu — Fri, 02 Jan 2009 09:38:01 -0800

Silly engineers, everyone knows the only way to make a computer behave is to casually place the scarred, dismembered corpse of one of its compatriots nearby. Pour encourager les autres.

By: benzenedream

benzenedream — Fri, 02 Jan 2009 10:38:49 -0800

Anti-vibration tables (and other more exotic gadgets) have been used for years in microscopy to protect the equipment from building vibrations.

By: BrotherCaine

BrotherCaine — Fri, 02 Jan 2009 10:44:39 -0800

No, you need to kick it on that black 'X' we taped on the side. The Magic Switch. When I used to work in IT one of the default (forced change on next logon) passwords I handed out was propitiate.

By: buzzman

buzzman — Fri, 02 Jan 2009 11:10:29 -0800

Flagged as noise.

By: blue_beetle

blue_beetle — Fri, 02 Jan 2009 11:18:34 -0800

noised as flag.

By: Cranberry

Cranberry — Fri, 02 Jan 2009 11:49:51 -0800

Finally a valid cause for a New Years resolution: I shall not shout at my computer because I might hurt its feelings.

By: slimepuppy

slimepuppy — Fri, 02 Jan 2009 12:12:56 -0800

So, uh, the whispered apologies afterwards make the latency go back down, right? I didn't mean it, baby! Boot up for daddy, please, sweetie. You can have the kids; I just want my backups.

By: weston

weston — Fri, 02 Jan 2009 12:14:13 -0800

The notebook is falling, thus, it shows zero G. That means you just dropped the notebook. When the sensor sensed that, it sends a command to the drive telling it to get the heads off the disk platters -- making it far more likely that your data will survive the fall, even if your notebook doesn't. Does anybody know if the standard MacBooks have these accelerometers? I'm wondering if this might be related to something I'm puzzling over. My mom works for a school district, and a couple of weeks ago they pointed out to all their staff that there's a warning on the recent MacBooks in the manual: Wait a few seconds until the sleep indicator light starts pulsing (indicating that the computer is in sleep and the hard disk has stopped spinning) before you move your Macbook. Moving your computer while the hard disk is spinning can damage it, causing loss of data or the inability to start up from the hard disk. Now they also said something else interesting: This only applies to those who received new Macbook laptops or use Macbooks produced since July 2007. It doesn't apply to Macbook Pros. I suspect that if the problem has developed recently, then it has to do with a change in the firmware/OS that determines how/when the heads are parked. But I haven't been able to figure out why the Pro models would be less susceptible than the plain MacBooks. Is it the accelerometer difference or might it be something else?

By: mrzarquon

mrzarquon — Fri, 02 Jan 2009 12:21:34 -0800

I don't know anything about the validity of their testing procedures, but I would say that there could be design issues just related to the fact that server and rack practices were built and tested around before people were sticking 42 3.5" drives in a 4U enclosure. That is a lot of spinning mass is a very small space, not to mention a lot of power and thermal load. (My friend had problems switching their server farms to ESX virtualized nodes, because the hosting company had not anticipating having 40U of server space taken up by 10 16 core boxes, each with 16+ gigE ports. Of course it was just was matter of the hosting company building out a new rack, but they had to delay deployment as it was put together.)

By: mrzarquon

mrzarquon — Fri, 02 Jan 2009 12:34:43 -0800

weston- that is the deep sleep mode on the machines. The contents of memory are being written to disk, to ensure that if the machine powers off completely while asleep, it can recover to where it left off once power is restored (ie, windows hibernation). Since the drive is being written to, you should not try to swing the laptop around and throw it into your backpack immediately after putting the machine to sleep. I think this is a 'problem' for both macbooks and macbook pros, but since the pros don't have shared video ram to be stored also (they have dedicated graphics cards) and they are less likely to be manhandled like a macbook is, the problem of bad sectors and disk damage from the machine being jostled during this hibernation stage manifests more frequently on the Macbooks.

By: DU

DU — Sat, 03 Jan 2009 18:57:06 -0800

I'm surprised this didn't end with a request to Purchasing to buy some quieter air conditioners. "We are losing $1eN every year to disk latency issues brought on by being in THE LOUDEST SERVER ROOM ON THE PLANET."