...give it a few seconds, then yank
July 22, 2018 11:02 AM   Subscribe

Do you really need to properly eject a USB drive before yanking it out? Popular Science says probably not. The internet rebuts. posted by not_the_water (103 comments total) 17 users marked this as a favorite
 
I mean, technically no, you don't have to. Not if you don't care about the consistency of the data on the USB drive. Is there nothing on there that you care about? And you don't mind that your OS bitches at you when you do it? No? Fine, yank away.

It's like, well no, technically you don't have to refrigerate milk. Hell you could leave it out on the counter all day if you want, and if you're lucky nobody will get sick drinking it. But if you plan on actually using it for anything important, and you care about whether or not it's safe to use, handling it properly would be a good idea, yes.
posted by Two unicycles and some duct tape at 11:17 AM on July 22, 2018 [23 favorites]


I ain't got time to wait those 2.5 seconds for it to unmount!
posted by greenhornet at 11:18 AM on July 22, 2018 [2 favorites]


I do it because I've done it for years and better safe than sorry while at the same time knowing it was probably unnecessary 99% of the time.
posted by downtohisturtles at 11:22 AM on July 22, 2018


“In terms of threats to worry about on a USB drive, unplugging it without ejecting the drive is way down there,” Waldo says. A bigger threat? Plugging one in.

Truth.
posted by Fizz at 11:22 AM on July 22, 2018 [37 favorites]


EVERY. FUCKING. TIME.
posted by Fizz at 11:22 AM on July 22, 2018 [3 favorites]


I've been just yanking them out for years and years and I've never had an issue. Worst case scenario I'm in the middle of moving some files and boneheadedly pull the drive because I frequently have the memory of a goldfish, and then the computer barfs at me. But moving files in Windows is really a copy-and-then-delete process, with the delete part not happening until after all copying is complete, so even if a file gets corrupted there's still a good copy in the original location. If there isn't any active read/write activity, the computer doesn't even complain.
posted by Anticipation Of A New Lover's Arrival, The at 11:23 AM on July 22, 2018 [3 favorites]


I'm usually pretty loose about following any directions, but this is an exception. I like the seat belt analogy- you almost never need to be wearing your seat belt, but I always do.
I am reminded of watching my specialist pull cards out of a mainframe with power on, which we were specifically told never to do. He just shrugged like, it's usually not a problem.
posted by MtDewd at 11:30 AM on July 22, 2018 [2 favorites]


Even leaving aside the software/filesystem stuff -- there is no chance that $20 USB drive has a capacitor that ensures that it can complete a block write, much less a block erase, before you yank the power. Those things are little computers inside, and they can pretty much decide to move stuff around (i.e. run their version of DEFRAG) whenever they want to.
posted by RobotVoodooPower at 11:41 AM on July 22, 2018 [2 favorites]


How would it defrag when my OS decides on the file system format?
posted by idiopath at 11:45 AM on July 22, 2018 [1 favorite]


As a multi OS user I say yes. On windows, no prob, on MAC OS, I've seen the OS no longer recognize the drive and demand that it be repaired, which of course means erasing all of the data. Not something I'm willing to play with because usually when I'm transporting files this way I'm on a deadline and can't afford to waste a day going back, reloading files and starting over.
posted by evilDoug at 11:46 AM on July 22, 2018 [6 favorites]


Back in the old (pre-USB) days, you could count on corrupting a drive if you unplugged it without properly unmounting it (also, this parked the head, which was important).

Firewire came along, and was specifically designed to tolerate that sort of abuse. I got a firewire drive and just for grins unplugged it in the middle of a write process. Plugged it back in and the write completed normally. A friend and I just gaped like we'd seen a UFO land.

So, yeah, I unmount drives because why tempt fate? But I'm still a little in awe that we can even have this argument.
posted by adamrice at 12:01 PM on July 22, 2018 [12 favorites]


The internet rebuts by saying "yes" without qualifying information.
posted by Brocktoon at 12:06 PM on July 22, 2018 [14 favorites]


2018: The internet rebuts by saying "yes" without qualifying information.
posted by Fizz at 12:07 PM on July 22, 2018 [18 favorites]


FAKE NUSB
posted by chavenet at 12:13 PM on July 22, 2018 [9 favorites]


There are two types of people, those who eject USB drives, and those who will.
posted by punilux at 12:16 PM on July 22, 2018 [19 favorites]


I thought this was a thing of the past. My laptop/OS has never even suggested there was an issue i don't think i even know what i have to do to undock anymore.
posted by biffa at 12:19 PM on July 22, 2018 [4 favorites]


It's now safe to turn off your computer.
posted by Vesihiisi at 12:23 PM on July 22, 2018 [12 favorites]


heh I remember a few years back my dad complaining that he always had a hard time getting his computer to unload his digital camera. He kept getting card readers and extra cables and it still seemed to get mucked up. So he tried getting pictures off my camera, and it worked perfectly fine. He said, huh, that's weird, and in one swift motion, just unplugged my camera.
posted by 5_13_23_42_69_666 at 12:29 PM on July 22, 2018 [17 favorites]


I sort of love the annoying "you didn't properly eject that" dialog because it is the designer's way of getting out in front of the "but but but it's faster and I don't care" argument.

You can tempt fate all you want, but either way you have to click something on screen.
posted by bradbane at 12:39 PM on July 22, 2018 [9 favorites]


My partner's version of this is when sending print jobs to our network printer. She often shuts down her laptop before her spooler has copied it to the printer and wonders why the print job hasn't appeared.

Sometimes she gets back from a trip, switches on her laptop and her travel documents start coming out on the printer.
posted by epo at 12:40 PM on July 22, 2018 [10 favorites]


It's like, well no, technically you don't have to refrigerate milk. Hell you could leave it out on the counter all day if you want, and if you're lucky nobody will get sick drinking it.

My parents leave milk out all the time but it doesn't matter because a) it's Ireland so for half the year the counter is colder than the fridge and b) it's Ireland so the vast quantities of tea consumed by family members and visitors means the milk never makes it to the end of the day anyway.
posted by kersplunk at 12:49 PM on July 22, 2018 [20 favorites]


I have never once done the "safe eject." I didn't even know it was a thing until I'd been using USB devices for years.
posted by The Underpants Monster at 12:51 PM on July 22, 2018


I don't eject volumes with MS-DOS/FAT32 filesystems on them. I always eject volumes with some kind of unix based filesystem; delayed writes are actually a thing on those.
posted by Xyanthilous P. Harrierstick at 12:56 PM on July 22, 2018 [5 favorites]


How would it defrag when my OS decides on the file system format?

From the computer's point of view, a USB flash drive appears to be a large array of 512-byte logical blocks (LBs), each one having a logical block address or LBA that's just an integer between 0 and some upper bound that depends on the size of the drive. The computer is able to read or write any one of these blocks at any time, just by handing its LBA to the drive and then reading or writing the associated 512 bytes of data. This is the same data addressing scheme that has long been used by hard drives, which USB flash drives are designed to emulate.

As you correctly point out, the computer does decide on the filesystem format: the choice of which particular set of LBAs to store any given file at is entirely the computer's, and when you tell the computer to defrag a drive, it will typically rewrite lots of LBs from lots of files to new LBAs so as to make any given file occupy LBs with a contiguous range of LBAs.

But that's not the whole story, because it turns out that the flash memory chips inside a USB flash drive can't actually write 512 bytes to any arbitrary logical block whenever the computer asks for that to happen. They simply don't work that way.

Internally, a flash memory write operation can only ever write zero-bits. The hardware is simply not capable of writing one-bits. The only way you get a one-bit out of a flash memory chip is by reading a bit that has never had a zero written to it since it was erased.

Erasure, for a flash memory chip, is not the same thing as writing. Erasure happens to a huge chunk of flash memory all at once; a typical size for that chunk, in a modern flash chip, is 8 megabytes. During erasure, every single bit inside that 8 megabyte erase block is set to the value 1.

So in order to write a 512-byte chunk of arbitrary data to some specified LBA, the flash drive's internal controller has to find a 512-byte block inside the flash memory chip that has not already been written since the 8MiB erase block containing it was last erased. If you write 512 bytes of arbitrary data to some LBA, and then re-write that same LBA with some other 512 bytes, the flash memory chip inside your USB drive does not see a request to re-write the same 512-byte region it wrote the first time. Instead, the internal controller chip finds another blank 512-byte region, and that's where the re-write actually goes.

And that means that the internal controller needs to keep track of a mapping between the LBAs the computer thinks it's reading and writing, and the real addresses inside the internal flash memory chips where the LB addressed by each of those LBAs is actually stored. The computer thinks it can just rewrite any LB to any LBA at any time, because that's how disk drives have always worked; it knows nothing about 8MiB erase blocks and the need for every write operation to hit a pre-erased region inside the flash memory chip.

The result of all these shenanigans is that writes progressively use up space inside the flash memory chip, even if they're all going to the same LBA from the computer's point of view. Every time a given LBA is rewritten, a 512-byte region inside the flash memory chip that used to hold its contents becomes "stale" and no longer addressable via any LBA over USB, as the controller's internal mapping is updated to map that LBA to the new spot where its data now lives.

And inevitably, at some point the flash memory is going to get full of erase blocks with a mixture of stale (superseded) and live (still addressable via some LBA) data, and a nice fresh 512 byte erased region to put the next write arriving over USB will simply not exist. And that's when the flash drive's internal controller has to run something very much like a defrag, where it copies all the still-live data out of some mixed stale+live erase block into a spare, clean erase block reserved for that purpose, then updates the LBA mappings, then erases what is now a completely stale block to make it able to accept more writes.

Now, in order to avoid gross amounts of data loss due to any unexpected power removal, the copy phase of that has to happen before the re-mapping phase and the re-mapping phase has to happen before the block-erase phase. But even with that ordering, if the power ends up getting cut right after the re-mapping is done but before the block erasure has had time to complete (erasing a block is relatively slow), then the chip can end up with a block that might well read as if erased but that has not in fact been fully erased. And when that block subsequently gets stuff written to it when the drive is next plugged in, some of the 1-bits inside it might be a bit marginal, and might actually read back as zeroes when the LB is read back out again, and that's data corruption.

Doing the recommended Safely Remove Hardware or Eject Device dance before physically removing the drive means that it will always have had enough time to complete any internal garbage collection and internal block erasure that it might still have been doing even after the computer has finished any given write operation. And that means you won't subsequently end up writing data back to partially erased memory, and that means it probably will come back out the same as it was when it went in, and that's a Good Thing.
posted by flabdablet at 1:17 PM on July 22, 2018 [169 favorites]


I remember this being a significant problem in the late 1990s and early 2000s when USB drives (and USB itself) were just becoming popular (displacing zip drives and CD-Rs as portable mass storage). If I remember correctly, contemporary versions of Windows were originally set to use a write cache so people who didn't "eject" their USB drive were genuinely risking data loss or corruption. I managed a large university campus lab during that time and we certainly saw it happen many times. It only took a few years for Microsoft to change that default setting so on Windows machines it's been a much less dangerous behavior for about 15 years.
posted by ElKevbo at 1:25 PM on July 22, 2018 [2 favorites]


The internet rebuts by saying "yes" without qualifying information.

Here's a really good reason to properly eject: if you have a program running that has files open from the usb drive that you've forgotten about, the program may not gracefully handle the file system going away and crash, losing whatever changes you'd not saved yet and useful state in the program.

As for file file system level issues, there are occasionally delayed write issues in pretty much every file system but it's been far more common that I have to repair file system metadata that didn't get saved properly. Chromebooks were really bad about that for a while.
posted by Candleman at 1:37 PM on July 22, 2018 [2 favorites]


After years of politely asking my PC and waiting for permission to eject, I rudely yanked my camera's SD card when I was in a hurry after copying over files a few years ago. It turns out that the files weren't done copying, I corrupted my card, and nearly lost all of my toddler's birthday photos. After tons of time stressing and running various recover processes, I got it all back, but I learned my lesson. That couple of seconds to do it properly won't kill me and are worth the wait.
posted by hessie at 1:40 PM on July 22, 2018 [5 favorites]


The "warning" dialogue is a perfect example of a developer "fixing" a "problem" by controlling the behavior of the user. Instead of popping up annoying "are you sure?!" windows, display a "Please wait while we unmount your drive" progress window.
posted by Brocktoon at 1:48 PM on July 22, 2018 [3 favorites]


I treat USB drives the way I treat vinyl albums. I was taught that you handle albums by the edges. Always have, always will. I learned to unmount USB drives before removal. Still do.
posted by Splunge at 1:49 PM on July 22, 2018 [1 favorite]


on Windows machines it's been a much less dangerous behavior for about 15 years

That's true. The tradeoff is that with that change in place, USB drives have got a fair bit slower to write to than they would otherwise be, and they also see more write traffic than they strictly need to, which means they don't last quite as long as they could.

And although this newer write-through rather than write-back caching behaviour does guard against corruption due to cached data never even having attempted to make it back onto the drive before removal, it doesn't alter the fact that yoinking a USB stick without warning causes power loss that could well happen at a times when the drive is still doing internal remapping/erasure for space recovery and/or wear levelling purposes, and this risks creation of partially erased regions in the flash chips that increase the chance of silent corruption on any subsequent write.

I rudely yanked my camera's SD card when I was in a hurry after copying over files a few years ago. It turns out that the files weren't done copying, I corrupted my card, and nearly lost all of my toddler's birthday photos.

Ayup. Reefing out a flash device when it's actively being written to is very, very risky, and it's not always obvious when that's happening. Doing Safely Remove and then waiting until the OS tell you explicitly that it is safe is always the Right Thing, mainly because that's a process that takes almost no time whenever it is actually safe and is only ever slow when it's not.
posted by flabdablet at 1:50 PM on July 22, 2018 [9 favorites]


I too treat USB drives the way I treat vinyl albums: I threw most of them away and keep a few around for mostly nostalgic reasons
posted by signal at 1:51 PM on July 22, 2018 [23 favorites]


Doing the recommended Safely Remove Hardware or Eject Device dance before physically removing the drive means that it will always have had enough time to complete any internal garbage collection and internal block erasure that it might still have been doing even after the computer has finished any given write operation.

Yeah, but block erases and block writes are typically less than 500 microseconds. Even 1000 of those operations should take less than a second. If you wait a second or two after a file transfer, its likely that all of those operations are long since completed, barring some weird caching in the OS.
posted by JackFlash at 1:51 PM on July 22, 2018 [3 favorites]


I too treat USB drives the way I treat vinyl albums: I threw most of them away and keep a few around for mostly nostalgic reasons
posted by signal


So sorry to hear about your loss.
posted by Splunge at 1:54 PM on July 22, 2018 [6 favorites]


I've spent the past 15 years removing USB thumb drives and external hard drives without unmounting, on a weekly basis, and I have yet to encounter any issues. I realize that's confirmation bias, but I feel that my sample size is fairly large. Although I should add that I've never used a Mac.
posted by exolstice at 1:55 PM on July 22, 2018 [2 favorites]


After using SCSI devices (and ridiculous SCSI chains) throughout the 90s, I’ve always had zero problem with waiting a few seconds for a USB drive to dismount.
posted by porn in the woods at 2:02 PM on July 22, 2018 [4 favorites]


I was taught that you handle albums by the edges. Always have, always will. I learned to unmount USB drives before removal. Still do.

I know people whose minds are just completely blown when they hear vinyl that doesn't grind and hiss and pop and crackle. None of those people ever get to touch my vinyl. Not ever.

If you wait a second or two after a file transfer, its likely that all of those operations are long since completed, barring some weird caching in the OS.

A lot depends on the drive. I have encountered several USB memory sticks where it's completely normal for a good ten seconds to elapse between the time Windows says a transfer has finished and the time the almost-invisible pinpoint activity light stops flashing on the drive. More noticeable on drives that are nearly full.

I've spent the past 15 years removing USB thumb drives and external hard drives without unmounting, on a weekly basis, and I have yet to encounter any issues. I realize that's confirmation bias, but I feel that my sample size is fairly large.

I regularly use power tools without ear or eye protection and I can still hear and I can still see. That doesn't make it a good idea for me to go around advising other people to do likewise.
posted by flabdablet at 2:05 PM on July 22, 2018 [11 favorites]


Yeah there's a mindset here that I just don't get, this idea of 'it's probably safe for you to do this thing therefore go ahead and do this thing', followed by a bunch of special cases and exceptions that no one pays attention to because they just wanted confirmation that it's probably okay for them to continue doing the lazy shortcut thing.

Folks at the end of the day it's your computer, or phone, or camera or whatever. It's your data. Yank it out whenever you feel like it. If waiting a few seconds is a problem, fuck it, yolo, grab it and go. Worst case scenario, you'll lose some or all of what's on the drive. Does that matter? Maybe not if you're in yolo omfg-why-do-i-have-to-click-and-wait you're-not-the-boss-of-me mode. Relish that ten seconds that you might have otherwise wasted.

Those of us who are suggesting caution here are talking about best practice. In other words, what's the best approach to handling removable storage, given the wide array of USB devices, protocols, operating systems, file systems, &c one might conceivably run into? What habit should I learn to minimize problems? In this case, best practice is, unmount through the operating system and then remove. Works reliably with all OSs, all file systems, all USB devices. Is it always strictly necessary? Maybe not. Is it always strictly necessary to wear your seatbelt or to flip the circuit breaker before messing with electrical wiring? Maybe not. But you want reliable and consistent outcomes, you take a few seconds to do the boring safe best practice thing.

Or don't, if you don't want to. It's your data, dude. Do what you want.
posted by Two unicycles and some duct tape at 2:23 PM on July 22, 2018 [14 favorites]


I feel the risk-taking levels of "taking out a USB key without unmounting" and "working on wiring while it's live" are perhaps slightly different.
posted by jeather at 2:28 PM on July 22, 2018 [5 favorites]


I've wondered if there might be an old cultural thing at play--if you come from a background of mounting and unmounting unix partitions 20 years ago, "just pull the plug" could sound pretty jarring.

I'm personally on "team unmount", but I'm not fanatically prescriptive about it. I do a formal unmount 97% of the time. So far, the other 3% hasn't caused any crises.
posted by gimonca at 2:38 PM on July 22, 2018


I deal a lot with students who want to give me files or get files from me. Their flash drive etiquette is appalling. They just yank that thing out any old time and are shocked when I unmount first.
posted by hydropsyche at 2:41 PM on July 22, 2018


It's bizarre reading this article and most of this discussion. It's like it's entirely uninfomed by the engineering of file systems, the precise ways these systems have been built. The risk of immediate disconnect is well understood, it's not a mystery. Even consumer systems like a Windows box have very well defined filesystem behavior. (I can't vouch as much for the $10 USB drive's controller.)

Any modern filesystem should be able to recover from a random disconnect, but it might take a long time to repair the volume and it's not really a great idea. The big risk is you may pull the filesystem with delayed writes still waiting to write your buffered user data. Hopefully that delay is at most a few seconds, but it's remarkably hard to be sure. It's much, much less likely you'll corrupt the filesystem itself. ext4fs, ntfs, and apfs all mostly handle disconnects fine.

FWIW I managed to actually corrupt an NTFS volume on Windows 10 about a month ago, unplugging it. Bunch of files missing when I plugged it in again. Windows did detect it was hinky and prompted me to manually run Disk Repair; that fixed it. Not sure why Windows didn't just do that when the drive mounted.
posted by Nelson at 2:44 PM on July 22, 2018 [3 favorites]


flabdablet: "Now, in order to avoid gross amounts of data loss due to any unexpected power removal, the copy phase of that has to happen before the re-mapping phase and the re-mapping phase has to happen before the block-erase phase. But even with that ordering, if the power ends up getting cut right after the re-mapping is done but before the block erasure has had time to complete (erasing a block is relatively slow), then the chip can end up with a block that might well read as if erased but that has not in fact been fully erased. And when that block subsequently gets stuff written to it when the drive is next plugged in, some of the 1-bits inside it might be a bit marginal, and might actually read back as zeroes when the LB is read back out again, and that's data corruption."

Y'know, if you have to explain your joke...
posted by chavenet at 2:52 PM on July 22, 2018 [1 favorite]


Not sure why Windows didn't just do that when the drive mounted.

Can't speak for what happened in this particular case but in general it's better for the OS to ask rather than make an assumption and initiate repairs, because it doesn't necessarily know whether it's looking at corruption or intentional modification from some other source (another device, OS, whatever).
posted by Two unicycles and some duct tape at 2:55 PM on July 22, 2018 [7 favorites]


This conversation is mind blowing, I thought the ability to unmount manually had disappeared with the advent of windows 7. I didn't know you still could, the os certainly never complains.

I will reevaluate my practices but haven't had any problems in the intervening years.
posted by deadwax at 2:56 PM on July 22, 2018 [5 favorites]


christ this is going to turn into the next metafilter sit vs stand isn't it

except, standing with certain toilets and certain butts can cause poop corruption
posted by Two unicycles and some duct tape at 3:06 PM on July 22, 2018 [10 favorites]


Any modern filesystem should be able to recover from a random disconnect, but it might take a long time to repair the volume and it's not really a great idea. The big risk is you may pull the filesystem with delayed writes still waiting to write your buffered user data.

Nobody is talking about "random disconnects". They are talking about disconnecting several seconds after the OS indicates it has completed the file transfer.

And you have provided exactly zero evidence of how "delayed writes" work or quantified in any manner how long a "delayed write" might last.

And with zero evidence, you accuse others of being "uninformed." Please inform us with actual data.

I'm not saying corruption is impossible but so far all I have heard is vague theoretical possibilities.

And so far I am the only one who has provided actual time numbers and although they may not be the complete picture they are a starting point for discussion.
posted by JackFlash at 3:06 PM on July 22, 2018 [2 favorites]


Windows 7 improved the process - before then you had to close any Explorer window that was looking at the device in question, or the eject would fail. 7 and later just closed them for you.
posted by reprise the theme song and roll the credits at 3:07 PM on July 22, 2018 [2 favorites]


I actually do have to actively dismount my ereader or it gets fucked up but good, but as a rule, I am moving videos to a usb key to watch on my tv, and I will take this incredibly high risk action as I like to live on the edge.
posted by jeather at 3:08 PM on July 22, 2018 [3 favorites]


My father killed his computer because rather than "powering down" his machine, he just pushed the on/off button. Like you do with a light swtich, or the tv. Why would anything be any different?
posted by armacy at 3:09 PM on July 22, 2018


My father killed his computer because rather than "powering down" his machine, he just pushed the on/off button. Like you do with a light swtich, or the tv.
I do this all the time. My computer then powers off or hibernates or whatever. I don't see the problem.
...unless I hold down the power switch or am using an old computer with a real power switch, like a C-64 or a PS2.
Speaking of which, it sounds in like the process flabdablet describes, in addition to potential problems from yanking, there would be the same issue for immediate power down, like loss of building power.
I haven't had a UPS for years. (although I mostly use laptops)
posted by MtDewd at 3:22 PM on July 22, 2018 [1 favorite]


I thought the ability to unmount manually had disappeared with the advent of windows 7. I didn't know you still could, the os certainly never complains.

Windows 7 changed the look of the old Safely Remove Hardware system tray icon, made it hidden by default, and IIRC gave it a different label as well. It's pretty clear that Microsoft's marketing folks think yoinking drives without warning is fine and dandy.

Right-clicking a drive in any Explorer window that exposes it, and then choosing Eject from the context menu, works too.

Windows never did complain about spontaneous removal, near as I remember. It will complain if you tell it you want to do a safe removal on a drive that's busy. And it has assorted kinds of indexing service that can keep drives busy for many minutes without you having touched a thing yourself, as well as a really irritating tendency to unpack Windows Updates sloooooowwwwlllly into temp folders on USB drives if those happen to be the ones with the most free space available.
posted by flabdablet at 3:25 PM on July 22, 2018 [3 favorites]


It's pretty clear that Microsoft's marketing folks think yoinking drives without warning is fine and dandy.

Putting one's faith in the technological savvy of any company's marketing folks is an awesome idea. I highly recommend it.
posted by Greg_Ace at 3:47 PM on July 22, 2018 [8 favorites]


Windows never did complain about spontaneous removal, near as I remember.

way way back in the evil 95 98 2000 me time period windows never did complain about lots of things - it just keeled over and died when you weren't expecting it - (vista wasn't much better - win 8 was a dog)

sometimes i forget to wait and take out the usb flash stick - one time the file system was corrupted because of this (but i may have taken it out of my imac, i can't remember) - anyway, a quick format took care of that - i just transfer files with them because networking on windows is pathetically slow and sucks up all my internet bandwidth
posted by pyramid termite at 3:49 PM on July 22, 2018 [1 favorite]


Windows ... has assorted kinds of indexing service that can keep drives busy for many minutes without you having touched a thing yourself.

Windows 7 will outright refuse to index a removable drive (without some specific contortions) and Windows 10 will not index a removable drive by default unless a user explicitly commands it to do so. These would be special cases and presumably only done by someone who knows the consequences.
posted by JackFlash at 3:53 PM on July 22, 2018 [3 favorites]


christ this is going to turn into the next metafilter sit vs stand isn't it

you are not supposed to put a usb drive there
posted by poffin boffin at 4:11 PM on July 22, 2018 [24 favorites]


Metafilter: Fine. Yank away.
posted by hal9k at 4:20 PM on July 22, 2018 [1 favorite]


I have never once done the "safe eject." I didn't even know it was a thing

They taught us about safe eject in health class, but they did warn us that it's unreliable, to say the least.

Fine. Yank away.

That too.
posted by duffell at 4:22 PM on July 22, 2018 [10 favorites]


you are not supposed to put a usb drive there

it's a good way to get a virus
posted by pyramid termite at 4:23 PM on July 22, 2018 [1 favorite]


> JackFlash:
"Windows ... has assorted kinds of indexing service that can keep drives busy for many minutes without you having touched a thing yourself.

Windows 7 will outright refuse to index a removable drive (without some specific contortions) and Windows 10 will not index a removable drive by default unless a user explicitly commands it to do so. These would be special cases and presumably only done by someone who knows the consequences."


It doesn't matter. I have yet to be able to unmount any drive if I have used it in the last 24-36 hours. Or worse. Just tried to unmount a drive I left plugged in (so I don't lose it) and haven't touched it in at least a week, but was just told it was busy and couldn't be unmounted. Not even a CLUE what is using it.
posted by Samizdata at 4:26 PM on July 22, 2018 [1 favorite]


i've gotten that sort of thing, too, where it's being used by some mysterious process

sometimes i just wait until i've shut the computer down and THEN remove it - what it doesn't know won't hurt it
posted by pyramid termite at 4:28 PM on July 22, 2018 [3 favorites]


Yank it out whenever you feel like it. If waiting a few seconds is a problem, fuck it, yolo, grab it and go.

technically you don't have to refrigerate milk. Hell you could leave it out on the counter all day if you want


MORE! My overloaded conscience needs more permissions to do what is flagrantly ill advised. Tell me I can stop brushing my teeth!
posted by justsomebodythatyouusedtoknow at 4:32 PM on July 22, 2018 [6 favorites]


I have yet to be able to unmount any drive if I have used it in the last 24-36 hours.

That sounds like a different problem. You have some misbehaving application that has a dangling file handle open to the drive. You need to close that application to close the file handle. It does not necessarily mean that the application is still writing to the file.
posted by JackFlash at 4:41 PM on July 22, 2018 [2 favorites]


> JackFlash:
"I have yet to be able to unmount any drive if I have used it in the last 24-36 hours.

That sounds like a different problem. You have some misbehaving application that has a dangling file handle open to the drive. You need to close that application to close the file handle. It does not necessarily mean that the application is still writing to the file."


I know. One could possibly hope that the OS could at the least identify the process in question and tell me what it is. Nope, it's STILL in use. And this is despite Windows having been shut down and restarted more than once since the last use.
posted by Samizdata at 4:59 PM on July 22, 2018 [2 favorites]


sometimes i just wait until i've shut the computer down and THEN remove it

Heh. Whenever I do this there's this fully irrational thought that runs through my head wondering if shutting down and pulling the drive without explicitly ejecting it still leaves me at risk of data being left in an inconsistent state. You know, what if the computer's doing verrrrry delayed writes, just batching then all up until it knows someone cares?

My brain goes to silly places sometimes.
posted by Nonsteroidal Anti-Inflammatory Drug at 5:04 PM on July 22, 2018


That was a great expository on Flash Defragging by flabdablet.

Flip a coin. Maybe flashes defrag. Maybe they don't and he was just pulling our leg that they do.
posted by surplus at 5:09 PM on July 22, 2018


I tend to agree that yanking a USB stick that definitely hasn't been written to in the last few seconds is conceptually in the realm of you almost surely won't incur write loss given how many opportunities the OS has to flush its write buffers in that time, and even more so that you almost surely won't encounter filesystem corruption with a journaling filesystem, but I still eject every time. It's a hand-on-the-stove deal for me, since my trust has been violated too many times by asynchronous systems that propagate writes through multiple layers of caches. It's a small price to pay for peace of mind, whatever the vanishingly small chance is of aberrant behavior.
posted by invitapriore at 5:26 PM on July 22, 2018 [1 favorite]


There was a big old USENIX paper "Understanding the Robustness of SSDs under Power Fault" where they describe corrupting a bunch of SSDs with power faults -- one was "bricked", i.e. totally inoperable. Doesn't mean it's going to happen to you tomorrow, but means that it *could* happen.

It's hard to even discuss properties of these things, because the Flash Translation Layer (FTL) is proprietary and occult; it might as well be a faster-than-light drive. Your only hope is that the engineers tested the "eject" case more thoroughly than the "yank" case.
posted by RobotVoodooPower at 5:33 PM on July 22, 2018 [6 favorites]


All the safe-unmounters will be horrified by the behavior of some devices. I've got a GRE scanner radio that had the very bare minimum amount of work done required to get its USB port to even work; and one of the side effects of that is that safely unmounting it (it appears as a USB drive for the microSD inside) causes it to unmount and then instantly remount; resulting in NO WAY to avoid the nag window about "you didn't safely unmount this device!"
posted by Xyanthilous P. Harrierstick at 5:38 PM on July 22, 2018 [1 favorite]


I still thin that computer manufacturers missed a trick by not using solenoid-locking USB ports. Apple could have added a physical eject motor …

(and yes, there are Lockable USB Connectors: that's what those two little holes on top of a USB A connector are for …)
posted by scruss at 6:31 PM on July 22, 2018 [7 favorites]


I remember an old Wang minicomputer that locked the 8" floppy drive door with a loud THUNK every time it was accessed.
posted by RobotVoodooPower at 6:38 PM on July 22, 2018 [1 favorite]


YOU CAN STOP BRUSHING YOUR TEETH
posted by Two unicycles and some duct tape at 6:46 PM on July 22, 2018 [3 favorites]


> It's hard to even discuss properties of these things, because the Flash Translation Layer (FTL) is proprietary and occult;

Eponysterical
posted by idiopath at 7:16 PM on July 22, 2018 [1 favorite]


"Understanding the Robustness of SSDs under Power Fault" where they describe corrupting a bunch of SSDs with power faults -- one was "bricked", i.e. totally inoperable.

This study applied a power fault in the middle of a deliberately intensive read/write load to see if they could inject errors. It doesn't apply to the case of a flash drive at idle.

The question here is how long after the OS indicates that a transfer is complete does the drive become idle. Milliseconds? One second? Three seconds? Never?
posted by JackFlash at 7:25 PM on July 22, 2018 [1 favorite]


TL;DR: don't yank unless you sync, and even then it's problematic leaving the other yanked-from unsatisfied. Practice Safe Eject and go all the way, better for both.
posted by zengargoyle at 7:35 PM on July 22, 2018 [1 favorite]


You'd need to threaten me with violence to make me safely eject a USB drive.
posted by zymil at 7:38 PM on July 22, 2018


The question here is how long after the OS indicates that a transfer is complete does the drive become idle. Milliseconds? One second? Three seconds? Never?

That's a good question, and I guess it depends on how the device does garbage collection -- the erasure and reclamation of blocks. The NOR flash I have experience with (not often used in consumer flash drives) did GC in the background, which took seconds to several minutes to complete IIRC.

If the device does GC in the foreground, you should have less stuff going on after writes complete. And if there's no deleted or overwrited files, you shouldn't see much GC. But it all depends on the flash tech and the whims of the engineers.
posted by RobotVoodooPower at 7:58 PM on July 22, 2018 [2 favorites]


except, standing with certain toilets and certain butts can cause poop corruption

I think in that area, the best analogy would be whether you check that there's paper on the roll before pooping or not. You can get by in life for years and years with no problems, and there's ways to recover if you do get the problem, but there's value in always being careful.
posted by traveler_ at 8:15 PM on July 22, 2018 [4 favorites]


You'd need to threaten me with violence to make me safely eject a USB drive.

Data corruption will angrily drive a stake through your spleen if you don't safely eject your USB drive!
posted by Greg_Ace at 9:03 PM on July 22, 2018 [2 favorites]


That's a good question, and I guess it depends on how the device does garbage collection -- the erasure and reclamation of blocks. The NOR flash I have experience with (not often used in consumer flash drives) did GC in the background, which took seconds to several minutes to complete IIRC.

Erase times for NAND flash are 10 times faster than NOR flash so garbage collection for a new block takes only a few milliseconds and background garbage collection isn't really necessary for good performance. And most algorithms are going to do garbage collection in the foreground, on demand basis, because background garbage collection inherently results in unnecessary wear from copying pages that may later be deleted if you just wait.

NAND flashes are over-provisioned to reduce the need for garbage collection until a drive is almost full and then typically only trigger garbage collection when free blocks fall below approximately 5%. My best guess is that drives will be idle within a second of finishing a file transfer.
posted by JackFlash at 10:19 PM on July 22, 2018 [2 favorites]


It's not the write time of the USB technology. NAND nor NOR make no difference. It's the OS and filesystem caching. Windows may not say done until it has done a sync... (but I doubt it, otherwise they wouldn't have the Eject Drive option). The truth is that the things you read from USB are read in giant chunks into memory and kept there virtually and your modifications to data-blocks and directory-blocks is all in memory for speed and not taxing the USB with a bazillion tiny read/write for single bytes and such. That cache only actually gets written to the USB when you select the eject (or you sync, or you unmount, etc). Before then.... your data is in memory that's just pretending to act like a USB. Doing a sync to physical media is too-damn-time-consuming to do all the time. You gotta wait, or at least try, or know how to force the OS-filesytem-cache to write changes before you can yank. If you can yank.... you have a crappy OS (sucks to be you), because it's slow because USB is slow, and that's why there's a OS specific filesystem cache. If you yank, you're playing with fire because if your machine was busy or you yanked too quick... your data is GONE BABY GONE because it was never on the USB in the first place.

It's bad form and terribly slow to sync/fsync after output to force writing to physical media. Makes things slow to a crawl... physical media is slow as f*** and best avoided.

Yanking prematurely... bad idea, or your OS sucks balls.
posted by zengargoyle at 12:44 AM on July 23, 2018 [2 favorites]


I'd love to see the follow up of people who say "I've never done a proper unmount and never had any problems" and how many post mysterious computer problems on Ask.
posted by bongo_x at 1:35 AM on July 23, 2018 [3 favorites]


One way of preventing this problem would be to have a physical interlock that prevented the drive from being removed - even if you had requested it - until all the operations had been completed. Rather like the way a washing machine door can't be opened until a few seconds after the end of the wash cycle (apparently to make sure that you don't get injured by a drum that may still be spinning).

Ironically this kind of interlock was a feature of much earlier computers: Macintosh disk drives and Zip disks (any maybe even writeable CDs?) would allow you to select "eject" - but would not give you their media until they were done with them. Sometimes you would lose the argument of whether you could have your media back and have to resort to persuasive user of a paper clip.
posted by rongorongo at 2:00 AM on July 23, 2018 [5 favorites]


ok. This will sound stupid, but how does one unmount a USB drive? Lordy knows I've tried! I've closed the files window, I've looked for any type of dialog box that might say close or eject or anything that looks like it might wrap up that USB drive and make it ready to pull out and I still get the Oops! dialog box popping up. Maybe if that box also included a sentence that said something along the lines of "Next time do A B and C before removing the USB..." but it doesn't.

So i fall back on just closing the files window that is showing me the USB device once the files have finished transferring hoping it's good enough (and that isn't enough to satisfy this silly little chromebook I'm currently using).

I mean, I'm sure it's something easy and obvious, but they just won't tell me what it is...

(why no, I'm not tech savvy, why do you ask?)
posted by newpotato at 3:36 AM on July 23, 2018 [1 favorite]


It's bad form and terribly slow to sync/fsync after output to force writing to physical media. Makes things slow to a crawl... physical media is slow as f*** and best avoided.

Yanking prematurely... bad idea, or your OS sucks balls.


There are some more accurate discussions of whether write caching exists in modern OSes. The (well-sourced) conclusion was that it's very rare in removable media.
posted by ambrosen at 4:31 AM on July 23, 2018


ok. This will sound stupid, but how does one unmount a USB drive? Lordy knows I've tried!

Hello fellow traveler! Did you by any chance grow up on a Mac? It took me ages to figure out how to eject a flash drive on Windows. I was used to having the USB icon pop up on the desktop when it was inserted so it was easy to find and right click to eject. Or just command+E.

I'm a tidy person so I always close open file windows when I've finished finding or working on something and therefore missed the left side panel in a file (Windows Explorer) window that contains the USB icon along with a clutter of annoying icons that I don't need and can't delete. So there's one place to look.

Doesn't always work though so go down to the Task Bar and on the right side of the screen at the bottom, there should be an arrow which pops up to show teeny tiny hidden icons. One of them is for the USB stick. Right click to choose the delete option.

why yes, I do have a shortcut to my Documents folder in the upper right corner of my Desktop
posted by TWinbrook8 at 5:29 AM on July 23, 2018


> Right click to choose the delete option.

Eject option! Eject option! :-)
posted by Nice Guy Mike at 5:48 AM on July 23, 2018 [2 favorites]


samizdata, I'm not really up to date but I remember sysinternals being good for tracking down which process is hanging on to your file or drive. If that doesn't work, there are some other suggestions here.
posted by trig at 5:49 AM on July 23, 2018 [2 favorites]


Also, on Windows, at least, use File Explorer, right click on the drive and select Eject.
posted by MtDewd at 6:53 AM on July 23, 2018 [1 favorite]


Nick Burns, your community weblog's IT guy
posted by thelonius at 7:17 AM on July 23, 2018 [2 favorites]


My two most common unplug-without-eject scenarios:
1) I sent the computer to sleep and forgot that the drive was plugged in. I'm fairly confident that there's no activity going on when I yank the USB out.
2) #$*(@!()%#@# ereader refuses to acknowledge that NOTHING IS TOUCHING IT DAMMIT no matter how many programs I close.

(I'm currently using a Sony PRS-300, which is 9 years old and no longer produced, because nothing on the current market has button-based page turning instead of a touchscreen. My computer occasionally lets me know that it does not like this ancient piece of tech interacting with its sleek modern interface.) (My computer stopped being sleek and modern over 5 years ago.)
posted by ErisLordFreedom at 8:02 AM on July 23, 2018 [2 favorites]


Waaay back in the old days, I bought a cheap 3rd party external floppy disk drive for my parents' original Macintosh. Unlike real Apple floppies which had a little motor to spit out the disk when it was safe, this cheap one had a mechanical eject button - you had to hit command-E, wait a few seconds, and then push the button. Every once in a while I would forget to wait a few seconds, and the disk would be ruined.

In conclusion, you kids get off my damn lawn.
posted by moonmilk at 10:06 AM on July 23, 2018 [3 favorites]


I think a big part of the problem is that, at least in Windows XP, the UX around unmounting a drive before removing it was fucking terrible. You had to find the weird little icon in the task bar (why the task bar? who knows? it's apparently the part of the UI where shit they didn't know what to do with went), right click it (or was it a left click? better try both!), pick the drive you want to eject (don't pick the wrong one!), tell it to do its thing, then wait some indeterminate amount of time, then pull the drive. It was an absurdly "clicky" process for something that a user might do a bunch of times a day. I can only suppose that maybe the engineers didn't think that users would be using USB drives as a replacement for floppies when they designed it.

On more modern systems where the safe-remove process is just a right-click "eject" or something similar, I suspect more users do the right thing, because it's easy to do. (Particularly, I think, if you were used to doing that to physically eject media, as was the case on Macs—they didn't have hardware eject buttons on floppy drives in the old days, and the physical ejects on CDs and Zip drives generally weren't something you used normally.)

But a whole lot of people got trained to do things the 'unsafe' way, and since the chances of corruption are fairly slim, there's not a lot of compelling reasons to get them to change now.
posted by Kadin2048 at 11:03 AM on July 23, 2018 [1 favorite]


I learned in computer class that flash memory is the most fragile and have since always treated it accordingly. I've had a few UBS sticks ruined for no apparent reason so I rarely use them anyway (other than to quickly transfer backed-up data from one machine to the one right next to it).

I have one attached to my keys. The only use I have for it is to give me peace of mind: it has a read me text file with a contact e-mail in case someone finds my keys and wants to return them to me.
posted by ipsative at 11:04 AM on July 23, 2018


MORE! My overloaded conscience needs more permissions to do what is flagrantly ill advised. Tell me I can stop brushing my teeth!

As my former dentist always said: You don't have to brush all of your teeth; just the ones you want to keep. I've got a new dentist now.
posted by The Bellman at 11:43 AM on July 23, 2018 [3 favorites]


> rongorongo:
"One way of preventing this problem would be to have a physical interlock that prevented the drive from being removed - even if you had requested it - until all the operations had been completed. Rather like the way a washing machine door can't be opened until a few seconds after the end of the wash cycle (apparently to make sure that you don't get injured by a drum that may still be spinning).

Ironically this kind of interlock was a feature of much earlier computers: Macintosh disk drives and Zip disks (any maybe even writeable CDs?) would allow you to select "eject" - but would not give you their media until they were done with them. Sometimes you would lose the argument of whether you could have your media back and have to resort to persuasive user of a paper clip."


Yeah, you apparently didn't read my comments about not being able TO eject a drive. I am going to need a BOX of paperclips by my computer, and that is NOT good, as I can GUARANTEE I will spill them at some point.
posted by Samizdata at 11:43 AM on July 23, 2018




except, standing with certain toilets and certain butts can cause poop corruption

There’s this naked rooftop bar in Key West, I don’t wanna get into it but things happen
posted by RobotVoodooPower at 5:52 PM on July 23, 2018 [1 favorite]


I still have a bunch of unbent paperclips in my desk drawer, all of them having far outlived the computer equipment that they were originally unbent to perform some emergency ritual on.

Very recently I did discover that there's a new use for them: opening the SIM card slot in a cellphone. If you don't have the little tool-thingy that the phone comes with, getting the SIM out is a real bastard without something like a paperclip to push in the magic hole.
posted by Kadin2048 at 12:22 PM on July 24, 2018


I have one attached to my keys. The only use I have for it is to give me peace of mind: it has a read me text file with a contact e-mail in case someone finds my keys and wants to return them to me.

The other good thing you could stick on that drive is a copy of your KeePass database that's at least up to date enough to have your current Dropbox credentials in it.
posted by flabdablet at 10:47 PM on July 25, 2018


I think a big part of the problem is that, at least in Windows XP, the UX around unmounting a drive before removing it was fucking terrible. You had to find the weird little icon in the task bar (why the task bar? who knows? it's apparently the part of the UI where shit they didn't know what to do with went), right click it (or was it a left click? better try both!), pick the drive you want to eject (don't pick the wrong one!), tell it to do its thing, then wait some indeterminate amount of time, then pull the drive.

Or you could right-click the drive in Windows Explorer, choose Eject from the context menu, wait for the pop-up that says it's safe to remove, then remove it.

That's what I've done for years in 7, 8 and 10, and having just now popped open an XP+SP3 VM to check that the context-menu Eject option is indeed there for USB drives, apparently what I could have been doing in XP as well had I not been so accustomed to using the system-tray Safely Remove Hardware thing that's been in Windows since Windows 95.

Over the 15 years I spent working as a school IT technician, the main reason I found for teachers failing to use this option was straight-up ignorance that it was even a thing. I found that showing teachers that it was there and explaining why they should use it caused about 80% of them to start doing so as a matter of course.

I also noticed that every single one of the dozens of intensive data recovery exercises I performed on teachers' external hard disks and USB memory sticks over those 15 years was for a teacher who did not habitually do the proper safe-removal dance before reefing out the USB plug.

My success rate on those exercises was something like 80%, but some teachers sadly did lose months of work that I could not help them recover; turns out there's a pretty strong correlation between a cavalier attitude towards unplugging your drives and a cavalier attitude towards backing shit up.

In short, as Two unicycles and some duct tape says: It's your data, dude. Do what you want. But you should be aware that the support folks you're going to ask to get it back for you when you break it (a) might not be able to and (b) will be filing you under "beyond help" and pushing your future issues a fair way down the priority list.
posted by flabdablet at 2:35 AM on July 26, 2018 [1 favorite]


There should be a much easier process than finding the USB logo or opening exploring, choose eject, wait. I have certainly done it, but I try to be patient.
posted by theora55 at 4:04 PM on July 26, 2018


Technical solution: implement a mechanical clamp inside each USB port, combined with a high-sensitivity voltage output pressure sensor that can gauge whether the physical connection is about to be severed. If so, initiate the clamps and send an interrupt that causes an fsync or equivalent to be issued to the relevant device. I imagine this would be about as reliable as the vacuum-actuated door locks on old Mercedes models, but hey, repair revenue.
posted by invitapriore at 6:55 PM on July 26, 2018


There should be a much easier process than finding the USB logo or opening exploring, choose eject, wait.

There's an app for that.
posted by flabdablet at 5:24 AM on July 27, 2018


« Older LBJ orders a pair of pants.   |   “No Restrooms, No Bare Feet, No Directions to the... Newer »


This thread has been archived and is closed to new comments