Why Can't MS Do This?

August 17, 2009 6:50 AM

An 8 year old critical security bug in the Linux kernel? No problem, we can fix that without even rebooting. You heard me, it is possible to apply a source code patch to a running kernel without reboot.
posted by DU (54 comments total)

If MS did it, we'd all call it a security hole.
posted by Western Infidels at 6:54 AM on August 17 [4 favorites]

If MS did it, it would be a security hole, since users have Administrator-level privileges by default.
posted by Plutor at 6:56 AM on August 17 [1 favorite]

Shit, I fell for the holy war trolling.
posted by Plutor at 6:57 AM on August 17 [7 favorites]

But does that feature exist on my eight year old Linux distro?
posted by furtive at 7:04 AM on August 17

A man goes into a pet shop to buy a parrot. The shop owner points to three identical looking parrots on a perch and says, "The parrot on the left costs $500." "Why does the parrot cost so much," asks the man. The shop owner says, "Well, the parrot knows how to use a computer."

The man then asks about the next parrot to be told that this one costs $1,000 because it can do everything the other parrot can do, plus it knows how to use the Linux operating system.

Naturally, the increasingly startled man asks about the third parrot to be told that it costs $2,000. Needless to say this inspires the question, "What can it do?" To which the shop owner replies, "To be honest I've never seen it do a thing, but the other two call him boss!"
posted by netbros at 7:06 AM on August 17 [17 favorites]

Yes, but in order to install ksplice so that kernel patches do not require a reboot, you have to... reboot to set up ksplice.

I really don't want to bogart my uptime stats (e-penis):

07:03:30 up 1075 days, 13:09, 3 users, load average: 0.12, 0.13, 0.05

posted by amuseDetachment at 7:06 AM on August 17 [3 favorites]

But does that feature exist on my eight year old Linux distro?

I think so. The ksplice thing is exploding my brain, but I'm pretty sure it'll work there. If you have a patch for your 8 year old source code, anyway.
posted by DU at 7:07 AM on August 17

in order to install ksplice so that kernel patches do not require a reboot, you have to... reboot to set up ksplice.

Apparently not.
posted by DU at 7:08 AM on August 17 [2 favorites]

DAMN YOU, amuseDetachment, i thought i had you till i checked right now:
10:09am up 1037 days, 6:16, 1 user, load average: 0.00, 0.00, 0.00

Just think, there are kids that were conceived, born, and now can walk and talk since our servers were last rebooted.
posted by Mach5 at 7:10 AM on August 17 [12 favorites]

ksplice may be a god send. It may revolutionize maintenance on live systems. It still sets off little warning bells in my head, though.
posted by crataegus at 7:10 AM on August 17

Using the service sets off warning bells. But I think if you download and run it yourself on patches from a known source you should be fine.
posted by DU at 7:12 AM on August 17

DU: Okay thanks for the correction, when I saw modifying the kernel source in the instructions, I assumed it was the installation process (not the patching process).

This is awesome. Thanks for the post.
posted by amuseDetachment at 7:17 AM on August 17

This sort of thing is common in a lot of realtime operating systems. I think you can do all sorts of crazy things in QNX.
posted by chunking express at 7:20 AM on August 17 [1 favorite]

Most Windows patches do not require reboots if you stop the correct services before installing and start them afterwards.
posted by LouieLoco at 7:32 AM on August 17

LOLWINDOZEAMIRITE?
posted by Afroblanco at 7:48 AM on August 17

I think the word kernel is adorable. Let's all be squirrels! I mean SQLs!

*stuffs cheeks with winter time treats, scurries away*
posted by Lipstick Thespian at 7:53 AM on August 17 [6 favorites]

I really thought it said:

"An 8 year old found a critical security bug in the Linux kernel?"

These kids today.
posted by ALongDecember at 7:54 AM on August 17 [6 favorites]

You heard me, it is possible to apply a source code patch to a running kernel without reboot.

Kids today . . .
posted by Ironmouth at 7:55 AM on August 17

ouch
posted by Ironmouth at 7:55 AM on August 17 [1 favorite]

> "An 8 year old found a critical security bug in the Linux kernel?"

No no no. An 8 year old is criticizing a security bug in the Linux kernel. She thinks you're a bunch of dummies for letting it slip.
posted by ardgedee at 7:59 AM on August 17 [1 favorite]

Holy crap, that's cool.
posted by l33tpolicywonk at 8:04 AM on August 17

To be fair the bug wasn't critical for the whole 8 years.

Most of the time it was just a quiet bug. It was only after it caught you looking at OSX images that it became so nasty.
posted by srboisvert at 8:06 AM on August 17

Yeah, micro-kernels give us this win because you can reset the individual chunks that need to be reset upon patching them. Windows 5.0 (NT/XP/Vista/Whatever) was originally designed as a micro-kernel, so this was the intention back in the day. But both the QA and the sheer number of code that had to be written to realize it was too much, and they had to scale back to a micro-kernel that acts like a monolithic kernel. (At the time, Windows NT 4.0/5.0 was the largest software project ever attempted -- larger even than any military project up to that point. Projects of this size are, now, quite quaint.)

One of my favourite environments, BeOS, was a 100% multi-threaded micro-kernel. BeFolk could, and would, apply kernel patches without restarting all the time.

And now, hand-waving: I'm not sure how Linux is doing this, since /technically/ it is not a micro-kernel, even if Torvalds and Tanenbaum had a long public argument about this a few years back. Monolithic kernels are just easier, and Torvalds was more interested in bootstrapping using simpler and better understood technology rather than tearing /all/ the pages from MINIX.

But, yes. Linux has grown up a lot over the years. I abandoned it about a decade ago because I got tired of being an OS tweaker instead of an OS user, but I see that it has finally reach some sort of usability. As a coder and an end-user, though, I'll stick to OS X.
posted by clvrmnky at 8:14 AM on August 17 [2 favorites]

metafilter: the way kernel-level routines such as sock_sendpage react when they are left unimplemented.

I recompile my kernel every morning and have never rebooted. This computer was built from the ground up already running, even before the cpu was properly seated. Also, it's over a thousand years old and the case is covered with Sanskrit etchings.
posted by blue_beetle at 8:55 AM on August 17 [6 favorites]

clvrmnky: I recently switched over to Ubuntu, and I have to say I've been finding it awesome. The occasional UI weirdness is far more than made up for by intelligent patching, software repositories, Compiz Fusion, and general freeness. Oh, and I can use VirtualBox to run Windows in a window if need be.

I'm not saying it doesn't have problems. But my turning point came when I realized that Windows has those problems too. Every Windows XP installation I've ever had (and it's even cropped up before XP, if I remember correctly) has had this hardware detection bug that causes it to forget about a CD drive occasionally on boot. This bug is upwards of ten years old now! While on the Linux side of things, a wireless card that didn't work when I first installed Ubuntu suddenly started working one day after a system update.

Ubuntu has just about got me convinced we really are living in the 21st century.
posted by JHarris at 9:17 AM on August 17

Not to be an old geek, but patching running kernels goes back to at least BSD 4.2 running on a VAX 11/785. It was scary, and involved loading new object code via a debugger attached to the kernel. But it was doable.
posted by Nelson at 9:33 AM on August 17 [1 favorite]

Hmm, I wonder if there's any reason ksplice wouldn't let you update and modify the running kernels of EC2 instances? One of the (few) frustrating things about running in Amazon EC2 is that you can only boot their approved kernels, which are sometimes a bit old — but if you can let them update themselves after booting...
posted by nicwolff at 9:41 AM on August 17

My (extremely limited) understanding is that all you need is root access to a running 2.6 kernel.
posted by DU at 9:43 AM on August 17

My (extremely limited) understanding is that all you need is root access to a running 2.6 kernel.

So you are saying you could use this local vulnerability to install ksplice?
posted by idiopath at 11:07 AM on August 17 [1 favorite]

So, yeah, how do you patch your kernel if you're using, say, Ubuntu 7.10? And you're not sure whether or not to trust Ksplice?
posted by jiawen at 12:22 PM on August 17

Let me know when I can de-minimize a window in less than 3 seconds on a system with an ATI graphics card.

I love jolicloud/Ubuntu on my netbook though.
posted by Memo at 2:19 PM on August 17

An 8 year old critical security bug

As I know fuck-all about technology, I just pictured an eight-year-old kid in a ladybug costume, with a flashlight and badge, frowning vigorously.
posted by the littlest brussels sprout at 3:30 PM on August 17 [7 favorites]

Look - I give Linux the benefit-of-the-doubt and try a new distribution every 6-8 months. Unfortunately I've never had reason to leave it on there. The latest Ubuntu killed my bluetooth mouse which worked previously and I cannot get it to connect to my wireless network - both of which worked under the previous version.

Let me know when it supports full Cut/Copy/Paste (including HTML, RichText) across ALL graphical applications... Oh, and CUA key combinations should work flawlessly.

It really bugs me, because Linux as a concept is powerful - yet Be (BeOS) was able to make something rock-solid with a tiny fraction of developers...
posted by jkaczor at 3:37 PM on August 17 [2 favorites]

16:12:50 up 1666 days, 2:10, 22 users, load average: 0.03, 0.29, 0.47
RIP ridiculous uptime :(
posted by kdar at 4:13 PM on August 17

I've been using this for a few months on my desktop at work, and I can say that it appears to work flawlessly.

It's a clever bit of work, definitely.
posted by robzster1977 at 4:16 PM on August 17

So, yeah, how do you patch your kernel if you're using, say, Ubuntu 7.10?

I don't know much about ksplice, but I imagine you download the patch for Ubuntu 7.10 and apply it.

And you're not sure whether or not to trust Ksplice?

Just recompile another kernel?
posted by DU at 5:03 PM on August 17

Ksplice replaces a function by linking a new version of the function, called the replacement code, into the kernel and by placing a jump instruction in the running kernel’s memory, at the start of the obsolete function, to direct execution to the replacement code.

I once worked on an embedded system project where we needed completely thread-safe versions of some of the ANSI C file handling functions, but the proprietary toolchain we were burdened with only supplied a non-thread-safe ANSI library, without source code, that it forcibly linked everything we built against.

Rather than re-implement an entire ANSI library with new names, I wrote a patch_function() function that

• accepted a pointer to an existing function and a pointer to a replacement function
• copied the first instruction from the existing function to a heap location, and followed it with a jump back to the second instruction of the existing function, thereby giving the existing function an alternative entry point
• wrote a jump to the existing function's old entry point, pointing to a small piece of wrapper code that added the existing function's new entry point to the parameter list of its caller, then jumped to the replacement function.

The net effect was that any call made to one of patch_function's victims would get redirected to one of our own functions, which would find itself passed a pointer to the original function, along with the original function's own parameters. We could surround any existing library function with mutex processing, or even completely replace it, and even calls made from inside the inscrutable proprietary library would get our additional processing applied.

As far as I know, that code is still running, somewhere out there...
posted by flabdablet at 7:20 PM on August 17 [1 favorite]

I don't know much about ksplice, but I imagine you download the patch for Ubuntu 7.10 and apply it.

From where? Ubuntu certainly isn't providing such a thing.
posted by jiawen at 8:06 PM on August 17

Go ahead [y/N]? y
Installing [vzb58txi] Multiple bugs in filesystem core.
Installing [m3gwjflz] Possible erroneous memory overcommit in program start.
Installing [wibq2aaj] Performance regression in filesystem buffer code.
Installing [woc3oq2f] CVE-2009-2692: Local privilege escalation in sock_sendpage.
Your kernel is fully up to date.

Well, hell. One little apt-get install and it works for me. Sweeeet.
posted by caution live frogs at 8:09 PM on August 17

I remember reading of some experimental linux kernel work that was like this, turned up to eleven— it was actually attempting to be able to snapshot the entire kernel's state so that you could restore the state to a different kernel version. I think the original goal was to support migrating a running system from one physical machine to another, but rebootless upgrades and suspend-to-disk were neat freebies. This was ten years ago; I don't think they ever finished the work.

The crazy extreme of this are things like old telecom switches or Tandems with a decade of uptime and so many upgrades that it's not likely they even can boot from a powered-off state any more.
posted by hattifattener at 8:54 PM on August 17 [2 favorites]

Let me know when it supports full Cut/Copy/Paste (including HTML, RichText) across ALL graphical applications... Oh, and CUA key combinations should work flawlessly.

You don't have to phrase tech support requests like that here. This isn't a Linux forum.
posted by Pope Guilty at 10:35 PM on August 17 [2 favorites]

From where? Ubuntu certainly isn't providing such a thing.

I don't understand what you are asking for. Ksplice doesn't patch "Ubuntu". It patches a kernel. If you have a really old kernel, such as the one that came with Ubuntu 7.10 and want to run a really new kernel, such as the one that comes with 9.04, you'll need diffs.

Ksplice is a piece of software that applies diffs, not a source of the diffs themselves. (Except fot their online service, which handles the most common cases of recent diffs.) You'll have to find some or just upgrade the whole kernel yourself.
posted by DU at 4:14 AM on August 18

OK, this is strange. Now when running Aptitude my system is trying to remove ksplice-uptrack and replace it with ksplice - what gives? I can't find anything telling me what the difference is between the packages.
posted by caution live frogs at 7:29 AM on August 18

Let me know when it supports full Cut/Copy/Paste (including HTML, RichText) across ALL graphical applications... Oh, and CUA key combinations should work flawlessly.

Linux is a kernel, cut and paste should definitely not be implemented in the kernel.

The Gnu/Linux OS that usually gets badged with the name "Linux" comes with about 4,000 packages of available applications (on my small to medium install the aptitude program tells me "195302 files and directories currently installed."). Some of those packages don't have a UI. Some are based on a non-interactive command line UI. Some are based on the Athena widgetset (or the raw Xlib that athena is built on) and use the X11 cut and paste conventions from the '80s (left button selects, right button resizes selections, middle button pastes the current selection). Everything newer than that is compatible (to some degree) with the previous conventions that are built into the specification for X11. Many work however the particular developer wanted, following no particular convention. Apps written after the year 2000 or so mostly use CUA bindings, but those are incompatible with legacy keybindings of a number of older apps.

If you feel like putting together a team to fix these 10,000 or so programs in these 4,000 or so packages so they all are retrofitted to use CUA bindings, feel free, but most people consider it a waste of their time. Or you could pare down that list of programs and pick a few thousand or a few hundred that all work with CUA. Who has the energy to maintain that kind of sheer volume of forked code? The problem with Be was they had no fucking apps. I don't even think Be could maintain that many forked apps with that kind of quality, or coerced that many maintainers who think nothing at all is wrong with the UI on their particular app to do UI in a consistant manner. When you have no power to hire or fire, and rely on donated time, or the freely shared fruits of donated time that wasn't even originally devoted to you, you make do. A Gnu/Linux system is really really for the large part a collection of scavenged and discarded code. A system that has made do with what is available for free. Of course the best rises to the top, and you get a very reliable and powerful system. But if you want consistency, start from scratch like Beos did, and best of luck to you, but Gnu/Linux is probably not for you.
posted by idiopath at 7:31 AM on August 18 [3 favorites]

And regarding my (admittedly snotty) attitude in the post above:

You seem to be trying to bribe the Linux community with the idea that you would be a potential user if X got fixed. This is a common approach, and has almost zero leverage. The Linux community doesn't need users, we have enough already. Users with the kind of "customer" attitude you display are not an income source, and do nothing to improve app quality or sustain the community. If you want to get our attention, mention that you might beta test or report bugs or help write documentation or write code. Offering to be a potential user offers us nothing.
posted by idiopath at 7:46 AM on August 18

Ksplice replaces a function by linking a new version of the function, called the replacement code, into the kernel and by placing a jump instruction in the running kernel’s memory, at the start of the obsolete function, to direct execution to the replacement code.

so every patch applied this way bloats your kernels wired memory requirement... nice for managing security fixes when a reboot can't be scheduled, but I'm curious how large the image would grow if you made a practice of applying every kernel minor version update diff this way...
posted by russm at 8:05 AM on August 18

As a fellow Unix lover, let me summarize idiopath's two comments. "Yeah, cut and paste sorta sucks in Linux. You can fix it yourself!"
posted by Nelson at 9:05 AM on August 18

jkaczor: "Let me know when it supports full Cut/Copy/Paste (including HTML, RichText) across ALL graphical applications... Oh, and CUA key combinations should work flawlessly."

Let me know when any Operating System forces that set of features across all software available for it.

"It really bugs me, because Linux as a concept is powerful - yet Be (BeOS) was able to make something rock-solid with a tiny fraction of developers..."

Not to mention a tiny fraction of the applications.
posted by Plutor at 9:33 AM on August 18

Users with the kind of "customer" attitude you display are not an income source, and do nothing to improve app quality or sustain the community.

Yes - typical linux booster response - fix it yourself and/or we don't want you in our playground.

Really? Users do nothing to improve app quality?

I understand the technical challenges (I am not a typical end-user), I even understand the difference between "kernel" and "userland". Hell, if I donate my time to code on something for fun, I code what I want.

Yet - this is not how Linux is "marketed", by either it's "boosters" or it's corporate interests.

For servers and embedded systems it is amazing - for the desktop, it still has a bit of work needed. If you can't admit that, then you are more evangelist than realist.
posted by jkaczor at 2:49 PM on August 18

Users with the kind of "customer" attitude you display are not an income source, and do nothing to improve app quality or sustain the community.

Wow, so we've gone from "Is it the year of Linux desktop for grandma" to more or less fuck off if you dont like it.
posted by damn dirty ape at 3:00 PM on August 18

I am not a Linux evangelist, I am a Linux user. And yeah, personally, as far as I am concerned, either pitch in and help or fuck off. I contrasted users to someone who submits bug reports, or helps with documentation, or contributes code, etc. that is, only a user, someone who offers nothing else. In that sense, a user does nothing for Linux, and losing someone who cannot at least offer one of these things is no skin off my back. Of course I don't speak for the greater Linux community, but personally I think the Ubuntu attitude of recruiting users for its own sake is seriously misguided.
posted by idiopath at 3:12 PM on August 18

I got flamey here, and I apologize.

Linux has strong points, but consistency is not, and will not ever be, one of them. If you want a consistent UI, Linux will probably never be the right OS for you, not because you are not good enough or not smart enough, but because it is the nature of GNU/Linux to be inconsistent and mostly rough around the edges.

That doesn't mean anybody needs to fuck off or any other rude stuff like that.

There is this cliche that you cannot get help with Linux problems by asking for help, but if you insult Linux in a place where Linux users hang out, you will get a bunch of people trying to defensively prove you wrong and helping you with your Linux problem. This works, and people do it, and it leads to a bunch of pissed off and defensive Linux users.

One cannot help having a little ego investment in one's favorite OS or brand of beer or whatever. So when I see someone yet again putting down Linux, my instinct is to try to make sure they never ever use Linux again. Which may be appropriate on IRC or something, but was probably out of place here.
posted by idiopath at 3:32 PM on August 18

I contrasted users to someone who submits bug reports, or helps with documentation, or contributes code, etc. that is, only a user, someone who offers nothing else. In that sense, a user does nothing for Linux, and losing someone who cannot at least offer one of these things is no skin off my back

You know what? I'm quite smart enough to work around all kinds of UI brokenness, but that doesn't mean I enjoy having to do so. To the extent that complaints from "customers" serve to remind developers of how people expect things to work, I think that those complaints are valuable.

personally I think the Ubuntu attitude of recruiting users for its own sake is seriously misguided

I'd like to encourage you to reconsider that position.

One of the reasons that malware causes so much trouble for so many people is that desktop computing, in 2009, is still pretty much a monoculture: Windows has an overwhelming market share in desktop computing environments. This is bad in and of itself, but it's made far, far worse by the Windows cultural legacy of influential apps that cause assorted kinds of trouble when run without administrative privileges. Most people seem to end up relying on at least one of these, which means that in practice, most Windows boxes are run without the OS's inbuilt security controls turned on.

I know for a fact in my own community that awareness of Linux in general and Ubuntu in particular is growing rapidly, and that Windows is starting to be perceived as a probably-necessary evil rather than as the only available choice. And even though most of the people I've convinced to give Ubuntu a whirl are unlikely to end up contributing code, the fact that they are happily using something that isn't Windows, but runs on the same cheap commodity hardware that Windows runs on and doesn't require AU$200 for a Windows licence, is not lost on their friends and relatives.

This growing awareness - that there are perfectly workable alternatives to the Microsoft product - is, I think, the main thing that Mark Shuttleworth intended to achieve by setting up the Ubuntu project. I'm not sure that Microsoft intended the release of Windows ME 2008 Vista to contribute as much to that awareness as it undoubtedly has.

As long as the desktop computing environment remains a de facto monoculture, (a) malware will continue to cost everybody significant time, money and irritation and (b) the simple fact that the number of Ubuntu users is growing will continue to make a positive contribution to diversifying that environment.

In the unlikely event that the percentage of Linux users grows until it's in the same ballpark as the percentage of Windows users, the Unix cultural legacy of strict separation between root access and user access that those users bring with them will encourage more app vendors to do things properly, and this will in turn make it more likely that Windows users will feel less pressure to turn off their security controls.
posted by flabdablet at 7:18 PM on August 18 [1 favorite]

So when I see someone yet again putting down Linux, my instinct is to try to make sure they never ever use Linux again

Ah, but see - I wasn't truly putting it down. As I said, I give it the benefit of the doubt and try it every 6 months or so, to see if it has progressed to something I feel will work for me.

I think it has potential - open source and free software are a powerful concept. I've been involved in computers both as a hobbiest and professional for a very long time and few things have excited me as much as Linux. (For instance - I have every month of Linux Magazine & Linux Journal for the past 5-6 years on a shelf behind me at the moment) Heck, I remember when Windows was literally a bolt-on to DOS. Right now I would peg the Linux desktop usability at about a "Windows 3.11" level - but I expect more because I have seen more.

(... as well, I do have to admit a tendency to root for the underdog, no matter who is paying my bills... )

Finally - what "irks" me is that we know it can be done. You can build a stable, solid GUI on top of an open kernel - if I really want the power of Unix with a great UI, I can always get a Mac....
posted by jkaczor at 9:58 PM on August 19