Harder Better Faster Stronger
June 18, 2012 6:27 AM   Subscribe

Breaking records at 16.32 petaflops, Blue Gene/Q is the world's new fastest supercomputer.

Code named Sequoia pdf, Blue Gene/Q pdf is built around a derivative of the IBM PowerPC A2 processor. A 45nm, 16 core (Sequoia derivative has 18 cores: 16 compute + supervisor + hot spare), 64 bit system on a chip, the PowerPC A2 pdf is highly parallel and highly energy efficient, enabling Sequoia to achieve 2.097 Linpack GF/watt. In comparison, the Fijitsu K Computer (the now second fastest Top 500 HPC system) gets only 0.825 Linpack GF/watt. The A2 derivative in Sequioa also features an interesting proprietary network interconnect pdf that allows for extremely low latency, very high throughput messaging. In total, there are 98 304 of these chips in Sequoia, each furnished with a luxurious 16GB of DDR3 RAM. Should any other records need to be smashed, the Blue Gene/Q platform can scale to a theoretical 262 144 nodes (possibly more). InfiniBand switch fabric connects the compute nodes to 58PB (65PB raw) of storage via 768 IO nodes running Lustre and ZFS (ported for Linux) on 3TB nearline SAS disks housed in NetApp E5400 arrays (ZFS stripe on top of hardware RAID6). The combination yields 1 TB/s of sustained IO, a staggering number for a storage system of this size and complexity. On the software side, Sequoia runs a special patched version of RHEL 6 for front end and IO nodes. Compute nodes run CNK, a lightweight operating system designed by IBM pdf for Blue Gene.

Blue Gene/Q is the fastest, most powerful, most capable high performance computing system on the face of the earth. Currently, it cannot run Crysis.

PS: here's the manual
posted by tracert (40 comments total) 7 users marked this as a favorite
 
This reads like IT porn. I can fondly recall falling asleep with a Computer Shopper magazine in my lap, my pants down around my....

You know, I share too much.
posted by thanotopsis at 6:38 AM on June 18, 2012 [5 favorites]


Man, I bet Team Fortress 2 would run awesomely on that.
posted by mightygodking at 6:41 AM on June 18, 2012


No bench? No sale.
posted by seanmpuckett at 6:42 AM on June 18, 2012 [3 favorites]


Currently, it cannot run Crysis.

Damnit, IBM. *puts away credit card*

More seriously though... huh, I didn't think anybody used nearline SAS for serious applications. I know it's cheaper than real SAS but you'd figure they wouldn't skimp on that. Also, I don't know enough about ZFS. Does it mitigate the speed hit of RAID 6 vs RAID 10?

Now, just imagine a Beowulf cluster of these...
posted by kmz at 6:43 AM on June 18, 2012 [1 favorite]


The most interesting thing about this to me is that it doesn't sound like technical difficulties but rather financial difficulties that are keeping the processing power lower. Seems odd that though this is the cutting edge of processing power its not currently what we're actually capable of doing.
posted by Carillon at 6:48 AM on June 18, 2012


kmz: "Currently, it cannot run Crysis.Now, just imagine a Beowulf cluster of these.."

I came in here to make that very joke. Clearly we were both reading /. too much a decade ago!
posted by barnacles at 6:49 AM on June 18, 2012


42
posted by GallonOfAlan at 6:52 AM on June 18, 2012 [4 favorites]


Could it be used for dating?
posted by anazgnos at 6:53 AM on June 18, 2012 [3 favorites]


Clearly we were both reading /. too much a decade ago!

I recently came across two techies who didn't get my "Netcraft confirms, *BSD is dying" reference.

/spills a 40 bit register
posted by DU at 7:00 AM on June 18, 2012 [1 favorite]


Yeah, see? This is a supercomputer. Bespoke silicon and lots of it, running a special MPP supercomputer OS designed for speed, with its own network of Linux hi-po systems to handle I/O and command-and-control duties.

For a while, it looked like the future of HPC was bog standard x86 chips on commodity motherboards running linux, crammed into endless rows of racks, a'la Google. I'm happy those days are done - this stuff is much more interesting.

More work needs to be done with traditional vector-processing supercomputers, tho - Amdahl's law hasn't gone away just because silicon's cheap these days.
posted by Slap*Happy at 7:00 AM on June 18, 2012 [2 favorites]


Whoa. ZFS for Linux was not stable at all when I used it a few years ago, and there were questions about the filesystem's future under Oracle (along with the possibility of patent issues).

Glad to see that the US government finally stepped up to the plate to develop it further. It really is a very nice filesystem that can do things that nothing else can. I wasn't aware it was a particularly great performer, or that you'd want to use it alongside hardware RAID (which seems to defeat half the purpose of ZFS), but to each his own, I guess....
posted by schmod at 7:03 AM on June 18, 2012


Well, theoretically, yes. But the computer matches would be so perfect as to eliminate the thrill of romantic conquest, anazgnos.

Instead, this machine is designed for the ultimate purpose of mankind: to model nuclear weapons.
Which, by the way, is no match for the real thing as pictures in the last link will show.
posted by coachfortner at 7:08 AM on June 18, 2012 [1 favorite]


Breaking records at 16.32 petaflops...

Whoa. That's MORE FLOPS THAN ADAM SANDLER.
posted by bicyclefish at 7:10 AM on June 18, 2012 [18 favorites]


Imagine a Beowulf cluster of those.

Sorry.
posted by shotintoeternity at 7:11 AM on June 18, 2012


Ah, nevermind. The RAID is temporary until ZFS gets a bit more stable, and IO performance was not their primary motivation behind selecting ZFS.
posted by schmod at 7:11 AM on June 18, 2012


Even more sorry now that I just read that comment twice above this.
posted by shotintoeternity at 7:12 AM on June 18, 2012


Could it be used for dating?

I don't think you need that much computing power just to do the simple calculations required for carbon dating, but sure, go for it.
posted by kmz at 7:14 AM on June 18, 2012 [1 favorite]


Instead, this machine is designed for the ultimate purpose of mankind: to model nuclear weapons.

I would hope that the U.S. government could find better uses for all this computer power than making bombs. Freedom would be far better served if this computing power was reserved for the real time government monitoring and real time "moderation" of all internet communications of everyone, everywhere.
posted by three blind mice at 7:16 AM on June 18, 2012 [2 favorites]


To start, the cluster is on a relatively open network, allowing many scientists to use it. But after IBM’s debugging process is over around February 2013, the cluster will be moved to a classified network that isn’t open to academics or outside organizations. At that point, it will be devoted almost exclusively to simulations aimed at extending the lifespan of nuclear weapons.

Yeah, fuck that. Maybe they will let the liberal tree huggers do something for the betterment of mankind once it is a few years out of date.
posted by Meatbomb at 7:23 AM on June 18, 2012


I think you're right, TBM.
The true purpose of this system must eventually be employed to monitor that our freedom is being utilized correctly.

But the NSA already seems to be on top of that.
posted by coachfortner at 7:29 AM on June 18, 2012 [1 favorite]


boots Windows 7 in under a three minutes!
posted by quonsar II: smock fishpants and the temple of foon at 7:39 AM on June 18, 2012 [1 favorite]


I don't think you need that much computing power just to do the simple calculations required for carbon dating, but sure, go for it.

I tried carbon dating once, but it just got my face, hands and clothes all smudgy.
posted by Strange Interlude at 7:46 AM on June 18, 2012 [1 favorite]


Surely, this takes us to the very brink of the singularity! Kurzweil vobiscum!!
posted by mondo dentro at 7:48 AM on June 18, 2012


It's interesting to see old school supercomputing pop up again like this. The power reduction is a significant advance. Not just saving the electricity bill, but also managing all the heat these things generate.

The whole beast takes 8 megawatts, or roughly the amount of power consumed by a big electric locomotive. A large wind turbine generates 1-3 megawatts. It's using about 5 watts / CPU core; a modern desktop PC uses roughly 20-100 watts / core when running under heavy load.
posted by Nelson at 7:59 AM on June 18, 2012


Are you by any chance a . . . pleasure model?
posted by awenner at 8:01 AM on June 18, 2012 [3 favorites]


The trouble with petaflops is feeding them. Very few problems can be addressed using this architechture. If it takes you an hour to stage a run that is finished in 4 seconds and another hour to recover the results how fast is that?
posted by pdxpogo at 8:03 AM on June 18, 2012


The whole beast takes 8 megawatts, or roughly the amount of power consumed by a big electric locomotive. A large wind turbine generates 1-3 megawatts. It's using about 5 watts / CPU core; a modern desktop PC uses roughly 20-100 watts / core when running under heavy load.

All this at a 45nm process, with intel currently using a 22nm process I wonder what IBM could produce using a similar technology.
posted by Harpocrates at 8:12 AM on June 18, 2012


Man, these are ugly. You would think for that money you'd get some style like the Cray.
posted by sourwookie at 8:30 AM on June 18, 2012


To start, the cluster is on a relatively open network, allowing many scientists to use it. But after IBM’s debugging process is over around February 2013, the cluster will be moved to a classified network that isn’t open to academics or outside organizations. At that point, it will be devoted almost exclusively to simulations aimed at extending the lifespan of nuclear weapons.
The LLNL machine will be for classified uses - but it's worth noting that Argonne National Lab is getting a bluegene/Q as well, and that machine will be open to the scientific community.
posted by kickingtheground at 8:42 AM on June 18, 2012


Man, these are ugly. You would think for that money you'd get some style like the Cray.

Yeah cause this thing doesn't look totally ridiculous with its crazy cat logo...

I actually kind of love IBM industrial design. Always have, always will. When you handle an IBM server it's just... right. It feels right. It's crazy expensive and they don't make sense to buy, but they still feel good when you take em apart. Give me a Thinkpad X series over a Macbook any day. A matte black, near featureless rectangle. Perfect.
posted by tracert at 8:52 AM on June 18, 2012 [2 favorites]


Blah blah blah AM I IMMORTAL YET???

Running out of time here, people...
posted by LordSludge at 9:05 AM on June 18, 2012


Bet it can't even play Youtube vids smoothly, either
posted by slater at 9:07 AM on June 18, 2012


Wow, could you imagine a Beowulf cluster of those things??

I have no idea what that means.
posted by slogger at 9:28 AM on June 18, 2012


Beowulf clusters are the aforementioned racks of ordinary computers networked together, with a parallel virtual machine running on the lot of them. The Blue Gene/Q isn't precisely "bespoke"--the cores are different from what you'd use in a desktop PC, but they are manufactured en masse, and you can buy them yourself if you really want to. At a guess I'd say the real challenge in putting this thing together was to do with the local network. The refrigerator-sized racks are standard, but you have to connect them together with ordinary cables. I think there are around 6,000 racks in this thing.
posted by LogicalDash at 9:40 AM on June 18, 2012


I'm still trying to solve Tic-Tac-Toe...
posted by Theta States at 9:50 AM on June 18, 2012


Nice to see mainframes water-cooled again.
posted by MtDewd at 10:57 AM on June 18, 2012


Giant cluster built to test nuclear weapons stockpile breaks HPC speed record

So after all that expense it just becomes a pile of molten glass?
posted by hal9k at 11:06 AM on June 18, 2012 [1 favorite]


Of course they say it will be used to make bombs, because that is more palatable than saying that it will be used to mine Americans' communications for data.
posted by hellphish at 11:41 AM on June 18, 2012 [3 favorites]


At that point, it will be devoted almost exclusively to simulations aimed at extending the lifespan of nuclear weapons.

Surely they could have harnessed some of this vast computing power to come up with a more plausible cover story than that.
posted by ook at 2:56 PM on June 18, 2012 [2 favorites]


Instead, this machine is designed for the ultimate purpose of mankind: to model nuclear weapons

Penultimate, surely?
posted by pompomtom at 6:50 PM on June 18, 2012 [1 favorite]


« Older Vacation Clips Without Context   |   Hollywood - Celebration of the American Silent... Newer »


This thread has been archived and is closed to new comments