A cluster of raspberries

September 12, 2012 2:57 PM

University builds inexpensive computer cluster with Raspberry Pi and Lego, and publishes a guide if you want to build your own.
posted by a snickering nuthatch (52 comments total)

The first test run on the cluster: calculating the value of Pi.

That's old hat. I wish someone would calculate the value of Raspberry.
posted by benito.strauss at 3:01 PM on September 12, 2012 [5 favorites]

Imagine a beowulf cluster of these.
posted by Ad hominem at 3:01 PM on September 12, 2012 [11 favorites]

I wish someone would calculate the value of Raspberry

My hypothesis: delicious.
posted by curious nu at 3:02 PM on September 12, 2012 [6 favorites]

Is calculating Pi even parallelizable?
posted by Malor at 3:08 PM on September 12, 2012

Yes, it is parallelizable.
posted by a snickering nuthatch at 3:10 PM on September 12, 2012 [2 favorites]

Malor: "Is calculating Pi even parallelizable?"

You betcha.
posted by idiopath at 3:10 PM on September 12, 2012 [1 favorite]

I am no expert. But I believe there is a way to calculate Pi that is a sum of various simple calculations. I think it may be called a digit-extraction algorithm
posted by Ad hominem at 3:11 PM on September 12, 2012

So... when are these Raspberry Pi thingies going to be reliably available in the USA? Every time I've looked, they've been not yet available or out of stock after their first production run.
posted by InsertNiftyNameHere at 3:13 PM on September 12, 2012 [1 favorite]

I believe that right there is a thicket.

Now I don't feel so impressive with my two Pis... (Accidentally ordered a second one)

Of course, I still need to find stable uses for them beyond "This is fun to tinker with".
posted by CrystalDave at 3:14 PM on September 12, 2012 [2 favorites]

That was great, cheers.
posted by Blazecock Pileon at 3:16 PM on September 12, 2012

Of course, I still need to find stable uses for them beyond "This is fun to tinker with".

Isn't that pretty much their stated purpose?
posted by Holy Zarquon's Singing Fish at 3:20 PM on September 12, 2012 [2 favorites]

My old Uni, so proud.

Admittedly I was in the Geology department, so we'd have just collaborated on the Lego build.
And then hit it with a hammer
posted by arcticseal at 3:21 PM on September 12, 2012 [6 favorites]

If I had unimpeded access to their system for just fifteen minutes, I'd fill it with confused-looking Lego minifig hobbits. For science.
posted by JHarris at 3:24 PM on September 12, 2012 [8 favorites]

Anyone see benchmarks? I wonder how much speed they gained factoring in the overhead of MPI and network latency.

I have a "cluster" and I imagine there would be an inflection point where it goes from being faster on one processer to being faster in parallel. I wonder around how many digits it is.
posted by Ad hominem at 3:27 PM on September 12, 2012 [1 favorite]

InsertNiftyNameHere: "So... when are these Raspberry Pi thingies going to be reliably available in the USA? Every time I've looked, they've been not yet available or out of stock after their first production run."

I just got two from MCM Electronics yesterday after ordering on Thursday of last week. I think they are sold out now, but maybe I just got lucky on a restock.
posted by wcfields at 3:27 PM on September 12, 2012

Edit: Newark has over 100 in stock ready to ship today.
posted by wcfields at 3:30 PM on September 12, 2012

All well and good, but for distributed shared memory applications you are paying big bucks to have good communications between nodes. You can afford to have slightly crappy processors if you're doing something that is easily domain decomposed.

Slow processors and a slow network? I am sure there are applications for that, but then you could also run it as a volunteer computing project.

All that said, give me a good compiler with good floating point performance and I'm sure we'd get along fine.
posted by Talkie Toaster at 3:33 PM on September 12, 2012

Of course, I still need to find stable uses for them beyond "This is fun to tinker with".

They make a pretty decent thin client. I'm currently typing this on a Raspberry Pi that's running OpenNX to display my desktop located on a remote linux machine. It's not perfect (hardware accelerated X11 is still a work in progress), but the power savings compared to using a full PC are worth it.
posted by RonButNotStupid at 3:40 PM on September 12, 2012 [2 favorites]

Ad hominem: "Anyone see benchmarks?"

Yeah, that was my first question. (Well, beyond "did they use transparent Legos to make it look extra-cool?") I don't see any performance stats on their website. Are they using "supercomputer" to mean a bunch of networked nodes? Isn't there a minimum number of flops required to be a supercomputer?
posted by jiawen at 3:43 PM on September 12, 2012

It's a super computer
posted by stbalbach at 3:44 PM on September 12, 2012

... the team used 64 Raspberry Pi computers, each equipped with a 16-gigabyte SD card to construct a functioning computing cluster for under £2,500 (a bit over $4,000)—not including the Ethernet swtiches used to connect the nodes.

Also not part of the price: THE LEGO. Those fuckers are expensive.
posted by nushustu at 3:45 PM on September 12, 2012 [2 favorites]

I think it's super enough that they did this! (but yeah, my nerd side wants raw data)
posted by antonymous at 4:22 PM on September 12, 2012

I am no expert. But I believe there is a way to calculate Pi that is a sum of various simple calculations. I think it may be called a digit-extraction algorithm

Maybe that's why they chose it as a first test run, because if you've ever stepped on a Lego that someone has left lying on the floor, it too is a digit-extraction.
posted by BigHeartedGuy at 4:30 PM on September 12, 2012 [2 favorites]

This is all good fun, but I will eat my hat if its performance comes anywhere close to the equivalent $4000 spent on a single high-end commodity Intel box. Maybe performance-per-watt is competitive, though, for data center applications. There's been a lot of talk about using ARM racks for applications that bottleneck on disk I/O instead of CPU.
posted by qxntpqbbbqxl at 4:32 PM on September 12, 2012

All well and good, but for distributed shared memory applications you are paying big bucks to have good communications between nodes. You can afford to have slightly crappy processors if you're doing something that is easily domain decomposed.

Slow processors and a slow network? I am sure there are applications for that, but then you could also run it as a volunteer computing project.

So, now that Acorn Computing has figured out CPUs with low watts per flop, we know the next challenge is dedicated memory transfer buses that are low bits/s/watt.
posted by ocschwar at 4:32 PM on September 12, 2012

Computing pi is a classic parallel code example for MPI because it's only a few lines of C or Fortran.

These days, I don't think you can really call your cluster a supercomputer unless it's made the top500. Entry as of this July means your machine can do more than 60 teraflop/s peak on the LINPACK Benchmark. Sadly enough, this little cluster is 64 nodes and 700 MHz per node. At about 45 GFlop/s theoretical peak, it doesn't even outperform a modern sized workstation, and a single nVidia GPU-accelerated node is more than 20 times more powerful than the entire Pi cluster.

But, hey, Legos are cool, and so are kids and others getting excited about distributed computing.
posted by onalark at 4:34 PM on September 12, 2012 [1 favorite]

All well and good, but for distributed shared memory applications you are paying big bucks to have good communications between nodes.

MPI uses a distributed memory model. Also, depending on the communications requirements of this particular problem, gigabit ethernet may be perfectly sufficient. There's still quite a yawning chasm between locally switched gigabit ethernet (which may or may not even be running TCP/IP) and something like SETI@Home running over people's broadband internet connections.
posted by indubitable at 4:39 PM on September 12, 2012

MPI uses a distributed memory model. That's not quite right, but close enough. MPI uses a message-passing model as opposed to the shared address space model we frequently associate with tools like pthreads and OpenMP. MPI is usable on both distributed-memory and shared-memory architectures, and frequently processs are mapped to cores instead of nodes to avoid the complications of hybrid programming models (e.g. MPI+OpenMP).

As for how the gigabit ethernet is switched, what other stacks are people running besides TCP/IP (assuming we're not talking about custom interconnects like Infiniband)?
posted by onalark at 4:48 PM on September 12, 2012

Whoops, looks like it only has 10/100 ethernet. Still, at $35 a pop, it's a cheap way to gain experience with setting up and running an HPC cluster, even if not really viable as a high performance platform.
posted by indubitable at 4:48 PM on September 12, 2012 [1 favorite]

Yeah, I believe this is mainly useful for learning about distributed computing rather than as a powerhouse.

Still though, a great way to practice with it.
posted by CrystalDave at 4:58 PM on September 12, 2012

Yeah it may not be a supercomputer but using MPI and 100baseT it would be good at tasks with bigger unit of work. It would be great for stuff like bayesian email classification for a high volume MTA, virus scanning for email attachments, processing log files. Anything besides pure number crunching.
posted by Ad hominem at 5:12 PM on September 12, 2012

As for how the gigabit ethernet is switched, what other stacks are people running besides TCP/IP (assuming we're not talking about custom interconnects like Infiniband)?

OpenMX is an option, it seems to cut out a lot of the overhead associated with TCP/IP but which is unnecessary on a closed network. I've never really had a chance to try it out myself, though; my office upgraded to an Infiniband cluster before I got to that point.
posted by indubitable at 5:32 PM on September 12, 2012

Anything besides pure number crunching.

No, it wouldn't be better at any of these things than a quad-socket workstation of commodity-class CPUs. This is an interesting academic exercise, but 64x700MHz is not going to even come close to the 64x2.6GHz AMD Opteron workstation you could get for a similar amount of money.
posted by onalark at 5:40 PM on September 12, 2012

If you want to learn Linux clustering and MPI for sub $5000 you could also buy thirty-two of:

This motherboard, which is about $100 a piece, with CPU.

32 x 4 GB memory modules (cheap these days)

32 x $60 SSD

Each motherboard consumes a max power draw of 11W. They can be powered directly from 12VDC. Rig up a 12VDC power supply to multiple motherboard on non conductive shelves.


Then you need a 48-port switch, some cables, etc. It can run x86_64 code.
posted by thewalrus at 6:06 PM on September 12, 2012 [2 favorites]

No, it wouldn't be better at any of these things than a quad-socket workstation of commodity-class CPUs

I think it would be. I've done a lot of testing with things like unzipping files, an operation you would need to scan email attachments. Using the same file, copied off a share to each machine, 10 concurrent unzips, one per machine, goes an order of magnitude faster than 10 simultaneous unzips on one machine. It isn't even close. I'm not a scientist so I go by benchmarks and I think task switching overhead and IO bound processes kill performance when run on one machine, which is why I use 10 machines to unzip uploads.
posted by Ad hominem at 6:10 PM on September 12, 2012

zip and unzip is not a good test at all as the bottleneck is usually disk I/O, particularly on physical spinning HDDs (vs SATA3 SSDs with 400MB/s sequential read or write speeds).

The CPU in a Raspberry Pi can barely run Firefox and scroll a web page, it's about equivalent in real computing power to a Pentium Pro 166 MHz.
posted by thewalrus at 6:14 PM on September 12, 2012

zip and unzip is not a good test at all as the bottleneck is usually disk I/O, particularly on physical spinning HDDs

Which is what I said. That is why I said it would be good for that particular application. I think we are saying the same thing. I said it would be good for stuff that didn't involve pure number crunching, CPU bound applications.

If I had one, I would test it. I wonder if I can get VMs running that slow.
posted by Ad hominem at 6:22 PM on September 12, 2012

You can artificially throttle the CPU speed of a Virtualbox VM, at least in my setup (Virtualbox 4.1.22, Linux 3.2.x or 3.5.x kernel host machine, various 32-bit kernel guest OSes).
posted by thewalrus at 6:50 PM on September 12, 2012

I'm really curious. You may be right. I'm going to try to figure out how to test this at some point. Really, it is the Lego that makes it neat. If I was doing a startup I would tell techcruch our entire data center was made out of Lego.
posted by Ad hominem at 6:58 PM on September 12, 2012

Ok, if I were inclined to build an inexpensive cluster with the provided guide, what would I be able to do with it?
posted by crunchland at 7:36 PM on September 12, 2012

As a side note, if I wanted something like the Raspberry Pi (size, power, and relative inexpensiveness) but wanted something more powerful what would one get?
posted by wcfields at 7:40 PM on September 12, 2012

Does it have to be ARM? Because you can get a 'real' x86 computer that consumes only 11 watts max (see the link I posted to a mini-ITX Atom motherboard above) for around $150...

Many other ARM development boards like the Beagleboard are $150-200.
posted by thewalrus at 7:51 PM on September 12, 2012

Mountains of Pi.
posted by neuron at 7:58 PM on September 12, 2012 [1 favorite]

IO bound processes kill performance when run on one machine, which is why I use 10 machines to unzip uploads

Instead of getting 10 machines, get a 10-drive RAID array and attach it to one machine. Or just get a few solid state drives. If you have an i/o bound application, invest in improving i/o performance. A cluster of Raspberry Pis isn't the answer.

I don't meant o deny that in some circumstances there's an argument to be made for many slow CPUs instead of fewer fast ones, but places like Google which have workloads which might benefit aren't convinced.

The Hacker News discussion goes into the performance of efficiency of the cluster in great detail. I think focusing on that misses the point that doing this is a fun learning exercise. Sure , you get get a single server with 64 cpu cores for not much more money and test constructing a cluster using virtualization, or even just spin up some instances on EC2, but it wouldn't have the same geek factor.
posted by PueExMachina at 9:22 PM on September 12, 2012

Slow processors and a slow network? I am sure there are applications for that
posted by flabdablet at 9:31 PM on September 12, 2012

if I wanted something like the Raspberry Pi (size, power, and relative inexpensiveness) but wanted something more powerful what would one get?

There's a lot of hardware out there, as browsing liliputing will show.

I recently got an MK802 II. $65 for a 1GHz Cortex-A8 single core processor, Mali 400 graphics, 1GB of RAM, 4GB of storage, a microSDHC card slot, 802.11b/g/n WiFi, 1 full-sized USB port, 2 micro USB ports, and a full-sized HDMI connector. It has Android 4 installed on the internal storage and can boot other linux distributions from the SD card. I haven't had much time to play with it, but I've got it running Lubuntu.

If you're looking for something closer to the developer board spirit of the Pi, there's the cubieboard. For $45 it has a 1Ghz Cortex-A8 single-core processor, Mali 400 graphics, 512MB of RAM, 4GB of storage, an HDMI port, a SATA port, 3 USB ports, an SD card slot, an ethernet port, an IR receiver, and a lot of expansion pins
posted by PueExMachina at 9:40 PM on September 12, 2012

Mountains of Pi.
posted by neuron at 7:58 PM on September 12 [1 favorite +] [!]

! I remember reading this! It was a crazy and fascinating story, about these 'genius' brothers who, hold your hats, Built Their Own Supercomputer!!!1!!!11! A supercomputer so big, it took over the apartment! And they calculate Pi to several billion places and found on a macro scale that it behaves in peculiar ways. I've thought about this story often, the description of Pi as a sort of territory into which one could go and explore was a powerful analogy.
posted by From Bklyn at 11:46 PM on September 12, 2012

What's support for the cubieboard like, though? Probably my favourite thing about my Pi is that I can run Debian on it. (Or Arch. Or Fedora. Or Gentoo. etc...) Last I looked at the mk802 for example, it just ran Android.

There's a pretty awesome community around the Raspberry Pi, doing lots of cool things, and generally being great at sharing them. It's not just about the hardware (which yes, is fairly weak on the Pi - though the GPU isn't to be sniffed at) but also the entire ecosystem that supports it.
posted by Dysk at 1:24 AM on September 13, 2012

Last I looked at the mk802 for example, it just ran Android.

It comes with Android installed on the internal memory but the A10 in general is designed around booting whatever operating system you install on the SD card. All you need is an SD card image of whatever operating system you want (such as the ones here) and you're good to go. The community around MK802 and A10 development is smaller than Raspberry Pi's but it's still just as supportive and collaborative.
posted by burnmp3s at 5:40 AM on September 13, 2012

@qxntpqbbbqxl performance per watt isn't great either. They are not particularly powerful boards. There were plenty embedded boards that existed before the Pi, that had a community and easy purchasing. But of course, they sell for 90$ and the Pi was SUPPOSED TO BUT NEVER SOLD FOR 25$. It looks like they are only going to produce the 35$ model and the actual manufacturers that licence, build and ship the design (the foundation changed it to this model) tack on enough extra charges to reach about the same price 85-90$.
posted by Napierzaza at 7:58 AM on September 13, 2012

thewalrus: "

This motherboard, which is about $100 a piece, with CPU."

Looks good, except there has been some lack of drivers for Linux.

It seems the Intel PowerVR SGX545 GPU's are almost unusable (mostly; no 3D acceleration and 2D can only allocate 8MB of memory in binary-only drivers) in Linux based upon the comments on the Intel Forum.
posted by wcfields at 8:11 AM on September 13, 2012 [1 favorite]

wcfields: "Edit: Newark has over 100 in stock ready to ship today."

Thank you, kind sir. Much appreciated!
posted by InsertNiftyNameHere at 11:45 AM on September 13, 2012