Sixth Sense: wearable tech

March 11, 2009 9:10 AM

MIT demo of some very interesting wearable tech (~7 mins vid from TED)
posted by peacay (49 comments total)

Impeach Obama and make that guy President.
posted by DU at 9:25 AM on March 11, 2009

OHHHHHH yeah. Love the bit about drawing your own calculators and clocks on your hands.

Fantastic post.
posted by Lipstick Thespian at 9:38 AM on March 11, 2009

Impeach Obama and make that guy President.

I'm pretty sure he's a good bit younger than 35. :)
posted by cerebus19 at 9:38 AM on March 11, 2009

Also, I had the idea of "eliminate the display device and show it on the wall" just like 3 weeks ago. So I get to be VP.
posted by DU at 9:39 AM on March 11, 2009

I was incredibly sceptical until I saw the number pad on the hand, and some of the drop out info on items in stores or books. A device that could include that kind of interactivity while being even less cumbersome than that would be fantastic. The issues of privacy exist, but if it was a device that could project like that but also be used as a normal cellphone or something it would deal with that.

Apparently more than a few cellphone manufacturers are considering phones with projectors built in, which seems like a ghastly idea considering all the idiots on trains who use their cellphones with loudspeakers on to play music...
posted by opsin at 9:44 AM on March 11, 2009

THAT IS SO DAMN COOL.

Anyone else apprehensive about the tag cloud that would projected upon your person?


This is also going to do interesting things to the dynamic of dialing the phone in dreams.
posted by louche mustachio at 9:48 AM on March 11, 2009 [1 favorite]

Actually, the store/book info was what made me skeptical. Not skeptical that I might like such a thing (which I think is what you mean) but skeptical that this could work. That's some super kickass recognition software that can tell a book from a package of toilet paper, at any angle and under any lighting conditions. Also recognizing all pages from every book? Getting a little blue sky.
posted by DU at 9:50 AM on March 11, 2009

Yeah, somehow I don't think it's going to be able to do a lot of the things in the demo. I still want one for Christmas.
posted by louche mustachio at 9:53 AM on March 11, 2009

Every single surface is going to be NSFW from now on.
posted by Dumsnill at 9:59 AM on March 11, 2009 [3 favorites]

Just remember: spoon computing while waiting in line will be considered rude unless you ask first.
posted by effwerd at 9:59 AM on March 11, 2009 [1 favorite]

Awesome. TED feels like the Ikea of Singularity.
posted by CynicalKnight at 10:00 AM on March 11, 2009 [5 favorites]

This is kind of cool, but I give it about five minutes in the real world before we're all demanding that they fuck off and take their projectors with them.
posted by bokane at 10:01 AM on March 11, 2009 [1 favorite]

"spoon computing"
posted by DU at 10:03 AM on March 11, 2009

That's some super kickass recognition software that can tell a book from a package of toilet paper, at any angle and under any lighting conditions. Also recognizing all pages from every book?

I think this is just a demo of what the device COULD do, if the right software is written for it. The clip of them inside the media lab, with colored tape on their fingers- moving pictures around like on a multi-touch screen- i think THAT is what it can actually do at the moment.
posted by TechnoLustLuddite at 10:07 AM on March 11, 2009

I think this is just a demo of what the device COULD do, if the right software is written for it.

Right, I get that. In fact, I guarantee it--because that software very much does not exist right now. But that's like saying Wall-E is how robots COULD be, if the right software were written for it. If anything, the "right software" would actually be a much bigger achievement than the hardware stuff the video (TED or Pixar) shows.
posted by DU at 10:12 AM on March 11, 2009

* Looks down at shirt *

ɐɟɯʇp
posted by steef at 10:14 AM on March 11, 2009 [4 favorites]

Awesome idea, but can you imagine if they actually took off? Every damn surface would be covered in people's shifting, overlapping, illegible projections. Stand talking to a group and have dozen projections of tag clouds overlapping on your T-shirt.

Instead of projecting onto a surface, it'd be much better to show the graphics on the inside of your glasses. Plus that way no-one could read what your terminal is showing (unless you choose to share displays?), and it could watch your pupils to see which parts of the visual field -- whether objects in the real world or displayed graphics -- you're focusing on. Maybe a sort of "look-and-double-blink" gesture to bring up information about whatever you're looking at?

Assuming the product recognition could be made to work (presumably pretty trivial if you show it the product's barcode - I've heard some phones already do this), some huge centralised server(s) would need to be set up as a backend. Which would need to be paid for somehow, so would likely be plastered with ads overlaid on every goddamned thing you look at. And can you imagine accidentally installing adware on it?

If you're in the tinfoil-hat brigade, one could imagine a camera system connected to a lookup database being very useful for building up a detailed profile of where you like to shop, relax, work, etc. Great for unscrupulous marketers (c.f. Phorm) or eight foot lizards who are secretly running the country!

Er, in summary: A gadget with awesome potential, but get off my lawn!
posted by metaBugs at 10:18 AM on March 11, 2009

Dumsnill : Every single surface is going to be NSFW from now on.

From now on?

*looks around*

Once again, I see I'm ahead of the curve!
posted by quin at 10:36 AM on March 11, 2009 [1 favorite]

Picoprojectors are all the rage in the projector community right now and they seem to be the latest inovation making this presentation possible. There are, however, a few major drawbacks beyond the obvious "I can't stand teenagers with laser pointers, now this ?(*&(*&!"

First, picoprojectors can run HOT. We complain about cell phones getting warm through high use, imagine these things. Oh, and yes, there are alsreay mfgs with these built into phones.

Second, the batter life is generally about an hour. To run these things with computing and mobile connectivity, there will have to be HUGE battery advancements.

So, I hate to say it, but ruling for today: Vaporware.
posted by Muddler at 10:40 AM on March 11, 2009

*note to self - spell check* :)
posted by Muddler at 10:41 AM on March 11, 2009

Instead of projecting onto a surface, it'd be much better to show the graphics on the inside of your glasses. ...it could watch your pupils to see which parts of the visual field -- whether objects in the real world or displayed graphics -- you're focusing on.

Do it for under $350. Part of the genius of their demo is that they've really done almost no work at all. It's a cellphone, a webcam, a mirror and a few pen caps. The only really expensive (and by that I mean "not immediately accessible COTS") thing there is the tiny projector.
posted by DU at 10:42 AM on March 11, 2009

Oh now now - I think it's awesome. I LIKE the idea of a tag cloud on my shirt - think of the possibilities when you go clubbing/to parties/whatever.

You could have feminists who have womensrights, equalpay, glassceiling, lesbianation come up on their chests.

I could see people tailoring their tags and wearing them around to suit their mood.

Of course, I'm one of those people who thought the interactive ads in Minority Report were one of the coolest things about the movie. I also like the idea of living in an urban environment where little bits of text are moving all over the place.

When I went to NYC recently (for the first time) I was blown out by the big walls on buildings that had video on them at ground-level.

Go Yay Future!
posted by Lipstick Thespian at 10:44 AM on March 11, 2009

WAAAAAAAAAAANT!
posted by Lord_Pall at 11:04 AM on March 11, 2009

Sixth sense? Haven't you people been keeping track? This is like the Dozenth Sense by now. Sixth Sense 3.0. Where is the haptic compass I was promised? Where's my surgically implanted Bone Fone? Where's my neural jack (sorry, neuromotor prosthesis)?

Don't believe the hype. I even stopped using the bluetooth earpiece my wife bought me because I can't be bothered to charge it, and it doesn't fit in my pocket as comfortably as the iPhone's wired earbuds. Curse you future!
posted by anotherpanacea at 11:05 AM on March 11, 2009 [1 favorite]

Brings to mind Stephenson's Gargoyles (from Snow Crash), and that BMW augmented reality mechanic video. I agree with everyone above who thinks the projector is a little offensive, especially in the context of a 'sixth sense'. When I go around touching things, everyone around me doesn't have to worry about feeling what I'm feeling; why should they have to experience my sixth sense too? Those BMW tech specs would work fine.

Another thing, about the machine-readable reality part (book annotations, etc.), is that for many of her examples (tp at grocery, eg) the products are already machine readable; it would be a matter of changing convention—barcodes on the front!—not developing new software or wevs. If your chestcam can pick up an ISBN or the dateline off the front of the paper, that's all you really need.
posted by carsonb at 11:06 AM on March 11, 2009 [1 favorite]

Also, whichever company picks up production of this sweet little idea gets dibs on interstitials. As you're bustling to work, chestcam detects blank wall ahead? HAVE A NICE PEPSI-COLA!
posted by carsonb at 11:08 AM on March 11, 2009

heh. my rant got away from you there. it should have been "Curse you future! You are only amazingly cooler, with your cellphones and GPS and digital cameras and netbooks, rather than incredibly, absurdly, surreally cooler as I was promised by Bruce Sterling and William Gibson!"
posted by anotherpanacea at 11:11 AM on March 11, 2009

When I went to NYC recently (for the first time) I was blown out by the big walls on buildings that had video on them at ground-level.

Thespian, get thee to Tokyo.
posted by rokusan at 11:16 AM on March 11, 2009

This system would be far more useful with an accelerometer and a second camera for stereoscopic vision for depth estimation.

Construction workers with the ability to project a laser level, accurate ruler in front of them with a gesture. The ability to pull up the floor plans for the entire structure with a gesture and display it on the nearest piece of drywall.

Reading a book on one of those off-white airplane seatback trays using something like the iPhone's Kindle app...
posted by Ryvar at 11:25 AM on March 11, 2009 [1 favorite]

I fear that my initial response, "There's not a lolcat Do Not Want big enough to express my feelings on this" would earn me a giant cloud of LUDDITE, CURMUDGEON and MR. GRUMPYPANTS tags.
posted by Spatch at 11:31 AM on March 11, 2009

That's some super kickass recognition software that can tell a book from a package of toilet paper, at any angle and under any lighting conditions. Also recognizing all pages from every book?

I think this is just a demo of what the device COULD do, if the right software is written for it. The clip of them inside the media lab, with colored tape on their fingers- moving pictures around like on a multi-touch screen- i think THAT is what it can actually do at the moment

Sure it could just read barcodes? Or read the RFID chips that are supposed to become ridiculously ubiquitous (ridiquitous™?) any day now?

I love the contrast though - the Microsoft Surface is a coffee table that costs $12,500, this prototype was assembled from off-the-shelf components, is totally portable and cost $350.
posted by kcds at 11:33 AM on March 11, 2009

It would be annoying to have to always aim everything you want to project on just so.. like, hmm I wonder what time it is let me check the 'watch' on my arm... ok now get my arm to the exact right position so it looks like there's a watch there.. oh shit a little to the left.. wait wait the car hit a bump whoaaa! ok ok now I got it.. oh shit, the sun glare makes it hard to see.. uh oh.. the car swerved again. Wow.. how fucking useful.
posted by ChickenringNYC at 11:37 AM on March 11, 2009

It would be annoying to have to always aim everything you want to project on just so

As long as you are close-ish, it should be able to correct for you. In fact, it must already be doing the first step of this in the watch demo. It displayed the watch face where he drew the circle. Now it just needs to track your arm and keep the display at the same location until you move so far that it can't anymore.
posted by DU at 11:45 AM on March 11, 2009

Two words: ubiquitous goatse.

Nostradamus, Verne, Huxley, Orwell, Stephenson, ... no one could foresee a dystopia as cruel as this. My bones grow cold.
posted by buzzv at 11:49 AM on March 11, 2009 [1 favorite]

So most that demo was actually faked? Yes, hardware is interesting, but nowhere near as unsolved a problem as the software they purport is. I assumed it would be a bar-code scenario, but they suggest image recognition, which I find hard to believe works for a domain larger than "this one toilet paper or that one book".

Yes, the idea is neat, but if its all smoke and mirrors it was just as neat in Ghost in the Shell, which they allude to at the end.
posted by pwnguin at 11:53 AM on March 11, 2009

>
Er, in summary: A gadget with awesome potential, but get off my lawn!

The potential for info-overload jumps out at you... but maybe that's an indicator not of how wildly stressful the future will be, but of how radically the human mind will have to adapt, just to accommodate the kind of data we'll be plugging into it. Our writing and speech aren't as ornate as they were in the past; I wonder what present functions we'll downgrade, as our brains evolve to incorporate Instantaneous Web Access.
posted by darth_tedious at 12:02 PM on March 11, 2009

I'm pretty sure that this is what Hitler had in mind when he invented HDTV.
posted by blue_beetle at 12:21 PM on March 11, 2009 [1 favorite]

That's some super kickass recognition software that can tell a book from a package of toilet paper, at any angle and under any lighting conditions.

Alternatively, it can perhaps use this.
posted by DreamerFi at 12:25 PM on March 11, 2009

Just wait til this actually becomes commonplace. We'll all be communicating via iconography instead of actual language.

CLIPPY WINS AFTER ALL!
posted by WolfDaddy at 1:09 PM on March 11, 2009

I really would like to build my own augmented reality system. Never thought about projection. Just the stereotypical glasses.

My main desire is a servo'd laser pointer on the side. And then I can say "computer - find 'text'" and it will point to the text I'm searching for.

*sigh*

Will AR ever take off? I think it might.
posted by symbioid at 3:01 PM on March 11, 2009

metaBugs: "Awesome idea, but can you imagine if they actually took off? Every damn surface would be covered in people's shifting, overlapping, illegible projections. Stand talking to a group and have dozen projections of tag clouds overlapping on your T-shirt."

I enjoy imagining what a future where these devices are commonplace might look like. Public spaces lined with freestanding blank panels for passersby to project on. New fashions that emphasize white space for personal projections -- white gloves could come into the vogue -- or shirts that incorporate grids to facilitate image stabilization. And disco balls could make a comeback with people who can't stand the new devices and want to smother them, like contemporary restaurants with cellphone-blocking insulation.
posted by Rhaomi at 3:13 PM on March 11, 2009

I agree with everyone above who thinks the projector is a little offensive

Not to mention the risk of having Mark Hamill shove a #2 Philips up your nose.
posted by CynicalKnight at 3:18 PM on March 11, 2009

.I love the contrast though - the Microsoft Surface is a coffee table that costs $12,500, this prototype was assembled from off-the-shelf components, is totally portable and cost $350.

Of course, Surface actually works and is fairly polished, this is a demo and half of this stuff doesn't actually work like they show, plus in its current form would be rather clunky. Not to mention that this involves a fraction of the hardware. In fact, I'm not really sure what your point is, since this is a completely different usecase / tech than Surface.

(Surface's real use: to be the most badass gaming table EVER).
posted by wildcrdj at 3:18 PM on March 11, 2009

I'm not really sure what your point is, since this is a completely different usecase / tech than Surface.

Yeah but... gestural user interface! Microsoft totally owns that! They invented it, just like they invented the graphical user interface before it! Call it a... GUI 2.0.
posted by anotherpanacea at 3:23 PM on March 11, 2009

I was one of Pattie's grad students at the MIT Media Lab, 1996-1999. She was a very kind professor, I really like how she goes out of her way to give her student credit here.

It's a good demo reel. A lot of the basic ideas were things my colleagues were working on there back in late 1990s, the Wearables computer groups with the likes of Brad Rhodes and Thad Starner. It's nice to see how the work has evolved.

One big change is the apparent ease of computer vision for tracking the coloured thimbles. Being able to locate hands in front of you isn't an easy problem and would have been very difficult in a portable device ten years ago.

The other big change is the switch towards projecting the information on a surface, rather than using a private display visible only to the user. I used to hang around with the wearables geeks and their private displays in the corner of their eyes (hung from a hat). It was really fucking irritating when they'd detach and start reading their screen: think someone whipping out a cell phone and staring at the screen while you're trying to have a conversation. Projecting the workspace so it's visible to all makes it a mch more social form of media and seems like an important decision to me. (But boy, that bright bulb must be hell on the battery life of a portable device.)

If you're interested in gestural interfaces, check out Oblong. It's a startup by some other MIT Media Lab folks, working on gestural user interfaces for computers. They have some frightfully smart people and interesting ideas, I've enjoyed watching them turn this kind of research into real product. (Oh yeah, and since everyone references Minority Report; one of the founders is the actual guy who designed a lot of that movie sequence.)
posted by Nelson at 3:54 PM on March 11, 2009 [1 favorite]

Like Nelson I was also a visitor to the Media lab in the mid 90s. In particular I remember a guy called Steve Mann there who was one of the pioneers of wearable computing. If you look at his wikipedia page you can see how things have come on since then in terms of miniaturisation. What impressed me was not only that he constructed and programmed all his own equipment - but also that he refined the design after hours of walking about with it taking street photos, getting into arguments with waiters and shopkeepers, cycling through Boston - and so on.

I think Pattie Maes' student shows that same spirit. It is quite easy to pick holes in the demo its overall appeal is in its broad ideas. And I am willing to bet that behind each idea lies many hours of gradual improvement based on practical use of prototypes. The Media Lab has always gone by the motto of "Demo or Die" - where the audience is the media, business sponors or conferences like TED. In the background, however the students can only succeed by having their actual engineering and programming skills examined and published in peer reviewed sources.
posted by rongorongo at 5:35 PM on March 11, 2009

"In the background, however the students can only succeed by having their actual engineering and programming skills examined and published in peer reviewed sources."

I'm a current student at the lab, and from my perspective this is not strictly true. Your level of academic engagement is sort of up to you. In general, PhD students are pretty academically engaged, but it definitely depends on the group. Some groups are more into doing interesting design work than they are publishing. It also depends a lot on the student. If, as a student, you don't enjoy writing/publishing it's possible to plot a low-publishing course through the phd program.

Within the lab, though, cred is all about demos. We all operate in semi-distinct academic realms so we may not know when people get papers published, but we can all respect a good demo. Pranav gets a lot of respect for pulling this off. It's not even about the tech — as pointed out this is not really much custom tech — it's a really clever design idea. And that's worth gold.
posted by heresiarch at 8:19 PM on March 11, 2009

That's some super kickass recognition software that can tell a book from a package of toilet paper, at any angle and under any lighting conditions.

There's an iPhone app that does something similar to this with CDs, DVDs, and video games. It's called Snaptell.

Snaptell:

When you see a book, CD, DVD, or game at a friend's house you want to look up and bookmark instantly, fire up SnapTell Explorer on your iPhone and take a photo of it. Similar to a bar code scanner (except you photograph the item cover, not its bar code), SnapTell automatically looks up your item and gives you links to Amazon, Barnes and Noble, Wikipedia, and straight-up search engines so you can compare prices and find out more about it.

It's pretty amazing.

The TED device would just be a more robust version of Snaptell (it would include more products than just CDs, DVDs, and video games).

Also, I don't think it's necessary for the device to be able to operate under 'any' lighting conditions or identify packages from 'any' angle. I presume you meant 'within reason', but I will still make the obvious comment that it would be sufficient for the device to operate under normal lighting conditions and require a more-or-less straight on view of the product package.

Certainly the better the device can operate with extreme viewing angles the more accurate and useful it will be. But I think most people would accept having to play with the angle for a few seconds for the device to identify it if it then provided them with the desired information.
posted by Davenhill at 12:52 AM on March 12, 2009 [1 favorite]

Snowcorn - sweet design.
posted by tellurian at 7:34 PM on March 25, 2009