Touch Me Baby One More Time

March 14, 2007 7:08 PM

Jeff Han, shows advances in his multitouch interfaces a year later. YT video 1. YT video 2.
posted by Blazecock Pileon (36 comments total)

Errant comma there; whoopsie.
posted by Blazecock Pileon at 7:12 PM on March 14, 2007

This is a year old not a year later.
posted by humannaire at 7:18 PM on March 14, 2007

That's fucking sweet.
posted by delmoi at 7:27 PM on March 14, 2007

This is awesome. Can't wait until it becomes more mainstream.
posted by Zephyrial at 7:30 PM on March 14, 2007

Want to see a new feature? Check this post out here.
posted by mathowie at 7:35 PM on March 14, 2007 [5 favorites]

Want to see a new feature? Check this post out here.

Is this the CSS icon feature I asked about a year back, or rendered on the fly?
posted by Blazecock Pileon at 7:37 PM on March 14, 2007

It's an on-the-fly thing (click on it!) to let video play inline without having to go to youtube. It'll be a user option in a day or two.
posted by mathowie at 7:44 PM on March 14, 2007

I believe Apple's going to claim a bunch of territory in this area, too. The iPhone has some sort of multitouch thing going on, Jobs made a multitouch mistakes during a presentations (ie. trying to use multitouch on a 30" display), and there's been some staking of claims ("we got patents").

It's about time. Adds a whole new dimension to the system.
posted by five fresh fish at 7:50 PM on March 14, 2007

It's an on-the-fly thing (click on it!) to let video play inline without having to go to youtube. It'll be a user option in a day or two.

Wicked! Sign me up...
posted by Blazecock Pileon at 7:55 PM on March 14, 2007

Way to hijack the thread, Matt. Did you get the idea from my inline pic demo from a while back?

This is a year old not a year later.

The second video is new, as far as I'm aware. I've been following Han's work with interest since the initial demo. Christ, I hope we get this stuff soon (but given the glacial pace of true innovation on the desktop (yeah, MS, I really wanted flippy 3-D windows when I ALT-TAB; you have completed me), I'm not going to hold my breath.)
posted by stavrosthewonderchicken at 8:02 PM on March 14, 2007 [1 favorite]

The photo light table still seems like the killer app.
posted by smackfu at 8:11 PM on March 14, 2007

That bit where he scatters the photos across the screen is pretty goddamn sweet.
posted by schoolgirl report at 8:34 PM on March 14, 2007

The videos? Awe-inspiring. Can't wait to get to the Future.

Inline YouTubing on MeFi? Awesome. The future is now.
posted by sappidus at 8:37 PM on March 14, 2007

Yeah, forget Multi-touch, I want the inline video...

Admittedly, I'm looking forward to seeing the touch tech in some new devices.
posted by DMan at 8:39 PM on March 14, 2007

Two cool things in one thread.
posted by nola at 8:59 PM on March 14, 2007

I'm writing my congressperson to nip this research in the bud — I'm afraid we develop this technology at our peril. And it's not sweat-stained visions of potential misdeeds by the CIA or Al Qaeda that keep me awake at night. Nay, friends, rouse yourselves: In the tuna-breathed, sweet-faced furbearers that share our living spaces, we mere decidigitals have a foe much closer to home.

Through appparently innocent redoubts such as cuteoverload.com, the seemingly humble, yet worrisomely ubiquitous Felis catus has demonstrably taken over the Web. Just imagine the havoc MeFite oflinkey's opposably-thumbed "pet" Melon — or the 26-toed Überkatze Extra linked in the same thread — would be able to wreak on a Hanpad. How do we know that they are not bankrolling Jeff Han's work through off-shore corporations and blind trusts even now?
posted by rob511 at 9:16 PM on March 14, 2007

Jobs made a multitouch mistakes during a presentations (ie. trying to use multitouch on a 30" display)

He did whats now? When?

YouTube think didn't seem to work on Safari+White theme here, Matt. Just got a blue box, then after a while a white square in the middle of it.
posted by bonaldi at 9:21 PM on March 14, 2007

I get sound from the inline videos, but no picture. Firefox on Vista.
posted by aerotive at 9:42 PM on March 14, 2007

And again as I said last time, this has been around for a while.

The MS Research version has been showing up at every tech fest for years now, and yet we only hear about Jeff Han and others on Metafilter and other blogging sites. It pisses me the hell off: the MS version is superior because it doesn't just do sensing of fingers, it does true interactivity- you don't have to touch the screen, and can use it as a scanner as well. Heck, you don't even need a screen: they've had prototypes that project the image on your desktop or wall, instead of a piece of polarized glass.

Here's the home page of Andy Wilson, the researcher at MS Research working on Touchlight- which you can see he's been playing with since 2004 or earlier!

Good video: here is a Channel 9/MSDN clip of Andy demonstrating an early prototype of the Playtable/Touchlight in 2004; he makes mention of the "you could scan a document on the same screen you interface with". Here he shows off a variation on that theme in this Robert Scoble clip, dispensing with the screen altogether and letting his hands in the air be the full interface. Granted, the "video interface" has been done, but the software that takes the images from two cameras and is able to essentially determine via stereovision when something is moving right by the screen is the brilliant part! And unlike capacitance models or other physical interfaces, it can readily be extended to accurately have real-world space be the interface; the screen and interfaces is the size and shape you want, not just the size and shape of the screen you buy.

Fuckers... I'm pissed because MS gets no credit, but the work Andy Wilson's done is fantastic: the Playtable was the talk of techfest several years ago, and it was all Jeff Han could dream of and more. And even then, they said you could build it for under $1000, since it was just a couple of decent webcams and a video projector; all the work was done in the software. If MS weren't so stodgy about making breakthroughs- always trying so hard to come up with how this would best be marketed- we'd already have these in market. MS could write the SDK and drivers, and let the masses come up with new applications. That's how Google does it, and it works: come up with something, and let other people drive it forward by inventing new ways of using it (see: google maps apps).
posted by hincandenza at 9:53 PM on March 14, 2007

I'm bad enough about obsessively cleaning fingerprints off my screens already, even BEFORE multitouch....
posted by mrbill at 10:02 PM on March 14, 2007

Oh- if you go to Andy's home page, about halfway down you can find downloadable videos of Touchlight from October of 2004- which again, that video shows off the "scan and manipulate" of actual objects/photographs, as you can see with the magazine cover at the 1:40 mark. And if that weren't bad-ass enough, right below it is a 2002 video of GWindows, which dispenses with the screen altogether- and that is very Minority Report already.

And again, it's all software- there's no special or custom hardware besides a pair of $30 webcams, you don't have to touch the screen itself or even have a screen, and it could be running on Windows right now. Alas... :(
posted by hincandenza at 10:11 PM on March 14, 2007

That's damn sexy.
posted by TheDonF at 12:59 AM on March 15, 2007

hincandenza,
Thank you so much for posting those links. I'm astounded. Wilson is brilliant...and it's amazing that this has been around for at least, what 7 years? And no one outside of a small group of tech experts has even heard about it? It's insane that MS hasn't gotten this out, it would revolutionize the computer. If they could get some decent designers they could completely blow away Apple's "wow" cred...

I think you may be being a bit unfair to Han and his team, though. Han's brilliance is more in the user interface than in the hardware of the touchscreen, as he himself says. The Wilson videos show an amazingly innovative new way of interacting with a computer, but pretty much the same interface software-wise. Han seems to have developed new ways of visualizing and interfacing with data. Perhaps if Han and his company could work with Wilson's developments we could achieve something that really would be an evolutionary jump in computer interface.
posted by Sangermaine at 1:00 AM on March 15, 2007

So the MS version requires you to make the internationally recognised symbol for a wanker over your keyboard? Or a non-intuitive live-long-and-prosper type gesture for other features?
disclosure: only the YouTube link worked for me

I know which I'd rather have, and which is more impressive. I'll take Han's applicious-looking and intuitive one. And to everyone that says it'll be too expensive, sure, for now.
posted by imperium at 3:21 AM on March 15, 2007

Oh- if you go to Andy's home page, about halfway down you can find downloadable videos of Touchlight from October of 2004- which again, that video shows off the "scan and manipulate" of actual objects/photographs, as you can see with the magazine cover at the 1:40 mark. And if that weren't bad-ass enough, right below it is a 2002 video of GWindows, which dispenses with the screen altogether- and that is very Minority Report already.

Major downsides:

• Three cameras surrounding the user at all times
• You have to speak at the computer to get it to do things
• There's no force feedback to add to the intuition

You're shaking your fist at the screen, and have no feeling for where the computer thinks your fist happens to be.

His research is interesting, and I can see applications for ASL or other sign language work, but Han's interface is miles ahead, is inexpensive, and gestures can be employed on a portable device (e.g. iPhone) that does not require a room filled with expensive digitizing equipment.
posted by Blazecock Pileon at 6:15 AM on March 15, 2007

How does iPhone use multi-touch? It looks like just a normal touch screen interface.
posted by smackfu at 6:37 AM on March 15, 2007

How does iPhone use multi-touch? It looks like just a normal touch screen interface.

iPhone multitouch
posted by Blazecock Pileon at 6:47 AM on March 15, 2007

So it seems like Apple only uses the two finger zoom thing.
posted by smackfu at 6:52 AM on March 15, 2007

Their portables also use two-finger trackpad scrolling, so multitouch has been a part of their interface plan for a while.
posted by Blazecock Pileon at 6:55 AM on March 15, 2007

It's a good thing that there are (at least) two mainstream competing approaches to haptic interfaces. I doubt either Han or Wilson would say that one or the other is a waste of time.

Lovely post, Blazecock.
posted by owhydididoit at 7:03 AM on March 15, 2007

I forgot - Wow, Matt.
posted by owhydididoit at 7:06 AM on March 15, 2007

Great post Blazecock!
posted by Mister_A at 7:29 AM on March 15, 2007

imperium: So the MS version requires you to make the internationally recognised symbol for a wanker over your keyboard? Or a non-intuitive live-long-and-prosper type gesture for other features?

disclosure: only the YouTube link worked for me

Uh... well then perhaps you should have watched the other videos before commenting. The hand gesture one with the "wanker" symbol was an old prototype, I think from 2002; the ones from even a few years ago are more like Han's, with the interactive screen.

If you search Youtube for "Touchlight", you'll find the 2004 touchlight demo from Andy Wilson, as well as the Eon Touchlight (apparently, MS finally licensed the technology to a 3rd party in order for them to get it to market in some form).

Blazecock Pileon: Major downsides:

• Three cameras surrounding the user at all times
• You have to speak at the computer to get it to do things
• There's no force feedback to add to the intuition

You're shaking your fist at the screen, and have no feeling for where the computer thinks your fist happens to be.

His research is interesting, and I can see applications for ASL or other sign language work, but Han's interface is miles ahead, is inexpensive, and gestures can be employed on a portable device (e.g. iPhone) that does not require a room filled with expensive digitizing equipment.

*sigh* Again, this is the frustrating part: people don't even watch the videos or do a valid comparison, they just knee-jerk a "Well MS's version must suck!". Han's is not "miles ahead" by any means, but yes it is more suitable to things like phones and small devices. The Touchlight is not 3 cameras surrounding the user, it's 2-3 cameras and a rear projection system, all behind the semi-transparent glass. Put it in a casing, and it's basically a regular TV/monitor that is also Minority-Report interactive.


I agree with owhydididoit, that having multiple groups investigating this form of interface is a good thing. What I resent is that people are dismissing Andy Wilson's work, which- if you bother to look at the more recent TouchLight videos- do everything Han's does and more, for quite cheap, and could and was easily done in a simple flat-screen format.

I didn't mean to dismiss Jeff Han's work, but it's extremely frustrating knowing that an internal SDK is floating around Microsoft, that the hardware (about $100 worth of webcams, some semi-transparent glass, and a used video projector off Ebay) could be whipped up fairly easily for cheap to make a vertical or horizontal screen/table, and you'd be doing everything Han is doing right now. Han's work, and others, seem to focus on making a new monitor that has multitouch capabilities. Wilson's work dispenses with touch altogether- whether in open-air gestures like GWindows, or in the TouchLight which is similar to Han's interface- to allow a much broader use of applications.


For example, when the Playtable (a horizontal touchlight, suitable as table/desk or monitor) was being demo'ed a few years ago at Techfest, the applications were already much cooler:

• Ability to draw or paint on the surface using normal brushes and a virtual palette, with surprising detail and accuracy
• Ability to have interactive objects placed on the tabletop, which adjust what's displayed and where. An example was a tilted plastic strip akin to a Scrabble tile holder, which the player would orient as normal but the table was smart enough to display the letters on the side that only they would see; similar for trivial pursuit question/answers
• Using simple glass squares with a tiny bit of infrared-visible ink to tag them, toss them on the table top. Because it's using complex visual recognition, it was able to in real-time display a video image to the various squares- but broken up to the appropriate square. The 12 or so glass squares became a motion-picture jigsaw puzzle, where that section of the video associated with that piece was displayed under the glass piece, which could be rotated, flipped, moved around, etc, and its section of the video file stayed under it and oriented as the piece
• In addition to the usual two-finger resizing of video, you could make any object an interactive dial: using a small film canister, with the little printed tag on the bottom, you could drag and drop pictures 'to" the canister, so that whenever it was placed on the tabletop the system would recognize it as a unique object, and call up the files/pictures associated with it. Plus, you could rotate the canister like a dial, and the images would "carousel".

These are just some of the simple applications, hard to describe, but when you see them are far far more than just "I can touch the screen with two hands!"
posted by hincandenza at 12:37 PM on March 15, 2007

Again, this is the frustrating part: people don't even watch the videos

I did watch the videos, but perhaps we're not referencing the same ones. Which can also be frustrating.
posted by Blazecock Pileon at 1:29 PM on March 15, 2007

Jobs made a multitouch mistakes during a presentations (ie. trying to use multitouch on a 30" display)

Gack. Must have had a brain fart there.

Jobs was doing a presentation. There was a 30" display. He walked up to it and tried to use the UI using multitouch. Which indicates to me that Apple has it already up and working, to the point that Job's is so used to using it that it's become natural.

BP's comment about trackpads surprised me -- turns out I've been using multitouch for a couple years! I'm so used to two-fingered trackpad scrolling that it frustrates heck out of me to use not-a-Mac laptops. It's a completely natural, nearly-intuitive action.
posted by five fresh fish at 6:47 PM on March 15, 2007

I'm imagining Starcraft on that thing and I think I just got a stiffy.
posted by LordSludge at 8:05 PM on March 15, 2007