Pictures Under Glass

November 9, 2011 1:53 PM

A Brief Rant on the Future of Interaction Design - "The next time you make a sandwich, pay attention to your hands. Seriously! Notice the myriad little tricks your fingers have for manipulating the ingredients and the utensils and all the other objects involved in this enterprise. Then compare your experience to sliding around Pictures Under Glass. Are we really going to accept an Interface Of The Future that is less expressive than a sandwich?"
posted by Defenestrator (96 comments total)

Heh. I'd have thought that too, but it turns out a lot of touchscreen stuff works fine without haptics. Not that people aren't working on that.

Anyway, everyone knows the future of UI is frantically waving your arms in thin air.
posted by Artw at 1:59 PM on November 9, 2011 [2 favorites]

The other problem not mentioned is that your fingers (or at least mine) aren't transparent.
posted by Keith Talent at 2:01 PM on November 9, 2011 [2 favorites]

He's right, but notice that Kay needed that 16x16 grid to extrapolate into the iPad. Where, I wonder are the seeds of tomorrow? We had force-feedback gloves in the VR heyday; they were dead-ends; the uber-stylus.

A place to start looking might be tools for musicians, although, despite a fantastic history of physical UI design, a lot of their digital interfaces leave something to be desired. Where are the concept instruments?
posted by fightorflight at 2:01 PM on November 9, 2011

The other problem not mentioned is that your fingers (or at least mine) aren't transparent.

They were working on that but then Kevin Bacon fucked it up big time.
posted by shakespeherian at 2:02 PM on November 9, 2011 [4 favorites]

The trick to make more use of hands is for a device to somehow present some kind of force feedback from interaction, or a dynamic three-dimensional interface, all while keeping the shrinking form-factors that people expect. This will require technology we don't have, yet.
posted by Blazecock Pileon at 2:03 PM on November 9, 2011 [1 favorite]

BINGO BINGO BINGO!

Touchscreens aren't even that good an interface to *computers*, let alone anything else.
posted by DU at 2:04 PM on November 9, 2011 [4 favorites]

It's a pretty good rant. What would his perfect input device look like? Would it be infrared tagged geometric primitives with motion sensors that we could use like a 3D mouse? Rotate a cube for simple angle control. Roll a sphere along the desk for incremental plus and minus changes?
posted by codacorolla at 2:04 PM on November 9, 2011 [1 favorite]

I was considering posting this myself, but was afraid it'd be deleted with a message to take it to the thread about the video it's responding to. DON'T DO IT, MODS.

Still, it does a better job addressing issues about future-UI than any other response, even if it's flawed.

And hey, I dig styluses, but then, I've spent most of my adult life with a pen in my shirt pocket, so I'm not typical.
posted by oneswellfoop at 2:06 PM on November 9, 2011

So then this new UI of the future... it will vibrate?

... or just be ribbed?
posted by Kabanos at 2:06 PM on November 9, 2011 [2 favorites]

It will simulate being ribbed by vibrating, silly!
posted by Artw at 2:07 PM on November 9, 2011

Touchscreens aren't even that good an interface to *computers*

This, of course, assumes computers in their current state are How Things Should Be Forever. I'm not arguing in favor of touchscreens as an interface, just asking why I should care about how computers want me to do things. That kind of thinking is what keeps us from making progress.
posted by yerfatma at 2:08 PM on November 9, 2011

I believe that hands are our future

Nike won't be pleased.
posted by omnikron at 2:08 PM on November 9, 2011 [1 favorite]

Touchscreens aren't even that good an interface to *computers*

When I finally got a smartphone I made damn sure to get one with a physical keyboard because I hate talking on the phone but I text like a goddam preteen and the thought of trying to do it on a touchscreen make me want never to do it again.
posted by shakespeherian at 2:09 PM on November 9, 2011 [5 favorites]

I mean, I'd love to have an all-purpose, morphing-on-the-fly, physical interface. I think we all would. Something that could BECOME a guitar instead of just showing you a guitar, wow, yeah, you'd rule the world. But mass can't be pulled out of thin air.
posted by Brainy at 2:10 PM on November 9, 2011

frantically waving your arms

Figured that would link to Minority Report.
posted by Trurl at 2:10 PM on November 9, 2011 [2 favorites]

If "expressive as a sandwich" means having (delicious) goop ooze out onto your fingers while layers of greens and vegetables slide past one another as the breads strains beyond the breaking point to contain it all, then I really do not want a UI that's as expressive as a sandwich.
posted by treepour at 2:12 PM on November 9, 2011 [5 favorites]

When I finally got a smartphone I made damn sure to get one with a physical keyboard because I hate talking on the phone but I text like a goddam preteen and the thought of trying to do it on a touchscreen make me want never to do it again.

Thiiiiisss. I fucking hate touchscreens. I hate using the new ATMs since I pretty much have to stab the screen just to get it to work. If I had a touchscreen phone I would've hurled it against the wall in the first 24 hours of use.
posted by Anima Mundi at 2:12 PM on November 9, 2011 [5 favorites]

I love switches, knobs, dials and buttons. My touchscreen phone irritates me, but it looks cooler and that's why the screens will win.
posted by dickasso at 2:12 PM on November 9, 2011 [3 favorites]

Cheaper, ultimately, too.
posted by Artw at 2:14 PM on November 9, 2011

Figured that would link to Minority Report.

Heh. I have heard so many UX designers reference Minority Report... it is their ultimate spaff movie.
posted by Artw at 2:15 PM on November 9, 2011 [3 favorites]

Eye tracking could be one thing to watch out for. Cameras are getting pretty cheap and even a rudimentary idea of where the user is looking would allow menus where you'd pick from, say, nine options just by looking and confirming your choice with some suitably unambiguous method. Even four choices would work for some uses. Sure, there are lot of problems to solve, people wear glasses and so on, but I don't think he idea is totally unworkable. (Maybe there is some research already?)
posted by tykky at 2:18 PM on November 9, 2011 [1 favorite]

I love touchscreens. They don't break, you can do multitouch, etc. With a well designed interface (like touchosc) I can control things more precisely than I can with some analog to digital knobs (continuous rotary encoders suck for a lot of things)...

Why compare touchscreens to other stuff. It's not like having a touchscreen removes sandwiches from the world. They're exceedingly convenient and durable. If someone can make transparent semiconductors with controllable volume change (by phase or piezoelectricity) then... ok, that would be neat, but not game changing for me.

I thought I would always want a tactile keyboard until it went away and was replaced with something nicer. Same with buckling spring for chiclet keyboards. The membrane keyboards in between sucked, as did the screen/interface on an iPAQ in 2000, but an iPad in 2010... now you're talking.
posted by nutate at 2:20 PM on November 9, 2011 [2 favorites]

Goop on a sandwich is just wrong. Mayo, I'm looking at you.

Yeah touchscreens are the digital watches of our era. They're cool, but they're just a step. Maybe a lot of you guys have endlessly clean hands all the time, but I use computers in my shop and if I had a smartphone it would be used when my fingers were greasy. Or, in the winter, encased in gloves.
posted by maxwelton at 2:21 PM on November 9, 2011 [2 favorites]

My touchscreen phone irritates me, but it looks cooler

Back in the '80s , sleek black panels with tiny push-buttons and little red LEDs looked cooler than anything else. The vintage audio gear from that era is just terrible in terms of UI. I used to have a Yamaha TX81Z rack synth, which piped a reasonably useful synth engine through a front panel consisting of ten buttons. The wave surged, but eventually broke, and modern audio gear once again has knobs you can turn.

Touchscreens are the same thing. The wave is surging, and people are trying to redo everything in sight using a touchscreen, but five or ten years from now it will all seem a bit of a cliché and cutting edge design will have moved on. Touchscreens will be with us forever but they are nothing like the last word in interface design.

(No, I don't have an iphone, or an ipad. Gee, how could you tell? I mostly use my phone for texting, and trying to type on a touchscreen makes no sense.)
posted by Mars Saxman at 2:24 PM on November 9, 2011 [1 favorite]

I am in complete awe of Bret Victor. His web page is a complete treasure trove, including things like a fun, beautiful, and clear explanations of control systems. I wish someone would give him a lifetime fellowship for him to explore his ideas.
posted by Llama-Lime at 2:24 PM on November 9, 2011 [8 favorites]

I love the thought process in this article, and I love the argument. I, too, hate touch screens, and I long for the tactile pleasure of feeling the things I interact with.

However, I think that the problem with this line of thinking is that it seems to think that the human body will still be relevant in the future. Who needs their fingers in the matrix?
posted by pazazygeek at 2:26 PM on November 9, 2011

Goop on a sandwich is just wrong

You had me at "Goop on a sandwich".
posted by ericost at 2:27 PM on November 9, 2011

After a little while you get used to typing on a touchscreen. While you'll probably never be nearly as fast or accurate as you are on, say, a fullsized PC keyboard, chances are that you will eventually get pretty good at it.

Those tiny smartphone keyboards are usually pretty awful anyway - key choice and placement can be incredibly frustrating, build quality/durability is often extremely poor, and they add bulk and additional moving parts to things that are at their best when small and simple.

This is like the transition from the little ThinkPad-style eraser pointer to a touchpads - it might seem awkward at first but just invest a little time and you'll come to prefer it.
posted by The Lamplighter at 2:29 PM on November 9, 2011 [1 favorite]

Where are the concept instruments?

There's a musical instrument that's a table where you slide things around it. I can't remember what it is called though. It's probably the most cutting edge.

On the touchscreen end, the Korg Kaoss Pads use a straight up touchscreen. The cheaper versions offer little in the way of guide to where you are touching, with the more expensive ones lighting up where you touch. As synths, they can be hard to play, but are awesome for drums, or changing two effect parameters at once.
posted by drezdn at 2:30 PM on November 9, 2011

When I finally got a smartphone I made damn sure to get one with a physical keyboard because I hate talking on the phone but I text like a goddam preteen and the thought of trying to do it on a touchscreen make me want never to do it again.

Thiiiiisss. I fucking hate touchscreens. I hate using the new ATMs since I pretty much have to stab the screen just to get it to work. If I had a touchscreen phone I would've hurled it against the wall in the first 24 hours of use.

This is exactly what I thought from the moment I got my first phone with a full physical keyboard until ten minutes after I got an iPhone. I was faster almost immediately.
posted by villanelles at dawn at 2:31 PM on November 9, 2011 [9 favorites]

Yeah, I was firmly in the "must have physical keyboard" camp, and even kept one foot in that pool with my Samsung Galaxy S. But once I got my iPhone 4, I was amazed at how fast and accurate the keyboard was. Can't imagine going back. And my iPad2 still seems like voodoo magic, that it works well enough that I actually use Pages on it.
posted by xedrik at 2:35 PM on November 9, 2011 [4 favorites]

This will require technology we don't have, yet.

Wait, you don't have a holodeck yet? I thought everybody had one.

This is like the transition from the little ThinkPad-style eraser pointer to a touchpads

I used touchpads for years before I was able to get a Thinkpad. You'll have to pry my TrackPoint from my cold dead home-row-staying fingers.
posted by kmz at 2:42 PM on November 9, 2011 [2 favorites]

It would be great to have an interface that made more use of hands. Then again, as the author said years ago, it would be completely unnecessary in many cases.
posted by ignignokt at 2:42 PM on November 9, 2011

drezdn: the original table is called reactable and is a tactile interface on top of a touchscreen. It is now also available as an iPad app.

The korg kaossilator and other kaoss pad type apps (including basic x/y over midi) are available on iPad as well (android too..?)
posted by nutate at 2:43 PM on November 9, 2011

The Del Taco I went to yesterday had a new soda dispenser with one spot for your cup and a touchscreen, from which you could select any of about a dozen or so different beverages, as well as different flavoringss.

I figured out how to use it straight away, but when I went back for a refill later, another gentleman was struggling to figure it out. Meanwhile, a line grew behind him.
posted by Celsius1414 at 2:44 PM on November 9, 2011 [1 favorite]

I can sum up my reply to the article very easily: sounds good, but easier said than done. Trying to be tactile and generic (i.e. for programmable devices) sounds incredibly tricky to me. I note that he doesn't actually propose anything in the vein of what he discusses. That should tell him something.
posted by Edgewise at 2:47 PM on November 9, 2011 [2 favorites]

Google informs me the machine is a Coca-Cola Freestyle machine:

http://www.deltaco.com/index.php?page=12&item_id=83

http://en.wikipedia.org/wiki/Coca-Cola_Freestyle
posted by Celsius1414 at 2:47 PM on November 9, 2011

I've used those Coca-Cola Freestyle machines. The only thing confusing about them is that you have to press a physical button to dispense the soft drink. The first time I used one, I didn't even see the big round button because it just felt like you were supposed to use the screen for everything.
posted by treepour at 2:50 PM on November 9, 2011 [2 favorites]

Those are neat ideas, but I'm not sure of where they're supposed to lead eventually. Real-life devices that require lots of nerves, bones and muscles to be operated also require also a lot of training - sometime years - before one manages to use them properly and safely. What sort of devices would benefit from a physical interaction process more complex than the current "simple" ones (keyboard, mouse, touch etc.)?
posted by elgilito at 2:53 PM on November 9, 2011

You know, the *only* thing I've ever found to not work well with modern interfaces is manipulating 3d objects. Pretty much everything else works quite nicely with a mouse, trackball, joystick, keyboard or graphics tablet.

Still, find me a better way to view and manipulate crystal structures and I will be *very* happy. Touching a flat screen doesn't seem like it would be any better for this, possibly a bit as you can touch two places at once, but you lose the ability to use modifier keyes.

Touch keyboards stick me as a dead end: They have no give. Can you imagine what it would feel like to type on a ST:TNG keyboard for 6 hours? Yikes.

The fact we have a perfect laptop interface device (touchpoint) and everyone still is using touchpads shows me that whatever interface we settle on will probably not be the best one, but the most cool-looking to a non-power user.

Related: I hear great things about trackballs, but I don't want to sink $60 on one if it turns out I don't like it. Anyone have a suggestion?
posted by Canageek at 2:53 PM on November 9, 2011

If you think the touchpoint is the perfect HID for laptops, use a Mac trackpad sometime.
posted by The Lamplighter at 2:55 PM on November 9, 2011

There is an art to the business of making sandwiches which it is given to few ever to find the time to explore in depth. It is a simple task, but the opportunities for satisfaction are many and profound: choosing the right bread, for instance. The Sandwich Maker had spent many months in daily consultation and experiment with Grarp the Baker and eventually they had created a loaf of exactly the consistency that was dense enough to slice thinly and neatly, while still being light, moist and having the best of that fine nutty flavor which best enhanced the savor of roast Perfectly Normal Beast flesh.

There was also the geometry of the slice to be refined: the precise relationships between the width and height of the slice and also its  thickness which would give the proper sense of bulk and weight to the finished sandwich -- here again, lightness was a virtue, but so too were firmness, generosity and that promise of succulence and savor that is the hallmark of a truly intense sandwich experience.

The proper tools, of course, were crucial, and many were the days that the Sandwich Maker, when not engaged with the Baker at his  oven, would spend with Strinder the Tool Maker, weighing and balancing knives, taking them to the forge and back again. Suppleness,  strength, keenness of edge, length and balance were all enthusiastically debated, theories put forward, tested, refined, and many was the  evening when the Sandwich Maker and the Tool Maker could be seen silhouetted against the light of the setting sun and the Tool Maker's forge making slow sweeping movements through the air, trying one knife after another, comparing the weight of this one with the balance of another, the suppleness of a third and the handle binding of a fourth.

Three knives altogether were required. First, there was the knife for the slicing of the bread: a firm, authoritative blade, which imposed a  clear and defining will on a loaf. Then there was the butter-spreading  knife, which was a whippy little number but still with a firm backbone to it. Early versions had been a little too whippy, but now the combination of flexibility with a core of strength was exactly right to achieve the maximum smoothness and grace of spread.

The chief among the knives, of course, was the carving knife. This was the knife that would not merely impose its will on the medium  through which it moved, as did the bread knife. It must work with it, be guided by the grain of the meat, to achieve slices of the most  exquisite consistency and translucency, that would slide away in filmy folds from the main hunk of meat. The Sandwich Maker would then flip each sheet with a smooth flick of the wrist onto the beautifully proportioned lower bread slice, trim it with four deft strokes and then at last perform the magic that the children of the village so longed to gather round and watch with rapt attention and wonder. With just four more dexterous flips of the knife he would assemble the trimmings into a perfectly fitting jigsaw of pieces on top of the primary  slice. For every sandwich the size and shape of the trimmings were different, but the Sandwich Maker would always effortlessly and without hesitation assemble them into a pattern which fitted perfectly. A second layer of meat and a second layer of trimmings, and the main act of creation would now be accomplished.

The Sandwich Maker would pass what he had made to his assistant, who would then add a few slices of new cumber and fladish and a touch of splagberry sauce, and then apply the topmost layer of bread and cut the sandwich with a fourth and altogether plainer knife. It was not that these were not also skillful operations, but they were lesser skills to be performed by a dedicated apprentice who would one day, when the Sandwich Maker finally laid down his tools, take over from him. It was an exalted position and that apprentice, Drimple, was the envy of his fellows. There were those in the village who were happy chopping wood, those who were content carrying water, but to be the Sandwich Maker was very heaven.

And so the Sandwich Maker sang as he worked.
posted by zamboni at 2:56 PM on November 9, 2011 [12 favorites]

I've used iPhones and Android devices, and I still love having a physical keyboard. I got a tablet, but prefer the ThinkPad x120e I got because of everything the linked essay says. I absolutely loathe Apple's clickpads, and any laptop that tries to emulate them.

Glassy, borderless, contextless touchscreens are kinda neat, but they're not the end-all tool that everyone seems to think they are. I agree with everything in the linked article.
posted by fnerg at 2:59 PM on November 9, 2011 [1 favorite]

Also, as I've ranted about many time before, two-finger clicks fail to register on every single MacBook I've tried about 10% of the time, which is FUCKING ANNOYING!!! I love my Thinkpad's right mouse button! The trend towards fewer buttons, and having the computer try to guess what you're trying to do is idiotic.
posted by fnerg at 3:03 PM on November 9, 2011

Really, it's all about the logitech cyberman and cyberman2 controllers.
posted by nutate at 3:07 PM on November 9, 2011

No, it's all about the OpenOffice Mouse.
posted by entropicamericana at 3:15 PM on November 9, 2011 [3 favorites]

I always loved the stylus on my various Palm Pilots and Treo phone. It acted pretty much like you'd expect a pen would and was way more accurate than my finger on my Droid phone. But a stylus is old-fashioned nowadays and capacitive touchscreens are the norm so there's no going back, unfortunately.
posted by tommasz at 3:15 PM on November 9, 2011 [2 favorites]

True, it required a stylus, but does anyone remember Graffiti?

I got pretty good at it, and I'm sure the stylus was only required because the input area was so small - are there any shorthand handwriting recognition apps out there for tablets? With the larger area, it could easily be sensitive enough to distinguish finger-tip sized input.
posted by porpoise at 3:18 PM on November 9, 2011 [1 favorite]

No, it's all about the OpenOffice Mouse.

Let's take it all the way.
posted by Artw at 3:19 PM on November 9, 2011

The guy who choreographed the Minority Report sequence seems to have done the thing for real. Looks more versatile than Microsoft's.
posted by CaptainCaseous at 3:20 PM on November 9, 2011 [1 favorite]

Seems hating touchscreens is the new "I don't own a TV".
posted by zardoz at 3:27 PM on November 9, 2011 [11 favorites]

the original table is called reactable and is a tactile interface on top of a touchscreen

FWIW, if you're interested in seeing one and live near NH, MeMail me as we're apparently going to be hosting the US debut.
posted by yerfatma at 3:35 PM on November 9, 2011

While I follow his reasoning, I completely disagree. Interfacing with a computer inherently depends on unnatural motion. I don't want to delete half of my document and turn the rest into green underlined Comic Sans if I get a sudden urge to scratch my elbow. I don't want my voice-operated car to brake hard and take a left-turn into a tree if I sneeze. If my cat or child jumps into my lap, I want to be able to exclude its actions from the input. And so on.

I want to interact with the computer deliberately, and its interface must be tolerant of and intelligently discount undeliberate input.

A useful analogy for this is the old-style D&D magic system, in which spells (and magic item activations) were classified as having verbal, somatic, and/or material components. To activate the spell, the caster would have to speak a set of words, make a set of gestures, and/or have in their possession an item (which is assumed to be used during the somatic gestures if applicable). Cutting-edge computer interfacing is similar. If the computer is monitoring sound in the car for spoken commands, the user may be required to physically possess an item (eg the "key"), or be pressing a button on the wheel, or precede the command with an introductory phrase, in order for the computer to obey. "Traditional" computer interfaces require somatic input onto a material device - the keyboard and mouse.

As anyone who played that kind of game knows, if the activation word is spoken accidentally while the item is being held, and the GM is in the mood for it, hilarity can ensue; and if the author of the linked article had his way, this would be the world we lived in.

The converse example is the disobedient interface - despite saying the command word, and holding the item, it does not activate. A real-life example available to us all is voice-recognition systems used by telephone companies and banks:
Did you say "Accounts Payable?"
Yep.
I'm sorry. I did not get that. Please say "YES" or "NO".
YES!
I'm sorry. I did not get that. Please say "YES" or "NO".
FUCK YOU! YESSSSS!!!!! YES YES YES YES YES!!!!
I'm sorry. I did not get that ...
God help the Scots.

I'm not familiar with Siri on the Apple iPhone 4S however I presume it requires some action to be taken to make it realize when it is being addressed, and when not. That is the salient point that the author of the article seems to be missing.
posted by aeschenkarnos at 3:37 PM on November 9, 2011 [5 favorites]

I agree with the article to the extent that it disagrees with people who think that the current touchscreen interface is the Final Evolution of Interfaces. But isn't that a bit of a straw man? I would make the argument that the current touchscreens are an excellent response to a large number of factors, including technology, the cost of materials, what idioms current users can expect to understand intuitively, and very importantly, the form factor of the current generation of mobile devices. As these factors change, more ideal interfaces will present themselves, if not demand to be created.

For instance, I wouldn't be surprised if we some kind of lightweight AR headgear becomes feasible within the next decade or so. In that case, it doesn't even make sense to talk about a touchscreen. We would need a completely new input device, probably something more like a wiimote (though hopefully not too much like a wiimote). Also, you'd want some kind of 3D cursor so you could designate real-world objects, so three elements - input, output and idiom - would change dramatically.
posted by Edgewise at 3:48 PM on November 9, 2011 [1 favorite]

I think it's implied that the interface has to be "addressable".Like, my jamjars don't suddenly open when I scratch my elbow, why should my computer be different? A well-designed interface has to include the intentionality.
posted by fightorflight at 3:48 PM on November 9, 2011 [2 favorites]

And while we're at it, what's with Lenovo putting that evil touchpad on the bottom of the keyboard, leading to people having to reach over to use the far superior Trackpoint, giving themselves carpel tunnel in the process?

And while they're at it, OH GOOD GOD HOW ARE YOU SUPPOSED TO TYPE ON A CAPACITIVE SCREEN? AT LEAST PROVIDE A STYLUS YOU SADISTIC "USER EXPERIENCE" ... people...
posted by Yowser at 3:50 PM on November 9, 2011

The actual dial, mouse or dohickey is vastly less critical than other element such as sub-millisecond latency. Consistency and reasonableness and familiarity are also vital. Waving arms will be fine, if the human organsm that is using it becomes familiar that a zigzag gesture diase what is viscerally expected with the "muscle memory". Mouse has become familiar, and that translated pretty easily to a nub or pad. All three actually make a lot of sense as a metaphor for a pencil on paper, mapped to a rectangular screen; do artists need a twirly thing when a pencil works very effectively?

Now there are an awful lot of UI abominations, pretty much every 'clever' web page with a clever icon. It's not cool, it's obscure, has no feedback, poor delineation, annoying design, good for a 13 year old that likes sparkles.

Good UI design should be invisible, as in it just works, gets out of the way, does the job and is actually a bit boring.
posted by sammyo at 3:56 PM on November 9, 2011 [2 favorites]

Here;s something interesting... Denny's, of all people, have gone and made their site one of those multi-directional scrolling panorama things. Might be more clever than actually useful.
posted by Artw at 4:00 PM on November 9, 2011 [1 favorite]

I personally expected to hate the touch screen droidX, but went with it for screen real estate. Don't love bit it's workable.

What I want is a great left handed chording keyboard. I want to hold the phone in my right hand and do all the typing with my left (or reverse). I went looking for one and it seems it's still a research topic. You'd think a hand sized molded ergonomic keyboard would be available by now, but they are all klunky ad hoc boxes with funky standard buttons from an electronics supply store or worse.
posted by sammyo at 4:02 PM on November 9, 2011 [1 favorite]

No, it's all about the OpenOffice Mouse.

what
posted by Blazecock Pileon at 4:04 PM on November 9, 2011

shakespeherian: "The other problem not mentioned is that your fingers (or at least mine) aren't transparent.

They were working on that but then Kevin Bacon fucked it up big time."

Did not. He did the thing any guy would do if invisible - Go creeper on the hot neighbor lady. Then screw it up.
posted by Samizdata at 4:11 PM on November 9, 2011

True, it required a stylus, but does anyone remember Graffiti?

I got pretty good at it, and I'm sure the stylus was only required because the input area was so small - are there any shorthand handwriting recognition apps out there for tablets? With the larger area, it could easily be sensitive enough to distinguish finger-tip sized input.

Heh, was looking into this the other day, as I too was a Palm addict back in the day. It's available as a free software keyboard on android.

Surprising how quickly it comes back to you. You do use a fingertip, but in a small area like the old days. The stylus was needed because the screens back then were single-touch resistive, whereas modern screens are multi-touch capacitive. The former worked on pressure, the latter work on capacitance across your finger tip, which is why you don't get stylus tips for (most) modern touch screens; you have to use a rubber tip that's roughly fingertip sized for it to register any touch - and also why you can't use an iphone while wearing gloves. But also makes it much easier to use; it works out where the middle of your finger is, and applies a bit of helpful 'this is what you probably meant to touch' and frankly, can be scarily accurate.

Still, writing in one letter a time is significantly slower than what I can crank out on my android in landscape mode with double-thumb operated and a good prediction/autocorrect/type ahead keyboard (such as swiftkey X), or even with something like swype. Sometimes, inventions of the past probably deserve to stay there. That goes for hardware keyboards too - a good software keyboard on a decent capacitive multitouch screen can equal or beat a phone hardware keyboard for speed and precision - especially once you include rarer symbols, and I honestly never wanted a keyboard-less phone. I reluctantly made the sacrifice to get something that didn't weigh a ton in my pocket, and was very surprised that my typing speed on it was just fine.

Currently using a filco cherry brown mechanical keyboard though - you can keep your fancy scissor-switch thin ones, I like a keyboard that has some spring to it. Though I have married it to a magic trackpad. While it might not be suitable for making a sarnie, I'm also not going to accidentally slice half a finger off resizing a window.
posted by ArkhanJG at 4:13 PM on November 9, 2011

Well I read the blog rant, love it to some degree but have some issues. Let's get back to an artists pad, yes there are many types of brush, pencil, stylus but they are all sticks that are held that touch the rectangle. I don't see any serious call for a brush that is held differently or manipulated with the shoulder (although some artist probably has one).

As long as our display is a (moving, glowing) rectangle, the mouse is not all that far from some ultimate interactive device. When the display is truly 3D, not a projection within that rectangle but all around the user, "look up for info", "look over your shoulder for a different kind of feedback", then the radical gesture interfaces will be needed.
posted by sammyo at 4:19 PM on November 9, 2011

I've had the good fortune to see CAVE demos, and I think the Wii and Kinect suggest that they're just a few years away from broader use. While it has some quirks, I thought the ability to manipulate Virtual 3-D objects with a wand solved many of the problems of trying to move those objects via a 2-D screen.

It seems to me that there's no lack of specialized interfaces for computerized systems. Even though most aircraft and some ground vehicles are fly-by-wire, I'm not certain many are fly-by-touchscreen. The music world is dominated by MIDI instruments and adapters. High-end digital photography is dominated by hybrid systems. And of course, accommodating people with motor disabilities has led to the development of many alternate forms of electronic input.

But most of the tasks demoed in the video seemed to involve selecting and interpreting items from complex menus, where a touchscreen is likely one of the better interface decisions.
posted by CBrachyrhynchos at 4:20 PM on November 9, 2011

The Kinect Killer will look just like a BopIt.
posted by LogicalDash at 4:24 PM on November 9, 2011

One more detail, at a siggraph or sigchi presentation I saw a very simple interface, just a dowel on a stand, about the size of a rolling pin. Two of them, across a table. roll one and the other was in perfect sync, if someone at the other added some friction, you could sense it in yours. It was the most incredible spooky visceral sensation. Get that hooked to a remote system that needed precise control and it would be as good as being there.
posted by sammyo at 4:26 PM on November 9, 2011

And we all know how the dynamic remote interactive 'bopit' will used and become really really popular...
posted by sammyo at 4:28 PM on November 9, 2011

Can we get back to the sandwich, please? Is there mutton? Is it nice and lean? Are the tomatoes ripe? What's our perkiness rating?
posted by obiwanwasabi at 4:28 PM on November 9, 2011

I'm a little confused at the direction he wants to head. Physical representaions of "things" we manipulate as if they were the things they represent? Doesn't that get you into the dilemma of a 1:1 scale map: perfect representation but not functionally different enough from what it represents to be of use?
posted by sourwookie at 5:11 PM on November 9, 2011 [3 favorites]

I was pretty convinced by his rant, but then I followed the link to his webpage and OMG, his website interface is ironically annoying.

I spent a while staring at it trying to figure out how to get "in" to the site, before realising there was a heap of stuff down lower to scroll down to. You can't right-click to open anything in tabs, but have to open in the same window and then return. The dark shading on either side of the page makes things hard to read. The whole page doesn't fit inside my browser window, so I have to scroll right to get to things, and the animation as all the icons for each project "slide" back into place when I return to the list is slow and irritating.

I get that a lot of these annoyances are because I am trapped in the current paradigms (of right clicking to open in tabs, clicking "enter" to get into a website, or having a recognisable page header at the top, etc), but when you break all conventions and force your user to follow new ones, the new ones had better be an OBVIOUS improvement. And these are not.
posted by lollusc at 5:18 PM on November 9, 2011 [4 favorites]

Not related to the post really, but it's times like this that I really miss Douglas Adams. His knack for writing about technology and interfaces is a voice sorely lacking these days. Is there anyone out there writing today that has that sense of fun and amazement and, when appropriate, total annoyance?
posted by Edogy at 5:19 PM on November 9, 2011

(And half my usual shortcut keys seem to be disabled on his site, too. Like spacebar to page down, etc).
posted by lollusc at 5:19 PM on November 9, 2011

A loud clatter of gunk music flooded through the Heart of Gold cabin as Zaphod searched the sub-etha radio wavebands for news of himself. The machine was rather difficult to operate. For years radios had been operated by means of pressing buttons and turning dials; then as the technology became more sophisticated the controls were made touch-sensitive, you merely had to brush the panels with your fingers; now all you had to do was wave your hand in the general direction of the components and hope. It saved a lot of muscular expenditure of course, but meant that you had to sit infuriatingly still if you wanted to keep listening to the same programme.

posted by XMLicious at 5:20 PM on November 9, 2011 [14 favorites]

I love touchscreens. They don't break

DISPUTE.
posted by JHarris at 5:22 PM on November 9, 2011

Not related to the post really, but it's times like this that I really miss Douglas Adams. His knack for writing about technology and interfaces is a voice sorely lacking these days. Is there anyone out there writing today that has that sense of fun and amazement and, when appropriate, total annoyance?

Stephen Fry.
posted by Blazecock Pileon at 5:53 PM on November 9, 2011 [1 favorite]

Ultimate UI is the video game the kid is playing at the dinner table in front of Guernica in "Children of Men".
posted by nathancaswell at 6:17 PM on November 9, 2011 [1 favorite]

I bought the first gen Droid with a keyboard two years ago because I couldn't imagine that I'd be happy with the software keyboard. After six months, I realized that I almost never bothered to slide out the keyboard and was always using the screen to type. When I replaced it, I bought an Incredible 2 without a keyboard and don't miss it in the slightest. I'm not giving up my desktop keyboard, you can't code on a touch screen but they work great on a small scale.
posted by octothorpe at 6:27 PM on November 9, 2011

The next time you make breakfast, pay attention to the exquisitely intricate choreography of opening cupboards and pouring the milk — notice how your limbs move in space, how effortlessly you use your weight and balance.

This man has never seen me stumble around the kitchen in a hungover stupor. Nothing balletic about it.
posted by thsmchnekllsfascists at 6:58 PM on November 9, 2011

Surprised more people haven't mentioned Swype as a better way to input text on touchscreens.

Seriously, it's like discovering T9 all over again.
posted by DLWM at 9:18 PM on November 9, 2011

Yeah, I use Swype. Better than any of the predictive keyboards I've used so far. It's only problem is that it frequently pops out words that aren't words. Some of them I may have accidentally typed in in the past, but shouldn't it recognize that I corrected it or didn't use it again in the future.

I know I can also delete those words, but I'm pretty sure I forgot how.
posted by Defenestrator at 9:26 PM on November 9, 2011

Yikes, take out that apostrophe and add a question mark.
posted by Defenestrator at 9:31 PM on November 9, 2011

–Sent from my Android phone.
posted by thsmchnekllsfascists at 9:40 PM on November 9, 2011

No, it's all about the OpenOffice Mouse.

what

*eyeroll* Not an official OpenOffice product, if it was even ever a real product at all.

(Not that I have much love for OpenOffice, or any office suite, other than as something I begrudgingly have to occasionally use.)
posted by kmz at 10:19 PM on November 9, 2011

Hi everyone. New here.

his website interface is ironically annoying.

Thought I would stick up for the webpage. In Chrome it resizes to the width perfectly. And the transitions are quite smooth for me (granted I am using a newer computer).

Considering the fellow is an UI designer, and that this is his own personal page, I think it is reasonable to expect things to be on the experimental side. The site is a nice showcase for the kinds of things that are becoming possible in browsers these days. I dont find it annoying at all.
posted by rabbitfufu at 10:35 PM on November 9, 2011

frantically waving your arms

Figured that would link to Minority Report.

Figured that would link to Kermit the Frog.
posted by mazola at 11:01 PM on November 9, 2011

Surely I can't be the only one who thinks this guy spent an evening getting stoned and marvelling at his hands?
posted by i_cola at 12:37 AM on November 10, 2011

We use lots of sensory input in order to manipulate things, and our brains are really good at figuring out what the relevant ones are at any given point of time. This is why touch screen ATMs are so often awful — there is absolutely sensory feedback at all, and the reaction time of the interface is too slow to associate correctly with behavior. On the other side, we have all seen the babies and cats play with iPads. This works fine, because we are hard wired to associate our actions with simultaneous reactions, be they tactile, auditory, or visual. For example, I just saw a talk on self-tickling via a time delayed robot arm that found roughly that a delay of more than two tenths of a second was processed as "other" and less was "self." The most important thing I can see in touchscreen interfaces is thus immediacy of response across as many senses as can be considered useful. The real world never lags, and I suspect that it is the computer lag that is the biggest factor in reducing fluidity, not the input method.
posted by Schismatic at 1:38 AM on November 10, 2011 [5 favorites]

The problem with virtual keyboards is only half that they're worse to use than the crappiest physical keyboards. The other half is that they take away a dismaying fraction of the screen from the display of information.

One of the apps I constantly on my android phone with slide-out keyboard is Shortyz, a crossword puzzle. The onscreen keyboard, even though it's optimized for the task (letters only, no digits or punctuation) occupies fully 50% of the screen in landscape orentation. If the app didn't also hide the notification bar, you'd be able to see something like 3 letters of a "down" clue. Slide out the keyboard and you can see 12 letters "down", enough to see almost every individual down word at once without scrolling. (physical keyboard plus portrait orientation isn't so bad, except that the area of each key becomes proportionally smaller which seems to magnify my error rate)

I've had a smartphone for about 18 months now, and I'm definitely looking for one with a keyboard next time around.

(I actually dreamed that I bought a smartphone with IBM type-M keyboard keys with the beautiful travel and click. they were still just as small as smartphone keys, though, so the phone was a bear to use, even ignoring the fact that it was 1 1/2 inches thick...)
posted by jepler at 7:56 AM on November 10, 2011

The article gets it completely upside down for me. I don't want to have tactile response from my screen. I want to be able to touch my peanut butter jar and have the lid slide open on its own. I want my car door to sense my presence and automatically open its door and start the engine. I want my shower to remember my settings for water temperature and flow rate so I don't have to manually set it every single time.

I don't want a touchscreen to behave more like reality. I want my reality to behave more like a touchscreen.
posted by euphorb at 8:21 AM on November 10, 2011 [3 favorites]

I'm torn. I don't know when I've read something that's so simultaneously thought-provoking and depressingly banal.

On the one hand (no pun intended) he's quite right that nobody should be impressed by a little video about glorified signage, which retreads a lot of old ideas, and there are other aspects to information design that we can easily overlook while our lives are so dominated by these tiny computers we've started carrying around lately, and it's good to be reminded that the world is full of neat devices we thought up to apply our opposable thumbs and ingeniously shape our environment and aren't humans are amazing etc etc

On the other, I'm flabbergasted and slightly disturbed that such a brilliant designer should find it necessary to state the obvious. Hands? Uh, yeah. They're really useful, thanks for reminding me. By obsessing over novelty or lack of it, he totally misses the point that this kind of screen-based design is no more or less than the next logical step in the tradition that brought us the written word, the printing press, photographic reproduction, cinema and telephony. It's taken 4000 years, not 40, and it's always been about language and communication. Physical agency is secondary to perceived meaning, which is why so many input devices act as proxies for real-world counterparts (styli, nunchuks, piano keyboards, steering wheels etc).

Touch screens are cheap, easy to use and versatile. If they aren't ubiquitous already they will soon be. They're every bit as frustrating as their antecedents, but they're just too useful to ignore. They have their limitations, but breaking out of them in the direction he suggests requires us to move from a universal way of thinking into something which is infinitely diverse and not particularly well suited to these kinds of tasks. If we want real alternatives to screen-based interaction, solutions need to focus on the context of things themselves, and not on how we can manipulate their physical space.
posted by Elizabeth the Thirteenth at 8:36 AM on November 10, 2011 [2 favorites]

That's the fundamental gesture in this technology. Sliding a finger along a flat surface.

There is almost nothing in the natural world that we manipulate in this way.

This bugged me for a long time yesterday, but it wasn't until this morning that I figured out why. And the question it comes down to is so what?

There is nothing in the natural world that I manipulate the way that I manipulate a pen or a pencil. There is nothing in the natural world that I handle in any way like I handle a computer keyboard (the QWERTY ones themselves designed around the limitations of the technology of the past, not the operator. They are still widespread because that's how we've all learned to use them. When does that stop? The technology will support a faster interface now, but how do we phase out the standard?). And a computer mouse? Making motions along the horizontal plane to affect the motion of something else on a vertical plane? Not much like that I do "naturally."

We're a tool using species - tool upon tool upon tool. My mouse is a tool designed to make my interface with another tool - the computer - easier. Touch screens are easier still (I was thinking about this article this morning while watching my 2 year old son attempt to interact with the TV. It was one of those shows that asks the audience to pick the right answer from choices on the screen, and my son's instinctive response was to point and tap the screen. Same instinct with the computer. It's why both my kids love getting their hands on the iPhone or their uncle's iPad). It seems to me that there is something more instinctive about a touch screen.

No, it's not ideal, and we will (as the article says) transition to something better. But this notion of something we do "naturally"? We use tools naturally. Those tools can be crude or sophisticated, but I would argue all of them are to some degree natural for us. As is the transition to better and better tools.
posted by never used baby shoes at 9:07 AM on November 10, 2011 [1 favorite]

It's nice to know how far one is into a book by weight, but it would be pretty nice if you could press on a word to get to a page in another book in another country translated from another language.

Bananas are easy to peel, but fish are a lot harder to get out of water, remove the scales, and eat without choking on translucent, tiny, sharp bones.

Opening a jar is a masterpiece of a tactile performance, except when the lid is so tight you need to find some other sharp object to jam in there to pry it open. But maybe you can ask another pair of hands to do it for you.
posted by romanb at 9:10 AM on November 10, 2011

Welcome to the future – we will not be opening jars to schedule an appointment. Unless that appointment is with our marmalade.
posted by romanb at 10:05 AM on November 10, 2011 [1 favorite]

You know who opens jars like motherfuckers? Octopi.
posted by Artw at 10:27 AM on November 10, 2011

The most important thing to realize about the future is that it's a choice. People choose which visions to pursue, people choose which research gets funded, people choose how they will spend their careers.

Despite how it appears to the culture at large, technology doesn't just happen. It doesn't emerge spontaneously, like mold on cheese. Revolutionary technology comes out of long research, and research is performed and funded by inspired people.

And this is my plea — be inspired by the untapped potential of human capabilities. Don't just extrapolate yesterday's technology and then cram people into it.
posted by infini at 11:57 AM on November 10, 2011

If you think the touchpoint is the perfect HID for laptops, use a Mac trackpad sometime.
posted by The Lamplighter

I have to take my fingers off the keyboard to use that. I also have to keep dragging my fingers around to use that. I'm have a mouse hooked up right now, I'm still using the trackpoint since it is faster for quick tasks then reaching for the mouse.

-----
And while we're at it, what's with Lenovo putting that evil touchpad on the bottom of the keyboard, leading to people having to reach over to use the far superior Trackpoint, giving themselves carpel tunnel in the process?
posted by Yowser

I used to hate having it there (I leave mine disabled so I don't hit it with my thumbs, which happens all the time if I leave it on), but it means you get some nice wrist-space when typing, which is a good thing.
posted by Canageek at 5:32 PM on November 10, 2011