Learning to see via an iPhone

September 19, 2010 4:02 PM

A blind man uses a mobile phone to "see":

I have never experienced this before in my life. I can see some light and color, but just in blurs, and objects don’t really have a color, just light sources...I went outside. I looked at the sky. I heard colors such as “Horizon,” “Outer Space,” and many shades of blue and gray. I used color queues to find my pumpkin plants, by looking for the green among the brown and stone. I spent ten minutes looking at my pumpkin plants, with their leaves of green and lemon-ginger. I then roamed my yard, and saw a blue flower. I then found the brown shed, and returned to the gray house.

posted by nomadicink (45 comments total)

(Hmm. Er... is there a reason you linked the cache of that page, instead of the actual page itself?)
posted by koeselitz at 4:07 PM on September 19, 2010

EyePhone?
posted by Rhaomi at 4:08 PM on September 19, 2010 [2 favorites]

(Hmm. Er... is there a reason you linked the cache of that page, instead of the actual page itself?)

Because it was easily fireballed about 8 hours ago and Metafilter would just smoosh it into the ground.
posted by Talez at 4:14 PM on September 19, 2010 [5 favorites]

Wow. Beautiful.
posted by k8t at 4:16 PM on September 19, 2010

As Gruber mentioned in his write-up, the author recently got an iMac as well.
posted by Ian A.T. at 4:19 PM on September 19, 2010

It was interesting when he said he went out at night and then remembered you needed light to see things.
posted by thorny at 4:23 PM on September 19, 2010 [10 favorites]

Apple's VoiceOver sounds like an impressive technology. It started life as a MacOS thing. Apple has a long history of design for accessibility to people with limited sight, no simple thing in a company that's so visual design oriented. I had no idea it was this effective and it would never even occur to me it might work on an iPhone.
posted by Nelson at 4:26 PM on September 19, 2010

The site hasn't been up since at least yesterday, when I first tried to read this post. Thanks for the cache links.
posted by mathowie at 4:29 PM on September 19, 2010

(Hmm. Er... is there a reason you linked the cache of that page, instead of the actual page itself?)

Yeah, I got the cache link off Daring Fireball, since I figured Mefi would crush it too. It was just amazing and moving to hear how some technology was really helping people.
posted by nomadicink at 4:42 PM on September 19, 2010

Nelson: "Apple has a long history of design for accessibility to people with limited sight"

A distinct lack of physical buttons and haptic feedback makes me doubt this.
posted by meehawl at 4:47 PM on September 19, 2010

That last part actually made me cry.
posted by empath at 4:48 PM on September 19, 2010 [2 favorites]

There's some other neat tools for helping the blind also, along with legislation that would give the Federal Trade Commission the power to set regulations for content providers and equipment involving the Internet.
posted by nomadicink at 4:51 PM on September 19, 2010

This reminds me of a project that puzzled me when I was a young freshman art student. I saw a photography workshop for the blind. I thought that was ludicrous. But it wasn't. There are legally blind people with "low vision" that can't see 2 inches beyond their nose. But give them a Polaroid point-and-shoot and they can hold the print up 2 inches from their eyes, and they can see it. Now digital cameras make it instant. My little niece is like that, she has low vision due to retinal detachment, oh she loves her iPhone, she even posts videos to her Facebook page.

I spent a lot of time over my years in the computer biz helping handicapped people get computers. My bosses were always furious at me for spending time with them because it always took a long time to interact and help them out. I loved to help them, nobody else would, and I built up a fair clientele. And Apple was always the first choice, even back to the days of the Apple ][. Apple was the first computer company to have a division specifically dedicated to assistive products. It was almost a requirement for their goals in educational markets. Apple often demonstrated their advanced concepts as assistive technologies, like their "Future Shock" concept video. I was a little surprised when they disbanded the assistive devices team, but they just rolled it into the standard development system. Now assistive technology is a standard feature, not an accessory.
posted by charlie don't surf at 4:51 PM on September 19, 2010 [8 favorites]

Just discovered the blind cool tech Podcast, which touches on tech of interest to blind people. It touches on all platforms, Android and Windows are mentioned several times too!
posted by nomadicink at 4:59 PM on September 19, 2010

give the Voiceover thing on your iphone a shot if you have one, it is pretty impressive. It's in settings->general->accessibility.
posted by neustile at 5:04 PM on September 19, 2010 [1 favorite]

>Nelson: "Apple has a long history of design for accessibility to people with limited sight"

A distinct lack of physical buttons and haptic feedback makes me doubt this.

Umm.. I was selling text-to-braille devices in the 1980s, before the Mac existed. You really wouldn't believe the profusion of innovative applications written by microcomputer hackers in the early days, like braille editors, early primitive speech synthesizers, etc. Many of these devices were custom written by family members, desperate for assistive solutions for their family members.

Now computer assistive devices are so good, braille is dying out, nobody needs it anymore. But you can still buy "braille panels" and now support is built in to the OS. You might want to look at that web page's descriptions of gestural input options that go way beyond the need for physical buttons and haptic feedback.

While looking for that web page, much to my surprise, I discovered that Apple revived their accessibility division. Good for them.
posted by charlie don't surf at 5:08 PM on September 19, 2010 [5 favorites]

FYI the iPhone also supports braille devices over bluetooth
posted by neustile at 5:28 PM on September 19, 2010

This post & the comments have made my day. Thanks for the great post!
posted by pointystick at 5:54 PM on September 19, 2010

Thanks so much for the post. I've been interested in assistive tech since working with my Master's advisor, who has been blind his whole life but has a tremendous ability to visualize concepts in Functional Analysis and Wavelets. As a Graph Theorist I have yet to find a use for my type of math in these sorts of projects. It would be cool to do some related research.
posted by monkeymadness at 5:58 PM on September 19, 2010

Awesome.

Just awesome.
posted by SLC Mom at 6:23 PM on September 19, 2010

This is great.
posted by alms at 6:33 PM on September 19, 2010

I love assistive technology. Not only is it useful for those with disabilities, it is useful to those who don't have disabilities. It's a hurdle to get things to work sometimes, but in the end it is worth it. What a beautiful story.
posted by Brent Parker at 6:56 PM on September 19, 2010

that's awesome.
posted by gaspode at 7:08 PM on September 19, 2010

Wow. I just tried the VoiceOver feature, and it's pretty damn impressive. It was tempting to rearrange all my apps to take advantage of the way it cycles through your screens and apps - it reads the screen number and if you don't 3-finger-scroll on, it reads the name of the first app on that screen. The voice speed is adjustable, there's a practice area where you can just tap the screen in different ways and it tells you what that gesture does.

It even coped pretty well with Twitter - mispronounced some of the names, and says "number" for the hashtags, but honestly it's very good.

But I never would have thought it was good enough to play with if Austin Seraphin hadn't written such a glowing account. I guess you really have to be in that situation to know exactly how to make the most of it, and to have the knowledge of other (presumably not as good) assistive technologies.
posted by harriet vane at 7:29 PM on September 19, 2010

It even coped pretty well with Twitter - mispronounced some of the names, and says "number" for the hashtags, but honestly it's very good.

It gets better, actually, because the application can deal with the special hash tag thing by providing an accessibility information that replaces relevant instances of "#" with a word that is more semantically accurate.
posted by Mikey-San at 8:06 PM on September 19, 2010

My question is this: Why is iTunes still so crappy (by hearsay, I've never used it)?
posted by maxwelton at 8:39 PM on September 19, 2010

This is really cool, thanks for posting it.
posted by LobsterMitten at 9:15 PM on September 19, 2010

Meehawl wins the troll award for this thread.
posted by joeclark at 9:19 PM on September 19, 2010 [2 favorites]

I really like the way he just instinctively uses the words "see" and "look". Extended perception!
posted by painquale at 9:20 PM on September 19, 2010

I was really surprised when blind users started contacting us with suggestions for our iPhone app. If you use the standard UIKit widgets, your app is mostly-accessible by default. It takes just a little extra effort to add the appropriate hints to make it truly accessible. Kudos to Apple for making this painless for developers (as opposed to the web and the 508 compliance boondoggle, which often is a checkmark and extra billable hours for Federal contractors rather than being truly assistive to the disabled).
posted by RobotVoodooPower at 9:46 PM on September 19, 2010

My question is this: Why is iTunes still so crappy (by hearsay, I've never used it)?

I imagine he has problems with the lack of consistent metadata associated with the music, which would make it easier to select songs either by name or by genre etc.

I have a love/hate relationship with iTunes. On one hand, it has an attractive interface and a great, customizable database system. I enjoy making playlists etc. and the ease of syncing devices.

On the other hand, its file management is awful. I bleed from the ears when I can't copy music from my iPod back to iTunes, for example. I haven't the foggiest why they tie their digital rights management to file locations. It's the most dumbed-down system imaginable, and yet totally unreliable. I've had to completely restore my libraries a handful of times for things which just shouldn't have happened.

And, every once in a while things just get buggy. And not just innocently buggy. Upon my update to iTunes 10, it just decided to blow away my Audiobooks folder from my iTunes directory. Gone. No warning. I selected one and it just gives me this awful little stomach-sinking exclamation point, prompting me to browse to something that it just blew away. Glad I have a backup - because the iTunes store only allows you to download music you've bought once.

It also inexplicably defaults to blowing away podcasts after a certain amount of time. Oh, you wanted that? Too bad - even though you ask it to save all podcasts. Even if the answer is 'oh, you just need to set this to blah blah blah' it shouldn't just nuke shit without you saying so - especially if its own settings imply that it's keeping it for you.
posted by jimmythefish at 9:47 PM on September 19, 2010 [1 favorite]

Trying not to derail here: If the "Episodes to Keep" setting isn't doing what you expect (keep in mind that you can have separate settings at the per-podcast level), you should file a bug. It's always worked for me, but I am not everyone.
posted by Mikey-San at 12:33 AM on September 20, 2010

Not to derail, but is it possible for non-developers to file bugs? 'Cause there's a whopper of a bug in artist sorting in iOS 4.x, but I've never been able to find a way to submit it.
posted by Lazlo at 12:42 AM on September 20, 2010

This is so cool, I had no idea the iphone could do this.
posted by dabitch at 1:17 AM on September 20, 2010

Blind people have generally used the Nuance TALKS system on Symbian 'phones for the last few years. The new VoiceOver in the iPhone and iPad is great. My understanding from some blind presenters from the RNIB is "No sight - possibly too steep a learning curve versus TALKS at the moment. Some sight - the iPhone and especially iPad is absolutely great." You can also plug a Braille line directly into an iPhone/iPad. And of course with an iPad (compared to a laptop) you can easily hold it as close to your face as you need.

See, mobile assistive technology has been around for a few years. What's really different is the fact it comes as standard on every iPad/iPhone/Mac, and because the Apple ecosystem is more rigorously controlled than other platforms, the accessibility standards needed to make everything work are more strictly applied. I think the iPhone/iPad will be the de facto standard for technology for blind users who aren't developers/geeks/at work (who'll be on JAWS on Windows) or broke (who'll be on NVDA/Thunder on Windows). Great stuff.

Watch out for the politics, though. Disability is also a tool in the Apple/Microsoft/IBM wars. Let's just say that each have made contributions to accessibility.
posted by alasdair at 3:25 AM on September 20, 2010

The term is Braille display (not “line” or any such nonsense) and the only thing you can “plug” them “into” is a desktop computer, not an iPhone (which has no compatible port) or an iPad (which also doesn’t). Bluetooth Braille displays are supported on iPhone. (Actually, on Mac too.) iPad will apparently have to wait till 4.2 for such support.
posted by joeclark at 6:04 AM on September 20, 2010

If you're playing with the VoiceOver feature on the iPhone, be sure to try the screen curtain. Three triple-finger taps and it turns the screen off. Makes it a little bit clearer what using the phone is like for a blind person.
posted by fungible at 7:08 AM on September 20, 2010 [1 favorite]

I was uncomfortable with how branded this post was. I thought I would be treated to a post on the science or medicine of blind vision. Instead, I got "check out how awesome the iphone is!" with a paragraph at the end about how he used some app that reads colors out as english words.

This was just an advertisement for the iphone, except that there exists the possibility that the person who wrote it was not paid by apple and that the person who posted it on Metafilter was not paid by apple or a viral marketing firm.
posted by Galaxor Nebulon at 11:13 AM on September 20, 2010

I was uncomfortable with how branded this post was.

Well, if anything, it's an "advertisement" for how having quality accessibility tools on smartphone devices can make a real difference for those with impairments who wish to participate in modern society. The fact that Apple's iPhone is a device which has these built in at the OS level is central to the story, because there really aren't any other devices which can demonstrate the impact having these tools makes. It's an amazing tale of how a device has changed the world for the author. Sharing the make and model of that device would only come naturally as part of that report.
posted by hippybear at 11:20 AM on September 20, 2010 [1 favorite]

Yes, but on the other hand, grumble grumble grumble. Whaddya say to that?
posted by Galaxor Nebulon at 11:26 AM on September 20, 2010

Um... that I wasn't meaning to step on your lawn, I was retrieving my frisbee, and I will be careful not to have it fly toward your yard again.
posted by hippybear at 11:41 AM on September 20, 2010 [1 favorite]

I watched the sun set, listening to the colors change as the sky darkened"

Wow.
posted by eggman at 12:14 PM on September 20, 2010 [2 favorites]

Thanks for the clarification, joeclark. I thought Braille lines were supported on iPad now, but now you mention it I think that blind presenter was onto the iPhone by that point.

I understand the on-screen keyboard is also getting an update in the autumn - will that be 4.2, do you think? It's very exciting anyway.
posted by alasdair at 4:05 PM on September 20, 2010

Galaxor Nebulon: I was uncomfortable with how branded this post was.

Hardly the first FPP on a single company's product. Sometimes information isn't communicated in completely unbiased ways. Deal.
posted by IAmBroom at 5:41 PM on September 20, 2010

Alasdair, there is no such thing as a Braille “line.” One more time: They’re called Braille displays.
posted by joeclark at 11:08 AM on September 21, 2010