Seeing at the Speed of Sound

March 7, 2013 10:17 AM

Rachel Kolb, deaf Rhodes scholar, on lipreading: "Even the most skilled lipreaders in English, I have read, can discern an average of 30 percent of what is being said. I believe this figure to be true. There are people with whom I catch almost every word—people I know well, or who take care to speak at a reasonable rate, or whose faces are just easier on the eyes (for lack of a better phrase). But there are also people whom I cannot understand at all. On average, 30 percent is a reasonable number. But 30 percent is also rather unreasonable. How does one have a meaningful conversation at 30 percent? It is like functioning at 30 percent of normal oxygen, or eating 30 percent of recommended calories—possible to subsist, but difficult to feel at your best and all but impossible to excel."

Previously bridging the Deaf/hearing divide on metafilter:

Lydia Callis enthralled New York City with the beauty and expressiveness of sign language.

Bad Lip Reading is courtesy of a music producer who, impressed by his mother's skill at lipreading, attempted to pick it up himself and found he was terrible -- but hilarious! -- at it.

posted by Eyebrows McGee (29 comments total)

When I answer that, yes, I can lipread, they relax. Then they prattle on as if all preconditions are off. Because I can "read" their lips, I must therefore be able to "read" everything they say. After all, it would be absurd for me to protest that I can sometimes read the words in a book, but sometimes not. Either you can read, or you can't.

Fascinating. I suspect I'm guilty of that myself. If someone I was interacting with was lipreading, I wouldn't want to patronize them by speaking slowly and using simplified language, but I never realized how difficult a feat it was. The Bad Lip Reading guy is a goddamned genius by the way. An orange peanut? For me?
posted by Rock Steady at 10:34 AM on March 7

This makes me feel a tiny bit better: I suck at lip reading. I can't do it at all. I guess it's a good thing I'm not deaf, though maybe if I was, I'd somehow acquire the skill. I actually do have a bit of a hearing loss, and I generally can't understand what people are saying at bars, when the music is loud. When people realize that, they start over-annunciating and moving their lips in broad gestures, which doesn't help. I have no idea what they are saying.

One of my favorite movies is "All that Jazz." I saw it when it first came out, and I've maybe seen in 50 times altogether. There's a scene in it in which a Broadway-musical producer announces that the new song for his show is "catchy and bouncy." Two other characters, who are more sophisticated than the producer, roll their eyes when they hear this. Then one of the sophisticates mouths something to the other.

When I first saw the movie, I was baffled. People around me laughed at the mouthed words, and the filmmakers obviously assumed the audience would understand them. But I had no idea what they were. Each time I watched the film after that, I paid close attention to that scene, watching the actress's lips very carefully. In the end, it took my about 20 views to get it.

She was just repeating "catchy and bouncy," mocking the phrase. Even the context didn't help me.
posted by grumblebee at 10:50 AM on March 7 [1 favorite]

"I never realized how difficult a feat it was"

Yeah, I thought her article did a really good job explaining something very foreign to me in a clear and interesting way, and she had a lot of thoughtful things to say about communication and personal identity. It was one of those things where I didn't know I was interested in the topic until I read the article. :)
posted by Eyebrows McGee at 10:51 AM on March 7

Even with perfect hearing, spoken language is ambiguous and hard to perfectly decipher. That's why computers are still pretty terrible at parsing voicemail messages and whatnot, a lot of what we think of as just hearing what someone is saying is actually a complicated process where we have to guess what exactly is being said based on our knowledge of the language being spoken and the context in which things are said. It's absolutely not the same thing as reading clearly printed text for anyone. Also everyone knows how to lipread to some extent, in that when you are talking to someone your eyes naturally focus on their face and seeing them talk helps your brain process their speech better than if you couldn't see them.
posted by burnmp3s at 10:52 AM on March 7 [3 favorites]

There appears to be an implicit assumption that people who are not hearing impaired operate at 100% which is pretty untrue. I don't have any diagnosed hearing problems but am pretty certain that in most environments where there is any background noise I operate at around 70-80% and fill in the rest. At home watching TV I often rewind or ask my wife what someone just said and neither of us can tell even after rewinding.

Also any kind of accent and all bets are off. I used to live in Birmingham and met people I could barely understand. Then there was the gentleman from Wolverhampton who I spent an entire afternoon with without even be able to understand what he said his name was.

I shared an office with a woman who was hearing impaired in grad school and it was an interesting experience. It was impossible not to startle her when I came into the office (I tried flicking the lights for a while). I actually asked about etiquette when she had an interpreter because I felt bad if I looked at her and ignored her interpreter and vice versa (look at the person you are talking to - not the interpreter). Of course the feedback is also strange because she had to watch the interpreter. In my second year I grew a beard - that changes things. I'm a mumbler and I have the kind of pitch that can sometimes be hard to hear so in some ways lip-readers have an edge over hearing people with me because I will speak louder, enunciate more clearly and face them when I am speaking. I also found I became very aware of and self-conscious of my facial muscles.

I switched advisers and labs and moved out after the first year and I am pretty sure my former office-mate ended up gravitating toward people who knew sign language or who were willing to learn it.

There is, if I recall correctly, research showing that everybody uses lipreading to a certain extent to differentiate between similar sounding words.

That story about Daniel using the cell phone makes think there must be a really simple speech to text phone app solution for those situations now.
posted by srboisvert at 11:03 AM on March 7 [1 favorite]

She repeats herself. This time I understand that it is a question. Well, most questions are easily answerable with "yes" or "no."
I decide fast, "Yes." Surely a positive response will make the girl happy.

As a (majority-not-totally) deaf guy, I'm in this situation dozens of times a week. Don't hear something, ask them to repeat it, still don't get it, and rather than push things further and embarrass us both, I fake some non-answer. Just read their tone and context, and go with that as best I can. I couldn't begin to count the number of non-conversations I've had in bars, crowded rooms, on the bus...

Standard way to go through life as a deaf person, I expect. Just this whole parallel existence that goes along with but never fully connecting with the larger world.

Enh. It's life.
posted by Capt. Renault at 11:14 AM on March 7 [7 favorites]

MetaFilter: one of those things where I didn't know I was interested in the topic until I read the article.
posted by Rock Steady at 11:17 AM on March 7 [6 favorites]

There is, if I recall correctly, research showing that everybody uses lipreading to a certain extent to differentiate between similar sounding words.

Yep! It's called the McGurk effect, and it's pretty cool.

The way we interpret speech is actually extremely complicated and depends on many factors, not just the sounds of the phonemes strung together. There's the visual cues, a la the McGurk effect, there's all sorts of complicated things with formants and overtones (for example, if you cut the stop off the front of a word, your brain will still hear the stop because of the transition of the overtones on the following vowel), and the gestalt effects that happen when we interpret a word even if part or all of it is masked (like in a bar, called the "picket fence effect").
posted by Lutoslawski at 11:19 AM on March 7 [9 favorites]

Metafilter: ask them to repeat it, still don't get it, and [...] push things further and embarrass us both
posted by eritain at 11:34 AM on March 7

The way we interpret speech is actually extremely complicated and depends on many factors, not just the sounds of the phonemes strung together.

Like my mom always said: "90% of communication is half mental."
posted by Floydd at 11:35 AM on March 7 [1 favorite]

This is how I hear French, and I'm lucky if I can get to 30%. I can generally tell what the subject of the conversation is, but not what exactly is being said concerning it. I try to fill in that part by guessing what those particular speakers would say.
posted by StickyCarpet at 11:36 AM on March 7 [1 favorite]

This is super interesting, thank you for posting it. Especially the parts about accents and darkness.

I remember on Survivor: the Amazon, there was a deaf contestant, Christy, who read lips, and complained of basically being isolated when the sun went down. There is to this day a controversy about whether or not she understood the instructions at the final tribal council, which were just spoken to her, and not put in writing. She ended up voting for someone who, just a day before or something) she said she wouldn’t vote for in a million years. A lot of fans speculate that she misread Jeff Probst’s lips and meant to vote against the winner, instead of for her.
posted by roomthreeseventeen at 11:42 AM on March 7

As a (majority-not-totally) deaf guy, I'm in this situation dozens of times a week. Don't hear something, ask them to repeat it, still don't get it, and rather than push things further and embarrass us both, I fake some non-answer. Just read their tone and context, and go with that as best I can.

As someone easily embarrassed in social situations I SO understand that response, but I have to say as the hearing-person side of the exchange it's actually very alienating. I've had family members and friends lose their hearing with age and I would so very much prefer a repeated "I'm sorry, I just didn't catch that" for however many times it takes than the vague "hm yes....(airy wave of head)." You just never know if you have actually conveyed your point, so you don't know what basis the conversation is continuing on. You don't know if they know who the story you're telling them is about, say, or what it's supposed to prove or what have you. And then when you realize, often quite a long time later, that they made a bad guess at what you were saying you're faced with the "do I go back over all that again or do I just forget it" problem. It gives every communication a rather tenuous and provisional quality that makes it so much more of an effort than any social chit-chat ought to be.

Whereas simply repeating something in various ways until it's understood is just a mechanical 'problem-solving' exercise. It has none of the erosive effects on your sense of actually bonding with the person that the "take a guess at the meaning and hope it comes clear later" trick does.
posted by yoink at 11:47 AM on March 7 [1 favorite]

There is to this day a controversy about whether or not she understood the instructions...

This is pretty common. The deaf person may think they've understood, and the hearing person think that the deaf person has understood, but only later does it become clear that they were each understanding different things.
posted by Capt. Renault at 11:47 AM on March 7 [3 favorites]

Whereas simply repeating something in various ways until it's understood is just a mechanical 'problem-solving' exercise.

If only it would happen that way.

For the vast majority of situations, asking someone to repeat something more than twice, or to repeat it using different words -- even if they're aware of your disability -- is met with people not treating you like you're deaf, but a moron. In my experience, most people have no idea how to make the accommodation you need, even if you tell them exactly what you need, to separate their words, use different words, whatever. And that's if you even want to get into describing your physiological condition to some stranger in the first place -- that in itself invites a conversation where they want to know exactly what you can and cannot hear, how it happened, blah blah blah, when it's intensely personal information and none of their business anyway.

As a practical matter, it's much simpler to just pretend, and deal with whatever minor consequences later.
posted by Capt. Renault at 12:00 PM on March 7 [3 favorites]

As someone easily embarrassed in social situations I SO understand that response, but I have to say as the hearing-person side of the exchange it's actually very alienating. I've had family members and friends lose their hearing with age and I would so very much prefer a repeated "I'm sorry, I just didn't catch that" for however many times it takes than the vague "hm yes....(airy wave of head)."

You say that now. I find that most people are really only *willing* to repeat something twice - maybe thice - before they say "Never Mind." And then they walk away and the conversation is over, and it's less likely they'll start another one sometime in the future. There really is a quite limited number of times most people are willing to repeat something without frustration tinging the interaction.
posted by stoneweaver at 12:05 PM on March 7 [7 favorites]

On preview: per Renault and stoneweaver.

as the hearing-person side of the exchange it's actually very alienating. ... I would so very much prefer a repeated "I'm sorry, I just didn't catch that" for however many times it takes than the vague "hm yes....(airy wave of head)."

Capt. Renault's report from the mostly-deaf world strongly resembles my own memories from foreign language immersion. Before I had that experience I would have agreed with you 100%: Get everything clear at once before proceeding to anything else, it's the only way to be sure to make sense of things.

But on the other side you learn: You learn that oftentimes a later restatement will come along, one better conceived and better articulated than the initial one you were struggling with. You learn that most things people are talking about don't actually matter. You learn, pardon me for saying so, that most people are less patient with repeating themselves than they think they would be. You learn that you yourself have a finite capacity for asking for repetition, for the effort of communicating. And all of those things conspiring yield a powerful pressure to just let some crap slide.

you're faced with the "do I go back over all that again or do I just forget it" problem

The other party has already voted for "forget it." That's not an infallible guide, especially since you understand what the subject is better than they do, but it's something to consider.
posted by eritain at 12:20 PM on March 7

Don't hear something, ask them to repeat it, still don't get it, and rather than push things further and embarrass us both, I fake some non-answer

...and end up wearing a puffy shirt on The Today Show.
posted by Knappster at 12:23 PM on March 7 [7 favorites]

There is so much in this article that I identify with. Especially that thrill of Daniel pulling out a phone to write to her - someone else taking some responsibility for the communication barrier for once! One of the most interesting parts for me of going to Gallaudet for grad school was that while there were still some communication barriers (many of the grad students are hearing non-native signers), the assumption was no longer that the breakdown was because of me, or something I had to take the lead in figuring out.
posted by spaceman_spiff at 12:30 PM on March 7 [2 favorites]

Amazing article! I'm fortunate enough to have exceptional lip-reading skills as a deaf person (70-80% without context), but even at that, it's not the flawless solution many hearing people think it is. It is so fucking draining. Seriously, the level of concentration you need to put into lipreading is off the charts, and it's more or less your default mode that you can't turn off. Even ten minutes of talking to someone drains me completely, and it's a constant endurance race to talk to someone without looking visibly tired. Which is a weird thing because I suspect I'm naturally extroverted, as even then, I like to talk to talk to people. No wonder I spend most days holed up in my room, though.

You can't do it when it's dark; you can't do it when you're tired; and you can't do it in group conversations because it takes a few seconds to "refocus" on people... But people only talk for a few seconds anyway.

And yet somehow, people assume lip-reading is a good substitute for being able to hear. Really, no - even if you can do it perfectly, you would just die doing it all the time.
posted by Conspire at 12:44 PM on March 7 [10 favorites]

Even with perfect hearing, spoken language is ambiguous and hard to perfectly decipher. That's why computers are still pretty terrible at parsing voicemail messages and whatnot, a lot of what we think of as just hearing what someone is saying is actually a complicated process where we have to guess what exactly is being said based on our knowledge of the language being spoken and the context in which things are said.

Gustave Flaubert famously wrote, “La parole humaine est comme un chaudron fêlé où nous battons des mélodies à faire danser les ours, quand on voudrait attendrir les étoiles.” ("Human speech is a cracked kettle on which we tap out crude rhythms to make bears dance, when we long to make music that will melt the stars.")
posted by ricochet biscuit at 1:08 PM on March 7 [7 favorites]

Metafilter: a complicated process where we have to guess what exactly is being said based on our knowledge of the language being spoken and the context in which things are said.
posted by herbplarfegan at 1:09 PM on March 7 [2 favorites]

The deaf person may think they've understood, and the hearing person think that the deaf person has understood, but only later does it become clear that they were each understanding different things.

I know that you believe you understand what you think I said, but I'm not sure you realize that what you heard is not what I meant.
posted by dhartung at 1:09 PM on March 7

"I thought you said you could read lips."

"I assumed I could."
posted by moorooka at 1:17 PM on March 7

I have pretty bad ringing in my left ear (that eardrum has popped several times and is heavily scarred) and my partner mumbles a bit, and when she stands on the left, I will sometimes hear her say two different things simultaneously-- one that she actually did say, and another which resembles it (often not all that much).

The most recent one was when she said "je suis fatigué" and I heard that and "just a sucky day" layered in right on top of it.
posted by jamjam at 1:17 PM on March 7 [3 favorites]

makes think there must be a really simple speech to text phone app solution for those situations now

Given my Android smartphone's success rate with voice dictation (definitely >%50 accuracy) and the soon-to-be creepy reality of Google Glass, I would hope that it's only a matter of time before mobile technology enables closed-captioned conversations for the deaf/hard of hearing.
posted by hot soup at 1:22 PM on March 7 [4 favorites]

Android 4 ships with a voice keyboard. There's nothing to install, you just whack the button shaped like those microphones from old movies. Thing is, you really want to use the *speaker's* phone, because it's learned to recognize their voice, specifically.
posted by LogicalDash at 3:22 PM on March 7

Standard way to go through life as a deaf person, I expect. Just this whole parallel existence that goes along with but never fully connecting with the larger world.

This sums it up well for me. I'm also mostly deaf. I wear hearing aids in both ears and I lipread but I really only do okay in one-on-one conversations with native English speakers in quiet situations. This has caused a lot of social isolation and missed opportunities throughout the years. I am not a shy person, but I tend to avoid conversations rather than guessing at what they said or making them repeat it a dozen times. I don't function well in business meetings* or schmoozing at conferences, which has no doubt hurt me professionally.

Imagine your boss saying to you in a meeting, "I need you to modify the ------ (what?) numbers on page -- (one? ten?) of the report and email it to ------ (who?) and Sandy (Cindy? Sammy?) by ----day (Monday? Wednesday?)" I'm put on the spot. Either I have to stop the meeting to clarify what he said, or I say "Okay" and catch up with him later. I repeat back what I think I heard, he corrects me, I repeat that back to him, etc. I know he doesn't think I'm stupid, but it's still frustrating. I do as much as I can by email and IM, but there are an amazing amount of people who eschew those in favor of a phone call or dropping by.

*Someone hilariously asked me to take notes on a conference call with people from another country. I offered to be someone else's backup for one meeting. I sent an almost-entirely blank document. I was not asked again.
posted by desjardins at 4:02 PM on March 7 [5 favorites]

You say that now. I find that most people are really only *willing* to repeat something twice - maybe thice - before they say "Never Mind." And then they walk away and the conversation is over, and it's less likely they'll start another one sometime in the future. There really is a quite limited number of times most people are willing to repeat something without frustration tinging the interaction.

Due to a degenerative disease, my dad lost the ability to speak comprehensibly. I pretty much didn't understand anything he said after the first year. And let me tell you, having to say, "I'm sorry, Dad, I didn't understand that," five, ten, fifteen, twenty fucking times in a row, and I didn't understand him any more than I did on try number one.... He just got angrier and angrier with me. I didn't want to make him madder by asking him to repeat, repeat, repeat, but that happened anyway because he was my dad and disabled and "Never fucking mind" wasn't an option.

To this day I am kind of nervous and scared when I have to deal with a foreign speaker who doesn't know much English and has a thick accent. I hate, hate, hate being the asshole who can't understand them after multiple tries of them saying whatever the hell it was over the phone. But since it's work, again, "never fucking mind" isn't an option.
posted by jenfullmoon at 7:42 PM on March 7 [2 favorites]