Sign Aloud
May 10, 2016 10:07 AM Subscribe
Gloves that Translate Sign Language into Text and Speech Award winning new tech. Make sure to watch the video.
I swear we had a Hampshire College alumn working on this back in 1995...
posted by gusandrews at 10:30 AM on May 10, 2016
posted by gusandrews at 10:30 AM on May 10, 2016
Reddit's /r/movies just had a thread about Congo, and that cannot be a coincidence.
I'm pretty sure that's what coincidence is.
posted by The Tensor at 10:33 AM on May 10, 2016 [3 favorites]
I'm pretty sure that's what coincidence is.
posted by The Tensor at 10:33 AM on May 10, 2016 [3 favorites]
Keeping in mind that these are undergraduates (and maybe we shouldn't beat up on them too hard), there are some major problems with this project.
The first set of problems are linguistic. ASL relies on more than just hand shape and movement of the hand-- facial expressions matter, as do things like where exactly the sign is being signed. Also, among other things, ASL "word order" is different from English. It's not clear how or how well their system will work with longer utterances. At the very least, the way things are presented in the video ends up (intentionally or not; I'm not saying that the creators believe this) perpetuating the myth that ASL is simply English, with signs instead of individual words, which is very much not the case.
The second set of problems really boil down to "Who is this really increasing access for?". See here (which also reiterates/outlines some of the linguistic issues with projects like these).
What is this for, anyway? Oh yeah, why are we doing this? So that Deaf people can carry a device with a camera around, and every time they want to talk to a hearing person they have to mount it on something, stand in a well-lighted area and sign into it? Or maybe someday have special clothing that can recognize their hand gestures, but nothing for their facial gestures? I’m sure that’s so much better than decent funding for interpreters, or teaching more people to sign, or hiring more fluent signers in key positions where Deaf people need the best customer service.
So, yeah. I'm not saying that technology like this is fundamentally bad or won't end up helping people somewhere down the line, or that the inventors are terrible people, but this whole problem is a lot more complex then the link makes it out to be.
posted by damayanti at 10:46 AM on May 10, 2016 [24 favorites]
The first set of problems are linguistic. ASL relies on more than just hand shape and movement of the hand-- facial expressions matter, as do things like where exactly the sign is being signed. Also, among other things, ASL "word order" is different from English. It's not clear how or how well their system will work with longer utterances. At the very least, the way things are presented in the video ends up (intentionally or not; I'm not saying that the creators believe this) perpetuating the myth that ASL is simply English, with signs instead of individual words, which is very much not the case.
The second set of problems really boil down to "Who is this really increasing access for?". See here (which also reiterates/outlines some of the linguistic issues with projects like these).
What is this for, anyway? Oh yeah, why are we doing this? So that Deaf people can carry a device with a camera around, and every time they want to talk to a hearing person they have to mount it on something, stand in a well-lighted area and sign into it? Or maybe someday have special clothing that can recognize their hand gestures, but nothing for their facial gestures? I’m sure that’s so much better than decent funding for interpreters, or teaching more people to sign, or hiring more fluent signers in key positions where Deaf people need the best customer service.
So, yeah. I'm not saying that technology like this is fundamentally bad or won't end up helping people somewhere down the line, or that the inventors are terrible people, but this whole problem is a lot more complex then the link makes it out to be.
posted by damayanti at 10:46 AM on May 10, 2016 [24 favorites]
This tech is relevant to my interests, so the FPP and informed opinions given or linked are much appreciated.
posted by comealongpole at 11:31 AM on May 10, 2016 [1 favorite]
posted by comealongpole at 11:31 AM on May 10, 2016 [1 favorite]
To be fair, the facial expression thing could easily be solved by the signer wearing a bulky mask already attached to a computer, which would not be at all cumbersome or creepy....
In another world, a couple of students are working on a hat with a microphone and a pair of robot arms for signing for the hearing but ASL illiterate. That is also going well.
posted by GenjiandProust at 11:32 AM on May 10, 2016 [1 favorite]
In another world, a couple of students are working on a hat with a microphone and a pair of robot arms for signing for the hearing but ASL illiterate. That is also going well.
posted by GenjiandProust at 11:32 AM on May 10, 2016 [1 favorite]
Bit one-sided, isn't it?
continued muttering about how interesting it is that D/deaf people continue to bear the burden of communication between themselves and hearing people. hm.
posted by lineofsight at 11:43 AM on May 10, 2016 [5 favorites]
continued muttering about how interesting it is that D/deaf people continue to bear the burden of communication between themselves and hearing people. hm.
posted by lineofsight at 11:43 AM on May 10, 2016 [5 favorites]
As what seems to be the token mefi Deaf voice, I hate this. So much. I would like to congratulate hearing people on getting together, once again, to perpetuate the proud history of hearing people deciding what it is exactly that Deaf people need, without any input or participation from Deaf people. The hearing people in my life keep sharing this on social media, and I've had to cringe every single day, as all of you stoke your savior complexes.
The problem is not that we have two different languages. The problem is that hearing people refuse to acknowledge any form of communication beyond spoken language to be viable or intelligent, and look down upon any Deaf person who does not attempt to communicate through somehow twisting their words into vocalization. The simplest solution, if you want to communicate with a Deaf person as an able-bodied hearing person, is for you to learn sign language; the playing field is perfectly level that way, and as much as you have been socialized to think otherwise, there is no obligation for you to solely communicate through vocalization. Barring time constraints, I'd like to point out that Deaf people have been doing everything they can to communicate with hearing people already. We carry around pads and pens and write notes back and forth. We ask you for your texting number so we can text you in a conversation instead - I have a friend who hosts parties with mixed Deaf/hearing crowds by asking everyone to bring a device to connect to a group chat. We're perfectly happy to gesture. The point is, there are a billion and one ways for people to communicate. I can't find a citation, but if you put two Deaf people in a room who have different languages, they'll work out a way to communicate within 30 minutes on average.
But no, you think spoken language is the only way. And you're so immersed in your own linguistic superiority, that you make our lives hell by imposing your thinking on Deaf people, every day, on a daily basis.
No one should be applauding this. Period.
posted by Conspire at 11:51 AM on May 10, 2016 [39 favorites]
The problem is not that we have two different languages. The problem is that hearing people refuse to acknowledge any form of communication beyond spoken language to be viable or intelligent, and look down upon any Deaf person who does not attempt to communicate through somehow twisting their words into vocalization. The simplest solution, if you want to communicate with a Deaf person as an able-bodied hearing person, is for you to learn sign language; the playing field is perfectly level that way, and as much as you have been socialized to think otherwise, there is no obligation for you to solely communicate through vocalization. Barring time constraints, I'd like to point out that Deaf people have been doing everything they can to communicate with hearing people already. We carry around pads and pens and write notes back and forth. We ask you for your texting number so we can text you in a conversation instead - I have a friend who hosts parties with mixed Deaf/hearing crowds by asking everyone to bring a device to connect to a group chat. We're perfectly happy to gesture. The point is, there are a billion and one ways for people to communicate. I can't find a citation, but if you put two Deaf people in a room who have different languages, they'll work out a way to communicate within 30 minutes on average.
But no, you think spoken language is the only way. And you're so immersed in your own linguistic superiority, that you make our lives hell by imposing your thinking on Deaf people, every day, on a daily basis.
No one should be applauding this. Period.
posted by Conspire at 11:51 AM on May 10, 2016 [39 favorites]
This is a senior-year design project - thousands of projects like this get churned out every year and maybe one actually becomes something useful. No one did any analysis of whether deaf people need this - it was done to show the students' technical competence.
If senior design projects actually went anywhere we'd have pool-playing robots in every bar in America by now.
No one should be applauding this. Period.
I get what you're saying but I think you're overestimating the amount of political speech contained in this project. I think it's at least an effort towards a project with social meaning, even if misguided. It beats another wireless 3d mouse, pool-playing robot or coloured marble sorter.
posted by GuyZero at 11:58 AM on May 10, 2016 [4 favorites]
If senior design projects actually went anywhere we'd have pool-playing robots in every bar in America by now.
No one should be applauding this. Period.
I get what you're saying but I think you're overestimating the amount of political speech contained in this project. I think it's at least an effort towards a project with social meaning, even if misguided. It beats another wireless 3d mouse, pool-playing robot or coloured marble sorter.
posted by GuyZero at 11:58 AM on May 10, 2016 [4 favorites]
Oh my bad, they're sophomores, not seniors. Good job on getting that design project done early guys.
posted by GuyZero at 12:00 PM on May 10, 2016 [2 favorites]
posted by GuyZero at 12:00 PM on May 10, 2016 [2 favorites]
Without going into too much detail, I can offer a little context here: these are UW undergrads who developed this technology in a new campus makerspace. The milieu around makerspaces and tech hacks in general has oversaturated this particular effort; undergrads are encouraged to be entrepreneurial, to 'change the world,' and to help those [perceived to be] in need.
This project suffers from a number of unfortunate blind spots: undergrads looking for a cool engineering project and then seeing how it could 'benefit society;' UW and the makerspace looking for a project to hype, and so beating the world-changing innovation drum loud and proud in press releases galore; this then feeds back to the undergrads, who start to believe their own hype and their engineering awards etc.
I hope that these kids hear and understand the perspective of the deaf community and learn about their own privilege and to talk to the folks they think they're helping BEFORE starting a project next time. I don't really hold out hope that UW or the makerspace will learn such a lesson. They'll be off looking for the next thing to hype.
And thanks for providing your perspective, Conspire.
posted by Existential Dread at 12:01 PM on May 10, 2016 [12 favorites]
This project suffers from a number of unfortunate blind spots: undergrads looking for a cool engineering project and then seeing how it could 'benefit society;' UW and the makerspace looking for a project to hype, and so beating the world-changing innovation drum loud and proud in press releases galore; this then feeds back to the undergrads, who start to believe their own hype and their engineering awards etc.
I hope that these kids hear and understand the perspective of the deaf community and learn about their own privilege and to talk to the folks they think they're helping BEFORE starting a project next time. I don't really hold out hope that UW or the makerspace will learn such a lesson. They'll be off looking for the next thing to hype.
And thanks for providing your perspective, Conspire.
posted by Existential Dread at 12:01 PM on May 10, 2016 [12 favorites]
Kate's criticisms, while on the mark, feel like they have a bit of a linguist's bias towards using pre/descriptive techniques for translation -- whereas the state of the art is (slowly) moving towards gathering more and more data, and ignoring grammar as much as feasible and just letting the model learn the structure as it learns the mappings between phrases. That said, I agree with what she said about its ultimate utility (mostly none), I just feel it's a little too pessimistic on the ability for translation to work eventually (e.g. the "nobel prize" reference).
The best use I can think of for this kind of translation is for transcription for non signers to get the gist of videos that are only signed. e.g. in the same way I might turn on auto transcription for a non-english video (and probably with the same hilarious results :-)).
This is *much* more interesting if you drop the translation aspect entirely. Consider a deaf user interacting with other deaf users in VR. In that case, a way to monitor the motion of the whole upper body including the arms/fingers with very high precision solves a real problem, and is very practical. Of course, this is totally applicable to everyone else too. However if you refined the tech based on deaf users signing to each other in VR, it seems like you would naturally gravitate towards interaction devices that had tons of precision for everyone.
I feel like just a high precision monitoring device probably doesn't win prizes though.
posted by smidgen at 12:01 PM on May 10, 2016 [3 favorites]
The best use I can think of for this kind of translation is for transcription for non signers to get the gist of videos that are only signed. e.g. in the same way I might turn on auto transcription for a non-english video (and probably with the same hilarious results :-)).
This is *much* more interesting if you drop the translation aspect entirely. Consider a deaf user interacting with other deaf users in VR. In that case, a way to monitor the motion of the whole upper body including the arms/fingers with very high precision solves a real problem, and is very practical. Of course, this is totally applicable to everyone else too. However if you refined the tech based on deaf users signing to each other in VR, it seems like you would naturally gravitate towards interaction devices that had tons of precision for everyone.
I feel like just a high precision monitoring device probably doesn't win prizes though.
posted by smidgen at 12:01 PM on May 10, 2016 [3 favorites]
I get what you're saying but I think you're overestimating the amount of political speech contained in this project. I think it's at least an effort towards a project with social meaning, even if misguided. It beats another wireless 3d mouse, pool-playing robot or coloured marble sorter.
I've seen this shared on Facebook every single day by hearing people for weeks. Please don't minimize the impact this is having on Deaf people and the way we are perceived.
posted by Conspire at 12:03 PM on May 10, 2016 [20 favorites]
I've seen this shared on Facebook every single day by hearing people for weeks. Please don't minimize the impact this is having on Deaf people and the way we are perceived.
posted by Conspire at 12:03 PM on May 10, 2016 [20 favorites]
I hope that these kids hear and understand the perspective of the deaf community and learn about their own privilege and to talk to the folks they think they're helping BEFORE starting a project next time. I don't really hold out hope that UW or the makerspace will learn such a lesson. They'll be off looking for the next thing to hype.
So this sums it up a lot better than I did - Conspire, I hope you don't take my comments as dismissal of your comments. I don't have much to say about the politics or society of deafness and I don't think these kids do either, which is indeed problematic if you're going to build something for deaf people.
Please don't minimize the impact this is having on Deaf people and the way we are perceived.
Fair enough. Sorry about that.
posted by GuyZero at 12:04 PM on May 10, 2016 [2 favorites]
So this sums it up a lot better than I did - Conspire, I hope you don't take my comments as dismissal of your comments. I don't have much to say about the politics or society of deafness and I don't think these kids do either, which is indeed problematic if you're going to build something for deaf people.
Please don't minimize the impact this is having on Deaf people and the way we are perceived.
Fair enough. Sorry about that.
posted by GuyZero at 12:04 PM on May 10, 2016 [2 favorites]
"Mute"??
Also, what everyone else said about the difficulty of mapping ASL onto spoken sounds. I don't speak ASL, but as far as I can tell, given the amount you'd have to slow down/simplify your communication for the gloves to understand it, you might as well just get a pen and paper. Not to mention how condescending it is to use Sesame Street ASL and then claim that you're piggybacking on "how deaf people talk to each other already."
Yes, they're only sophomores, but that's exactly the time for someone to say "kids, your ambition is outpacing your sensitivity, and you need to take a step back."
posted by babelfish at 12:35 PM on May 10, 2016 [1 favorite]
Also, what everyone else said about the difficulty of mapping ASL onto spoken sounds. I don't speak ASL, but as far as I can tell, given the amount you'd have to slow down/simplify your communication for the gloves to understand it, you might as well just get a pen and paper. Not to mention how condescending it is to use Sesame Street ASL and then claim that you're piggybacking on "how deaf people talk to each other already."
Yes, they're only sophomores, but that's exactly the time for someone to say "kids, your ambition is outpacing your sensitivity, and you need to take a step back."
posted by babelfish at 12:35 PM on May 10, 2016 [1 favorite]
God I love this thread. As a hearing engineer learning ASL, I've been seeing this everywhere as "tech is saving Deaf people!" and haven't had a good response until now.
posted by the agents of KAOS at 12:38 PM on May 10, 2016 [5 favorites]
posted by the agents of KAOS at 12:38 PM on May 10, 2016 [5 favorites]
It could also be used for taking dictation? I speak faster than I type, I imagine most people who can sign are able to sign faster than they type as well?
posted by museum of fire ants at 12:49 PM on May 10, 2016 [1 favorite]
posted by museum of fire ants at 12:49 PM on May 10, 2016 [1 favorite]
God I love this thread. As a hearing engineer learning ASL, I've been seeing this everywhere as "tech is saving Deaf people!" and haven't had a good response until now.
I posted it to HN yesterday. It seems to have been pretty thoroughly raked over the coals there. But it made the front page and they had a good discussion. So I thought I would toss it out here and see if it failed to get deleted.
You can check the conversation on HN if you want more ammo: Linky
posted by Michele in California at 12:57 PM on May 10, 2016
I posted it to HN yesterday. It seems to have been pretty thoroughly raked over the coals there. But it made the front page and they had a good discussion. So I thought I would toss it out here and see if it failed to get deleted.
You can check the conversation on HN if you want more ammo: Linky
posted by Michele in California at 12:57 PM on May 10, 2016
I've wanted to learn ASL for a while, but haven't been able to. There are no open classes available at my university, and I couldn't make much progress using online materials. Like most people, I need interaction to learn a language. Also, the modality is so different than what I'm used to that I can't tell when I'm doing it right.
I honestly think that a lot of the hype around this project is because of misconceptions about ASL--namely, the idea that ASL is based on English, or is even just English words translated into gesture. If you believe that, then it sounds a lot more realistic.
Like, there are two huge research problems that need to be solved for a device like this to work: "speech recognition" for ASL, and automatic translation of transcribed ASL into English. Speech recognition of languages that are well-studied like English took decades of research and massive amounts of data to get where it is now. And automatic translation--well, Google translate is state of the art, and look at how well it does with any structurally different or under resourced language.
If they could make progress on the first problem, I would like to see some kind of learning aid for hearing people like me--perhaps something that could provide basic feedback on pronunciation, like the crude speech recognition used by programs like Rosetta. But, given how frustrating Rosetta is, it would have to be very good to get even that far.
The technology itself could be very useful, but the framing has been terrible. I really do wonder how much hype there would have been if people knew more about ASL.
posted by Kutsuwamushi at 1:01 PM on May 10, 2016 [2 favorites]
I honestly think that a lot of the hype around this project is because of misconceptions about ASL--namely, the idea that ASL is based on English, or is even just English words translated into gesture. If you believe that, then it sounds a lot more realistic.
Like, there are two huge research problems that need to be solved for a device like this to work: "speech recognition" for ASL, and automatic translation of transcribed ASL into English. Speech recognition of languages that are well-studied like English took decades of research and massive amounts of data to get where it is now. And automatic translation--well, Google translate is state of the art, and look at how well it does with any structurally different or under resourced language.
If they could make progress on the first problem, I would like to see some kind of learning aid for hearing people like me--perhaps something that could provide basic feedback on pronunciation, like the crude speech recognition used by programs like Rosetta. But, given how frustrating Rosetta is, it would have to be very good to get even that far.
The technology itself could be very useful, but the framing has been terrible. I really do wonder how much hype there would have been if people knew more about ASL.
posted by Kutsuwamushi at 1:01 PM on May 10, 2016 [2 favorites]
SignAll is a startup that employs Deaf and signing people, is actively soliciting the involvement of the signing community, and is focused on a camera-based technology that doesn't require the use of any awkward hardware*. The camera-based approach also means it could just as easily be provided by a hearing person (or by a business) as by the signing person, and it incorporates facial expressions and body movement, not just hand and arm movements.
It's a fairly new startup—and based in Hungary—so I couldn't find much about the company's reception in the Deaf community, but at least on the surface it seems like a much better informed, more inclusive approach.
* The prototype is a little bulky, but in theory it could be scaled down to a smartphone (or laptop) with stereo cameras, which is where smartphone cameras are headed anyway in order to take 3D pictures.
posted by jedicus at 1:04 PM on May 10, 2016 [3 favorites]
It's a fairly new startup—and based in Hungary—so I couldn't find much about the company's reception in the Deaf community, but at least on the surface it seems like a much better informed, more inclusive approach.
* The prototype is a little bulky, but in theory it could be scaled down to a smartphone (or laptop) with stereo cameras, which is where smartphone cameras are headed anyway in order to take 3D pictures.
posted by jedicus at 1:04 PM on May 10, 2016 [3 favorites]
I don't know much about ASL, although I confess I did cringe a bit when I first saw this video making the rounds on Facebook. The first thing that occurred to me was that it's funny, suggesting that deaf people might love to wear ridiculous talking gloves everywhere they went. The second thing that occurred to me is that it would actually make more sense to ask hearing people to walk around wearing light-up t-shirts with a little chip inside which magically displayed text captions for everything they said.
In fact, the more I think about it, the fewer objections I can imagine to that alternate course of action. Why not? Because deaf people are the ones who should wear silly things in order to communicate? That seems to be the only honest answer anybody could give.
also really I kinda do want a caption / translation t-shirt, that would be really cool I think, and probably useful in a lot of circumstances
posted by koeselitz at 1:06 PM on May 10, 2016 [1 favorite]
In fact, the more I think about it, the fewer objections I can imagine to that alternate course of action. Why not? Because deaf people are the ones who should wear silly things in order to communicate? That seems to be the only honest answer anybody could give.
also really I kinda do want a caption / translation t-shirt, that would be really cool I think, and probably useful in a lot of circumstances
posted by koeselitz at 1:06 PM on May 10, 2016 [1 favorite]
undergrads looking for a cool engineering project and then seeing how it could 'benefit society;'
So much this and I've been there. It seems to me that its at least as worthwhile a technical area as others in natural language processing - also, not something some undergrads are gonna solve in a deep way - but framing as "look what we're doing for the Deaf! Technology! Human interest!" is kind of insulting.
posted by atoxyl at 1:06 PM on May 10, 2016
So much this and I've been there. It seems to me that its at least as worthwhile a technical area as others in natural language processing - also, not something some undergrads are gonna solve in a deep way - but framing as "look what we're doing for the Deaf! Technology! Human interest!" is kind of insulting.
posted by atoxyl at 1:06 PM on May 10, 2016
I wish that, amongst all of the press-releasy, click-baitey, hype for this, there were some actual details about the process that Pryor and Azodi went through when developing this, and how the technology *actually* works (including current limitations, ideas for future refinements, etc).
I giggled to myself when, in the video, they talked about how they are changing the lives of 70 million people (there aren't 70 million people who use ASL, and even if pareto-principle levels of decent translation are feasibile, they are definitely discounting the challenge of translating to other sign languages).
I also wonder what their "extensive work with individuals in the disabled community" actually entailed. Both the blogger at katies.online, and Conspire here state that this kind of system is not what deaf people want or need. While it's entirely possible that they didn't actually talk to any deaf folks and just magicked the idea out of their own heads, I think it would be interesting to know if they did work with folks in the Deaf community, who think that this is a step in the right direction (but understand the technical limits of an undergrad project).
As an engineer, I really kind of hate the term "Engineer's Disease," though I think this is a place where it could apply. BUT, I wonder if this is more of a case of "Marketer's Disease," in that the technology is necessarily super limited (both in terms of what is feasible now, and, as long as it relies on the speaker wearing some kind of prothesis, in terms of what good it could ever do even if "perfected), but everything printed about it (except for some blog critiques) has been so breathless and uncritical.
When I look at this tech, I see the potential as being something like those augmented reality glasses that are supposed to be able to translate printed signs from one language to another. It would be great if motion capture technology could get to the point where a hearing person could say, wear glasses and an earpiece, with the glasses interpreting the gestures and facial expressions of sign language into an audio language that the hearing person could understand.
But I understand that the science isn't there yet, so I can see how these dudes thought that this was at least feasible as a proof of concept, and I can also see how the contest structure could encourage them to buy into their own hype for the purposes of the video/marketing and discourage discussion about limitations, whether this is really the right direction.
posted by sparklemotion at 1:09 PM on May 10, 2016
I giggled to myself when, in the video, they talked about how they are changing the lives of 70 million people (there aren't 70 million people who use ASL, and even if pareto-principle levels of decent translation are feasibile, they are definitely discounting the challenge of translating to other sign languages).
I also wonder what their "extensive work with individuals in the disabled community" actually entailed. Both the blogger at katies.online, and Conspire here state that this kind of system is not what deaf people want or need. While it's entirely possible that they didn't actually talk to any deaf folks and just magicked the idea out of their own heads, I think it would be interesting to know if they did work with folks in the Deaf community, who think that this is a step in the right direction (but understand the technical limits of an undergrad project).
As an engineer, I really kind of hate the term "Engineer's Disease," though I think this is a place where it could apply. BUT, I wonder if this is more of a case of "Marketer's Disease," in that the technology is necessarily super limited (both in terms of what is feasible now, and, as long as it relies on the speaker wearing some kind of prothesis, in terms of what good it could ever do even if "perfected), but everything printed about it (except for some blog critiques) has been so breathless and uncritical.
When I look at this tech, I see the potential as being something like those augmented reality glasses that are supposed to be able to translate printed signs from one language to another. It would be great if motion capture technology could get to the point where a hearing person could say, wear glasses and an earpiece, with the glasses interpreting the gestures and facial expressions of sign language into an audio language that the hearing person could understand.
But I understand that the science isn't there yet, so I can see how these dudes thought that this was at least feasible as a proof of concept, and I can also see how the contest structure could encourage them to buy into their own hype for the purposes of the video/marketing and discourage discussion about limitations, whether this is really the right direction.
posted by sparklemotion at 1:09 PM on May 10, 2016
I don't know why undergrad (or even postgrad, or whatever) design projects get hyped like this on a regular basis. Or, rather, I understand, but I wish it would go away. It's really not helping anyone other than university PR, maybe.
I feel a bit bad for the students involved that this has blown up as much as it has- the combination of hype and backlash has got to be less than optimal for their state of mind.
posted by BungaDunga at 1:13 PM on May 10, 2016
I feel a bit bad for the students involved that this has blown up as much as it has- the combination of hype and backlash has got to be less than optimal for their state of mind.
posted by BungaDunga at 1:13 PM on May 10, 2016
As an engineer, I really kind of hate the term "Engineer's Disease," though I think this is a place where it could apply.
I already agreed with the person who said it up thread but I think what happens a lot is that some technical people come up with a project they want to do at heart because it's technically interesting. But for one reason or another they end up feeling like they have to couch it in - or actually sell it to themselves as - some TED-attendee "changing the world" shit.
posted by atoxyl at 1:16 PM on May 10, 2016 [2 favorites]
I already agreed with the person who said it up thread but I think what happens a lot is that some technical people come up with a project they want to do at heart because it's technically interesting. But for one reason or another they end up feeling like they have to couch it in - or actually sell it to themselves as - some TED-attendee "changing the world" shit.
posted by atoxyl at 1:16 PM on May 10, 2016 [2 favorites]
But for one reason or another they end up feeling like they have to couch it in - or actually sell it to themselves as - some TED-attendee "changing the world" shit.
Yes! On Kickstarter at least there is a mandatory section for "Risks and Challenges" for a project, and most Kickstarter videos that I like address it in the video too. I think part of the flaw here is that this hype video has zero time spent on why this isn't the absolutely most perfect thing. So it leaves Pryor and Azodi looking like, well, smug assholes.
I'm willing to believe in a version of the world where Pryor and Azodi understand why this technology isn't actually "life changing," or "giving people the right to communicate with the world," but the parameters of the contest discouraged them from going into it in the promo video.
posted by sparklemotion at 1:25 PM on May 10, 2016 [1 favorite]
Yes! On Kickstarter at least there is a mandatory section for "Risks and Challenges" for a project, and most Kickstarter videos that I like address it in the video too. I think part of the flaw here is that this hype video has zero time spent on why this isn't the absolutely most perfect thing. So it leaves Pryor and Azodi looking like, well, smug assholes.
I'm willing to believe in a version of the world where Pryor and Azodi understand why this technology isn't actually "life changing," or "giving people the right to communicate with the world," but the parameters of the contest discouraged them from going into it in the promo video.
posted by sparklemotion at 1:25 PM on May 10, 2016 [1 favorite]
I also wonder what their "extensive work with individuals in the disabled community" actually entailed.
I'll put a paycheque down on a wager for "fuck all."
posted by mandolin conspiracy at 1:28 PM on May 10, 2016
I'll put a paycheque down on a wager for "fuck all."
posted by mandolin conspiracy at 1:28 PM on May 10, 2016
So that Deaf people can carry a device with a camera around, and every time they want to talk to a hearing person they have to mount it on something, stand in a well-lighted area and sign into it?
Computers used to take up a whole room too.
There are a bunch of problems that need to get solved in order for this to be a real product. But one of the biggest is just getting a computer to read and analyze hand movement and be able to understand it. Once that piece exists, you can work on solving the other problems (size, context, accuracy, etc.).
I worked in retail sales and then auto sales for something like 5 years and in that time, I helped out deaf customers three times (other than just showing someone where something was) and every single time was a huge PITA on both sides. Now, I'm the type of person that when a customer with special needs comes in and everyone else runs away, I jump in with both feet because it's an opportunity to really make a positive impact and it's challenging. One of those was a phone call (using a relay service) so knowing ASL wouldn't have helped me there and twice in five years is not likely to make me want to pick up ASL as a second language. In both cases, the other person could read lips reasonably well and that helped speed things up a lot because they could write out questions/responses and I could give them my whole sales spiel vocally. But there is still a language barrier because it IS a different language. While both I and the other person understood that we were both trying our best but we both got frustrated.
A fully fledged version of these, even if it's just a direct translation, would have made those interactions so much easier for both of us. Now, I only helped out two deaf customers over five years so if it comes down to it, I can deal and I'll do everything I can to make everything as good as it can be. A deaf person is going to have a frustrating time every single time they need to deal with a person that doesn't know ASL which in some cases is going to mean every damn day. Is it fair that the burden falls on the deaf? No, but it's not really realistic to expect everyone else to learn ASL either. I'm perfectly happy helping people via pen and paper or enthusiastic gestures. It's frustrating but it doesn't happen very often so I'll deal with it but other than this technology, there aren't really any good solutions.
posted by VTX at 1:32 PM on May 10, 2016
Computers used to take up a whole room too.
There are a bunch of problems that need to get solved in order for this to be a real product. But one of the biggest is just getting a computer to read and analyze hand movement and be able to understand it. Once that piece exists, you can work on solving the other problems (size, context, accuracy, etc.).
I worked in retail sales and then auto sales for something like 5 years and in that time, I helped out deaf customers three times (other than just showing someone where something was) and every single time was a huge PITA on both sides. Now, I'm the type of person that when a customer with special needs comes in and everyone else runs away, I jump in with both feet because it's an opportunity to really make a positive impact and it's challenging. One of those was a phone call (using a relay service) so knowing ASL wouldn't have helped me there and twice in five years is not likely to make me want to pick up ASL as a second language. In both cases, the other person could read lips reasonably well and that helped speed things up a lot because they could write out questions/responses and I could give them my whole sales spiel vocally. But there is still a language barrier because it IS a different language. While both I and the other person understood that we were both trying our best but we both got frustrated.
A fully fledged version of these, even if it's just a direct translation, would have made those interactions so much easier for both of us. Now, I only helped out two deaf customers over five years so if it comes down to it, I can deal and I'll do everything I can to make everything as good as it can be. A deaf person is going to have a frustrating time every single time they need to deal with a person that doesn't know ASL which in some cases is going to mean every damn day. Is it fair that the burden falls on the deaf? No, but it's not really realistic to expect everyone else to learn ASL either. I'm perfectly happy helping people via pen and paper or enthusiastic gestures. It's frustrating but it doesn't happen very often so I'll deal with it but other than this technology, there aren't really any good solutions.
posted by VTX at 1:32 PM on May 10, 2016
All I mean is, there's a huge graveyard of dumb student tech ideas. Most of them are told that they're actually dumb ideas before they hit thousands of eyeballs. Sometimes, the idea blows up, and they get to be told loudly, by everyone, how dumb their dumb idea is, rather than being told quietly.
I feel much worse for deaf people who have to deal with so many people's ignorance and misunderstanding of what ASL even is, with clueless people showing up to tell them how they will Save Them.
but other than this technology, there aren't really any good solutions.
Kinect-style full-body high-resolution realtime 3D imaging would be leaps and bounds better for this situation, since you could just install one in front of a customer service desk, rather then requiring deaf people wear dorky gloves that don't work because you need to capture facial expressions to even begin to get all the information you need to make a faithful interpretation.
posted by BungaDunga at 1:40 PM on May 10, 2016 [2 favorites]
I feel much worse for deaf people who have to deal with so many people's ignorance and misunderstanding of what ASL even is, with clueless people showing up to tell them how they will Save Them.
but other than this technology, there aren't really any good solutions.
Kinect-style full-body high-resolution realtime 3D imaging would be leaps and bounds better for this situation, since you could just install one in front of a customer service desk, rather then requiring deaf people wear dorky gloves that don't work because you need to capture facial expressions to even begin to get all the information you need to make a faithful interpretation.
posted by BungaDunga at 1:40 PM on May 10, 2016 [2 favorites]
If it were just something that was not useful to me, I (and every other Deaf person in my social group) would not be reacting so violently against this video. Rather, this video serves to further marginalize our language, culture, and position in society as people with full agency as opposed to objects of pity. Maybe that was not the intent of these students, and maybe they did not anticipate that it would blow up like this. But they still released this video out into an openly ableist society that took full advantage of this video to further deny the agency of Deaf people, a historical oppression that has continued over centuries. At very least, people need to understand that the anger of Deaf people is justified, rather than wagging fingers at them for making mountains over molehills.
A deaf person is going to have a frustrating time every single time they need to deal with a person that doesn't know ASL which in some cases is going to mean every damn day. Is it fair that the burden falls on the deaf? ...other than this technology, there aren't really any good solutions.
My frustration is not the communication break-down. I've spent every single day of my life honing these skills, and thanks to these experiences, I would very much consider myself more of an expert in bridging communication gaps than most hearing people. Where I am frustrated is with the refusal of hearing people to let me exercise these skills because they think I'm lesser, stupid, incapable, and in need of assistance. Which stretches the range from directly refusing to speak to me, or by poisoning the well in the first place by patronizing us by trying to make us into a learning example for you. For instance, I can't tell you the number of times a day I have to say "speak normally" to a hearing person who enthusiastically mouths every syllable they say in the misguided idea that it helps me lipread better - when my lipreading skills have been entirely practiced upon normal articulation in the first place. Let us tell you how to bridge the communication gap, instead of assuming there's no good solutions - we've practiced this skill every single day of our lives.
posted by Conspire at 1:50 PM on May 10, 2016 [10 favorites]
A deaf person is going to have a frustrating time every single time they need to deal with a person that doesn't know ASL which in some cases is going to mean every damn day. Is it fair that the burden falls on the deaf? ...other than this technology, there aren't really any good solutions.
My frustration is not the communication break-down. I've spent every single day of my life honing these skills, and thanks to these experiences, I would very much consider myself more of an expert in bridging communication gaps than most hearing people. Where I am frustrated is with the refusal of hearing people to let me exercise these skills because they think I'm lesser, stupid, incapable, and in need of assistance. Which stretches the range from directly refusing to speak to me, or by poisoning the well in the first place by patronizing us by trying to make us into a learning example for you. For instance, I can't tell you the number of times a day I have to say "speak normally" to a hearing person who enthusiastically mouths every syllable they say in the misguided idea that it helps me lipread better - when my lipreading skills have been entirely practiced upon normal articulation in the first place. Let us tell you how to bridge the communication gap, instead of assuming there's no good solutions - we've practiced this skill every single day of our lives.
posted by Conspire at 1:50 PM on May 10, 2016 [10 favorites]
So as not to derail overmuch, if anyone wants to memail me with ideas for how to improve day to day communications with my wife's deaf, but later in life so that she isn't knowledgeable in ASL and only has some basic 'fingerspelling' (my new vocab word for the day, thanks for that) skills, grandmother that would be something I'd very much appreciate. Dedicated laptop running transcription software didn't work all that great the last time we tried it. /derail
posted by RolandOfEld at 2:09 PM on May 10, 2016 [1 favorite]
posted by RolandOfEld at 2:09 PM on May 10, 2016 [1 favorite]
Time for another chorus of "Nothing about us without us. Nothing about us without us. NOTHING ABOUT US WITHOUT US."
posted by Lexica at 2:14 PM on May 10, 2016 [7 favorites]
posted by Lexica at 2:14 PM on May 10, 2016 [7 favorites]
Really, if they ever find intelligent alien life out there, they need to send an all-Deaf team of astronauts. My friends and I routinely play a version of charades where you're not allowed to even gesture (because that would be too easy) but have to express concepts with our facial expressions alone and our hands behind our back. Meanwhile, whenever hearing people encounter a Deaf person, they're all like "but they can't speak! There's no good way to communicate! All is lost!" And I'm just stunned at how different our attitudes and understandings of what communication fundamentally is are.
posted by Conspire at 2:44 PM on May 10, 2016 [13 favorites]
posted by Conspire at 2:44 PM on May 10, 2016 [13 favorites]
And we actually play this game with Deaf children a lot, because it helps them develop their non-verbal/signed communication skills. Like, I'm not kidding when I say Deaf people are trained experts in bridging communication gaps. Or when I say Deaf Culture is an actual culture.
posted by Conspire at 2:50 PM on May 10, 2016 [11 favorites]
posted by Conspire at 2:50 PM on May 10, 2016 [11 favorites]
"This is *much* more interesting if you drop the translation aspect entirely. Consider a deaf user interacting with other deaf users in VR. In that case, a way to monitor the motion of the whole upper body including the arms/fingers with very high precision solves a real problem, and is very practical. Of course, this is totally applicable to everyone else too. However if you refined the tech based on deaf users signing to each other in VR, it seems like you would naturally gravitate towards interaction devices that had tons of precision for everyone."My first thought, after disgust with the fremdschämenly arrogant failure to listen to Deaf people about this, the smug douchiness, and the shallow yet slick bullshitery, was that the underlying infrastructure could maybe be neat for teaching sign? I'm trying to pick up some BSL from my partner and it seems to be unusually difficult to learn sign languages outside of classes taught by a competent language teacher that not everyone has access to and, particularly outside of ASL, human teacher independent resources for learning sign are remarkably sparse.
Maybe gloves like these could be taught a dictionary of signs and be used to give a student feedback on how well they're signing? I'm thinking something like Rosetta Stone that would teach vocabulary and syntax by connecting pictures of objects and situations with videos of signing, which a student could interact with in a hands only approximation of the sign language being learned. The gloves would replace Rosetta Stone's headphones, microphone, and keyboard. Like any other, a mature teaching program would be incapable of teaching real fluency, but like Rosetta Stone could maybe give a student of a new sign language a similar running start, hearing or otherwise?
posted by Blasdelb at 4:36 PM on May 10, 2016 [1 favorite]
Instead of encouraging engineering students to "change the world" in makerspaces could we please encourage them to do co-design and/or take classes in the social sciences and/or do community service
we are raising such lopsided people
posted by gusandrews at 4:48 PM on May 10, 2016 [7 favorites]
we are raising such lopsided people
posted by gusandrews at 4:48 PM on May 10, 2016 [7 favorites]
My friends and I routinely play a version of charades where you're not allowed to even gesture (because that would be too easy) but have to express concepts with our facial expressions alone and our hands behind our back.
I'm intrigued. I tried to imagine doing this, I can only picture myself conveying emotions. How would you express a concept?
posted by storybored at 6:25 PM on May 10, 2016
I'm intrigued. I tried to imagine doing this, I can only picture myself conveying emotions. How would you express a concept?
posted by storybored at 6:25 PM on May 10, 2016
I'm intrigued. I tried to imagine doing this, I can only picture myself conveying emotions. How would you express a concept?
Lots of ways, we use different approaches. You can express something round, for instance, by puffing out your cheeks. Or something sudden by making a "PAH" morpheme and jolting back. You can connect it to pre-existing concepts - like, one of the concepts I got in a previous game was "plant", so I pretended to be the sunflower from Plants vs. Zombies. You can use your eyes to manipulate objects in space. The possibilities are limitless.
posted by Conspire at 6:31 PM on May 10, 2016 [1 favorite]
Lots of ways, we use different approaches. You can express something round, for instance, by puffing out your cheeks. Or something sudden by making a "PAH" morpheme and jolting back. You can connect it to pre-existing concepts - like, one of the concepts I got in a previous game was "plant", so I pretended to be the sunflower from Plants vs. Zombies. You can use your eyes to manipulate objects in space. The possibilities are limitless.
posted by Conspire at 6:31 PM on May 10, 2016 [1 favorite]
I mean, as a more concrete example, let's say you get the concept "Golden Parachute". So you could start off by miming a frog - everyone knows how to do that, just puff out your cheeks at various intervals and seem a bit out of it. Once they have that, establish what a frog eats - stick out your tongue and grab an imaginary fly, and chew on it. Once they have fly, convert it to the actual concept of flying by looping your head around and raising your nose into the air. Once they have that, drop down your head rapidly and then stop it in a soft way that puffs out your cheeks until they get parachute. Then you just have to connect it to the concept of golden - you can do that by referencing how bright something is by shielding your eyes from it.
It's not that hard, really. It's just that, again, hearing people aren't really trained in non-verbal communication compared to Deaf people. I've played this with hearing people who are just learning to sign on my team before, and they can never clue into the concepts and what I'm doing as fast as the Deaf people can.
posted by Conspire at 6:51 PM on May 10, 2016 [9 favorites]
It's not that hard, really. It's just that, again, hearing people aren't really trained in non-verbal communication compared to Deaf people. I've played this with hearing people who are just learning to sign on my team before, and they can never clue into the concepts and what I'm doing as fast as the Deaf people can.
posted by Conspire at 6:51 PM on May 10, 2016 [9 favorites]
That is very cool.
posted by storybored at 6:53 PM on May 10, 2016
posted by storybored at 6:53 PM on May 10, 2016
"The simplest solution, if you want to communicate with a Deaf person as an able-bodied hearing person, is for you to learn sign language"
I tried to learn sign language off and on (whenever I could get instruction) for years. I have books, I have flash cards. I still suck at learning ASL, just as badly as I sucked at trying to learn Spanish and French. I suck at learning languages--my brain refuses to speak any. I can glean words here and there when someone speaks/signs, but I sure as hell can't communicate back with you in any language I've tried to learn. I couldn't tell you my own damn name if you asked it of me in another language (even though I have a name sign, so come on, dumbass jenfullmoon), I freeze. I thought the language freezing issue wouldn't be an issue if my mouth wasn't a factor, but it continued to be. I had to IM my deaf coworker all the time anyway.
I've tried, but apparently I'm too stupid to learn how to communicate with others in their own language. There's just something fucking wrong with my brain on this topic. So much as I get "why don't you learn THEIR language," I tried and I was never able to manage it. Not everyone can and some folks are just better at picking up other languages than others.
I make no judgments on this technology whatsoever (can't watch the video right now at work, but I wonder exactly how well this would work in practice--I'd guess not that efficiently?), but hell, if there's any way to make it easier for people to communicate in any way, I can't really badmouth it in principle.
posted by jenfullmoon at 7:38 PM on May 10, 2016
I tried to learn sign language off and on (whenever I could get instruction) for years. I have books, I have flash cards. I still suck at learning ASL, just as badly as I sucked at trying to learn Spanish and French. I suck at learning languages--my brain refuses to speak any. I can glean words here and there when someone speaks/signs, but I sure as hell can't communicate back with you in any language I've tried to learn. I couldn't tell you my own damn name if you asked it of me in another language (even though I have a name sign, so come on, dumbass jenfullmoon), I freeze. I thought the language freezing issue wouldn't be an issue if my mouth wasn't a factor, but it continued to be. I had to IM my deaf coworker all the time anyway.
I've tried, but apparently I'm too stupid to learn how to communicate with others in their own language. There's just something fucking wrong with my brain on this topic. So much as I get "why don't you learn THEIR language," I tried and I was never able to manage it. Not everyone can and some folks are just better at picking up other languages than others.
I make no judgments on this technology whatsoever (can't watch the video right now at work, but I wonder exactly how well this would work in practice--I'd guess not that efficiently?), but hell, if there's any way to make it easier for people to communicate in any way, I can't really badmouth it in principle.
posted by jenfullmoon at 7:38 PM on May 10, 2016
You can express something round, for instance, by puffing out your cheeks. Or something sudden by making a "PAH" morpheme and jolting back.
Huh. This is utterly fascinating. I don't get how "round" connects to "puffed out cheeks" or "Sudden" to "PAH", but that speaks precisely to your point about expertise in cross-medium communication, I suppose.
posted by CrystalDave at 9:02 PM on May 10, 2016
Huh. This is utterly fascinating. I don't get how "round" connects to "puffed out cheeks" or "Sudden" to "PAH", but that speaks precisely to your point about expertise in cross-medium communication, I suppose.
posted by CrystalDave at 9:02 PM on May 10, 2016
« Older Vacillating mind inputs through text and video... | Rita Moreno drops the mic Newer »
This thread has been archived and is closed to new comments
posted by tobascodagama at 10:16 AM on May 10, 2016 [1 favorite]