Comments on: I have no idea what perceptual insight is, but this is pretty interesting

I have no idea what perceptual insight is, but this is pretty interesting

TheDonF — Sun, 16 Nov 2008 09:29:34 -0800

An Introduction to Sine-Wave Speech Play the first sound and you'll probably hear nothing but squeaks and bleeps. Play the second one and then go back to the first. Cool!

By: TravellingDen

TravellingDen — Sun, 16 Nov 2008 09:42:15 -0800

That's awesome...

By: Muddler

Muddler — Sun, 16 Nov 2008 09:42:51 -0800

Nifty examples of the theory. Anyone that has has done any foreign travel or had to cut through unfamiliar accents knows this phenomenon. We'll also watch the face of a person and gestures to help decode unclear speech. Try randomly raging at someone "I LOVE YOU" with fists in the air and you'll note the confusion - our brains try to decode the full message in total context, and discordance confuses us. Written communications, such as-email and text messages, lack a detailed web of visual context, which is one reason, I think, it becomes so easy to unintentionally flame others on the Internet. Now, if someone could come up with a website to train me in the various Indian accents, I might be able to understand tech support.

By: SPrintF

SPrintF — Sun, 16 Nov 2008 09:43:06 -0800

OK, that's just eerie!

By: DU

DU — Sun, 16 Nov 2008 09:45:13 -0800

I heard "Islam is the light"

By: andoatnp

andoatnp — Sun, 16 Nov 2008 09:46:01 -0800

I would be curious how much of what I (we?) hear in the scrambled audio clip has been influenced by just previously listening to the normal audio sample. It seems to work perfectly if you listen to the normal sample moments before the other one, but if you wait an hour, a day, etc. would it still work?

By: fearfulsymmetry

fearfulsymmetry — Sun, 16 Nov 2008 09:46:05 -0800

Got half the message first time through... all of it on the second, without the hint. And I got the picture too.

By: iamkimiam

iamkimiam — Sun, 16 Nov 2008 09:54:37 -0800

Awesome. Also reminds me of "I am sitting in a room" from this thread.

By: fcummins

fcummins — Sun, 16 Nov 2008 09:59:58 -0800

If you liked that, you might like this. Stimuli used in a psycholinguistics experiment, but again, compare your perception before and after listening to the original. Not by me, but by Franck Ramus.

By: dunkadunc

dunkadunc — Sun, 16 Nov 2008 10:04:54 -0800

I got half of it the first time around, and after hearing the original I was able to pick out consonants that I never picked up on the first time. What's really weird is that after hearing and understanding that one original, I can understand all the other sine-wave sentences, without having heard their originals.

By: brundlefly

brundlefly — Sun, 16 Nov 2008 10:06:24 -0800

Whoa. Pretty neat stuff.

By: dunkadunc

dunkadunc — Sun, 16 Nov 2008 10:07:33 -0800

Having heard several examples of sine-wave speech, your perceptual system has tuned into this form of distortion, so as to be able to perceive new sine-wave speech sentences more clearly. Ahh, there you go. Got the picture first time around, though.

By: scblackman

scblackman — Sun, 16 Nov 2008 10:10:01 -0800

This is absolutely fascinating. It has got me thinking that this example of the perception of degraded speech also explains how parents learn to understand the mis-shapen words of their toddlers. I find myself, after receiving a "training" example (e.g., my wife saying, "Anika learned how to say 'MetaFilter' and "mathowie'"), I am able to immediately recognize and understand what she means when she says "Meffimerr" or "Maowie". I never realized that it was such a well-described function of the human brain and senses. Once again, the child is the teacher of the parent. Excellent post.

By: chicken nuglet

chicken nuglet — Sun, 16 Nov 2008 10:10:07 -0800

Hah, that was cooler than I thought it would be.

By: Brandon Blatcher

Brandon Blatcher — Sun, 16 Nov 2008 10:10:21 -0800

When I played the first sound, I heard one of the Mac Os voices (deraged maybe?) saying "It was a sunny day and I was sure we were going to the park." Not perfect by close. Yeah, on playing the other sounds, I can definitely make out the sentences through the distortion. What's the big mystery?

By: isopraxis

isopraxis — Sun, 16 Nov 2008 10:12:22 -0800

This is so cool. I had always wondered exactly what the imperial probe droid on Hoth was transmitting. Now it all makes sense: 'The camel was kept in a cage at the zoo, The camel was kept in a cage at the zoo.'

By: Quietgal

Quietgal — Sun, 16 Nov 2008 10:12:28 -0800

I listened to the first sound clip repeatedly, without listening to the second "clear" version, and found that it became increasingly intelligible without any external hints. By the 3rd hearing I could start to make out the words and after 5 repetitions it was completely understandable. And I wasn't particularly trying - it just resolved into a proper sentence all by itself. (As far as I could tell from my top-level awareness; my brain was probably doing some pretty slick processing, but I wasn't putting any conscious "decoding" effort into it.) I wonder if Brits would pick up on it any faster - the speaker has an English accent, which might have made it a little harder for my American brain to interpret (park = "pahk"). Also, some parts of the other samples I couldn't make out, even after hearing the clear version - it seems like some words are just easier to grok than others (no spoilers, but it's probably a function of the actual sounds rather than how common the word is). Random Sunday morning AI-babble: would this be one type of test for sentience? The ability to decode something rapidly, without additional input? In other words, on-the-fly pattern matching with extreme distortion/filters? Or can machines do this easily (like in 5 iterations)? Obviously, software was used to generate the distorted sound files, but how easy is it to reverse-engineer the sound if you have no information about it? Any AI researchers out there?

By: WolfDaddy

WolfDaddy — Sun, 16 Nov 2008 10:14:14 -0800

I understood the first clip perfectly the first time, even to the fact that the speaker was female and that to my American ears she had an accent. I don't get it.

By: Beautiful Screaming Lady

Beautiful Screaming Lady — Sun, 16 Nov 2008 10:22:14 -0800

WolfDaddy: Most naive listeners hear this as a set of simultaneous whistles, or science fiction sounds Me and you, see, we're the clever ones.

By: Saxon Kane

Saxon Kane — Sun, 16 Nov 2008 10:24:24 -0800

PAUL IS DEAD.

By: Bookhouse

Bookhouse — Sun, 16 Nov 2008 10:24:57 -0800

Very interesting stuff. Thanks.

By: Solon and Thanks

Solon and Thanks — Sun, 16 Nov 2008 10:31:01 -0800

Yeah, on playing the other sounds, I can definitely make out the sentences through the distortion. What's the big mystery? It's not a mystery you're trying to solve, it's an example of how your brain learns to recognize speech. From the link: "As you listen to these four examples, you may find that you get better at understanding the sine-wave speech first time around. This is an example of perceptual learning. Having heard several examples of sine-wave speech, your perceptual system has tuned into this form of distortion, so as to be able to perceive new sine-wave speech sentences more clearly."

By: knile

knile — Sun, 16 Nov 2008 10:33:24 -0800

Robert Remez has also studied SWS. One study presented here. It's a favorite for comparing the brain's responses to speech and nonspeech, because it's "both". One study by Dehaene-Lambertz, et al, may be available to you here.

By: Navelgazer

Navelgazer — Sun, 16 Nov 2008 10:33:32 -0800

I buried Paul. Seriously, though, at least I wasn't the only one getting the messages the first time around.

By: Brandon Blatcher

Brandon Blatcher — Sun, 16 Nov 2008 10:42:28 -0800

As you listen to these four examples, you may find that you get better at understanding the sine-wave speech first time around. Nah, I understood it was speech the first time it was heard. Didn't hear any chirps or beeps, just distorted speech.

By: chelegonian

chelegonian — Sun, 16 Nov 2008 10:47:03 -0800

I work with people with learning difficulties, many of whom have very idiosyncratic modes of speech. After several years of working with these people i can now understand most of what is said to me and i find it easier to adjust to a new persons mode of speech. I thought it was interesting that new staff member could listen to a request that i had heard as "can we go out" and not understand it at all, i suppose this would explain why.

By: Poolio

Poolio — Sun, 16 Nov 2008 10:47:39 -0800

Wow. It's a schooner.

By: Gyan

Gyan — Sun, 16 Nov 2008 10:53:51 -0800

This phenomenon, perceptual insight, is also probably applicable to musical cognition. Time for some 12-tone torture again.

By: brundlefly

brundlefly — Sun, 16 Nov 2008 11:02:05 -0800

Wow. It's a schooner. Ha! I was thinking the same thing.

By: aliceinreality

aliceinreality — Sun, 16 Nov 2008 11:09:16 -0800

I thought the first said "It was a funny day and the children were going to the park," on the first listen. I think being one letter away from the real sentence is pretty good, and that I now deserve a cookie.

By: rebel_rebel

rebel_rebel — Sun, 16 Nov 2008 11:18:19 -0800

I just saw Remez give a talk this weekend, and he played an example of SWS and asked if anyone could identify the sentence. Someone immediately shouted "The watchdog gave a warning growl!" I think it must be like Name That Tune for psycholinguists.

By: Jpfed

Jpfed — Sun, 16 Nov 2008 11:22:45 -0800

This VST plugin can be used to produce the same effect on arbitrary audio. As far as I can tell, it partitions the audio signal into chunks, where each zero-crossing marks a new chunk. Then, it replaces each chunk with a simple wave (saw, triangle, square, sine) of your choice, with the same peak amplitude as the original audio chunk. The company that originally wrote it is defunct, and it's listed as freeware, so you can probably download to your heart's content.

By: Jpfed

Jpfed — Sun, 16 Nov 2008 11:25:04 -0800

Wait, they use formant tracking, which is very different. Ignore my silliness.

By: peacheater

peacheater — Sun, 16 Nov 2008 11:25:52 -0800

I understood most of the first one the first time around -- except for the children bit.

By: jack_mo

jack_mo — Sun, 16 Nov 2008 11:39:40 -0800

Didn't hear any chirps or beeps, just distorted speech. Really? I'm not too surprised that folk could make out the speech on first listening to the sine wave clips, but to not hear all the crazy swooping synthesiser-type sounds at all is inconceivable to me and my ears. They sounded like samples from some lost Louis & Bebe Barron soundtrack, until I heard the clear version.

By: nickyskye

nickyskye — Sun, 16 Nov 2008 11:39:48 -0800

Seeing a pattern through the distortion seems to be a double-edged skill, one that endows a person with the ability to make sense where there seems to be little but also to make sense where there may be none, such as in pareidolia or apophenia. Apophenia is the experience of seeing patterns or connections in random or meaningless data. The term was coined in 1958 by Klaus Conrad, who defined it as the "unmotivated seeing of connections" accompanied by a "specific experience of an abnormal meaningfulness". Pareidolia is a type of apophenia involving the finding of images or sounds in random stimuli. I think the brain is hard-wired to decode but decoding, like any perception may have a number of variations, such as being accurate, reality based, a misinterpretation or imaginary. Stimulating and informative post. Thanks TheDonF.

By: jfrancis

jfrancis — Sun, 16 Nov 2008 11:58:06 -0800

After one or two I was pretty accurate on the rest

By: penguinliz

penguinliz — Sun, 16 Nov 2008 12:04:53 -0800

I understood the speech the first time round, but it wasn't until I played the undistorted version I realised I know the woman who is speaking (she's a friend of mine who once worked at the MRC-CBU). I don't know if that makes a difference, although it is kinda weird to think that a bunch of Mefites are listening to her talk. When I was a poor student I used to get extra cash by being part of the volunteer panel they use for these experiments, and I think I may have done the noise-vocoded speech experiment. Nice to see something I helped with pop up on the blue.

By: Slack-a-gogo

Slack-a-gogo — Sun, 16 Nov 2008 12:06:57 -0800

Way cool.

By: UbuRoivas

UbuRoivas — Sun, 16 Nov 2008 12:08:27 -0800

ok, guys, quit joking around. there's no difference between the clips at all, and this is a big prank at my expense, right? right?

By: Solon and Thanks

Solon and Thanks — Sun, 16 Nov 2008 12:15:44 -0800

Nah, I understood it was speech the first time it was heard. Didn't hear any chirps or beeps, just distorted speech. When it says "chirps or beeps", that is distorted speech.

By: nickyskye

nickyskye — Sun, 16 Nov 2008 12:32:45 -0800

Related: Check this out: Mind-reading software developed in the Netherlands can decipher the sounds being spoken to a person, and even who is saying them, from scans of the listener's brain.

By: delmoi

delmoi — Sun, 16 Nov 2008 13:04:52 -0800

Yeah, I was able to understand it before hearing the first 'normal' speech the second time I listened to the track, but not the first time.

By: turgid dahlia

turgid dahlia — Sun, 16 Nov 2008 14:24:24 -0800

I recorded myself saying "Rex! Rex! C'mere Rex! Good boy!", then sine-waved it, and played it back to my dog, and he turned inside-out. So thanks a bunch for that.

By: pointilist

pointilist — Sun, 16 Nov 2008 14:57:02 -0800

I played in a french farce where several of the characters had speech inpediments. One character could not pronounce consonants due to a cleft palate, another had an intense lisp (thuppothedly Cathtilian Thpanish). About a third ot the way through the cleft palate character refers to the Spaniard saying, "Who hant huner han a her he hay!" Cracks us all up.

By: UbuRoivas

UbuRoivas — Sun, 16 Nov 2008 14:57:25 -0800

That comment is useless without video.

By: UbuRoivas

UbuRoivas — Sun, 16 Nov 2008 14:59:33 -0800

(i was referring to TD's comment, but it applies equally to pointilist's)

By: nickyskye

nickyskye — Sun, 16 Nov 2008 15:05:55 -0800

pointilist, I've been rereading that line, "Who hant huner han a her he hay!", a bunch of times and so want to get the joke. Can you do the tedious and translate it into English for me?

By: Belle O'Cosity

Belle O'Cosity — Sun, 16 Nov 2008 15:27:55 -0800

That was interesting. I found that during all of the clips I could understand the last few words quite clearly, but the initial was distorted. I am thinking my brain quickly figured out the code and made it work by the end. Then after I repeated played it, not hearing the clear version, I could make the whole thing out. Fascinating stuff.

By: UbuRoivas

UbuRoivas — Sun, 16 Nov 2008 15:52:46 -0800

Who hant huner han a her he hay: "Who wants butter in a hurf durf way?" (as spoken by somebody with a mouth full of rich creamery butter)

By: nickyskye

nickyskye — Sun, 16 Nov 2008 17:46:47 -0800

"Who wants butter in a hurf durf way?" zomg. So funny! Good one. Oy, just looked it up on Urban Dictionary and realized it's a MeFite in joke.

By: ericbop

ericbop — Sun, 16 Nov 2008 18:40:33 -0800

So that's how Luke understood R2 so well!

By: thisisdrew

thisisdrew — Sun, 16 Nov 2008 20:00:17 -0800

this makes me think of other "brainy" stuff. ...Like Robert Krulwich's audio piece on how a human, a rat, and a spider learned to live in space, or how my brain adapted to eyeglasses when I first started wearing them. Brains are crazy things.

By: eschatfische

eschatfische — Sun, 16 Nov 2008 20:17:33 -0800

I actually experienced this when moving to the Bay Area. The BART train announcements are done using an older speech synthesis application. I couldn't understand them at all on first listen -- they were mind-bogglingly weird sounding. On the second listen, I could understand a snippet or two. But if I heard my partner or a BART driver make the announcement, the machine-intoned version would make perfect sense the next time around. On the other hand, the MUNI Metro system, which uses voice samples for their announcements, was perfectly easy to understand on first listen. Now, of course, I understand the announcements by second nature. But I always wondered if tourists had as much difficulty as I did understanding what exactly the announcements were saying. Per this, probably... but they're learning.

By: pointilist

pointilist — Sun, 16 Nov 2008 20:41:01 -0800

Sorry. "You can't understand a word he says!" After listening to the guy for 20 minutes it was, like, obvious.

By: ooga_booga

ooga_booga — Sun, 16 Nov 2008 21:09:17 -0800

It's a schooner!

By: Burhanistan

Burhanistan — Sun, 16 Nov 2008 21:16:12 -0800

Yeah, I heard the example just fine on the first listen. I do have sensitivity levels bordering on insanity, though.

By: twoleftfeet

twoleftfeet — Sun, 16 Nov 2008 21:51:03 -0800

Rd zzss sndnz foist han din weed dur necks wan. Read this sentence first and then read the next one. Now go back and read the first sentence. Makes sense now, doesn't it?

By: ghost of a past number

ghost of a past number — Mon, 17 Nov 2008 00:31:34 -0800

I completely understood all of the clips, both distorted and undistorted BEFORE clicking on the link. My inherently superior brain is no match for your cheap tricks, Matt Davis.

By: minifigs

minifigs — Mon, 17 Nov 2008 02:53:00 -0800

You're all liars. I did get the picture though...

By: quin

quin — Mon, 17 Nov 2008 10:53:34 -0800

If I ever make an animated clip involving aliens or robots or something, I want to have them speaking plain English Sine Wave. The vast majority of people would think it's intentionally nonsensical, but over time, some might start realizing that the captions actually do match up with what they are hearing. (I think that for extra fun I'd have the captions convey the essence of the spoken text without actually saying the exact same thing.) I bet that for the people that caught on to this, the first moment of realization would have them questioning their sanity for a few seconds. "Did I actually hear that? Because for a moment there, I totally understood what that thing said..." It would be mind-fuck-tastic.

By: Xoebe

Xoebe — Mon, 17 Nov 2008 11:35:49 -0800

I have never been good at decoding aural input/output. Songs I have heard for years are still gibberish to me until I see the lyrics written down, unless they are very clearly spoken/sung. I also have a hard time decoding speech. It's not a hearing problem, I have been tested, and I hear sound just fine, but there is definitely something lacking in the processing. I hadn't thought about it much until just now, but when I have trouble hearing speech, and I ask someone to repeat the sentence, much of the time I don't understand even on the second, third, or Lord forbid, the fourth repetition. It's almost as if certain combinations of sounds just don't register. What I hear, when I misunderstand - is that the person speaking is speaking clearly, but the words are nonsense. "Dad, the point of curtains is banana seventy". What? Upon repetition, it's the same nonsense. Very frustrating. My Dad is the same way. We always thought it was just encroaching deafness. The sine wave speech, in each example to me, was gibberish. After listening to the clear speech, I could go back and hear the patterns hidden in the sine wave speech, but my initial recognition never improved. Gibberish each time. Interesting.

By: paisley henosis

paisley henosis — Mon, 17 Nov 2008 16:58:41 -0800

Worked exactly as advertised for me, and also for my wife, who was sitting across the room, and had no idea what was going on, until she herd the first clip the second time, and loudly exclaimed her surprise.

By: patrick rhett

patrick rhett — Mon, 17 Nov 2008 19:44:45 -0800

I heard beeps and whistles, my wife heard words. She is now, officially, better at everything.