Associative Musical Visual Intelligence

June 1, 2008 6:11 PM

"Associative Musical Visual Intelligence (or "amvi" for short) is a type of intelligence that's difficult enough to define, let alone test. Many creative people can associate across sensory domains: they "hear" hints of shapes and can "taste" the essense of colors. At its most extreme this phenomenon is called synesthesia. However, I believe that creative people subconsciously employ elements of synesthesia every day when attempting to think of things in new ways. This is a logic test that attempts to measure one's ability to correlate musical phrases with abstract shapes and symbols."

From Jake Mandell, creator of the tonedeaf test (previously), the rhythm test, and the adaptive pitch test, and researcher at the music and neuroimaging lab at Beth Israel/Harvard Medical School in Boston.

posted by carsonb (40 comments total)

I like this. I'll have to check it out when I'm not making 'above the table' dollars.

Also: No sound card. [WTF!? DAMN, EMPLOYER!! grumblegrumblemutter]
posted by Minus215Cee at 6:20 PM on June 1, 2008

Very interesting. I scored 75% overall, which was a bit lower than I expected. I found it took me the first three or four questions to understand the way the symbols and sound clips worked (sometimes there were two parts with a gap between, and it confused me) so I got the first few wrong straight off, but after that I think I got them all right. Cool.
posted by loiseau at 6:23 PM on June 1, 2008

Oops, a couple corrections:
Jake Mandell is a former member of the music and neuroimaging lab, and this is the undeleted original MeFi 'previously' post I meant to link to.
posted by carsonb at 6:25 PM on June 1, 2008

This is a logic test that attempts to measure one's ability to correlate musical phrases with abstract shapes and symbols Squares and Triangles and Arrows.

Music to me is much more about emotion and personal projection than it is inclined planes and red triangles.

75%.
posted by clearly at 6:26 PM on June 1, 2008

I don't get it. I every question wrong before I quit. I must find a test that determines one's ability to relate celebrity body parts to the faces of friends ("hey you have John Leguizamo's nose!").
posted by SassHat at 6:29 PM on June 1, 2008

er...got every question wrong.
posted by SassHat at 6:29 PM on June 1, 2008

Hmmm. . . this was interesting, but I'm not sure this has anything to do with synesthesia. It's just a matter of translating a few differnent aspects of music into a made-up, but intuitive language (a phrase with ascending pitch represented by an upwards arrow, etc.)

90%
posted by flotson at 6:31 PM on June 1, 2008 [2 favorites]

None of those damn sounds were red, except the few cymbal bits. The rest were sort of grey and maybe bluish.
posted by cobaltnine at 6:37 PM on June 1, 2008 [1 favorite]

I got 100% and it seemed really easy to me. This has nothing to do with synesthesia, it's just correlating pitch and tone with symbols.
posted by naju at 6:39 PM on June 1, 2008 [1 favorite]

I got 95% and so, independently, did my girlfriend. We both got the exact same percentage distribution on the categories in the final score, leading me to believe that we both got exactly the same one wrong.

Which would be interesting, except, as naju points out, this test as NOTHING to do with synesthesia or the ability of visualize sound as shapes and colors or vice-versa.

It's a very simple pattern recognition test where, once you figure out the intended pattern, it's very easy. Vertical positioning correlates to change in pitch, color correlates to differences in timbre, shape distortion correlates to small-scale melodic change, and shape change correlates to large-scale melodic change. Add a couple other fairly obvious curveballs (shape complexity equated to rhythmic complexity once) and it's not that hard. The one my gf and I both got wrong was probably unintentionally vague.

Leading me to think that this doesn't test much other than:
1) The ability to recognize differences such as why "up" is not the same as "down", and more importantly,
2) The ability to figure out what answer the person who made the test wants, as opposed to any "correct" answer, which is why I think some people aren't doing so well.

Which I find kind of annoying. If I wanted to take a test where the key is figuring out what they want you to say, I'd go back to grade school.

Real synesthetic visualization is much more personal, much more complex, and much harder to pin down. Yes, upward pitch movement does grossly correspond to upward spatial movement for a lot of people, but it's hardly so straightforward as that, and the relations of color and shape to melody are much richer and stranger. This test is like a "spot the differences" cartoon in the Sunday comics advertising itself as a means of identifying latent Picassos.
posted by kyrademon at 7:13 PM on June 1, 2008

85%, between excellent and exceptional, but I had to replay most of the excerpts several times. (I know that I clicked on one of the wrong answers by mistake, so maybe that balanced out). Still, this is pretty interesting, as I'm a visual/kinetic thinker, and have a lousy aural memory, but love music.
posted by maudlin at 7:17 PM on June 1, 2008

I got 100% and it seemed really easy to me. This has nothing to do with synesthesia, it's just correlating pitch and tone with symbols.

Same. It's pretty dumb.
posted by Jairus at 7:23 PM on June 1, 2008

65%, but I coulda done better if I'd cared to repeat the clips. I'm afraid I just picked at random if I didn't immediately intuit an answer. I agree that this is nothing to do with synesthesia, because the sounds had a totally different shape and color for me than the symbology he was using. (not a synesthete)
posted by mumkin at 7:24 PM on June 1, 2008

I managed a 0%, answering all questions.
posted by Mr. President Dr. Steve Elvis America at 7:40 PM on June 1, 2008

Woohoo! I am exceptional! Let's all smoke a bowl!
posted by Foam Pants at 7:50 PM on June 1, 2008 [1 favorite]

I think that it is related to synesthesia in that it is a topic that is about personal interpretations of music that is discussed almost exclusively on the Internet.

90% as well.
posted by dobie at 8:01 PM on June 1, 2008

Here Niefield puts Mandell's theories into practice. Better MP3 of a cool song here.

I can't believe I'm the first person to post this. I'm too young to be the only one old enough to remember when Carpark records was cool. (Cool on the internets, that is.)

Also, this test is stupid, because I failed miserably yet consider myself a creative, musically inclined dude.)
posted by scope the lobe at 8:05 PM on June 1, 2008

Also, this test is stupid, because I failed miserably yet consider myself a creative, musically inclined dude.

Sorry, but the test indicates that you're not creative after all. Now get yourself a pair of dockers and a cubicle.
posted by Mr. President Dr. Steve Elvis America at 8:12 PM on June 1, 2008

The implication here is that every one with synesthesia would agree that fives are green, twos are red and sevens are blue.

This is the musical version of ascii art where if you squint just right you can see it.

Carrying out the analogy, here's a GIF.
posted by Kid Charlemagne at 8:15 PM on June 1, 2008

Huh. I'm a musician and got a 90%, but I'm with everyone who says this has absolutely nothing to do with synesthesia. Actually, this looks like a computer programmer's take on creative intelligence (read: extremely left-brain). "Oh, I know, we'll measure creative intelligence with a multiple choice test!" If people can't see the irony in this, they should have their grant funding revoked.
posted by Baby_Balrog at 8:18 PM on June 1, 2008 [3 favorites]

I initially viewed the shapes while listening to the music and attempted to reason a match -- this only worked, however, when the arrangement of objects mirrored the tone changes. I tried again by listening while closing my eyes and was able to first establish my own shape and then could find the best match in the shapes presented -- I almost think the test would be better if the shapes didn't appear until the sound was complete.
posted by VulcanMike at 8:45 PM on June 1, 2008

That having been said, I agree with Kid C's observation about agreement in the "language" of synesthesia:
The implication here is that every one with synesthesia would agree that fives are green, twos are red and sevens are blue.

Though there may be natural intersections in language with certain types of synesthesia -- someone with emotion-color synesthesia might experience feelings of lust and the color red natually -- I always thought of sound->color/shape as a personally abstract form of synesthesia, much in the way that each person with the ordinal-linguistic personification form might experience a different personality for the letter "e".
posted by VulcanMike at 9:11 PM on June 1, 2008

Finally took it all the way through and have decided the test was fun (got an 80%), but it has little to do with synesthesia. I honestly don't see how it's much more abstract than sheet music -- well, perhaps sheet music rolled into a square, but it's still following the same basic changes that are core to simply perceiving music -- if a chord shifts a few octaves, most people can tell what direction it's moved. If I change instruments, it's also pretty obvious. If you've done bad at this test and you don't have exceptionally bad sound perception, does it really mean that you have an inherent Associative Musical Visual Intelligence deficiency?
posted by VulcanMike at 9:43 PM on June 1, 2008

Actually, this looks like a computer programmer's take on creative intelligence (read: extremely left-brain).

It's very right-brained of you to buy into that sort of pseudoscience.
posted by TheOnlyCoolTim at 9:56 PM on June 1, 2008 [1 favorite]

I didn't like the symbols at all, they were very ugly and ungainly. I started to get the hang of it because it's a pretty literal representation of the music, but until then, I got pretty much every one wrong. meh
posted by The Light Fantastic at 10:12 PM on June 1, 2008

I thought his "test" mostly tested your familiarity with the conventions of sheet music.
posted by fshgrl at 10:14 PM on June 1, 2008

Yeah, this is pretty dumb. It's a test of how well you can figure out what they 'mean' with these symbols.
posted by delmoi at 10:44 PM on June 1, 2008

90% for myself, but it didn't seem to have much to do with synesthesia to me. But then I let me 6-year-old son (who had just woken up from a nightmare) try it (to calm him down). He has demonstrated several tendencies toward synesthesia in the past, sometimes even considering instrumental music to be narrative in its visualizations. He got fewer right than I did, and his overall take on things: "I like the music, but they keep getting the pictures wrong. That music [the last one] should be spikes with green and then red." And the like....
posted by zeugitai_guy at 10:45 PM on June 1, 2008 [1 favorite]

KidC, that toccata and fugue makes me froth at the mouth. 1.mumpty million page views and a ton of positive comments and it's just so wrong wrongity wrong wrong.

Ok, I understand that the easy way to do that sort of visualization is to pipe MIDI through some graphics routines et voila: Virtual Player Organ. But they could have picked a MIDI that had slightly more expressiveness than Robbie the Freaking Robot. I had to cleanse my palette by manually syncing a no-name compilation T&FinD- with the video, which was instructive enough.

Maybe I'm just getting old - I don't think that would have bothered me as much even a decade ago.
posted by Kyol at 10:46 PM on June 1, 2008

Well at least y'all didn't get a 30%. At some point I started to feel like monkeys tapping on keys randomly would have had better luck. Maybe it's a total lack of familiarity with sheet music....that's a comforting thought.

As for whether or not this relates to synesthesia, I kept waiting for the rationale. I'm sure the guy's written a paper detailing it, but a very (very!) quick look didn't produce anything other than more tests. I'd really like to know the backstory to this, as well as the results.

(Also, I felt it was kind of bad form to tell me my results were being tabulated and sent on-even anonymously--AFTER the test was over. Unless I missed a warning upfront, that's no bueno for me)
posted by librarylis at 12:10 AM on June 2, 2008

I feel I should defend what he's doing against the bashing you're all giving him. He says he is testing an ability which 'at its extreme' is synaesthesia. So all you wacky creative synaesthetes can disagree about the colours or shapes as much as you want (btw synaesthetes don't generally agree on their associations anyway). He's testing something more basic, which is the ability to associate differences in colour, or shape or elevation with differences in the music. Now I wouldn't say the test is perfect, since anyone who has studied music theory will have an advantage in their familiarity with common theory-test 'tropes' (like recognising contreamotion, formal repetition). Nevertheless, the test does genuinely track cross-sensory associations. You can't infer the right answers very quickly (though this may be possible). You need first of all to intuitively perceive that the tune is 'mutating', that the pitch is 'rising', that timbre change implies a different colour.

Note that a tune does not literally 'mutate' or modulate because there is no fixed identity unless we perceive it so. Nor do pitches literally 'get higher'. In other cultures, high and low pitches are described as weak and strong (among the African Bashi), white and black (among the Lau of the Solomon Islands), or small and big (among the African Basongye) (Merriam 1964). Despite the differences here, these alternative descriptions make sense to us. They are both highly analogous to our descriptions of high and low, and indicate a universality of applying intermodal metaphors to musical pitch. That is, it's likely that quite fundamental neural associative processing is at work here.

If I were to improve the test, I would have things like 4 beats/lines go with squares, where 3 beats/lines go with triangles; differentiate 'square' cuts with smooth ones; maybe crank up the colour/pitch association (so higher phrases are lighter in colour rather than higher up); 'rough' timbres with rough looking textures; large shapes with slow movement etc... More along the lines of his example where the 'echoing' (repeated smaller and lighter) square went with the echoing beat.
posted by leibniz at 1:51 AM on June 2, 2008

By the way, there is a likely link between these capacities and creativity, but it's much much less direct than he is implying here (hello, does this test divergent thinking at all?).

Associating this test with creativity is just going to annoy people, but like a lot of psychological testing, it does very nicely fuck with the insecurities of arty people. (Phew I scored 90%, my lack of contributions to society are validated!)
posted by leibniz at 2:13 AM on June 2, 2008

Leibniz, I think the main objection that has been raised is that, in fact, you *don't* need to intuit the right answers. The answers here seem to imply that the answers can be inferred with relative ease by anyone who has had traditional Western musical training, and that anyone who hasn't finds them almost impossible to intuit.

The would seem to imply that the test is more a test of musical training than intuitive connection.

Perhaps a better test for that would use a number of different tropes such as the ones you raise (e.g., big and small or black and white for pitch instead of just up and down), with far more choices and two mirror images both being acceptable answers for any given question (in case someone does the reverse association). But even that probably wouldn't be a very accurate test for this.
posted by kyrademon at 2:25 AM on June 2, 2008

I don't think I need to bother with this. I don't think you can test this sort of skill with a webpage.
posted by chuckdarwin at 2:35 AM on June 2, 2008

The answers here seem to imply that the answers can be inferred with relative ease by anyone who has had traditional Western musical training

I have had a very minimal amount of musical training. What little there was of it terminated when I was 13 years old; we were not taught to read charts, for instance. Indeed, I still cannot read charts or play any melodic instrument.

However, I managed to get 90% on this test. Maybe a basic understanding of the physics of what is happening is helping me?

The musical 'visualisation' videos I see generally cause me to feel disquiet as they are made by people who don't seem to have even a basic understanding of whatever it is that this test is attempting to measure. This one by Michel Gondry is an exception, it might not get everything right, but it is fairly consistent.
posted by asok at 3:34 AM on June 2, 2008 [1 favorite]

I only got 80. I'd have done better, but the purple notes all tasted wrong.
posted by Goofyy at 3:54 AM on June 2, 2008

Agreed. I don't think this a test of anything other than the ability to hear pitch, timbre, and remember short musical phrases, and to quickly catch on to the simplified notation system the author made up. Interestingly, what time signature/tempo 'looks' like = not important I guess. I think on both the visual and musical sides, there was a strong suggestion that the strengths of the software tools that were used had a lot of influence on the scope of the idea.

I did well because I'm practiced in those three listening skills, and when I saw his first question, I consciously concluded that his symbols were intended to consistently describe the most simple elements of music. I can think metaphorically about sound, but there was no need to. People who did poorly, my guess is a) you probably just thought there was something more complex going on in the mind of the test-writer than there actually was, or b) one of those three skills I outlined above is challenging for you, which is a normal state of affairs unless you have practiced them at some point. I think it was particularly confusing because the test-writer heavily frames it as a test of creative association, getting you ready to be as intuitive as possible, and then he presents a very straightforward test of logic (with a dash of ear training). Maybe it's secretly a test of the test-taker's ability to dismiss incorrect contextual information?

Standard musical notation is a collection of consistently-applied abstract visual symbols, exactly like this is, but with much more granularity. I'd be interested in knowing whether people who report signs of synaesthesia having nothing to do with the grammar of music find it easier to master music notation than people who don't. My sight-reading and -writing are both poor, and when I use notation I always feel distracted by its bad infodesign, but some of it surely developed as an accretion of the ways that composers 'saw' what they wanted to communicate.

I suppose the test could be partially fixed by having the pictures use different imagery in every question instead of a consistent symbolic language, but of course then you'd be directly confronting the problem that no two people are likely to "see" music identically, and evaluating the correctness of an answer would be difficult. The current approach conceals that flaw with a different flaw.
posted by Your Time Machine Sucks at 4:12 AM on June 2, 2008

I started off by looking at the images while listening to the audio ... wrong every time. Stopped looking at the images = 100% ... overall score with the mistakes, 80%.

What is this really testing?
posted by aldus_manutius at 8:02 AM on June 2, 2008

100% -- it should be harder.
posted by swift at 9:37 AM on June 2, 2008

I'm pretty sure a computer programmer would not have measured creative intelligence this way. Good programmers rely very much on the intuitive vision of how the whole problem aligns with the facilities available to solve it, long before they go into all the reductionistic detail work, and even the half-decent ones are sensitive to mismatches between representation and reality (like when you have multiple-choice options which neither exhaust the whole field of possibilities nor systematically exclude one another). Every programmer I know would scoff.

I agree that this is a logic test, a figure-out-what-I'm-thinking test, not a creativity test and indeed not in a form in which you can test creativity. Jake Mandell, let not thy left brain know what thy right brain doeth; then thy right brain, which seeth in secret, at least has a fighting chance to reward thee openly (if only by reminding you that the experimental design of which your left brain is so proud can in no way be aligned with what you say you want to measure).
posted by eritain at 10:59 AM on June 3, 2008