Race and Intelligence, Redux
December 16, 2007 2:51 PM   Subscribe

About a month ago, a MeFi FPP on this article sparked a controversy here on the usefulness of the concepts of IQ and race in determining whether some ethnic groups can be shown to be intrinsically less intelligent than others. Now James Flynn, discoverer of the Flynn effect, has written a book, What is Intelligence?, that settles many of the issues of this controversy. In this week's New Yorker, Malcolm Gladwell summarizes Flynn's arguments succinctly in a review entitled "None of the Above: What IQ doesn't tell you about race."
posted by ubiquity (85 comments total) 11 users marked this as a favorite
 
Determining IQ is like determining the shape of a fire.
posted by ronin21 at 2:58 PM on December 16, 2007 [6 favorites]


How is racism controversial?
posted by cytherea at 3:01 PM on December 16, 2007 [1 favorite]


Sadly, it's unlikely to "settle the issues".
posted by Steven C. Den Beste at 3:02 PM on December 16, 2007


Those tests don't measure jack shit. I do very well on standardized tests because I can usually figure out the logic behind how questions are asked, but that doesn't prove anything other than I am a good little test taker. Some people don't test well at all and it is no reflection on their true smarts.
posted by 45moore45 at 3:04 PM on December 16, 2007 [1 favorite]


Don't tell Rupert Sheldrake this
posted by A189Nut at 3:05 PM on December 16, 2007


It is a grievous injustice that I did not have these results in my rhetorical arsenal when I was 14.
posted by enn at 3:08 PM on December 16, 2007


"Settles"? This is Metafilter, son.
posted by bonaldi at 3:09 PM on December 16, 2007 [4 favorites]


The notion that you can test someone's innate intelligence makes some pretty bold claims that you know more than anyone else about what is genetic vs environmental. And it seems pretty obvious environmental causes (e.g., lead paint*) can lower people's intelligence pretty seriously, which sort of ruins the "innate" argument right there.

I guess what I'm trying to say is "what the crap is wrong with these people? sheesh!" (the IQ testing folks).


* mmmmmm
posted by aubilenon at 3:09 PM on December 16, 2007


The real problem is we, as a species, WANT to be able to quantify "intelligence" and be able to rank ourselves against everyone else on the spectrum. The fact that it doesn't work that way hasn't kept us from doing so yet, and I doubt it will in the foreseeable future.
posted by yhbc at 3:15 PM on December 16, 2007


I am also dubious that this will settle any debates. Despite overwhelming evidence that "it depends" is the appropriate answer to virtually all divisive and controversial questions, large groups of people persist in adamantly insisting that there's one really true truth to everything.

And when what's at stake is whether or not my people are inherently smarter than those other people I don't like, well....
posted by lodurr at 3:18 PM on December 16, 2007


"these results" = the Flynn effect, of course
posted by enn at 3:18 PM on December 16, 2007


Flynn's work is brilliant.
Must be something genetic about being from New Zealand.

Seriously, what a pleasure to have a corrective to the usual genetic reductionism. Anyone who tells you they know that X% of some cognitive ability is inherited and Y% is environmental is doing bad science in the service of self-promotion. Bullshit! Bullshit!
posted by cogneuro at 3:19 PM on December 16, 2007 [1 favorite]


Two weeks ago, Flynn came to Manhattan to debate Charles Murray at a forum sponsored by the Manhattan Institute. Their subject was the black-white I.Q. gap in America. During the twenty-five years after the Second World War, that gap closed considerably. The I.Q.s of white Americans rose, as part of the general worldwide Flynn effect, but the I.Q.s of black Americans rose faster. Then, for about a period of twenty-five years, that trend stalled—and the question was why.

Murray showed a series of PowerPoint slides, each representing different statistical formulations of the I.Q. gap. He appeared to be pessimistic that the racial difference would narrow in the future. “By the nineteen-seventies, you had gotten most of the juice out of the environment that you were going to get,”


Couldn't have anything to do with, say, the G.I Bill.

Lord, do I hate Charles Murray.
posted by spiderwire at 3:22 PM on December 16, 2007


I join the accolade to Flynn. Here is to wishing that all the scientists, especially the statisticians, would stop jumping to stupid conclusions just to publish an idiotic paper in order to get tenure.
posted by francesca too at 3:33 PM on December 16, 2007


IQ tests that reflect an ability to match the correct question to a known answer may be measuring the programmability of the subject within a narrow range of focus, much like the education process. I would like to see a test that measures the ability to see ambiguity in statements, or that rank probable answers, because this feature is eliminated for standardized tests, but requires intuitive intelligence. This would handicap the programmables. This phenomenon has been joked about before by at least one genius who cited his ability to see flaws in corporate hiring test questions and had to second guess some of his responses in order to score highest.
posted by Brian B. at 3:37 PM on December 16, 2007 [1 favorite]


Forget race, an IQ score doesn't tell you much of anything.

Imagine you had the perfect test. One that could measure how fast someone learned something, how well they could visualize both concrete and abstract ideas (picures and math, for example), and any other measure of "intelligence" you care to name. Would someone who aced every category be perfect?

Of course not. Skills at taking a test, any test, don't translate perfectly to the real world. Success in life is determined by personality, dedication, and creativity (for n = success) just as much as intelligence, and all four are at the mercy of chance and opportunity. The difference between success on the test and success in reality could only widen as the test became imperfect and the link between test score and ability weakens.

Beyond that, what if you were tired the day you took the test? Sick? What if you spent the next three years doing mental exercises? The brain learns, and past performance is no indication of future results.

Our very definition of intelligence is flawed and our tests are almost completely unrelated to success in the real world (there is some correlation). Oh, and race is a stupid marker, as the effect of nature on intelligence is dwarfed by nurture.
posted by Orange Pamplemousse at 3:39 PM on December 16, 2007 [1 favorite]


Forget race, an IQ score doesn't tell you much of anything.

I don't know if you've ever taken a serious, professionally-administered intelligence test, but this isn't even remotely true. The BIA, for example, can tell you some fascinating stuff even if taken with the requisite grain of salt.

Beyond that, what if you were tired the day you took the test? Sick?

If it's being used to measure you personally, take it a different day. In the statistical sense, this makes no difference.

Our very definition of intelligence is flawed and our tests are almost completely unrelated to success in the real world (there is some correlation).

Not even remotely true. Regardless of whether this is a chicken-or-egg question, there is a very clear relationship between IQ test performance and "success" by any reasonable metric. IQ does not discriminate within domains, but it does discriminate between them.
posted by spiderwire at 3:50 PM on December 16, 2007 [1 favorite]


Here is to wishing that all the scientists, especially the statisticians, would stop jumping to stupid conclusions just to publish an idiotic paper in order to get tenure.

You know that if they don't get tenure, at most universities they get fired, right? And that if there were no tenure, it probably wouldn't affect the rate of published conclusions you think are stupid. Tenure is really neither here nor there in this discussion.
posted by grouse at 3:52 PM on December 16, 2007


Most potentially incriminating snippet of the article:

The difference, in that case, was not the fact of the children’s blackness, as a fundamentalist would say. It was the fact of their Germanness

Is that a Godwin?
posted by vorpal bunny at 3:56 PM on December 16, 2007


How much is the result of an IQ test biased by the boredom and frustration of the person taking it, and the relevance of the test to their life? I ask this because I'm bored shitless by the things - nothing is more frustrating than spending hours ticking boxes and rotating shapes in my mind and trying to figure out number patterns. When I've attempted them, after some period I've basically given up and rushed through the rest of the answers, anxious to get it over and done with and move onto something meaningful and interesting.

Other people, I suppose, are especially competitive and try extra hard to get that score.
posted by Jimbob at 3:56 PM on December 16, 2007


Jayne Mansfield was supposed to have had an IQ of 163. Now, I loves me some Jayne Mansfield, but her entire biography is an argument that intelligence and the messy details of life have only a passing familiarity with each other.
posted by Astro Zombie at 4:00 PM on December 16, 2007


This article is another refutation of the Saletan piece really illustrates how stupid it actually was.
posted by delmoi at 4:01 PM on December 16, 2007 [2 favorites]


Jimbob, are you talking about a professionally-administered IQ test, or some test you took on the internet? A diagnostic intelligence test is not a particularly competitive environment.
posted by spiderwire at 4:02 PM on December 16, 2007


I have done a professionally-administered test, and yes, I was bored and got to a point where I couldn't really bother trying very hard anymore. And it is competitive - people like to tell people "I've got an IQ of 135!" more than they like to tell people "I've got an IQ of 87!", no matter how professional the test was.
posted by Jimbob at 4:12 PM on December 16, 2007


I could have sworn Gladwell's article was FPP'd already. But I can't find it.
posted by Avenger50 at 4:12 PM on December 16, 2007


I could have sworn Gladwell's article was FPP'd already. But I can't find it.

Unlikely, since it's dated Dec. 17 2007.

I have done a professionally-administered test, and yes, I was bored and got to a point where I couldn't really bother trying very hard anymore. And it is competitive - people like to tell people "I've got an IQ of 135!" more than they like to tell people "I've got an IQ of 87!", no matter how professional the test was.

Perhaps you should consider taking a different test. The Kaufman BIT, for example, has almost none of the pictorial or pattern-matching tricks you discuss, nor does it require any writing at all -- and it correlates strongly with Stanford-Binet.

Also, by diagnostic test I mean a research test, or more precisely a test taken towards another purpose besides evaluating IQ. In either case, there's no competitive motivation since the subject's interest is in accurate diagnosis, not the outcome of the test.
posted by spiderwire at 4:19 PM on December 16, 2007


I have often wondered about the stress aspect of these tests. Isn't the faster you complete the test, the better the score? Or am I mistaken? Because any form of timing (even "you have 3 hours to complete this document") introduces an element of psychological stress, which affects individuals' performances differently (and is something that environment has a big part of).

I suppose I'm asking a question that is similar to JimBob's. Competitiveness, boredom, anxiety... How do you control for the extrinsic factors that affect a test that is trying to extract an intrinsic quality.
posted by kisch mokusch at 4:24 PM on December 16, 2007


Perhaps you should consider taking a different test.

Well...my point is, I don't really want to take a different test, because I'm not actually that interested in doing IQ tests to begin with. Just like I'm completely joyous that I will hopefully never have to sit another exam, and I love my calculator a hell of a lot more than I love doing long division on a piece of paper, and I'd rather be making music or playing with my son than doing a sudoku puzzle. Different strokes. Surely, whether people want to do an IQ test or don't want to do an IQ test is going to bias the outcome? I doubt the Kaufman BIT test is so innocuous and fun as to be equivalent to a double-blind study.
posted by Jimbob at 4:25 PM on December 16, 2007


Surely, whether people want to do an IQ test or don't want to do an IQ test is going to bias the outcome?

No, it won't — that's the point. It very clearly might bias the outcome in your individual case, but it's not a source of systemic error. As you say, people can be motivated both for and against taking the test.

I doubt the Kaufman BIT test is so innocuous and fun as to be equivalent to a double-blind study.

Well, it's at least a hell of a lot less boring than what you describe. And most of the battery isn't repetitive -- e.g. if you're being tested for auditory processing, I imagine that your ability to distinguish phrases in background noise doesn't change much depending on your mood. You either do it or don't.
posted by spiderwire at 4:34 PM on December 16, 2007


I suppose I'm asking a question that is similar to JimBob's. Competitiveness, boredom, anxiety... How do you control for the extrinsic factors that affect a test that is trying to extract an intrinsic quality.

Time constraints vary depending on the test or what's being tested for. But the answer to your question is the same as it is to JimBob's — those factors are both trivially easy to control for and not sources of systemic error.
posted by spiderwire at 4:41 PM on December 16, 2007


It very clearly might bias the outcome in your individual case, but it's not a source of systemic error.

Unless people within the categories they are testing have a different desire to complete the test. And if you're dividing your population across occupational lines, or economic lines, or national lines, or even racial lines, it's completely reasonable to think they might.
posted by Jimbob at 4:42 PM on December 16, 2007


Those tests don't measure jack shit. I do very well [...] because I can usually figure out the logic [...]

So no relation, then, between this hypothetical "intelligence" thing they claim to be measuring and your "being able to figure things out"-edness?
posted by you at 4:52 PM on December 16, 2007 [8 favorites]


Unless people within the categories they are testing have a different desire to complete the test. And if you're dividing your population across occupational lines, or economic lines, or national lines, or even racial lines, it's completely reasonable to think they might.

Assuming that there's a positive correlation between socioeconomic status and proclivity for boredom, or race and boredom (you realize what you just said, right?), which I doubt, that's still not a source of systemic error.

The error wouldn't be consistent across test regimes, nor even within a single test regime. A trivial reordering of the test battery would reveal a decline in performance not related to the content of the questions.
posted by spiderwire at 5:11 PM on December 16, 2007


For me, this is the takeaway:

An I.Q., in other words, measures not so much how smart we are as how modern we are.

Fascinating article. Thanks for posting.
posted by MythMaker at 5:21 PM on December 16, 2007


"Settles"? This is Metafilter, son.

Yeah, he was banned.
posted by homunculus at 5:22 PM on December 16, 2007


Well, boredom isn't really the issue. Desire to complete the test is. As I said, different strokes. Just to continue to play with stereotypes, a farm hand or an artist or a boilermaker might have less interest in doing a test like this than, say, a computer programmer or a lawyer or a teacher. We know occupation has an association (obviously) with income. We know that occupation also has some association with ethnicity (see today's thread about the decline in black ownership of farms). What you do for a living, and for recreation, has some association with, for example, how creative you are, how much you like manual work, whether you like being indoors sitting at a desk or working outdoors. These things are going to influence whether you're the sort of person who makes the most of an IQ test, or whether you can't be bothered. Which might lead to biased estimates of "intelligence" based on those demographic categories.

My best friend is a chronic pot-smoker, who worked on a dredging barge, then as a furniture mover before he recently got a job as a crane rigger. I know for a fact that he would hate doing an IQ test with every cell in his body. I'm waiting for my PhD thesis to be examined. This guy whips my ass in chess within 12 moves every time we play, and displays a kind of logic and reasoning that often escapes me. I would not be surprised if I did much better on an IQ test than him, but I also know in many ways he is much more intelligent than me.
posted by Jimbob at 5:23 PM on December 16, 2007


Your argument by anecdote is unpersuasive, and you have your causality reversed. And I assume that by "complete the test" you mean "take the test at all," because in the latter case the performance decline would still be obvious.

But, giving you the benefit of the doubt — even if the correlation you posit exists, it's still not a source of systemic error. Taking the previous reductio ad absurdum one step further: even if your hypothetical is correct, there will be a glaring statistical disparity between your friend's performance answering a question from an IQ test and his performance answering that same question when asked in isolation.

So long as he understands the question, his motivation simply does not affect the outcome of a cognition test in a systemic fashion. That's a trivial matter of testing methodology. It may be the case that his cognitive abilities are substantively different from yours, which is the issue that Flynn raises — but while you now hint at that possibility, it's qualitatively distinct from your initial objection, which was proclivity towards test-taking.
posted by spiderwire at 5:44 PM on December 16, 2007


What would be really great would be to have an organization that allows people who are preoccupied with how smart they are to take a test and, if they score high enough, be allowed to join. (Naming is tricky, but I suggest a greek word you might see on the window of a furniture store.) This would serve several purposes:

1) They can frequently remind you of their accomplishment;

2) It provides something to pad out a personal ad with;

3) It provides an air of unearned authority (see also engineering degree) which they feel allows them the final word on any subject.

It has the additional benefit of letting the rest of us know who in the hell to avoid at all costs.
posted by maxwelton at 5:46 PM on December 16, 2007 [3 favorites]


What would be really great would be to have an organization that allows people who are preoccupied with how smart they are to take a test and, if they score high enough, be allowed to join... It has the additional benefit of letting the rest of us know who in the hell to avoid at all costs.

Aww, law school's not that bad.
posted by spiderwire at 5:50 PM on December 16, 2007 [1 favorite]


I'm always amazed at the angst surrounding IQ tests.

IQ tests, like any test, measure the thing they're designed to measure in a fairly narrow sense. IQ tests measure "IQ." They are not some kind of magical window into a person's potential success, wisdom, or trivia. IQ tests have a non-zero correlation with intelligence (something thus far we haven't had a good definition of, but can kinda sorta recognize when we see it). That means that they aren't perfect, but that the number that appears is not random as compared with your intelligence. Yet people constantly point it out as valueless because it isn't perfect. I doubt we'll find anything in the next century that will be a perfect measurement of intelligence.

The IQ test is the BMI of intelligence: it's nowhere near the final say on anything, but dude, if your BMI is 40, you're in trouble. If your BMI is 15, you're in trouble. IQ tests are probably a bit more useful than that.

What have IQ tests been able to point out? Well, lead isn't a great idea for kids. Breastfeeding helps if they carry certain genes. It's a ballpark figure that comes up with a number on how well you did on some mental exercises that day, not some Gattaca-like measurement of your worth, and nobody but the people who hate it treat it that way.
posted by adipocere at 6:00 PM on December 16, 2007 [3 favorites]


Agree with the get easily bored with taking the test comments - have never taken one I didn't give up after 10 minutes.
posted by A189Nut at 6:01 PM on December 16, 2007


Wait, how would you adjust the methodology to control for people giving thoughtless answers due to their lack of motivation re: IQ testing? And I'd argue that it would be trivially easy to show a correlation of interest in IQ tests to race by examining the racial make-up of those who take the tests voluntarily and independently.

I have to say that it seems you're dismissing Jimbob's concerns with a combination of unearned condescension and out-your-ass hand-waving.
posted by klangklangston at 6:05 PM on December 16, 2007


You know that if they don't get tenure, at most universities they get fired, right? And that if there were no tenure, it probably wouldn't affect the rate of published conclusions you think are stupid. Tenure is really neither here nor there in this discussion.

For other reasons as well; one sure way to risk your generic tenurability is to risk engaging in this topic at all, if you are in the interested sciences (psychology, anthropology, biology, etc.). In any case, the worst offenders on most of this stuff tend *not* to be tenured professors, and often have a chip on their shoulders about "established" academic science and their purported exclusion from it which actually drives their interest in questions of absolute intelligence measurement.

Bunch of nutter racist wackos, in other words, pretending to be scientists.
posted by fourcheesemac at 6:21 PM on December 16, 2007


Me, personally, I love IQ tests. I ate them for breakfast. Since the first day of kindergarten when I was taken aside, my parents called, and I was immediately scheduled for my fist IQ test. From that point on it was It was agreed upon, "You are such a smart young man, you are so bright," and so forth.

That is until middle school, because when my grades had bottomed out, get this, now it was suspected I was retarded. So I was re-tested. The results? Almost twenty points higher than the last time I was tested (third grade). In fact, I was allowed to pass seventh grade with all "F's" based on my IQ!

From there, high school, where I dropped out but took the GED test and completed it in less than 40 minutes, missing a single question. I got to take the SAT, and scored 700 on the english with my very first hangover, even with a 2.0 GPA, a shoo-in for college and university. From there, LSAT (96% percentile) and ASVAB (99% percentile).

When the ARMY and I came into disagreement, and I demanded to be released on conscientious grounds, I was given the Minnesota Multiphasic Personality Inventory — hundreds of true or false questions! It was like heaven! The results of which leading to a personal letter of recommendation from the evaluating psychologist/Captain, "By no means let this soldier out...he will no doubt be a significant military leader." (Ha! Um, wait a minute...)

For me, every time it came down to being able to write well and manage standardardized testing, I have been very fortunately successful. Also, I have always been good with video games, computing, and incidentally communicating.

Moral of story? Some people are better at comprehending the methodologies of standardized tests than others. (Like I said, I ♥ standardized tests.) Advantage of such skills? Societally speaking, my case speaks for itself.
posted by humannaire at 6:43 PM on December 16, 2007 [3 favorites]


The problem I see, spiderwire, is that the whole process lacks the rigour of the sort of double-blind studies you see elsewhere in biology.

You say results can be adjusted for whether people are bored with the test, and whether or not they really want to take the test, and whether or not they are interested in and competitive about the result... why, then, do medical scientists need to hide who has been given placebos in drug trials? Surely they can just tell people whether they are getting a placebo or not and do some magical statistical adjustment on the results, because they know people so well?

IQ tests seem like potentially a very strong case of the means of observation influencing the outcome of the experiment. Some people aren't intelligent. Some people are intelligent and good at taking tests. Some people are intelligent and not good at taking tests. And these last two options are most likely correlated with aspects of ourselves that are not anything to do with intelligence.

However, I do have to revise my anecdote above - it has occurred to me that my friend may indeed do better than me on IQ tests, because I realize the reason I'm so bad at chess is essentially the same reason I hate IQ tests; boredom and frustration with something that I can't see the point of. If he's so good at thinking through chess moves and concentrating, he may well kick ass in an IQ tests, provided they let him outside for a bowl now and again.
posted by Jimbob at 6:55 PM on December 16, 2007


Time constraints vary depending on the test or what's being tested for. But the answer to your question is the same as it is to JimBob's — those factors are both trivially easy to control for and not sources of systemic error.

So what are the sources of systemic error? This isn't a topic that I'm close to, but the last I'd heard there were all sorts of biases in various tests. Are you saying that there is no longer any sex bias? No more education bias? No more bias towards Westerners?

And as a scientist, I've a small objection to the notion that controlling for anything is trivially easy, especially in studies with people.
posted by kisch mokusch at 7:09 PM on December 16, 2007



The thing all the critics of IQ keep leaving out is that they generally do predict who does well at school and at intellectually challenging jobs. Do they predict 100%? Absolutely not-- which is why everyone here can cite zillions of exceptions of brilliant people who do poorly on tests.

however, virtually no one can cite many examples of non-brilliant people who do brilliantly on the tests-- except, of course, if people cheat. You can find people who are failures in their pursuits who do well on the tests-- but that is usually due to very obvious emotional and behavioral challenges that don't have anything to do with cognitive intelligence.

What I find fascinating is how people fail to take into account the effect of the environment on IQ. Genes give you a range you can perform in-- environment can either put you at the high end of that range or the low end. There's some interesting work showing how many words kids are exposed to in middle class v. impoverished environments-- by age 3, the poor kids know something like 300 words while the middle class know something like 1000. Now think about how those little differences can become large differences over time and it becomes amazing that the race/IQ gap is as small as it is.

The fact that the gap is narrowing over time shows just how important the environment is. I found the stuff in the Gladwell piece about Asian immigrants and IQ fascinating-- showing how culture can affect IQ and school performance over time.

Stress is also a massively underplayed factor. When you are under extreme stress, your higher brain regions literally shut down and your intelligence is narrowed to a focus on surviving here and now. If that is repeated over time, the ability to control the stress regions via the higher brain regions can be compromised. You become what you practice-- if you practice having to fight for survival, you'll be great at that but not so good at math tests. Failing to take into account the impact of stress is another reason we wind up with seeming racial differences which are really poverty and racism-related.

What's hard, of course, is figuring out how to change the situations which are so stressful for kids and families and to create the hope, future-orientation and love of learning that is needed to escape those situations in the long run.
posted by Maias at 7:23 PM on December 16, 2007 [2 favorites]


yhbc: The real problem is we, as a species, WANT to be able to quantify "intelligence" and be able to rank ourselves against everyone else on the spectrum.

Only so long as we can be sure that we, personally, come out above average.

I'm always amazed at the angst surrounding IQ tests.

It has to do with the tendency among authority figures of rating a person as a person according to how they score.

As a kid who rated highly on them and got to be, honestly, probably a little smug in thinking of himself along the way, I have to say that, these days, I don't think it's ultimately decreased the number of times I've thought I must be the stupidest person in the world over that of most people. In fact, I probably think that more than average.
posted by JHarris at 7:41 PM on December 16, 2007


Maias: However, virtually no one can cite many examples of non-brilliant people who do brilliantly on the tests

Easy-- the majority of the members of Mensa, whose only claim to brilliance is belonging to Mensa.
posted by Pyry at 8:20 PM on December 16, 2007 [1 favorite]


Wait, how would you adjust the methodology to control for people giving thoughtless answers due to their lack of motivation re: IQ testing?

If you give someone a question outside the context of an IQ test and they answer it better or worse than they do in when taking an actual IQ test, it means that the test isn't controlling for a source of error. Flynn's actually talked about this at length. The adjustment in the methodology isn't trivial, but detecting it is.

You say results can be adjusted for whether people are bored with the test, and whether or not they really want to take the test, and whether or not they are interested in and competitive about the result...

No, I said that the results could be adjusted for declines in performance, assuming those declines are indeed systemic, and that you knew the question order. But obviously it would be preferable to adjust the test in order to prevent the performance declines. Overall interest in the test could, I suppose, be adjusted for, but probably not effectively. But it's certainly detectable.

I'll be a bit more concrete:
1. To detect performance declines, you merely reorder the questions for a given test -- then for any question, you can easily obtain a delta describing how more or less likely a person is to correctly answer it depending on its position in the test. If there is a detectable difference in rate-of-boredom between groups, it will be clearly reflected in that delta.
2. To detect test pressure, you could, for example, give single test questions to subjects outside the context of a test, and again observe easily whether test conditions tend to affect performance and whether the effect differs between groups.

why, then, do medical scientists need to hide who has been given placebos in drug trials? Surely they can just tell people whether they are getting a placebo or not and do some magical statistical adjustment on the results, because they know people so well?

Double-blind testing is an adjustment for the placebo effect.

This is exactly what I'm talking about -- if you never thought to look for the placebo effect, then it would be a source of systemic error, but it's trivially easy to detect and adjust for once you do, to a certain degree of statistical confidence.

Beyond that, the question is inapposite: if, for some reason, you introduce an intentional error, you cannot "adjust" for it without doing a separate test to quantify the error. If you're aware of the problem, it makes no sense to muck up your control group for no reason.

So what are the sources of systemic error? This isn't a topic that I'm close to, but the last I'd heard there were all sorts of biases in various tests. Are you saying that there is no longer any sex bias? No more education bias? No more bias towards Westerners?

Yes, those are all possible sources of systemic error, as the article discusses. My opinion is that they are -- but that's what the whole debate is about.

One's particular attitude toward the process of test-taking itself is generally not a source of systemic error. Though it is admittedly a possibility and has happened in the past, it's quite simple to detect and adjust for, and is. Cultural bias, on the other hand, is nearly impossible to quantify, and thus not trivial to detect or eliminate. This is the entire heart of the debate.

The thing all the critics of IQ keep leaving out is that they generally do predict who does well at school and at intellectually challenging jobs. Do they predict 100%? Absolutely not-- which is why everyone here can cite zillions of exceptions of brilliant people who do poorly on tests.

That is a highly misleading statement. Again, IQ tests discriminate between domains (average IQs for doctors are probably higher than for real estate agents), but not within domains (IQ does not predict success as a doctor or as a real estate agent). Additionally, IQ tests do not "predict" anything. Correlation is not causation.

Failing to take into account the impact of stress is another reason we wind up with seeming racial differences which are really poverty and racism-related.

Actually, stress is, as indicated, not that difficult to detect and control for. There are far more difficult issues in standardized testing to deal with -- for examples, there is some compelling evidence that even asking students to self-identify their ethnicity before a test (even if the question is optional) can have a statistically measurable effect on test performance. That is an extremely difficult problem to contend with from a test-design perspective -- much more so than stress.
posted by spiderwire at 8:39 PM on December 16, 2007


What would be really great would be to have an organization that allows people who are preoccupied with how smart they are to take a test and, if they score high enough, be allowed to join. (Naming is tricky, but I suggest a greek word you might see on the window of a furniture store.)
- maxwelton

You certainly wouldn't be allowed in. Mensa is a Latin word.
posted by matkline at 8:58 PM on December 16, 2007 [2 favorites]


You folks interested in how things like stress, anxiety, boredom etc. affect IQ should look in on Claude Steele's work on how African American students' test performance is affected by stereotypes and expectations. Basically, if you're in a situation where a negative stereotype is invoked (like, black students taking an IQ test), you respond by fulling the stereotype (and so perform worse than you could). If you remove the "stereotype threat" performance goes up. also applies to women taking math tests. That's some real deep stress there.
posted by cogneuro at 9:17 PM on December 16, 2007 [1 favorite]


Wait, how would you adjust the methodology to control for people giving thoughtless answers due to their lack of motivation re: IQ testing? And I'd argue that it would be trivially easy to show a correlation of interest in IQ tests to race by examining the racial make-up of those who take the tests voluntarily and independently.

And how exactly would you get people unwilling to take an IQ test to take an IQ test? Other then that, it's a very easy experiment.
posted by delmoi at 9:27 PM on December 16, 2007


So what are the sources of systemic error? This isn't a topic that I'm close to, but the last I'd heard there were all sorts of biases in various tests. Are you saying that there is no longer any sex bias? No more education bias? No more bias towards Westerners?

Yes, those are all possible sources of systemic error, as the article discusses. My opinion is that they are -- but that's what the whole debate is about.


They're not the sources. They're the read out. The source remains unknown. If two groups are matched for everything but factor, and there is a significant difference between the two there's only two conclusions: One is smarter than the other, or the test is biasing against one of the groups. But that doesn't make the source of error known. You don't know what it is about the test that discriminates one group and favours the other. Furthermore, that premise is made on the grounds that the two groups should be equal, which we don't know, because there's no reliable readout. Which leads me to:

there is a very clear relationship between IQ test performance and "success" by any reasonable metric.

Now that is a misleading statement.
posted by kisch mokusch at 9:53 PM on December 16, 2007


ugh. but one factor
posted by kisch mokusch at 9:54 PM on December 16, 2007


Upon re-reading it appears I'm also being misleading. The first read out refers to a read out of bias. The second read out refers to a read out of intelligence.
posted by kisch mokusch at 10:15 PM on December 16, 2007


I suppose I should clarify: "stress," as discussed here, is just a hypothetical cause. What we're really discussing are two possible effects, which could have any number of causes:
1. Decline in performance over the course of a test
2. Reduced performance as a result of the test environment

Both effects are fairly easy to detect, because they can be controlled for via simple methods, like asking questions in different orders (like any major standardized test does) or by asking questions out of context. And the people that design standardized tests are generally aware of that.

Flynn, however, is discussing an entirely different and more difficult question, regarding neither methodology nor quantification. He is proposing, essentially, that IQ tests evaluate acculturation (which shouldn't be confused with conformity). The assumption is that if we could just factor the "cultural" elements out of the equation, the tests would work -- and the bulk of the race-IQ debate is whether or not the tests adequately do that. Flynn's argument is that this is a canard -- all that's being measured is the societal valuation of intelligence, and the accuracy of the test is a function of how well it does that. "Accuracy" in this context means legitimacy, nothing more.

I should be clear that to the extent that JimBob is approximating this argument, I'm not trying to say he's wrong, but I think that the reasons are important. We should be wary of two things: First, the need to attribute cause-and-effect; Second, the cynical view that IQ tests reflect consensus reality.

As to the first: suggesting that certain people have an aversion to tests can be a way of tacitly acknowledging the validity of the metric even while discounting the accuracy of the test. It seems to me that pointing to someone's ability at chess as evidence against IQ tests confirms the central assumption of IQ tests: that in an ideal form, the test would be a predictor of performance -- that it would have a little readout at the end with categories like "Chess Ability," or "Doctor Potential." (Or the reverse criterion: that in order to be valid, the test should be able to predict such things.)

As to the second point: what Flynn is trying to get across, I think, is that IQ is in every respect the tail that wags the dog; the relationship between the test and the thing being tested is a closed loop, which via periodic renormalization creates a sort of centripetal force towards the intellectual norm.

This is not a way of saying that "intelligence testing" reflects a privileged consensus; in fact, attitudes towards IQ tend to indicate the opposite. Mensa is annoying; yet we individually all want to have a high IQ. We take the supposedly high IQs of successful tycoons as confirmatory; yet we're morbidly fascinated with the fact that some pop stars are reputed to have high IQ scores. Everyone has personal anecdotes about the accuracy or inaccuracy of IQ tests. If anything, the societal attitude toward IQ scores seems to be that they don't reflect the privileged consensus -- but they reflect a desire for a privileged consensus.

That's why we should be extra careful about ascribing causes and effects here. If Flynn's right and the whole thing really is a headless beast, then the causes and effects are driven by the process -- they don't exist in isolation, waiting to be accurately identified and measured. Disenchantment with IQ tests is usually a way of saying that the game is rigged. But that's part of the game as well, to the extent that it posits a world where the test could be separated from the bias; it assumes that the test can exist independently of the bias.

To be a bit pithy, the point is this. IQ is not the yardstick for cognitive ability; cognitive ability is the yardstick for IQ. Evaluating intellect is how we accommodate our desire that our reality reflects our intent.
posted by spiderwire at 10:23 PM on December 16, 2007 [1 favorite]


delmoi: And how exactly would you get people unwilling to take an IQ test to take an IQ test? Other then that, it's a very easy experiment.

Well, that's easy, you just need to find some cooperative organization that has lots of minions they can order into doing any kind of tedious test they like. Perhaps any school ever created would be able to help you.

Alternately, if you need adults, you can take get your subjects before you tell them what the testing entails - although that will still introduce selection bias.
posted by Mitrovarr at 10:24 PM on December 16, 2007


They're not the sources. They're the read out. The source remains unknown.

Very true.

there is a very clear relationship between IQ test performance and "success" by any reasonable metric.

Now that is a misleading statement.


Also true, but it is not inaccurate, either. There is a relationship between IQ and occupation. What I should have said is that IQ is a measurement of "success" in the tautological sense, and the metric's reasonability is defined in those terms.

That is,

If two groups are matched for everything but [one] factor, and there is a significant difference between the two there's only two conclusions: One is smarter than the other, or the test is biasing against one of the groups. But that doesn't make the source of error known.

There is also the possibility that the test doesn't measure an independent variable. If the test is reflexive but "sticky," then it will exhibit a systemic bias against non-dominant groups. However, in that case, the test does not have a systemic error — in fact, the assumption of systemic error actually reinforces the bias.
posted by spiderwire at 10:43 PM on December 16, 2007


spiderwire: The assumption is that if we could just factor the "cultural" elements out of the equation, the tests would work...

Just based on this article, that doesn't seem to me what he's saying. Not that that IQ tests evaluate acculturation but that their results correspond to acculturation. Rather than saying "accounting for cultural influences, all mental capacities are the same across all people" it seems like the significantly different statement "there is a mental capacity, whatever is measured by these tests, that can be enhanced or determined by culture."

Look at this quote from the article in the section where it's talking about IQ relating to "Germanness":

"The mind is much more like a muscle than we’ve ever realized," Flynn said. "It needs to get cognitive exercise. It’s not some piece of clay on which you put an indelible mark." The lesson to be drawn from black and white differences was the same as the lesson from the Netherlands years ago: I.Q. measures not just the quality of a person’s mind but the quality of the world that person lives in.

Re the OP's suggestion that the book will settle some points in the previous debate: this actually wouldn't deny a racial difference in IQ, it would simply say that the difference is cultural rather then hereditary.

It also sounds like he's trying to provide evidence that IQ as measured is definitely a particular faculty rather than general intelligence.
posted by XMLicious at 11:27 PM on December 16, 2007


Metafilter: "Success" in the tautological sense
posted by XMLicious at 11:30 PM on December 16, 2007


Moral of story? Some people are better at comprehending the methodologies of standardized tests than others. (Like I said, I ♥ standardized tests.) Advantage of such skills? Societally speaking, my case speaks for itself.

You're the guy who posted the human botfly video, aren't you?
posted by jokeefe at 11:57 PM on December 16, 2007


First. To be fair, I've read a fair amount about and by Flynn, and while I have my characterization of his argument that I think is accurate, the dude is all over the map, and I'm not convinced he's consistent — at the very least, he changes his mind a lot. (Nor is Malcolm Gladwell exactly the most rigorous scholar.) So to the extent that Flynn has said things that contradict what I said, no guarantees — I'm more interested in defending my argument than his, to the extent that he has one.

That said, I think that reading his findings and arguments is helpful for understanding the different dimensions of the debate going on here.

You'll also find this in the article, from Flynn: "Either the children of today were far brighter than their parents or, at least in some circumstances, I.Q. tests were not good measures of intelligence."

Gladwell overemphasizes the nurture part of Flynn's argument while missing the larger picture, I think. He comes close in this paragraph:
“If the everyday world is your cognitive home, it is not natural to detach abstractions and logic and the hypothetical from their concrete referents,” Flynn writes. Our great-grandparents may have been perfectly intelligent. But they would have done poorly on I.Q. tests because they did not participate in the twentieth century’s great cognitive revolution, in which we learned to sort experience according to a new set of abstract categories. In Flynn’s phrase, we have now had to put on “scientific spectacles,” which enable us to make sense of the WISC questions about similarities. To say that Dutch I.Q. scores rose substantially between 1952 and 1982 was another way of saying that the Netherlands in 1982 was, in at least certain respects, much more cognitively demanding than the Netherlands in 1952. An I.Q., in other words, measures not so much how smart we are as how modern we are.
That last sentence is correct, but also totally wrong. Note how Gladwell describes the shift Flynn's talking about as a "great cognitive revolution," or a more "cognitively-demanding" environment — but that's not at all what Flynn says in that first sentence. Flynn's talking about the different method of thinking that's most appropriate for a particular lifestyle; modern society trains the mind — like a muscle, remember — toward a more abstract, associational methodology, as opposed to a functional methodology.

[Flynn said a while back that "tests do not measure intelligence but only a minor sort of 'abstract problem-solving ability'"]
posted by spiderwire at 12:02 AM on December 17, 2007 [1 favorite]


Here, also, is a nice snippet from an Amazon book review that puts it well:
Flynn goes on capturing the dynamic interaction between the environment and cognitive capabilities over time. Flynn's "individual multiplier" stipulates that an individual influences the environment he operates in because the environment will respond to where the individual excels. Flynn's "social multiplier" describes the escalation of cognitive demands as we progressed from an agricultural to an information based society. The social multiplier is a giant mass wide positive feedback loop. Everyone's expectation of academic and professional achievement has risen over time.

Thus, Flynn concludes we have become far better at solving abstract problems because that is what society currently demands us to do. In his mind, this does not necessarily mean we are so much smarter than our ancestors (the physiological processing capability of our brains has not changed). It just means we have a different focus (abstract and post-scientific vs concrete and pre-scientific).
The FPP links the intro of the book here. It's recommend reading it if you haven't yet; I think it corroborates my characterization of the argument.
posted by spiderwire at 12:11 AM on December 17, 2007 [1 favorite]


Fair enough! Whatever Flynn's core position is Gladwell is definitely using a heavy touch in his characterization of it. Thanks for your insightful comments in this thread.
posted by XMLicious at 12:12 AM on December 17, 2007


Arrgh! The Google Books excerpt ends right at the point I was interested in. From p. 14:
Are IQ gains due to "cultural bias"? We must distinguish between cultural trends that render neutral content more familiar and cultural trends that really raise the level of cognitive skills.
I'll have to scare up a copy.
posted by XMLicious at 12:27 AM on December 17, 2007


Maias: ... virtually no one can cite many examples of non-brilliant people who do brilliantly on the tests....

Oh, I can. Lots of them. Mensa and HiQ are lousy with such folks.

I've worked with some pretty dumb folks who had really high IQs.
posted by lodurr at 3:36 AM on December 17, 2007


I'm certain this is something that has been discussed, debated and defined for eons, but I'm wondering what "intelligence" is, exactly. It seems to be a very abstract concept.

Is it related to knowledge?
Is it related to wisdom?
Is it related to insight?
Is it related to reasoning?
Is it related to intuition?
Is it related to perceptivenes?
Is it related to proficiency in a skill?
Is it related to memory?
Is it related to creativity?

Is it some combination of the above? I assume it is, but I know IQ tests are very much designed to test only a subset of these facets of the skillfullness of the mind.

We all know about people who are geniuses at the mathematics of set theory or something, but can't change a washer. And we all know of people who are highly adept at mechanical tasks but can't do their taxes. I just get the feeling that any notion that intelligence can be measured on some one-dimensional linear scale is a complete failure - as Flynn indicates, the kind of intelligence we need in modern societies has changed over the last century. If it's not static, and it's not a simple score, what's the point again?
posted by Jimbob at 4:43 AM on December 17, 2007


You're the guy who posted the human botfly video, aren't you?

Ain't fame great?
posted by humannaire at 6:14 AM on December 17, 2007


BTW, XMLicious,...

Whatever Flynn's core position is Gladwell is definitely using a heavy touch in his characterization of it.

...agreed.

And if you grok'd that from the one article (and thread), you should read one of his books. (Blink, The Tipping Point, et al).

As a writer, Malcolm Gladwell is an acupuncturist who's sends his points home with a ball-peen hammer.
posted by humannaire at 6:25 AM on December 17, 2007


Here's my favorite part of the article:
The psychologist Michael Cole and some colleagues once gave members of the Kpelle tribe, in Liberia, a version of the WISC similarities test: they took a basket of food, tools, containers, and clothing and asked the tribesmen to sort them into appropriate categories. To the frustration of the researchers, the Kpelle chose functional pairings. They put a potato and a knife together because a knife is used to cut a potato. “A wise man could only do such-and-such,” they explained. Finally, the researchers asked, “How would a fool do it?” The tribesmen immediately re-sorted the items into the “right” categories.

posted by exogenous at 7:01 AM on December 17, 2007 [3 favorites]


I took an IQ test in the 11th grade. It turns out it's 127, which means I'm pretty smart, but not super-smart. I also have ADD and some other difficulties. I can follow an advanced conversation or book-to a point, then my head starts to spin and my eyes glaze over and I get this urge to go play with a big rubber ball. Basically, I'm smart enough to know how smart I'm not, which can be a blessing and a curse. Sometimes, I wish I was just a notch dumber so I could be blissfully ignorant of the whole shooting match.

My point, IQ like traditional notions of intelligence is a bag of snakes, so be careful before sticking your hand in it.
posted by jonmc at 7:11 AM on December 17, 2007


My IQ's about like a million or something.
posted by Divine_Wino at 8:53 AM on December 17, 2007


My IQ's about like a million or something.

Mine's room temperature....in hell.
posted by maxwelton at 1:51 PM on December 17, 2007


The fact that the person with the highest recorded IQ, Marilyn Vos Savant, has found herself writing a weekly column solving brain teasers in Parade magazine, should give pause to anyone inclined to value their IQ very highly as a predictor of success.
posted by jayder at 12:19 PM on December 18, 2007 [1 favorite]


Maias: ... virtually no one can cite many examples of non-brilliant people who do brilliantly on the tests....

lodurr: Oh, I can. Lots of them. Mensa and HiQ are lousy with such folks.


A co-worker who belonged to Mensa took me to one of their local meetings. It was a bunch of pretentious dorks with mediocre jobs. The people I met seemed to be people who, clearly, hadn't amounted to much, but consoled themselves that they were intellectually a cut above 99% of the population. But meeting them, and finding out what they did, went a long way toward severing, in my mind, any presumed relation between IQ and real-world accomplishment.
posted by jayder at 12:28 PM on December 18, 2007


I do very well on standardized tests because I can usually figure out the logic behind how questions are asked, but that doesn't prove anything other than I am a good little test taker. Some people don't test well at all and it is no reflection on their true smarts.

True dat. Standardized tests measure not only the ostensive content abilities, but also the abilities you identify, viz., the ability to suss out the intention behind a questioner. Gladwell's example of the tribesman who could sort into categories according to the stated preference of the examiner is a great illustration of exactly that fact.
posted by Mental Wimp at 10:44 AM on December 19, 2007


[Flynn said a while back that "tests do not measure intelligence but only a minor sort of 'abstract problem-solving ability'"]

And it's that minor sort of abstract problem-solving ability that Flynn applies to the data to gain his conclusions.
posted by Mental Wimp at 10:46 AM on December 19, 2007


"The fact that the person with the highest recorded IQ, Marilyn Vos Savant, has found herself writing a weekly column solving brain teasers in Parade magazine, should give pause to anyone inclined to value their IQ very highly as a predictor of success."

That, and she's frequently wrong when it comes to any issue outside of anagrams and guest-seating puzzles. Her political and moral views boarder on the infantile, despite being in Guinness.
posted by klangklangston at 1:11 PM on December 19, 2007


That, and she's frequently wrong when it comes to any issue outside of anagrams and guest-seating puzzles. Her political and moral views boarder on the infantile, despite being in Guinness.

Oh well. At least we've still got Bobby Fischer to provide us with a moral compass.
posted by spiderwire at 1:10 PM on December 20, 2007


And Kasparov to vote for! (Well, if you're a Russian, and want to be beaten by Putin's thugs).
posted by klangklangston at 1:20 PM on December 20, 2007


And Kasparov to vote for! (Well, if you're a Russian, and want to be beaten by Putin's thugs).

Could be a write-in. Beats some of the alternatives.

Personally, I'm a fan of Vladimir "Vlad the Bad" Zhirinovsky, the guy who ran against Yeltsin on the "retake Alaska and wash the blood from our boots in the Indian Ocean" platform.

I saw him as a "Russia Consultant" on CNN a few months back. Like he was just, y'know, some dude. That was pretty sweet / weird.

We should send Ann Coulter to some overseas networks as a "U.S. Policy Consultant" and really scare some fools. I mean, assuming they noticed the difference.
posted by spiderwire at 3:22 PM on December 20, 2007


Ha! I saw Zhirinovsky on CNN and called my girlfriend over to take a look!

We might be better off sending Phelps. Can you imagine him at the embassy, telling people in the Sudan that God's killing them because they have gays?
posted by klangklangston at 5:15 PM on December 20, 2007


Or maybe Iran:
Phelps: "You're all going to hell because of gays."
Ahmanejihad: "We have no gay people in our country."
Phelps: "I'VE FOUND THE PROMISED LAND"
posted by spiderwire at 5:31 PM on December 20, 2007


*Ahmadinejihad

Or as Steven Colbert would say, Ahmandadinejehiddyhibahd.
posted by spiderwire at 5:31 PM on December 20, 2007


« Older "No, son, that is a transvestite."   |   fix or repair daily Newer »


This thread has been archived and is closed to new comments