Consider the human judgment

November 8, 2011 4:07 PM

The King of Human Error: Michael Lewis profiles Nobel laureate Daniel Kahneman

Excerpt:

In 1983—to take just one of dozens of examples—they [Kahneman and Tversky] had created a brief description of an imaginary character they named “Linda.” “Linda is thirty-one years old, single, outspoken, and very bright,” they wrote. “She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in antinuclear demonstrations.” Then they went around asking people the same question:

Which alternative is more probable?

(1) Linda is a bank teller.

(2) Linda is a bank teller and is active in the feminist movement.

The vast majority—roughly 85 percent—of the people they asked opted for No. 2, even though No. 2 is logically impossible. (If No. 2 is true, so is No. 1.) The human mind is so wedded to stereotypes and so distracted by vivid descriptions that it will seize upon them, even when they defy logic, rather than upon truly relevant facts. Kahneman and Tversky called this logical error the “conjunction fallacy.”

Don't miss: The Quiz Daniel Kahneman Wants You to Fail

posted by vidur (61 comments total)

As you enter the theater, you discover that you have lost the ticket. The theater keeps no record of ticket purchasers, so the ticket cannot be recovered. Would you pay $10 for another ticket to the play?

The question as written doesn't make it clear that I have to pay again in order to see the play. I answered No, because I thought I'd gotten my ticket stamped when I'd entered the theater.
posted by LogicalDash at 4:19 PM on November 8, 2011 [1 favorite]

I didn't fall into the traps on any question. That quiz just played into my personal confirmation bias that I am anything but a normal human being.
posted by Mister Fabulous at 4:19 PM on November 8, 2011 [1 favorite]

Eponysterical, Mr.Fab.
posted by leotrotsky at 4:33 PM on November 8, 2011

Hm, since I was warned going in I guessed the wrong trick about the "dates" and "ticket" questions altogether. I didn't know how many dates I've eaten recently, and I guessed that as the "ticket" question omitted to say that I couldn't enter without buying another ticket, that I should answer "no" and go to the show anyway.
posted by jepler at 4:36 PM on November 8, 2011 [2 favorites]

I don't see how #2 is logically impossible. Their interpretation of the question appears to diverge from basic common usage. The structure of the answers clearly implies an exclusive disjunction between them--as in, they're not both true--and people would tend to answer as if they were. Claiming that 2 is a more specific case of 1 is special pleading.
posted by fatbird at 4:38 PM on November 8, 2011 [6 favorites]

100% chance these guys never met Linda.
posted by humanfont at 4:39 PM on November 8, 2011 [1 favorite]

(Tversky had died in 1996, making him ineligible to share the prize, which is not awarded posthumously.)

rarely
posted by shoesfullofdust at 4:42 PM on November 8, 2011 [1 favorite]

The structure of the answers clearly implies an exclusive disjunction between them--as in, they're not both true--and people would tend to answer as if they were.

I think the phrasing of the question--which asks which alternative is more probable--works against the implication of an exclusive disjunction; since they are conceding that neither may be true, the assumption that one must be true and the other false doesn't follow as naturally as it might otherwise. I answered wrong, and it's because I fell victim to the conjunction fallacy, not because I thought they were asking which one and only one were true.
posted by layceepee at 4:48 PM on November 8, 2011 [1 favorite]

Q2 is also BS. Some of the details are probably informative.
posted by a robot made out of meat at 5:05 PM on November 8, 2011

I didn't know how many dates I've eaten recently

I read the question the same way, and thought "Zero, I don't eat dates."

Considering my tendancy to overthink questions, I'm surprised I did halfway decent on standardized tests in my lifetime.
posted by drezdn at 5:14 PM on November 8, 2011

If you phrase things in such a way that the common understanding of the question is entirely different than the actual question, you can get arbitrarily weird results.

Bob is holding a knife. Alice is asleep. Should Bob cut Alice with the knife?
A) Yes
B) No

Result: 94% of people oppose surgery.
posted by 0xFCAF at 5:17 PM on November 8, 2011 [12 favorites]

Yeah, I don't understand Q2 either. Say that an engineer is twice as likely to fit that description as a lawyer. (I don't know, I'm just pulling numbers out of my ass.) Then the prior odds of being an engineer are 3/7; the likelihood ratio is 2/1; so the posterior odds of being an engineer are 6/7, i. e. the posterior probability of being a lawyer is 6/13 or about 46 percent.

(And wouldn't you know I picked 40 to 60 percent for that question?)
posted by madcaptenor at 5:21 PM on November 8, 2011

Um, posterior probability of being an engineer.
posted by madcaptenor at 5:22 PM on November 8, 2011

I'm just pulling numbers out of my ass.

The definition of a posterior probability.


Exactly why the second question in the quiz is misleading; "the correct response being precisely 30 percent" is only correct if the attributes of Jack as described are entirely uncorrelated with the profession chosen. (Which they're not; about 2/3 of lawyers in the US are men, but almost 90% of engineers. This fact alone means -- if the sample is representative, and we have no information to assume it's not -- that there's a 37% chance that a randomly-selected man is an engineer. And an 88% chance that a randomly-selected woman is a lawyer.)

If the question was based on a survey of 30 opera singers and 70 sushi chefs, and their example, Jacqueline, spoke Italian but not Japanese, the correct response would not be precisely 30% chance. Or take it to an extreme; 70 jockeys and 30 NFL offensive tackles. Bob weighs 320 pounds.
posted by Homeboy Trouble at 5:28 PM on November 8, 2011 [6 favorites]

I always find these kinds of things to be obnoxious, in a "gotcha" kind of way.

"they documented a peculiar behavioral tendency: when people faced a gain, they became risk averse; when they faced a loss, they became risk seeking."

I really hope their research was better than this, because this example is just dumb. If I lose $900 or $1000 it doesn't really make much difference--I'm approximately equally screwed in terms of paying my rent. Taking a 10% of escaping completely unscathed isn't risk-seeking; it's praying that luck will save your poor broke ass. Likewise, the payout differential isn't big enough in the positive case.

Change the odds and numbers a little bit, and we can easily see that people take the opposite tack with mathematically equivalent statements:

0.0001% chance of getting $5 million vs guaranteed $5
or
0.0001% chance of losing $5 million vs guaranteed loss of $5

I'm pretty sure state lottos have pretty conclusively shown people take the opposite approach all the time in this scenario, even though it's mathematically equivalent.
posted by Dr.Enormous at 5:29 PM on November 8, 2011 [2 favorites]

Their answer to Q2 is really astonishingly fucked up when you think about it. By the same logic, they could have said "Jack has an engineering degree, works at a firm that employs thousands of engineers and one lawyer, has several engineering patents, and cannot name a single Supreme Court Justice, past or present" and we'd still have to say "30% is the precisely correct answer". That's madness.
posted by 0xFCAF at 5:31 PM on November 8, 2011 [3 favorites]

OK, Q2 is a really misrepresenting the phenomenon that Kahneman has studied. What the "representativeness bias" really means is that people follow the representativeness of the features while ignoring other relevant information. In the original studies, the bias was reflected by the fact that people said someone had a 90% chance of being an engineer when there was a 70/30 split, and didn't change their response at all when it was a 30/70 split in the other direction. You're all correct that the problem as stated in the article is BS, but Kahneman's original work is generally pretty fabulous.
posted by svenx at 5:41 PM on November 8, 2011 [1 favorite]

Oh, I'm not criticizing Kahneman. He has a Nobel Prize! Of course pointing this out and using it to mean "everything he says is right" is exactly the sort of thing he would warn against.
posted by madcaptenor at 5:46 PM on November 8, 2011

Here is the situation posed in the quiz:

A. Getting $900
B. 90 percent chance of getting $1,000
and
A. Losing $900
B. 90 percent chance of losing $1,000

Common answers are A and B.

And here is the situation posed by Dr.Enormous:

A. Getting $5
B. 0.001 percent chance of getting $5,000,000
and
A. Losing $5
B. 0.001 percent chance of losing $5,000,000

Common answers are (presumably) B and A.

There are two issues to consider in these situations:

(1) There is no difference in the "expected value" within each alternative. $900 is the same as 90% of $1000. Ditto for the second situation. That's why the flips A/B and B/A are of interest.

(2) The amounts involved are comparable in the original question, not so in the second one. There is only a 10% difference there, as opposed to 99.999% in the second situation. This leads to a flipping of responses from A/B to B/A. If we replaced $5 with $4,500,000 and adjusted the probability accordingly, I suspect we'd be back to A/B. And while I haven't read the paper, I suspect this has to do with what the terms "gain" and "loss" mean. Someone who already has $5,000,000 probably doesn't consider $5 to be much of a gain/loss.

Also, there is a note that says that the quiz is only loosely based on Kahneman's research. So, it is not clear who designed it.
posted by vidur at 5:50 PM on November 8, 2011 [2 favorites]

Also, some of the work on people being risk averse for gains and risk seeking for losses is really striking. In the classic Asian Disease Problem, they tell people that there's a disease that will kill 600 people if no action is taken. Then they give them a choice between two alternative: Plan A will save exactly 200 people, and Plan B has a 1/3 chance of saving everyone and a 2/3 chance of saving no one. People strongly preferred plan A. But they gave other people the exact same alternatives framed as losses: Plan A would definitely result in the death of 400 people, Plan B had a 1/3 chance of no deaths and a 2/3 chance of 600 deaths, now most people preferred Plan B. They were much more willing to take risks when it was described as x number of people dying rather than x number of people being saved.
posted by svenx at 5:51 PM on November 8, 2011 [1 favorite]

Also also, yes the "Linda the Bank Teller" problem in its original form has some definite flaws -- the most obvious being that might interpret the options as mutually exclusive. Plenty of followup work has been done to try to get around those issues, though. So for example, if you give the two alternatives to different groups of people and just ask them the rate their likelihood, people independently give higher probabilities to "bank teller who's active in the feminist movement" than to "bank teller."
posted by svenx at 5:58 PM on November 8, 2011 [2 favorites]

I...

1) read the article
2) jumped straight to the quiz
posted by shoesfullofdust at 6:02 PM on November 8, 2011

These reactions are absolutely hysterical if you read the actual article. Kahneman talks about how old men only see the forest because they've lost the ability to see the trees...
posted by graphnerd at 6:07 PM on November 8, 2011 [1 favorite]

Relevant XKCD

Anyway, yes, the fundamental problem in these questions is not how people think -- it's that the question they want the respondent to think they're asking, and the question they claim they're actually asking, are two different things.

The lawyer/engineer one is a classic example of this; what they hope is that you will read it as "how likely is it that these personality traits correlate to an engineer", so that they can then swoop in and say "what we were really asking is the mathematical definition of a percentage!"

Which ultimately tells us very little about the respondents and quite a bit about the people conducting the quiz...
posted by ubernostrum at 6:11 PM on November 8, 2011 [12 favorites]

Ah all of these responses remind me why I do what I do for a living.
posted by JPD at 6:20 PM on November 8, 2011

And my main beef with the article is that Kahneman and Tversky's work is very very readable by anyone who ever had to read an academic paper as an undergrad.
posted by JPD at 6:25 PM on November 8, 2011

There is only a 10% difference there, as opposed to 99.999% in the second situation.

That's my point: a 10% difference is stupidly small and leads to very different behavior, and I hope their work was based off something less pointless than a 10% difference.
posted by Dr.Enormous at 6:39 PM on November 8, 2011

Your comment is exactly illustrating the point they are trying to make. You'll risk losing 1000 in order to make 100, by only losing 900 but you probably wouldn't risk high odds of winning 1000 but rather prefer a sure win of 900, or a loss of 100. Your response should be the same to both. The actual sizing of the issue is a second question. Yes it scales, but the end result is still that humans are not rational risk takers.
posted by JPD at 6:54 PM on November 8, 2011 [1 favorite]

In 1983—to take just one of dozens of examples—they had created a brief description of an imaginary character they named “Linda.” “Linda is thirty-one years old, single, outspoken, and very bright,” they wrote. “She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in antinuclear demonstrations.” Then they went around asking people the same question:

Which alternative is more probable?

(1) Linda is a bank teller.

(2) Linda is a bank teller and is active in the feminist movement.

The vast majority—roughly 85 percent—of the people they asked opted for No. 2, even though No. 2 is logically impossible. (If No. 2 is true, so is No. 1.) The human mind is so wedded to stereotypes and so distracted by vivid descriptions that it will seize upon them, even when they defy logic, rather than upon truly relevant facts.

Seconding svenx. No one who cites this puzzle ever seems to consider that people might retroactively assume, upon hearing option 2, that the first option must have been shorthand for "Linda is a bank teller, but not active in the feminist movement."
posted by UrineSoakedRube at 7:14 PM on November 8, 2011 [2 favorites]

That's my point: a 10% difference is stupidly small and leads to very different behavior, and I hope their work was based off something less pointless than a 10% difference.

The 10% difference means that the two figures are close enough to create a perceptible "gain" or "loss", I think. Either way, the common gut/intuitive response flips between A and B. It is either A/B or B/A, but not A/A (implying a clear preference for certainty) or B/B (implying a clear preference for risk taking).
posted by vidur at 7:16 PM on November 8, 2011

It's only by doing insane things like talking about a $900 profit as a loss of 100 because you might have had 1000 that this sort of thing makes sense. Lord, it's like people have never met anybody who has a small savings account and bills to pay.

If we want to make a little game theory square or the like, and then find our optimal rational choices, you can't just treat the dollars as the numbers. You absolutely have to incorporate the impact of that dollar amount on their life into the expected values. People may behave irrationally, but this is a terrible example: all that happens is that any normal human factors in more variables than the stupid scenario is accounting for.
posted by Dr.Enormous at 7:31 PM on November 8, 2011 [2 favorites]

You absolutely have to incorporate the impact of that dollar amount on their life into the expected values. People may behave irrationally, but this is a terrible example: all that happens is that any normal human factors in more variables than the stupid scenario is accounting for.

Let $N be your total financial net worth. This is everything you have - cash, investments, inheritance, resale value of clothes you are wearing etc.

Now, I'm holding a fully-loaded gun to your head and forcing you to pick between:

A. Getting $0.9xN
B. 90% chance of getting $N

and

A. Losing $0.9xN
B. 90% chance of losing $N

Are your answers A/B?

Let us now repeat the same experiment with the percentage in B adjusted to 0.00000001% and the factor in A adjusted accordingly. Are your answers now B/A?

I was just kidding, man. You are free to go and keep your $N.
posted by vidur at 7:49 PM on November 8, 2011 [1 favorite]

Somebody needs to acquaint the writer of the article, or the researcher, with Grice's conversational maxims, which add up to the general principle that people in conversations are trying to cooperate with each other.

Giving all that information about Linda's background in order to ask a question for which it is completely irrelevant is a violation of Gricean conversational maxims, specifically, the maxim of quanity:

* Make your contribution as informative as is required (for the current purposes of the exchange).
* Do not make your contribution more informative than is required.

Also the maxim of relevance:

* Be relevant.

Therefore, an average reader will instinctively reinterpret the question in such a way as to assume the information *was* relevant, for example, by assuming the choice is between Linda being a bank teller who isn't a feminist activist, and one who is a feminist activist, as fatbird said.
posted by edheil at 7:51 PM on November 8, 2011 [3 favorites]

Heck, another fallacy is the when you read that he had received a Nobel Prize, most of you probably assumed he did science.
posted by benito.strauss at 8:12 PM on November 8, 2011

Heck, another fallacy is the when you read that he had received a Nobel Prize, most of you probably assumed he did science.

Um....he did/does.
posted by Philosopher Dirtbike at 8:49 PM on November 8, 2011

No one who cites this puzzle ever seems to consider that people might retroactively assume, upon hearing option 2, that the first option must have been shorthand for "Linda is a bank teller, but not active in the feminist movement."

See page 303 here.
posted by Philosopher Dirtbike at 9:03 PM on November 8, 2011

Question 2 is both correct and incorrect. It's correct that the probability the bio is an engineer is A (10-40%), but the explanation is incorrect because the explanation says:

"If you answered anything but A (the correct response being precisely 30 percent), you have fallen victim to the representativeness heuristic again, despite having just read about it. "

Here is my notation. Pr() means "probability", L "lawyer", E "engineer", M "male" and Pr(E | M) "given the bio is a male, what's the probability he is an engineer?" We are given the following unconditional probabilities:

Pr(L) = 0.70
Pr(E) = 0.30

We are also told in the bio that the person is a male. (" Jack is a 45-year-old man. "). We are asked, "What is the probability that Jack is one of the 30 engineers?" In other words, given that we know the person is a male, what is the probability he is an engineer? What is the conditional probability in other words, or Pr(E | M). Using Bayes Rule, that probability can be expressed in the following:

Pr(E | M) = Pr(M | E)Pr(E) / [ Pr(M | E)Pr(E) + Pr(M | L)Pr(L) ]
Pr(E | M) = Pr(M | E)(0.3) / [ Pr(M | E)(0.3)) + Pr(M | L)(0.7) ]

Notice the information we need that we do not presently have. We need to know Pr(M | E) and , Pr(M | L). In other words, given you are an engineer, what is the probability you are a male? Given you are a lawyer, what is the probability you are a male. The sex ratios by industry, in other words. I googled around and found around 20% of engineers are females, and 53% new JDs in 2010 were males. So, that means:

Pr(M | E) = 0.8
Pr(M | L) = 0.53

And therefore plugging those into the above, we get:

Pr(E | M) = (0.8)(0.3) / [ (0.8)(0.3)) + (0.53)(0.7) ]
Pr(E | M) = 0.39

That is, conditional on being a male, the probability you are an engineer is 39%, even though without the information that the person is a male, it was only 30%. Which is still within the range provided for letter A (10-40%), but is not "precisely 30 percent".
posted by scunning at 9:10 PM on November 8, 2011 [2 favorites]

Man I'm getting annoyed. Question 4 is poorly worded. As it's worded, Kahneman's wrong. If the person bought a ten dollar ticket, they are ten dollars poorer. Demand is a function of prices and income, and if movies are normal goods, then their willingness to pay for a movie - even for the first movie - could be lower than the price of the movie itself.

The correct way to ask this question is the way Robert Frank asks it in his principles of micro/macro textbook. It's like this: "Jack and Jill have the same preferences and income. Jack buys a ten dollar ticket to the movie but loses it before he gets there. Jill waits to buy her ticket til they get to the theater, but upon arriving finds she lost her ten dollar bill. is one of the more likely to buy a second ticket than another?"

And the answer to that is by the logic of homo economicus - no. They are equally likely to do the same thing. The decision to go to the movie at all is based on the marginal benefits of the movie versus the marginal cost. The marginal cost is $10. You ignore sunk costs, in other words. They are both $10 poorer regardless of their choice, so it's irrelevant.

But, the marginal benefit of the movie could have changed because marginal benefits are a function of income, as well as preferences. And if income fell, and the movies are normal goods, then it's possible marginal benefits are now below $10. Whatever it is, though, it's the same for both of them. If they are less likely to go or more likely to go, they are equally relative to one another likely to take that action because they face the identical situation.
posted by scunning at 9:21 PM on November 8, 2011

No one who cites this puzzle ever seems to consider that people might retroactively assume, upon hearing option 2, that the first option must have been shorthand for "Linda is a bank teller, but not active in the feminist movement."

See page 303 here.

Oops, I got the wrong page. Look at 299: "Surprised by the finding, we searched for alternative interpretations of the subjects' responses. Perhaps the subjects found the question too trivial to be taken literally and consequently interpreted the inclusive statement T as T&not-F, that is "Linda is a bank teller and is not a feminist. In such a reading, of course, the observed judgments would not violate the conjunction rule. To test this interpretation..." They then go on to discuss this alternative. (Look at 303 for an example of an experiment showing the fallacy in the absence of the easy linguistic explanation)

So, no one who cites this puzzle ever seems to consider that alternative? It was considered and rejected. I know that everyone on Metafilter thinks they're very smart, but if you think such an easy explanation has escaped consideration for literally decades, maybe that assessment should be rethought.

Somebody needs to acquaint the writer of the article, or the researcher, with Grice's conversational maxims, which add up to the general principle that people in conversations are trying to cooperate with each other.

I am sure that Kahneman is aware of Grice's conversational maxims. They are well known to cognitive scientists.
posted by Philosopher Dirtbike at 9:30 PM on November 8, 2011 [1 favorite]

See page 303 here.

Good, Kahneman considered it. Still, I've seen this puzzle multiple places -- the Vanity Fair article and in John Mauldin's Bullseye Investing among others -- and none of them seemed to pick up on this issue.
posted by UrineSoakedRube at 10:56 PM on November 8, 2011

So, no one who cites this puzzle ever seems to consider that alternative? It was considered and rejected. I know that everyone on Metafilter thinks they're very smart, but if you think such an easy explanation has escaped consideration for literally decades, maybe that assessment should be rethought.

Wait, are you sayng that Kahneman cited the puzzle? I was under the impression that he authored it.
posted by UrineSoakedRube at 11:04 PM on November 8, 2011

The root of the problem:

Human language is not math.

You can't do math with it. That's why we invented mathematical notation.
posted by jet_manifesto at 12:21 AM on November 9, 2011 [1 favorite]

Good lord.

Human language thought is not math.
is essentially the whole point of the exercise, that human beings are not good at solving probabilistic problems, instead we have a series of innate biases. That's like the point of their work.
posted by JPD at 3:58 AM on November 9, 2011 [1 favorite]

Exactly! Human language/thought introduces uncertainty into any argument. Case in point: you seem to think I'm disagreeing with you. Or am I misreading your "Good lord."?
posted by jet_manifesto at 4:14 AM on November 9, 2011

scunning - given the defintion of the representativeness heuristic is people rejecting a baysian solution in a preference for a more easily seen solution, you are just showing you are well educated enough and disciplined enough to do the math and grab the easy answer. Yeah, the Vanity Fair answer description is wrong.

The classic example K&T use is the likelihood of a false positive in a medical test.

And yeah, question 4 is sort of poorly worded, but the data shows that homo economicus is bullshit. Even phrased in the much clearer way you propose, the empirical data shows that given similar marginal demand/benefit functions people do not view losing a 10 dollar ticket = to losing 10 dollars

Instead of picking nits over an admittedly poorly worded 4 question quiz in Vanity Fair of all places why don't you all pick up one of their books.
posted by JPD at 4:17 AM on November 9, 2011

If you enjoy this stuff, The Drunkard's Walk by Leonard Mlodinow is an absolute must-read.
posted by Acey at 4:18 AM on November 9, 2011

ha - yeah we agree.
posted by JPD at 4:18 AM on November 9, 2011

JPD - re: question 2. Insofar as K&T do not claim that it's irrational to use Bayes Rule, then I'm fine. Medical tests included, Bayes rule will allow you to calculate the conditional probability given some information one learns - assuming you know those priors, the conditional probabilities, etc.

And yeah, question 4 is sort of poorly worded, but the data shows that homo economicus is bullshit. Even phrased in the much clearer way you propose, the empirical data shows that given similar marginal demand/benefit functions people do not view losing a 10 dollar ticket = to losing 10 dollars..

I am not claiming that people do not deviate from a simplistic model. I am claiming that insofar as that question is supposed to showing that to us, it fails to do so. Period. I ask that question I noted to my students precisely because I think my students view "losing $10" and "losing a $10 ticket" as different. On the latter, they typically think they are spending $20 on a ticket if they bought the second ticket; in the former, they just think of spending $10 on the ticket. So yeah - welcome to my world where I get to teach people that their intuition is wrong in some particular application. It is ultimately wrong, as K&T appear to be noting, to fail to see the equivalence of the two situations.

Instead of picking nits over an admittedly poorly worded 4 question quiz in Vanity Fair of all places why don't you all pick up one of their books.

Isn't this a thread where people are talking about the online questionnaire or isn't it? There's only four questions in this questionnaire, and yet two of the answers provided are incorrect technically. So why is it nit-picking to note that?

Regarding my exposure to K&T. I'm not a behavioral economist, by any means, but everyone has to have some knowledge of behavioral economics to do research in their field, especially if they do applied micro work where information is important. I don't read K&T anymore, but I have read their original prospect theory paper from the 1970s a few times, and have continued to read when something comes into view. But yeah - I wish I had more knowledge of K&T's work. I wish I had more knowledge of my own area of research, too, though. Opportunity costs keep me from doing a lot of stuff that I would do otherwise - reading more K&T included.
posted by scunning at 5:13 AM on November 9, 2011

the data shows that homo economicus is bullshit

Not really. The data show that homo economicus is not literally true for all actors, which would surprise nobody.

That doesn't mean it can't be literally true, or almost so, for some actors. Particularly, actors in situations where you can appeal to quasi-evolutionary forces that drive out actors who do not either do the math or behave as if they were doing the math. In general, rational choice models aren't so great at describing the consequence-free behavior of individual voters, and are better at describing the behavior of serious candidates who have something to lose.

It also doesn't mean that the assumptions of homo economicus can't be perfectly reasonable simplifying assumptions for even those cases where it's quite far from true, so long as you're satisfied with using very blunt-instrument assumptions to generate broad, stylized results. Even for mass voters, using simple, wrong, rational-choice models will probably get you the correct sign on most variables. You just really, really wouldn't want to make precise predictions of their effect size.
posted by ROU_Xenophobe at 5:50 AM on November 9, 2011

Insofar as K&T do not claim that it's irrational to use Bayes Rule, then I'm fine.

No I don't think their claim has anything to do with the rationality of using Bayes or not, but rather the untrained mind does not naturally reach to calculate conditional probabilities, instead looking for an easier answer that just sort of pops up out of a wall of numbers.

Isn't this a thread where people are talking about the online questionnaire or isn't it? I thought it was about a sort of interesting profile of Daniel Kahneman. I didn't even look at the questionnaire until people started complaining about. Certainly I agree its a crappy quiz. And I think we both completely agree about what is right and wrong with question 4.

Not really. The data show that homo economicus is not literally true for all actors, which would surprise nobody.

That doesn't mean it can't be literally true, or almost so, for some actors. Particularly, actors in situations where you can appeal to quasi-evolutionary forces that drive out actors who do not either do the math or behave as if they were doing the math. In general, rational choice models aren't so great at describing the consequence-free behavior of individual voters, and are better at describing the behavior of serious candidates who have something to lose.

No its more than that. In the case of relative risk aversion the entire population in aggregate is not a rational actor. Its one of the plausible explanations for the persistance of the B/M pricing anomaly.
posted by JPD at 6:44 AM on November 9, 2011

Instead of picking nits over an admittedly poorly worded 4 question quiz in Vanity Fair of all places why don't you all pick up one of their books.

Because what we have in front of us is the VF article? Just like if this were non-social science journalism with obvious nonsensical inaccuracies, people are going to hate on a misleading article. I think we're doing the real scientists a favor by not just ignoring when media get it wrong.
posted by a robot made out of meat at 6:51 AM on November 9, 2011 [1 favorite]

Stock pricing is not all choices, though.

And, me, I'd put stock markets closer to the low-consequence choices of voters than the high-consequence decisions of candidates. Sure, there's money involved, but pseudo-evolutionary forces are only going to drive out the absolutely most foolish investors.

I don't think we're really disagreeing.
posted by ROU_Xenophobe at 6:51 AM on November 9, 2011

Stock pricing is not all choices, though

I'm not quite sure what you mean?
posted by JPD at 7:29 AM on November 9, 2011

I'm quite surprised at how competitive our culture is to getting "right" answers on a quiz.
It may look like a quiz, but it is not. It's not about getting "right" answers, it's about how people perceive problems and talking about how the questions are worded is missing the point. The questions and quiz are designed to highlight and amplify the ambiguities of how people think about certain problems. For example, Linda the bank teller. Sure, it's reasonable to assume that by leaving out the feminism aspect implies lack of feminism activity, thus driving more people to select the second answer. The point is not about getting the "right" answer, its about the underlying assumptions that drive people to select 2. It highlights that most people usually don't take facts at face value, they read their experiences into those facts and create an internal story consistent with their experiences inspired by those facts, and then answer the question based on their internal stories weighted more heavily than the actual facts.

On a slightly different note, I find it interesting how, I too, was initially annoyed at the quiz. I was annoyed because it felt like the answers were discounting my experiences. Which, after some reflection and more reading, I realized was exactly the point of the quiz, to show how much people read into questions like these.

I think its good to be aware of this phenomenon, but I also think that it's a socially beneficial. What I mean by that this is exactly the type of quiz where two people can get two different answers and both still be right, and that if those two people were to discuss their reasoning, chances are that they will both have their minds expanded a bit.

It also illustrated something slightly evil that lurks in multiple choice questionnaires that I never quite noticed before. All in all, I found it quite thought provoking.
posted by forforf at 7:32 AM on November 9, 2011

I'm not quite sure what you mean?

The fact that some actors making some choices show clear and significant departures from the predicted outcomes of simple rational-choice models does not mean that all actors making all choices do likewise.

Homo economicus isn't bullshit. It's limited in application, and the severity of its simplifying assumptions puts sharp bounds on the precision of its predictions in many circumstances (and many economists may well pay insufficient attention to those limits). But that isn't the same as simple bullshit. Especially in circumstances where the rest of your theory is so highly abstracted away from reality that you can't expect to predict more than the signs on variables (or at most to make rough stylistic divisions between ``small'' and ``large'' effects) plain and simple rational choice models may well be good enough.
posted by ROU_Xenophobe at 8:15 AM on November 9, 2011

I think we are arguing at cross purposes here - and I was actually asking about your statement that
Stock pricing is not all choices anyway. But to address your comment

Yes absolutely in general homo economicus is fine and reasonable (and practical) general simplificiation, however much of the empricial evidence surrounding how humans react to risk and probability shows that the idea that humans are on average rational actors is false. There are of course a small % of people who either by training or some innate difference in brain wirings are able to remain totally rational actors but the population as a whole is not.

So yeah, in the context of this particular case its a flawed simplification.
posted by JPD at 8:27 AM on November 9, 2011

I was actually asking about your statement that Stock pricing is not all choices anyway

Didn't I address that? Even if stock pricing shows anomalies, that doesn't mean that all choices do. I didn't mean anything more than that.
posted by ROU_Xenophobe at 9:02 AM on November 9, 2011

Wait, are you sayng that Kahneman cited the puzzle? I was under the impression that he authored it.

These are not mutually exclusive. And given that the explanation was considered and rejected 30 years ago, complaining that no one mentions it now is silly. No one mentions the aether explanation for light waves anymore, either. Before you imply that researchers are blind to obvious expanations, a little reading might be in order.
posted by Philosopher Dirtbike at 9:08 AM on November 9, 2011

These are not mutually exclusive. And given that the explanation was considered and rejected 30 years ago, complaining that no one mentions it now is silly. No one mentions the aether explanation for light waves anymore, either. Before you imply that researchers are blind to obvious expanations, a little reading might be in order.

No, they are not, but generally when people use the word "cite", they mean quote, and generally not themselves.

You seem dead set on pointing out that Kahneman addressed this objection, because you seem to think that I've done a literature search on this puzzle/question/whatever and overlooked what researchers have to say about this argument. Well, you got me, I was lazy and skipped that step. Congratulations!

But Kahneman and Tversky's illustration/puzzle/question has a life of its own outside of the academic literature, and every time I've seen it cited there, the explanation is always that it's an example of conjunction fallacy, that people who answer B are "so wedded to stereotypes and so distracted by vivid descriptions that it will seize upon them, even when they defy logic, rather than upon truly relevant facts." Not once have I read anyone follow up with the probability of A and B variant, or the Borg example. I am one of those people who immediately thought, upon hearing choice B, that choice A meant "Linda is a bank teller, and not a feminist." And yes, I assume that there are others who think the same way I do. Kahneman and Tversky's paper argues, and I have no reason to disbelieve their conclusion, that this is not the source of most people's confusion. But then again, I never claimed that.

No one mentions the aether explanation for light waves anymore, either.

And this is the example which actually gives the lie to your argument. You seem to think that addressing this objection once, 30 years ago, is adequate. Hey, guess what? The aether explanation for light waves is mentioned all the time. You know why? Because in order to teach electricity and magnetism and what the Michelson–Morley experiment actually proved, as well as to draw a distinction between sound and light, the once-widespread and not-unreasonable-at-the-time belief that light had to travel through a medium is brought up quite a bit, in physics textbooks as well as in works addressed to laymen.
posted by UrineSoakedRube at 9:59 AM on November 9, 2011

Me: "Insofar as K&T do not claim that it's irrational to use Bayes Rule, then I'm fine.

JPD: No I don't think their claim has anything to do with the rationality of using Bayes or not, but rather the untrained mind does not naturally reach to calculate conditional probabilities, instead looking for an easier answer that just sort of pops up out of a wall of numbers.

Well, in the answer that was provided, note that whoever wrote the article claimed that K&T argue it's a cognitive error to be given priors about some event, then be told some information that is actually correlated with those events, and then use answer the question and note the posterior belief is no longer the same as the prior belief. I mean, imagine if the problem had read like this:

There's 100 people, 50 of whom are males. Psychologists have profiled them and here's a sample bio. "Person 82 has a vagina". What's the probability the bio is about a male?

Given the existence of transsexuals, the conditional probability that person 82 is a male isn't 1, but it's damn near close. And yet as far as I can tell, whatever this heuristic is that's being described, it seems to be the interviewer at least thinks K&T would say that drawing that belief after reading the bio is wrong - the probability even with that information is still 0.5. But that is stupid.

So either K&T are confused or the author is confused, but someone is confused and that someone isn't me.
posted by scunning at 10:00 AM on November 9, 2011

'tis the author I'm sure.

Even if stock pricing shows anomalies, that doesn't mean that all choices do. I didn't mean anything more than that.

Gotcha.
posted by JPD at 11:20 AM on November 9, 2011