But What Do They Know, Anyway

January 25, 2021 1:06 PM

According to two recent papers, the Dunning-Kruger Effect is probably not real. (Previously and previouslier.)
posted by cosmic owl (29 comments total)

sets off to scan the credentials of the writers to see if they set off any "of course you would say that Dunning-Kruger isn't probably not real" vibes.
posted by Sauce Trough at 1:11 PM on January 25, 2021 [7 favorites]

I will nevertheless continue to direct people to this video in which John Cleese explains stupidity.
posted by Sing Or Swim at 1:21 PM on January 25, 2021 [9 favorites]

If the "effect" (a strong correlation between not knowing much, and thinking you know a lot) is not real, nonetheless failures of self-assessment are a real phenomenon and can lead to serious real-world consequences. See for example the last four years in the United States Federal Executive Branch.
posted by Aardvark Cheeselog at 1:34 PM on January 25, 2021 [24 favorites]

The problem I see is that people frame the effect as being "novices see themselves as experts", which hasn't been my experience. Instead, it tends to be that an individual who has an unwarranted positive judgement in their skill will misjudge their actual skill because of this. One of the studies catches this in their abstract:

Our results further confirm that experts are more proficient in self-assessing their abilities than novices and that women, in general, self-assess more accurately than men.

The point that women more accurately self-assess comes from the fact that our society tends to encourage men to think highly of their skills in general (and, of course, to have women doubt theirs.) It's also why the biggest examples of the Dunning-Kruger effect in real life tend to be subject matter experts working outside of their field (or in other words, why doctors tend to be horrible small business owners.)
posted by NoxAeternum at 1:41 PM on January 25, 2021 [43 favorites]

Warning: If you're sure they're wrong before reading the papers, doing so might cause a universe-ending paradox.
posted by kleinsteradikaleminderheit at 1:43 PM on January 25, 2021 [24 favorites]

I feel very confident it is. I say this without even reading the article.
posted by From Bklyn at 1:47 PM on January 25, 2021 [12 favorites]

Next up: someone will publish a paper - probably entitled "But Actually" - showing that mansplaining isn't real.
posted by PhineasGage at 1:48 PM on January 25, 2021 [19 favorites]

the Dunning-Kreuger Effect may not be real, but nonetheless, Vonnegut was right when he said "the problem with dumb bastards is that they're too dumb to realize that they're dumb bastards."
posted by entropone at 1:55 PM on January 25, 2021 [16 favorites]

For another example of “random data makes plots like this too, so the effect can’t be real,” go back fifteen years to the debate about the “hockey stick” temperature graph produced in the context of climate change research. In that case, the effect turned out to be real.
posted by fantabulous timewaster at 1:55 PM on January 25, 2021 [3 favorites]

The jury is still out on the Done-in-by-Krueger effect, which hypothesizes that people in horror movies tend to overestimate their ability to stay alive.
posted by oulipian at 2:00 PM on January 25, 2021 [56 favorites]

There are several confusing elements in this article, but I think I understand the essential argument:

A quartile ranking of random data will produce a nice 1/1 sloped line, because you just sorted it.
Averages of blocks of random data will produce a flat or random slope (depending on the sample size) clustered around the mean because, well, they're random.

Plotting these against each other will give the illusion of the lowest quartile underperforming and the highest quartile overperforming.

A point of confusion for me was that in the random data example the actual ability appears correlated with the perceived. But with quartiles, that could happen by chance (or be cherry-picked).
posted by justkevin at 2:20 PM on January 25, 2021 [1 favorite]

The Dunning-Kruger Effect might be real, but we could never know that, because if it is real, that could only be determined by a legitimate expert in the Dunning-Kruger Effect, none of whom could exist prior to the determination.
posted by jamjam at 2:35 PM on January 25, 2021 [15 favorites]

Surely the D-K expert could exist, and we could know it, but the expert couldn’t know it?
posted by clew at 3:46 PM on January 25, 2021 [4 favorites]

someone will publish a paper - probably entitled "But Actually" - showing that mansplaining isn't real

Companion piece to the explanation of why manspreading isn't real, "Butt, Actually'.
posted by biffa at 4:05 PM on January 25, 2021 [6 favorites]

Other observations on the Dunning-Kruger effect.

Part of what we lump into the D.K. effect I think comes from "Idiot Insulation", where actions taken by more thoughtful individuals are ignored by more impulsive types with "on average" no consequences.

An example of this is motorcycle helmets. Riding a motorcycle is dangerous, but statistically just wearing a helmet and other protective gear drastically improves your survivability in an accident. Since motorcycle accidents are not common, any impulsive type can jump on a bit without gear, zip around, and come back unharmed proclaiming all that stupid gear unnecessary. Now the more thoughtful person is in the position of explaining statistics to someone who doesn't want to hear it, and the other has their one example to prove their point. At this point it becomes a matter of pride on who is right, an Godwin will be invoked at some point.

This doesn't quite fall into the category of the D.K. effect, but there is some sort of cognitive bias that goes along with willful ignorance; hence "Don't Play Chess with a Pidgeon".

(One of the most egregious examples I know of is the sport of drunk driving that people I grew up with insist on practicing, with one of the most outlandish excuses being "I'm doing this to protest the government". 10 kids that started high school with me didn't make it to the end because of that crap, yet people still do it with relish)
posted by The Power Nap at 4:16 PM on January 25, 2021 [2 favorites]

In tandem with this, there is a psychological theory that suggests all people are capable of learning any skill, across the board, depending on the way the skill is taught (basically saying any type of person is capable, aside slight biological differences in the mind).

I cannot recall the name and I'm trying to find it, if anyone knows the reference, please recall it. It's presented by a 60s psych, and well adopted in education.

Dunning-Kruger seems to compliment the, "most people are dense/under-educated," perspective (when in reality, it's most likely "most people are educated in a specialized or narrow perspective, relative to their culture needs and bias")
posted by firstdaffodils at 4:20 PM on January 25, 2021 [1 favorite]

The provably true aspect of DK is captured in the pithy saying emphasing experience over naive enthusiasm in hazardous workplaces e.g. "There are old electricians, and bold electricians, but no old+bold electricians."
posted by seanmpuckett at 4:59 PM on January 25, 2021 [2 favorites]

In Education studies, teachers with a high sense of self-efficacy tend to have better student outcomes than those with a low sense, even if the latter are better trained / more experienced etc. (See Tschannen Moran, Woolfolk Hoy, et al. - multiple papers and uses of Teachers self efficacy scale instrument).
posted by b33j at 5:36 PM on January 25, 2021 [5 favorites]

Maybe the Dunning-Kruger Tendency. I, for one more, never considered the DKE a clinical finding, or at least never used it as one, but it's OK, ADHD had to run through the same gauntlet and it turned out fine.
posted by rhizome at 5:39 PM on January 25, 2021 [2 favorites]

...and those times when there was one set of footprints, why those were the times that your belief in the Dunning-Kruger effect carried you.
posted by CynicalKnight at 6:50 PM on January 25, 2021 [7 favorites]

There is a definitely a pop-psychology version of the Dunning-Kruger effect that doesn't seem to be based on the original research. It's illustrated by the meme with the U-shaped graph showing a left peak labelled "mount stupid" for novice overconfidence followed by and a "valley of despair" in the middle (where you realise how little you know), followed by the "slope of enlightenment" towards a "plateau of sustainability" on the right. There's nothing like this graph in their paper, it's made up for humour purposes.
posted by L.P. Hatecraft at 7:47 PM on January 25, 2021 [2 favorites]

As far as correlation/causation + valid experience/skill versus confidence/etc, I'm not surprised the DK effect is a dash shilly.

It's a psychological personification of the Tarot's Fool Card (the Fool, a novice, full of enthusiasm but without experience): sounds as though it may be an old narrative repeated in a new light, or format.

Areas of learning are different within the brain. Personality doesn't always completely correspond with the person's acquired knowledge or skills. Additionally, I've met many, many, many instructors who are still absurdly passionate about what they do, late into their interests (why wouldn't they be?).

..the Trump ref is key, tho.
posted by firstdaffodils at 11:51 PM on January 25, 2021

..has anyone noticed this thread is very good.

10/10, not-done-in by Kruger-effect. (Author spreads cat food across floor for Schrodinger, the cat)
posted by firstdaffodils at 12:13 AM on January 26, 2021 [1 favorite]

Very interesting! I've just read the blog post and not the original articles yet, but from that discussion at least, one should have the takeaway that the DKE is not well supported by the original work by Dunning and Kruger, not necessarily that the DKE is actually contradicted by this new work. Without some modeling to show not just the "no-effect" model but also what the distribution of results is over various simulated levels of DKE bias, it's hard to judge whether the original data is or isn't within the range of what would be expected for the null model. Perhaps the authors of one of the papers criticizing the DKE actually did this, and the author of the blog post just didn't discuss it.
posted by biogeo at 1:19 AM on January 26, 2021 [2 favorites]

There is a definitely a pop-psychology version of the Dunning-Kruger effect that doesn't seem to be based on the original research. It's illustrated by the meme with the U-shaped graph showing a left peak labelled "mount stupid" for novice overconfidence followed by and a "valley of despair" in the middle (where you realise how little you know), followed by the "slope of enlightenment" towards a "plateau of sustainability" on the right. There's nothing like this graph in their paper, it's made up for humour purposes.

Alexander Pope nailed it long ago. Everybody knows the first line "A little learning is a dangerous thing" but very few people know the rest of the poem.
posted by srboisvert at 2:40 AM on January 26, 2021 [2 favorites]

A potentially relevant tweet from 2019 involving Serena Williams and the percentage of men in a survey who believe they could win a point against the Grand Slam tennis legend.
posted by Bella Donna at 8:44 AM on January 26, 2021 [5 favorites]

There is another theory widely used in education, emphasizing an instructor's ability to rely on the natural, or already present qualities of a student in order to hone those qualities.

Not quite like possessing extra enthusiasm or confidence from inexperience, but showing the many ways a person can possess or grow knowledge.


..btw, if anyone can find a video of Serena Williams just dunking on dudes (in a tennis match), please share.
posted by firstdaffodils at 12:45 PM on January 26, 2021

Serena Williams just smoking a bunch of dudes on the tennis court.

Marshawn Lynch is both more realistic and more respectful.
posted by Lexica at 1:14 PM on January 26, 2021 [3 favorites]

More beautiful than lions hunting in the African savanna. Wow. Thank you.

"Confident in my ability to properly tennis, I take the court. I smile at my opponent. Serena does not return the gesture. She'd be prettier if she did, I think. She serves. The ball passes cleanly through my skull, killing me instantly."
posted by firstdaffodils at 1:32 PM on January 26, 2021 [5 favorites]