Going Meta
July 6, 2016 1:09 PM   Subscribe

Metaknowledge is knowledge about knowledge. Metaknowledge means you are aware of what you know or don’t know, and of where your level of knowledge stands in relation to other people’s.

Dražen Prelec, a behavioural economist at the Massachusetts Institute of Technology (MIT), is working on a way to smarten up the hive mind. When you average everyone’s judgments, information that is known to all gets counted repeatedly, once for each person, which gives it more significance than it deserves and drowns out diverse sources of knowledge. In the end, the lowest common denominator dominates.

What is needed is a better way to measure the value of a person’s knowledge before putting it into the mix.

Whereas you might have no independent way to verify people’s knowledge, you can confirm their metaknowledge. Metaknowledge functions as a powerful bullshit detector. It can separate crowd members who actually know something from those who are guessing wildly or just parroting what everyone else says.
posted by NoRelationToLea (48 comments total) 38 users marked this as a favorite
 
Wait, what if I don't know what metaknowledge I know?

(no, wait, reverse that)

(no wait wait, let's code an algorithm to automatically re-reverse)
posted by sammyo at 1:20 PM on July 6, 2016


Or as Donald Rumsfeld might say, metaknowledge is a "Known known."
posted by deadaluspark at 1:44 PM on July 6, 2016 [8 favorites]


Came to make Rumsfeld joke. Can only add, also the "known unknown."
posted by allthinky at 1:52 PM on July 6, 2016 [5 favorites]


Gosh that guy was some kind of twisted poet, wasn't he?
posted by deadaluspark at 1:54 PM on July 6, 2016 [1 favorite]


The Unknown
As we know,
There are known knowns.
There are things we know we know.
We also know
There are known unknowns.
That is to say
We know there are some things
We do not know.
But there are also unknown unknowns,
The ones we don't know
We don't know.
posted by leotrotsky at 1:58 PM on July 6, 2016 [16 favorites]


So sweet.
posted by Kabanos at 1:59 PM on July 6, 2016 [7 favorites]


And so cold.
posted by Kabanos at 2:00 PM on July 6, 2016 [23 favorites]


If I'm getting the idea right, people who are best at predicting how many other people agree with them are also best at predicting the right answer.

People who know what the crowd's opinion is but disagree are the most valuable outliers.

No doubt this social psychology result will blow up into nothing in a decade or so, but it does suggest that people who are able to consider at least two opinions at once (other people's and their own) are more likely to get the right answer.

This is just the justification I need for reading both Metafilter and the National Review blog during election season.
posted by clawsoon at 2:01 PM on July 6, 2016 [15 favorites]


The unknown.
So sweet.
And so cold.

This is why I come to metafilter.
posted by brambleboy at 2:04 PM on July 6, 2016 [3 favorites]


We're 9 comments in and not a single metaknowledgefilter joke yet? What are we even doing here?
posted by BigHeartedGuy at 2:09 PM on July 6, 2016 [4 favorites]


/metamoron
posted by jonmc at 2:10 PM on July 6, 2016


People who know what the crowd's opinion is but disagree are the most valuable outliers

E.g., yoink -- gone, but not to be forgotten.
posted by jamjam at 2:11 PM on July 6, 2016 [1 favorite]


metaknowledgefilter is to MeFi as correcthorsebatterystaple is to XKCD.
posted by iamkimiam at 2:15 PM on July 6, 2016 [2 favorites]


for my part I'm more interested in the set of all knowledge not contained in the set itself.
posted by You Can't Tip a Buick at 2:27 PM on July 6, 2016 [7 favorites]


I teach history, and the concept of how limited most people's knowledge is on any given field of history is one of the major points I try to get across to my students, not to humiliate or intimidate them but to get them to recognize a very real set of starting limitations.

As per above, Rumsfeld's "known knowns" talk is a great way of starting a conversation on this, and I usually play that clip. It's about trying to convey just how enormous some areas are, about how there's whole side areas most never bother with that are fundamental to understanding how something that everyone is familiar with played out (e.g. you can't understand the American Revolution unless you understand the English Civil War; people charging machine guns for four years in WWI is unfathomable until you look at the earlier Franco-Prussian and Russo-Japanese Wars, etc). Things often resist right or wrong one-sentence answers, and the constant release of new knowledge (memoirs, archives, new studies in general) can change things back and forth even when we do "know" the answer for a while, but most people aren't even familiar with the idea of evolving knowledge: the tendency is for people to imagine a black and white final correctness. There's plenty of bad-faith ways people can go wrong, but failures of metaknowledge explain how people honestly and in good faith think they know most everything there is to know about something because they saw Braveheart, The Imitation Game, or a single one-hour History Channel documentary a few years back.

A good way, I've found, is asking how many in the class have a hard (complex) job. I ask if they think they could explain everything there is about their job in five or ten minutes and expect someone to be able to both understand it and do it properly from then on without further help. Usually the idea makes them laugh. I then point out that people imagine history works like this all the time.
posted by Palindromedary at 2:31 PM on July 6, 2016 [36 favorites]


I'd love to hear about some notable exceptions to this tendency — I could imagine that there are some topics where the data are skewed in some strange direction by an additional factor that messes with our knowledge and/or metaknowledge.
posted by iamkimiam at 2:31 PM on July 6, 2016


Well, as the newspaper example in the article illustrates, there's a worrisome disconnect between knowledge and meta-knowledge-- there's no intrinsic reason that someone who knows a lot about newspaper circulation figures should also be knowledgeable in forecasting economic growth.

In fact, it is not clear to me whether the correlation between meta-[subject]-knowledge and [subject]-knowledge is privileged in some way, or whether there's just a general correlation between all types of knowledge, and meta-knowledge just happens to be a convenient-to-measure proxy. That is to say, if you had the participants also answer basic questions about science or history, how strongly would that correlate with other knowledge? (Surely someone has done this study)
posted by Pyry at 2:46 PM on July 6, 2016 [3 favorites]


If I'm getting the idea right, people who are best at predicting how many other people agree with them are also best at predicting the right answer.

This sort of sounds like my Learned League scoring strategy. If I know the answer, it must be an easy question and I score accordingly.
posted by maryr at 2:46 PM on July 6, 2016


The people who can best think about how may others will overthink like them, are also the best at overthinking.

/Metabeanplating
posted by Kabanos at 2:53 PM on July 6, 2016 [1 favorite]


It has always mystified me why the known-knowns/unknown-knowns/unknown-unknowns aphorism gets attributed to Rumsfeld. It was, virtually verbatim, common parlance among the Landmark Education and/or NLP crowd long before Rumsfeld rattled it off.
posted by lastobelus at 3:00 PM on July 6, 2016


If I'm getting the idea right, people who are best at predicting how many other people agree with them are also best at predicting the right answer.

These are the people I want on my Family Feud team!
posted by ThePinkSuperhero at 3:02 PM on July 6, 2016 [6 favorites]


Gist:

Dumb people don't realize how much they don't know and think they are smart.

Smart people realize how much they don't know and think they are dumb.
posted by CrowGoat at 3:08 PM on July 6, 2016 [1 favorite]


The best lack all conviction,
while the worst
Are full of passionate
intensity.

posted by leotrotsky at 3:14 PM on July 6, 2016 [10 favorites]


Once you know that quote you start seeing variations of it everywhere.

It's the Dunning-Kruger Baader-Meinhof Effect.
posted by leotrotsky at 3:17 PM on July 6, 2016 [16 favorites]


So, applied epistemology?
posted by zinful at 3:26 PM on July 6, 2016 [1 favorite]


ok, that was incredibly flippant of me and I apologize. I'm actually pretty pleased to see someone from an Economics background explicitly doing some philosophy, and this practical application of a theory that too often is considered inexplicable or inapplicable to actual knowledge, rather than some Platonic ideal of knowledge, is pretty exciting!
posted by zinful at 3:39 PM on July 6, 2016 [2 favorites]


Behave yourselves, people. Nobody wants to have to open a "Going Meta" MeTa about metaknowledge, to my knowledge.
posted by duffell at 3:42 PM on July 6, 2016 [7 favorites]


I'm thinking of the crackpots like the sovereign citizens, or MRAs, etc. Some of those types have a persecution complex that under-represents how many people agree with them. The "everyone's against me" idea. Just because they are correct that "everyone" is against them, does not make them any more credible.

(although, 3% is still probably a gross overestimate of how many people will side with the three-percenters, so.)
posted by ctmf at 3:47 PM on July 6, 2016


Also, bad news. The only person who seems to have predicted how popular Trump would be is... Trump.
posted by ctmf at 3:50 PM on July 6, 2016 [1 favorite]


Now that I've read this article I feel that I'm armed with meta-metaknowledge. Yes, I'm willing to go turtles-all-the-way-down on this one.
posted by Abehammerb Lincoln at 4:03 PM on July 6, 2016 [1 favorite]


It was also neat to read how social researchers have used a variation on this to learn about socially unacceptable behaviour. If somebody tells you, "Oh, no, I would never do that, but everybody else does it," then they probably do it.
posted by clawsoon at 4:19 PM on July 6, 2016 [4 favorites]


I don't know much of anything but I know a little about everything.
posted by shockingbluamp at 4:29 PM on July 6, 2016




People who know what the crowd's opinion is but disagree are the most valuable outliers.

I'll be sure to mention this to my boss during the next staff meeting.
posted by ZenMasterThis at 4:37 PM on July 6, 2016 [2 favorites]


"Freon"

-Harold Finch.
posted by clavdivs at 4:39 PM on July 6, 2016


What we talk about when we talk about what we talk about when we talk about talking.
posted by duffell at 4:58 PM on July 6, 2016 [2 favorites]


Metaknowledge? I blame the tendency for 'young people' (starting in my generation, I can't blame others) to use "y'know" as a verbal placeholder. I much prefer "well", "yeah" or a long drawn-out conjunction: "annnnnnnd...", "sooooooo...".
posted by oneswellfoop at 5:11 PM on July 6, 2016


Meta
posted by nickmark at 6:38 PM on July 6, 2016 [7 favorites]


I haven't read the original article, but that doesn't matter, as I'm quite sure I know what the authors are up to, those cheeky bastards!

A simpler version of this would be: "We measured the IQ of all subjects and weighted the answer by their IQ". In other words, "listen to smart people."

The thing about IQ is that (depending whom you believe) it doesn't accurately measure a lot of human intellectual facilities. So using IQ as a proxy for "truth teller" is fraught, because we all know that "book smart" is not the same as "street smart" an neither is the same as "wise".

I commend the authors for their presumed logical approach to this topic.
posted by soylent00FF00 at 7:42 PM on July 6, 2016


I think you could be utterly wrong. The article actually supports a case that domain expertise should indeed entail sophisticated "street smarts"--that the people who are authoritative are those who demonstrably have scaled their theory of mind up to account for the entire community's thoughts and perspectives. This is a clear theme running through the author's argument.

I think the authors would clarify that IQ/intelligence are not metaknowledge. Your criticism may be on to something, but as you admit to not having read the article, the relationship between intelligence and the explicit subject of the article, namely, validity of knowledge, is unclear. Actually, I have not thought up a strong critique of the article yet, and that's something that subtly vexes me.
posted by polymodus at 8:24 PM on July 6, 2016 [2 favorites]




A simpler version of this would be: "We measured the IQ of all subjects and weighted the answer by their IQ". In other words, "listen to smart people."

The article is not about that at all.
posted by Miko at 9:44 PM on July 6, 2016 [5 favorites]


The last paragraph of the post immediately made me think of this E2 node I read years ago:

Foolproof method to determine how much a person knows about computers
posted by teatime at 12:17 AM on July 7, 2016 [1 favorite]




"That's us, dude!"
posted by MtDewd at 5:52 AM on July 7, 2016 [1 favorite]


A simpler version of this would be: "We measured the IQ of all subjects and weighted the answer by their IQ". In other words, "listen to smart people."

The article is not about that at all.

Apologies, sloppy writing on my part. I meant to say that a simpler version of the concept (of assigning some weight to an individual's knowledge) would be to simply weight by IQ - I was not intending to say that this is what the authors did in this particular paper. I should probably go read the paper now :-)
posted by soylent00FF00 at 7:23 AM on July 7, 2016


Actually, I have not thought up a strong critique of the article yet, and that's something that subtly vexes me.

After reading the article I'm not very impressed - it includes lots of anecdotes and internal inconsistencies - for example, they discuss a survey of climate scientists where they found that only 1 of 15 gave a prediction with small error bars, and that one scientist turned out to be a contrarian, they do not mention statistical power, confidence intervals, or (somewhat ironically) the error bars of their own prediction. In other words, my bullshit detector is going off on this "study" as it has a N of 16 and is apparently dependent on a single data point. Meh.

The other studies seem to largely use the method of asking a person a question and also asking them to judge what other people will answer, and observing that there's a correlation between correct predictions on the latter with correct answers on the former. This effect seems more pronounced when the answer to the question is contrary to popular opinion (what one could perhaps classify as "trick questions").
posted by soylent00FF00 at 7:34 AM on July 7, 2016 [3 favorites]




« Older A Diamond and a Kiss: The Women of John Hughes   |   What does it mean to be a modern woman? Newer »


This thread has been archived and is closed to new comments