The Complexity of a Controversial Concept

June 20, 2005 6:03 PM

The Logic of Diversity"A new book, The Wisdom of Crowds [..:] by The New Yorker columnist James Surowiecki, has recently popularized the idea that groups can, in some ways, be smarter than their members, which is superficially similar to Page's results. While Surowiecki gives many examples of what one might call collective cognition, where groups out-perform isolated individuals, he really has only one explanation for this phenomenon, based on one of his examples: jelly beans [...] averaging together many independent, unbiased guesses gives a result that is probably closer to the truth than any one guess. While true — it's the central limit theorem of statistics — it's far from being the only way in which diversity can be beneficial in problem solving." (Three-Toed Sloth)
posted by kliuless (6 comments total)

Suroweicki on "the wisdom of crowds" at work: he starts with the jellybeans, but moves on to discuss flattened information hierarchies and shared decision making. So there.

(self link!)
posted by paul_smatatoes at 6:22 PM on June 20, 2005

Technically, the central limit theorum refers to the characteristics of a sampling distribution.

Let's imagine we're looking at the heights of people in the population. There is a real value out there for the mean height, say, of British people. We could find it by measuring everyone, but to do so would be excessive. Instead, we take a random sample and measure the heights of those selected. The central limit theorum states that, for any collection of samples, the mean of the mean values for those samples will approach the mean of the population. (More samples will generally bring you closer to the true value) Meanwhile, the distribution of those sample means will follow a classic bell curve distribution.

The fact that this is true for any population, whether the underlying distribution follows a unimodal, symmetric bell curve or not, is one of the most surprising and useful facts of statistics.
posted by sindark at 8:55 PM on June 20, 2005

Hmmm, a few years ago there was a book called "Extraordinary Popular Delusions And The Madness Of Crowds"...
posted by blue shadows at 9:13 PM on June 20, 2005

I believe the "wise" masses have also bequeathed us with terms like "committee-think", embodying the idea that the output of a mass vote is as prone to be bland and uninspired as it is apt to be technically correct.

Nobody in a corporation truly believes that the leader at the top is the most qualified to be there. Hierarchies exist for reasons of efficiency, just like a military organization. How often do we hear that its more important to make some decision than to waver? Generals do not poll their troops.

Another example pointed out is how the crowd is often right when polled on "Who wants to be a Millionaire?" The important caveat is they are right on the easy, common sense questions. As the questions get harder and more specialized, the masses become less wise. If the million-dollar question is about a figure from 14th-century Italy, you are much better off calling your friend, the historian.
posted by vacapinta at 10:56 PM on June 20, 2005

"If the best problem solvers tend to think about a problem similarly, then it stands to reason that as a group, they may not be very effective," Page says.

This is a point that leapt out of Surowiecki's book for me. His point was that if an organization has a bunch of people all of whom come from similiar backgrounds, with similiar educations, they're all going to think about problems similiarly and come up with similiar solutions. Look at the FBI, the CIA, the government in general, and then look at their recent failures. Surowiecki argued these organizations need people who don't look, act, or think like the majority of the people in these organizations to better solve the problems they're facing.
posted by raaka at 12:29 AM on June 21, 2005

When Surowiecki's book came out I also immediately thought of Extraordinary Popular Delusions And The Madness Of Crowds, which actually came out over 150 years ago and is available to read online. Most famous is the Tulipomania chapter. Always be suspicious of mob mentality.
posted by Slack-a-gogo at 4:55 AM on June 21, 2005