“Easy read” should not mean “easy write.”

December 13, 2011 1:07 PM

One of the delights of the books and the blog is the authors’ willingness to play with ideas and consider alternative explanations. But unquestioning trust in friends and colleagues combined with the desire to be counterintuitive appear in several cases to have undermined their work. They—and anyone who wishes to convey economics and statistics to a popular audience—just need to take the next step and avoid, in any given example, privileging one story over all other possibilities.

Freakonomics: What Went Wrong?
posted by RogerB (52 comments total)

Isn't this a problem in the dissemination of all knowledge?
posted by grog at 1:11 PM on December 13, 2011

"As the authors of statistics-themed books for general audiences, we can attest that Levitt and Dubner’s success is not easily attained."

This has a touch of sour grapes about it, no?
posted by modernnomad at 1:14 PM on December 13, 2011 [4 favorites]

Between Freakanomics and the Prosperity Doctrine it's amazing the economy didn't tank harder in 2008.
posted by Artw at 1:19 PM on December 13, 2011 [3 favorites]

This is the science they call the dismal science, right? My epiphany was reading Sumner's Money Illusion blog and he tells us that having expertise in the subject of Economics generally implies no special competence at managing one's personal investments. I was like WTF? but have since seen this confirmed in a number of places as conventional wisdom.
posted by bukvich at 1:25 PM on December 13, 2011 [3 favorites]

Among other things, I learned from this article that these guys were the ones that came up with the birth month-soccer success idea that Malcolm Gladwell spouted on the radio the other day. Which is perfect because I think of these guys as "Malcolm Gladwell with numbers."
posted by ignignokt at 1:26 PM on December 13, 2011 [7 favorites]

The author didn't mention their biggest embarrassment on this front: their recounting the racist Orangejello and Lemonjello urban legend because their friend swears he met them at a grocery store.
posted by painquale at 1:26 PM on December 13, 2011 [11 favorites]

If you listen to the Freakonomics podcast, try replacing it with EconTalk. Russ Roberts is a great interviewer, and it's consistently enligtening (though the recent one on gender differences made me think less of it).
posted by painquale at 1:30 PM on December 13, 2011 [3 favorites]

I look forward to American Scientist: Freakonomics: What Went Wrong? : What Went Wrong?

A casual observation of the article reveals semantic quibbling and speculative or even erroneous claims presented with an air of certainty.
posted by Bovine Love at 1:30 PM on December 13, 2011

If you want competence in investments, you talk to someone in Finance, not Economics. Or if you're smart, you ignore both of them and throw your money into company-matched retirement funds, houses, and index funds.
posted by Yowser at 1:31 PM on December 13, 2011 [4 favorites]

This article raised my opinion of Freakonomics.

I thought it was totally dumb, now I think it's just mostly dumb.
posted by jamjam at 1:35 PM on December 13, 2011 [1 favorite]

To me, Freakanomics always seemed like the classic correlation is not causation issue. Or perhaps trusting too much in superficial numbers without understanding underlying causes. "These two things are close together and must be related!" No, perhaps you're missing the issue that is the real determining factor of bothvthose things.

I would be happy if we de-emphasized economics as the pre-eminent social science. From having to explain to Econ 101 students and libertarians that, no the consumer is not always rational and often makes completely irrational decisions, to the assumption that economic principles can be applied to and solve every problem. I am rather tired of it.

Can we get some sociology up in this shit?
posted by X-Himy at 1:36 PM on December 13, 2011 [13 favorites]

This post actually seems pretty appropriate to Freakanomics.
posted by COBRA! at 1:37 PM on December 13, 2011 [1 favorite]

having expertise in the subject of Economics generally implies no special competence at managing one's personal investments.

this is like how having expertise in the subject of Mathematics generally implies no special competence at the calculation of tips.
posted by madcaptenor at 1:37 PM on December 13, 2011 [5 favorites]

I don't know why economics is called a science, dismal or otherwise. Hypothesis? Experimentation? No.

They need to call it the dismal religion, instead.
posted by absalom at 1:43 PM on December 13, 2011 [10 favorites]

Does the increase of Somalian maritime piracy mean that climate change was recently mitigated? And if we suppress Somalian pirates, does climate change resume? Or is this only function of pre-industrial European piracy in Atlantic waters?
posted by Xoebe at 1:44 PM on December 13, 2011 [5 favorites]

Worse than a "dismal science", Economics these days is the equivalent of Astronomy in the pre-Gallileo era, so tied to long-established assumptions they can't admit the truth that the sun does not orbit around the earth. (And having powerful supporters ready to cut the head off anyone who disagrees)

having expertise in the subject of Economics generally implies no special competence at managing one's personal investments.
Using the "Principals" of Chicago School Economics in your personal investment strategy will pretty much guarantee you losing a fortune in the mid-to-long run. Luckily for them, most prominent Economists (including the Freakonomists) have enough powerful friends with inside connections to ensure their continued affluence.
posted by oneswellfoop at 1:45 PM on December 13, 2011 [8 favorites]

Economics is not a science...it is sometimes called a social science and often a humanities subject.
Being a thereapist does not mean you yourself never have problems etc.
posted by Postroad at 1:46 PM on December 13, 2011

Painquayle, for the record, my sister and I used the on air names Orangejello and Lemonjello on a pirate radio station around Austin back in the 80's, and there was no racial overtone at all to us doing it, and it probably predated whatever urban legend you reference. Just saying that you may be seeing racism where there is none.
posted by dejah420 at 1:48 PM on December 13, 2011

The author didn't mention their biggest embarrassment on this front: their recounting the racist Orangejello and Lemonjello urban legend because their friend swears he met them at a grocery store.

Through the recent MetaTalk discussion of this issue, I've discovered that I actually like Lemonjello as a name. Obviously, you'd need to work on the spelling, but it sounds very pleasant.
posted by Bulgaroktonos at 1:51 PM on December 13, 2011 [2 favorites]

dejah420: a primer.
posted by Navelgazer at 1:52 PM on December 13, 2011 [1 favorite]

There was one particular argument made in Freakonomics that I remember finding specious when I first read it. At some point in the 1980s, American income tax returns started requiring that you give the social security numbers of all your dependents. That year, the number of claimed dependents dropped significantly. This was taken to indicate that millions of people had been cheating on their taxes by claiming nonexistent dependents until they had to back up their claims.

Now, I'm old enough to remember getting my social security number. I was something like ten years old, and the only reason I needed one was because my parents had decided that it would be instructive for me to have a small savings account in my name. If they had not decided on this, I would no doubt have gone several years more without a SSN. My point is that before this change to the rules governing dependents, it was normal for children to not have SSNs. Which means it's pretty much a certainty that at least some of the disappearing dependents were the children of people who put off doing their taxes until the last minute, and who didn't know about the change until it was too late to register their real dependents.

Of course, I imagine that in other cases, it was just a matter of cheating, like Freakonomics claimed. But in what proportion? I don't know. Presumably the vanishing dependents who actually existed would come back the following year, once their parents or guardians registered them for social security. But the Freakonomics guys hadn't bothered looking at the statistics for that year, because they hadn't thought of other explanations, and apparently weren't in the habit of checking their conclusions for conclusions once they reached them. Recognizing this soured me somewhat on all their other findings.
posted by baf at 1:59 PM on December 13, 2011 [12 favorites]

When Freakonomics came out I was an economics major and enjoyed that it the book used economics to explore things that were more interesting than the equations in my econometrics class. I had the blog pretty high on my RSS-reading list.

But I think it was about two years ago when I finally just stopped reading it. It was clear they were publishing things just to be controversial and not really caring about whether or not the data was high-quality. There was one essay about how local food is so bad for the environmental that was pretty scarily bad, written by a history professor, and my agricultural econ professor at the time wrote in to Dubner and Levitt about writing a rebuttal on the blog. They never got back to her, even though she is the most cited author on the subject in academic journals. I guess it's pretty similar to how they treated Das Gupta. I guess people who carefully research subjects as a career and publish tight papers are just too boring.
posted by melissam at 2:07 PM on December 13, 2011 [5 favorites]

I liked Freakonomics. Someone gave it to me and it successfully propped up by my busted bookshelf for a year and a half.
posted by mannequito at 2:09 PM on December 13, 2011 [3 favorites]

Those who promise easy answers are simply idiots.

I'm looking at you, Ron Paul.
posted by Ironmouth at 2:10 PM on December 13, 2011 [5 favorites]

Using the "Principals" of Chicago School Economics in your personal investment strategy will pretty much guarantee you losing a fortune in the mid-to-long run.

Really? I'd have thought that the investment strategy to pick up from the efficient market hypothesis would be long-term diversification, regular rebalancing, and not trying to time the market. (That's basically what I got from Random Walk Down Wall Street.) Is that really a dominated strategy in the mid-to-long run? What do you recommend instead?
posted by anotherpanacea at 2:18 PM on December 13, 2011 [3 favorites]

ignignokt: "Among other things, I learned from this article that these guys were the ones that came up with the birth month-soccer success idea that Malcolm Gladwell spouted on the radio the other day. Which is perfect because I think of these guys as "Malcolm Gladwell with numbers.""

Not sure if you are referring to this and this or just generally that they play a bit loose to make some silly narrative work, but...

I've always thought Dubner looked like Gladwell, no?
posted by symbioid at 2:19 PM on December 13, 2011 [1 favorite]

The problem with pop economics isn't the "economics", it's the "pop". Dubner/Levitt's analysis is sloppy and suffers from the worst kind of contrarian instincts, but sloppy work exists everywhere, not just in economics. Regardless of the stereotypes, economists recognize that consumers are not always rational actors (and have models that include irrational consumers), recognize that correlation isn't causation, etc.

In my experience, sociology tends to suffer from quantitative and causation errors more than econ does, but whatever.
posted by downing street memo at 2:28 PM on December 13, 2011 [2 favorites]

In my experience, sociology tends to suffer from quantitative and causation errors more than econ does, but whatever.

I'm not sure it needs to be a contest. They're both pretty bad a lot of the time. I sure am glad I studied political science, which never makes these mistakes. Speaking of which, did you know democracies by nature will never, ever go to war with each other?
posted by Hoopo at 2:35 PM on December 13, 2011 [13 favorites]

He is definitely right about their favorite sources/friends -- I started listening to the podcast recently, and it seems like that Nathan Myhrvold guy has popped up in like 4 different episodes for 4 different topics.
posted by This_Will_Be_Good at 2:39 PM on December 13, 2011

Patent troll and chef Nathan Myhrvold.
posted by Artw at 2:40 PM on December 13, 2011 [4 favorites]

The superfreakonomics guys are idiots. It's like they're totally unaware of the existence sociology or social psychology, fields that have studied these issues (yes, using statistics) and decided that since they're 'economists' they can go out and 'analyze' things that people already know about using statistics. They also spout obvious nonsense. (The global warming thing was a good example)

If you want competence in investments, you talk to someone in Finance, not Economics. Or if you're smart, you ignore both of them and throw your money into company-matched retirement funds, houses, and index funds.

Because houses and index funds have done so well recently! (Also, company-matched retirement funds are just a method of investing, not an asset class. You can invest company-matched funds in whatever you want, provided the company provides that option. Some have a limited range and others let you invest in whatever you want)
posted by delmoi at 2:55 PM on December 13, 2011

symbioid, I had no idea what Dubner looked like! 2 out of 2 people that I know of that look like that play fast and loose with stats to get the story they want!
posted by ignignokt at 2:56 PM on December 13, 2011 [1 favorite]

Freakonomics: find a crazy correlation; write a highly speculative, sensational and unprovable correlation; profit.
posted by humanfont at 3:07 PM on December 13, 2011

I think one way Levitt and Dubner could address a lot of these issues and still be contrarian would be to pose their pop-science articles more as claims, and end each article or blog post with challenges or questions.

For example, for Oster's claim about children never having been born due to Hepatitis B, instead of just taking it at face value, they could talk about what kinds of evidence, experiments, and analysis would help support or refute that claim. That is, make it a bit more like MythBusters, and introduce people more to the thought processes, culture, and values of of scientists rather than the "proof by authority" approach that they're implicitly (and most likely accidentally) supporting.

What would be great for people in Levitt and Dubner's position is that they could transform the Freaknomics blog into a crowdsourced mechanism for alternative hypotheses and studies. Basically, turn it from an engine for disseminating papers into one for critically thinking about scientific results and proposing new ways of supporting/refuting arguments. It would be cool if they could also link things to crowdsourced analysis tools, like Many Eyes. Now that would be something really useful!
posted by jasonhong at 3:09 PM on December 13, 2011 [9 favorites]

Judging the entire discipline of economics based on the Freakonomics guys and the recent collapse of the economy is kind of absurd. It betrays a serious lack of understanding of what economics is, and a huge amount of diversity within the discipline.

absalom: I don't know why economics is called a science, dismal or otherwise. Hypothesis? Experimentation? No.

They need to call it the dismal religion, instead.

Seriously? The entire field of behavioral economics might have a problem with that statement.

Honestly, reading threads about economics on Mefi at this point is like hearing people describe Freud as the "founder of the field of psychology". It just betrays a huge amount of ignorance and intellectual laziness.
posted by MadamM at 3:23 PM on December 13, 2011 [13 favorites]

The article is okay, but it seems to imply that the Freakonomics guys may have, at some point, gotten something right. This seems highly unlikely to me.
posted by snofoam at 3:29 PM on December 13, 2011 [4 favorites]

The superfreakonomics guys are idiots. It's like they're totally unaware of the existence sociology or social psychology, fields that have studied these issues (yes, using statistics) and decided that since they're 'economists' they can go out and 'analyze' things that people already know about using statistics.

This is pretty typical. Economists consider themselves to be the only real scientists among the social sciences and don't trust anything put out by sociologists or psychologists, thinking they are innumerate.

Likewise complex mathematical DSGE economists look down on saltwater Keynesian economists.

It is the same reason that physicist Richard Muller reexamined all of the temperature data underlying global warming research because of his contempt for the mathematical abilities of lowly climate scientists. Turns out he was surprised to find they were just as good mathematicians as he was.

There is a definite hierarchy in status for scientists. The fancier and more obscure your mathematical techniques, the higher the status.
posted by JackFlash at 4:02 PM on December 13, 2011 [4 favorites]

Jackflash wrote: There is a definite hierarchy in status for scientists. The fancier and more obscure your mathematical techniques, the higher the status.

How does that fit with Levitt's work, though? I think this is really overstated to be honest. If anything, economists usually see their work as complementary, not substitutable, for others. The economic approach is one approach among numerous approaches (and even it is considerably heterogenous). Most of the time, economists produce work for themselves - no different from any other professional discipline.
posted by scunning at 4:13 PM on December 13, 2011

company-matched retirement funds, houses, and index funds.

1) disappearing
2) have you seen the real estate market for houses lately?
3) if you like whiplash
posted by spitbull at 4:26 PM on December 13, 2011

"For example, for Oster's claim about children never having been born due to Hepatitis B, instead of just taking it at face value, they could talk about what kinds of evidence, experiments, and analysis would help support or refute that claim. That is, make it a bit more like MythBusters, and introduce people more to the thought processes, culture, and values of of scientists rather than the "proof by authority" approach that they're implicitly (and most likely accidentally) supporting.``


This one thousand times over! Just moving the focus of their blog towards explaining processes rather than entertaining with novel or contrarian ideas would improve things for everyone. The world needs more writing that actually teaches the process of science with real examples, instead of just spoon feeding the public today's curious result.
posted by Kevin Street at 4:38 PM on December 13, 2011

"But unquestioning trust in friends and colleagues combined with the desire to be counterintuitive appear in several cases to have undermined their work."

It's probably lazy and contrarian of me, but I've long considered Freakonomics to be the Lost Colony of Slate:

Why everything you know about  trending keyword  is WRONG!

(Except Dahlia. She's the Lost Colony of some Webzine I Would Have Liked.)
posted by mph at 5:18 PM on December 13, 2011 [2 favorites]

I always thought the problem was "trying to make a living at it". As in;

"I have half a dozen ideas for interesting little essays about some counterintuitive facts".
"Those were really entertaining, please do that 100 more times".
"I really just have the few, I’m not sure there are 100 like that".
"Please do that 100 more times and we’ll give you money".

See also; Mortgage Backed Securities.
posted by bongo_x at 6:26 PM on December 13, 2011 [3 favorites]

I always thought the problem was "trying to make a living at it".

I think that's a good insight that applies to a lot of tight-focus blogs. When I worked in online tech publishing I saw my share of "somebody put this poor bastard out of his misery" efforts, and got roped into one of my own. I'm really glad that they just quietly salted the ground where my Movable Type install had been and redirected the blog.* subdomain to somewhere useful where I was paying freelancers who hadn't gotten around to riding their narrow interests into the dirt: There's no record left of the misery I felt trying to think of something novel to say about network security 3-5 times a day.
posted by mph at 7:58 PM on December 13, 2011 [1 favorite]

Using the "Principals" of Chicago School Economics in your personal investment strategy will pretty much guarantee you losing a fortune in the mid-to-long run.

Got it. Memo to Ronald Coase: stop fiddling with those assets, they're mine. Milton Friedman, what are you doing in my mutual funds? You've been dead since 2006. That explains all this talk about zombie investments. Richard Posner, why don't you go back to being a highly cited federal judge and leave my piggy bank be?

This one thousand times over! Just moving the focus of their blog towards explaining processes rather than entertaining with novel or contrarian ideas would improve things for everyone. The world needs more writing that actually teaches the process of science with real examples, instead of just spoon feeding the public today's curious result.

It does, but the problem is that most people don't want to do the hard work of thinking these things through, they want an amusing anecdote for exchange over lunch or the water cooler or suchlike. The Becker/Posner blog might be more to your taste - topical, accessible, but with solid arguments. You won't always agree and often the authors use the blog to just sketch the outlines of an idea (fully developing it would require a book), but it's always thought-provoking.
posted by anigbrowl at 8:03 PM on December 13, 2011

I found their original book interesting for some of the ideas but repellent in their constant gee-whiz-we're-smart self-promoting
posted by Bwithh at 8:07 PM on December 13, 2011

painquale: I listen to EconTalk -- actually I think I've heard almost every episode. It's informative and stimulating listening material. However, you should know that Russ Roberts has some extremely bizarre, right-wing/libertarian opinions. He's nowhere near the mainstream of the economics profession and would probably better be called a scholar of business/entrepreneurship than an economist. (His two institutions, Hoover and the Koch-brothers-funded Mercatus Center, are very far from ideologically neutral -- and most of the people he interviews are from those institutions.)

Not saying it's a bad source of thought-provoking conversation. Just that it's even less representative of the economics profession than the Freakonomics podcast.
posted by miyabo at 8:16 PM on December 13, 2011 [2 favorites]

Wow. I think y'all read a different book then I did.
posted by Bovine Love at 8:42 PM on December 13, 2011

miyabo: He's nowhere near the mainstream of the economics profession and would probably better be called a scholar of business/entrepreneurship than an economist.

That doesn't sound fair to me. He's an economist who has come to some very libertarian positions. It's not like his methods are so ideologically infected that he shouldn't count as an economist, and you certainly don't need to agree with most economists to be one.
posted by painquale at 9:28 PM on December 13, 2011

The single best thing I have ever read on MetaFilter, on this very subject:

Freakonomics: We do what social scientists have been doing for decades, exept we ignore all their previous work, use the word "economics" and "marginal" a lot. Also peer-review is for pussies the marginal value provided by peer review does not outweigh the opportunity cost of not blogging for the NYT.
posted by Mayor West at 5:18 AM on December 14, 2011 [5 favorites]

Classic example of "how do we turn a few interesting articles into a book?" with a bonus of "how do we write a sequel now?" Standards slip.
posted by smackfu at 7:11 AM on December 14, 2011

Thanks for the econtalk recommendation. I'm looking for a decent podcast to replace Planet Money (a bit too cute and slow moving) and Freakonomics (for the reasons stated here).

Any other podcast recommendations, something that's a step up from those 2 baselines?
posted by efbrazil at 8:45 AM on December 14, 2011

There are no reasons stated here, just assertions that haven't been peer reviewed.
posted by Bovine Love at 11:07 AM on December 14, 2011