Inside the paradox of forecasting

January 20, 2011 10:50 PM

That guy who predicted the big one? Don't listen to him. Stern School of Business (NYU) Professor Nouriel Roubini (wiki : twitter : prev : prev : prev) made waves when he predicted the Great Recession, but not all of his predictions have panned out. This needn't be a surprise, however: Predicting the Next Big Thing: Success as a Signal of Poor Judgment outlines how those who make big, accurate predictions are often worse than the general public at making predictions in general.

Nassim Nicholas Taleb, referenced in the third "prev" link, has of course written extensively on the folly of predictions in general.

posted by Sticherbeast (34 comments total)

Nassim Nicholas Taleb, referenced in the third "prev" link, has of course written extensively on the folly of predictions in general.

Actually I think Taleb predicted the crash as well, although I'm not sure. I saw an interview with both Taleb and Roubini. The seemed to agree with eachother quite a bit, with Taleb being even more negative about the chances for recovery and even more down on the big banks.

Thing about the crash is that it didn't really take a genious to see it coming, the problem was that no one in the banking industry (or few people) really looked at what was going on, or just assumed someone else had it covered.
posted by delmoi at 11:11 PM on January 20, 2011 [3 favorites]

Is it just my imagination, or is there some kind of historical revisionism going on here? I remember scores of analysts warning about the dangers of opaque derivatives trading and the inevitable collapse of the US housing market fully years before the, ahem, bubble hit the fan. The real weakness of most market observers is not ignorance or the unpredictability of the markets - it is denial.

"I know all indications point towards a ridiculous bubble...I know all historical evidence says that bubbles collapse...but maybe just this one time it will grow forever and we will all live in eternal luxury...maybe just this one time..."
posted by jet_manifesto at 11:36 PM on January 20, 2011 [14 favorites]

Better a fox than a hedgehog
posted by Philosopher's Beard at 12:29 AM on January 21, 2011 [1 favorite]

I actually saw a good bit of this coming, but YEARS ahead of that, late night talk radio callers were saying that it was going to happen and it was some kind of Chinese plot to take over America. Oooooopaaaa!
posted by Katjusa Roquette at 12:29 AM on January 21, 2011 [1 favorite]

There are no true geniuses, only some people who take longer to revert to the mean
posted by Philosopher's Beard at 12:32 AM on January 21, 2011 [8 favorites]

CEPR has been on the money for a long time.
posted by notion at 12:42 AM on January 21, 2011 [2 favorites]

"I know all indications point towards a ridiculous bubble . . ."

I read this as I know all indications point towards a ridiculous book . . .
posted by IvoShandor at 1:15 AM on January 21, 2011

Actually I think Taleb predicted the crash as well,

Glenn Beck has been predicting economic catastrophe for a lot longer than any of these guys.

When the great battle between good and evil comes and my rapture rations and dried beans run out - and gold becomes worth its weight in gold - I might have to admit he was right.
posted by three blind mice at 2:04 AM on January 21, 2011 [3 favorites]

-.-. --.- -..
posted by clavdivs at 2:18 AM on January 21, 2011

Glenn Beck has been predicting economic catastrophe for a lot longer than any of these guys.

Yeah but that guy doesn't count. It's like throwing a handful of darts at the board. Beck predicts anything crazy that can be thought up, sometimes several times per broadcast. Being kind of maybe right about one thing doesn't qualify him as a predictor of any sort. Predictions are like goober-ass uncles, everybody has one.
posted by IvoShandor at 3:24 AM on January 21, 2011 [3 favorites]

So if they are predicting that people who make big accurate predications may not generally be correct in more general predictions, should we trust their prediction about prediction accuracy?
posted by memebake at 3:25 AM on January 21, 2011 [1 favorite]

Seems like the same kind of problem discussed in Expert Political Judgment.

In my opinion anyone who predicts anything about anything is probably a witch, and should be dealt with accordingly. To the Bureau of Meteorology!
posted by Ritchie at 3:46 AM on January 21, 2011 [4 favorites]

Stopped clock right twice a day, news at 11.
posted by empath at 4:13 AM on January 21, 2011 [4 favorites]

I actually listened to him give a talk in efarly 2009 or late 2008. It was when all the talking heads were talking about green shoots. He agree with the common consensus that there was going to be a recovery. The only thing he differed on was the rate.
He was very interesting to listen to, because explained why he thought that way and went through all the likely scenarios. Unlike most economist who do a lot hand waving and cite garbage like efficient markets. You can learn by actually listening to him even if you disagree.
I would also guess that he does the public prediction thing to drive business to his website.
posted by roguewraith at 4:34 AM on January 21, 2011

It's like playing a group stock picking game. In real life, when you care that you don't actually lose too much of your money, you are probably best picking a well diversified portfolio such that any particular loser can not drag you down too far. When the money is funny and there is no real penalty for a big loss, and the object is to end up with the biggest pile, you win by betting big on some skyrocketing company. Many such bets go straight to the trash, much like buying a lottery ticket, but when they hit they can hit big. So it is with these public prognosticators. Pick a big story that will get attention and make a bold prediction contrary to current public thinking. If it comes true then you have hit the lottery so to speak.
posted by caddis at 4:38 AM on January 21, 2011 [2 favorites]

Macroeconimists have predicted nine out of the last five recessions.
posted by Evilspork at 4:42 AM on January 21, 2011 [6 favorites]

Armchair quarterbacking catastrophe is pretty easy. Being right about it pretty easy. Over identifying and expecting catastrophe far more frequently than it occur, thereby ruining your credibility by the time the big event occurs? Also easy.
posted by Nanukthedog at 5:03 AM on January 21, 2011 [4 favorites]

Seems like the same kind of problem discussed in Expert Political Judgment.

This New Yorker piece on that book gets to the root, I believe...

The experts’ trouble in Tetlock’s study is exactly the trouble that all human beings have: we fall in love with our hunches, and we really, really hate to be wrong.

posted by sexymofo at 5:04 AM on January 21, 2011 [2 favorites]

Anyone who didn't see this coming 5 years ago was blind.
posted by spitbull at 5:04 AM on January 21, 2011 [5 favorites]

Peter Schiff was right.
posted by alpinist at 5:30 AM on January 21, 2011

Nearly everyone posting in this thread is missing the point about these articles (read Tetlock on this, as suggested by Ritchie). The point is not which economists or pundits correctly predicted the recession. In any situation, there will nearly always be people who correctly predict major events contra prevailing wisdom of the time. We tend to focus on the people that got it right as evidence that they knew better than others what was going on -- and they may have, about that particular event. However, being right about a major event contra prevailing wisdom increases the likelihood that your denominator (that is, total predictions) increases which the numerator (accurate predictions) decreases. The point is not to say, "Well, so and so, got it right."

There is an analogous problem in intelligence. For example, not to defend Bush, but we all pointed to the fact that there was a NatSec briefing titled "Bin Laden determined to strike within the US." However, what we don't know is how many of those briefings exist and how many predictions they make that don't pan out. Thus, if we acted on every single one of those predictions we would quickly exhaust all of our resources, making the probability of one of those events occurring approach 1.
posted by proj at 5:51 AM on January 21, 2011 [4 favorites]

s/which/while
posted by proj at 5:52 AM on January 21, 2011

I agree with your main point, proj, but I believe you have misinterpreted the controversy surrounding the Bush NatSec briefing. It was not that such a briefing existed and insufficient action was taken; it was the fact that afterwards, the Bush administration attempted to claim that no one could possibly have made such a prediction, and that there were therefore no possible precautions they could have taken.

The equivalence might be -- to explain why there was insufficient action, it might be fair for an economist to say that no one could have predicted when the housing bubble would collapse. It might be fair for an economist to say that at the time, there were conflicting reports as to whether there was really a housing bubble at all. But if an economist declaims that no one in the entire world could ever possibly have believed at the time that there was a housing bubble, that is simply ass-covering and should be taken as such.
posted by kyrademon at 6:05 AM on January 21, 2011

That's a fair point, though I believe my larger argument still stands.
posted by proj at 6:15 AM on January 21, 2011

three blind mice: Don't you mean old crappy French coins from Goldline™ become worth their weight in gold?

That said, if the economy/environment/government/peace between the global superpowers goes to shit and there's a civil collapse of some sort, what good is gold going to be? I mean, it's not like people are going to want jewelry or materials for a FAB so much as say, a can of beans, guns/ammo or maybe the odd gallon of gasoline to run one of the many abandoned cars.

Personally, I say stock up on Muir Glenn Tomatoes. They'll keep, they're delicious, and they improve just about any dish, whether it be irradiated rat, dog, or even the occasional serving of long pig. [Normally, I don't trade with cannibals, but desperate times, you know? Besides, they are famous for their witty jokes vis a vis the Irish.]
posted by mccarty.tim at 7:25 AM on January 21, 2011 [1 favorite]

People make a lot of predictions. Some are bound to be right.
posted by inturnaround at 7:34 AM on January 21, 2011

[Normally, I don't trade with cannibals, but desperate times, you know? Besides, they are famous for their witty jokes vis a vis the Irish.]


I'm offended by this statement.

I'm not sure if I'm offended as a cannibal or as an Irishman, but let it be known... I said "harumph..." and crossed my arms.
posted by Bathtub Bobsled at 7:37 AM on January 21, 2011

The Big Short is all about this, how about 10 investors in the entire world not only saw it coming, but laid money down on it (shorted the market) and made billions of dollars. Predictions are cheap, putting big money down saying if and when it will happen, is a different story. John Paulson for example earned more money from a single bet than any person in history, far outdoing the famous Soros currency bets of the 80s and 90s. Michael Burry was probably the first to see it coming and understand why, he made a lot too, but could have made a lot more if his financial backers hadn't had cold feet and pulled out. Not bad for a Neurologist who did investing in his spare time.

> Peter Schiff was right.

Schiff is a permanent bear and self promoter, he will get it right occasionally, and be wrong most of the time. It doesn't cost him anything to keep shooting from the hip and hope one day he is shown right while pandering to peoples fears and ideologies.
posted by stbalbach at 7:58 AM on January 21, 2011

Sadly, Paul the Octopus was unavailable for comment.
posted by BitterOldPunk at 9:08 AM on January 21, 2011 [1 favorite]

Let me also chime in on the "it was obvious" wave. Not every crash is obvious by any means, but the housing bubble was so gratuitously overinflated at the same time that most people's wages were flat, and there didn't seem to be any way to have a soft landing.

I've only twice gone on record predicting a crash, once in 2007 and once in 1987... but both times it was glaringly, blatantly obvious!

Who did they think were going to sell all these houses to, anyway?
posted by lupus_yonderboy at 10:06 AM on January 21, 2011

Woah, Baader-Meinhof'd! The Skeptic's Guide to the Universe podcast recently did an episode -- #283, December 15, 2010, now that I check their site -- that included an interview with a guy about this very thing. It was oddly fascinating, possibly due in part to his soothing Canadian accent.
posted by Jinkeez at 10:10 AM on January 21, 2011

Bad Credit? No Credit? Bankruptcy? No job? No problem, you can get a house! For nothing! No money down! The value of the house will pay for the house and your lifestyle!

It didn't take a genius, really, to see what was wrong with this from the beginning. A few hundred late night commercials for CountryWide Mortgage should have made it clear to anyone.
posted by spitbull at 1:57 PM on January 21, 2011 [1 favorite]

Anyone who didn't see this coming 5 years ago was blind.

And yet hardly anyone made money on it. Nassim Taleb shorted the banks just before their big swan dive and cleaned up. (Don't know what if anything Roubini did.)
posted by IndigoJones at 7:19 AM on January 22, 2011 [1 favorite]

I was talking to a friend of mine the other day about stuff similar to this, and he had mentioned a rock star finance guy who had supposedly cleaned up through shorting the crisis.

Thing was, this rock star guy then started his own finance group, and he was carefully watched to see how much he'd make in the market, seeing as he was so smart.

His return was zilch. Being right about one thing didn't mean he could keep on cranking out correct predictions.

It's driving me nuts that I can't remember the name of this guy...
posted by Sticherbeast at 10:50 AM on January 22, 2011