What is the future of the US stock market?

May 14, 2004 6:14 PM

Why Stock Markets Crash : Critical Events in Complex Financial Systems. Professor Didier Sornette of UCLA has some very interesting things to say about stock markets. In his book, he explains how his "theory of cooperative herding and imitation [...] has detected the existence of a clear signature of herding in the decay of the US S&P500 index since August 2000 with high statistical significance, in the form of strong log-periodic components." Although his timing has been just a bit early, the theory, the predictions to date and the pictures are all pretty uncanny. This is easily the most interesting book on the stock market I have ever read and provides interesting and believable hypotheses about things I never imagined could have rigorous explanations. For an overview, here is an interview with the author.
posted by muppetboy (19 comments total)

[This is good.]
posted by Kwantsar at 7:38 PM on May 14, 2004

So.. it looks like he is predicting downward for the US market?
posted by stbalbach at 8:05 PM on May 14, 2004

But...he is a Professor of Geophysics?????? That is so cool!

It reaffirms my belief that Artists would make better politicians than Lawyers.
posted by jaronson at 8:41 PM on May 14, 2004

Theory of cooperative herding - hey! That's a band name!
posted by troutfishing at 9:17 PM on May 14, 2004

I wanna hear MidasMulligan's take on this.
posted by Vidiot at 11:04 PM on May 14, 2004

"So.. it looks like he is predicting downward for the US market?"

at this time, yes. we are in the middle of a massive "antibubble". the herding behaviour in this decline is occurring with logarithmic periodicity (did you look at those charts!?). this means that the peak we just hit in the long run-up in the NASDAQ which began in march 2003 should mark the beginning of a much longer and deeper decline. in other words, it's time for the herd to gradually turn pessimistic again (or in my opinion, simply realistic). it should do this until the pessimism hits its limit and the stock market hits dead bottom. then we get a bounce (and probably the last one and a buying opportunity of a lifetime... although who knows for sure...)

the ugly part for people long in the market will be the likelihood that when we finally do hit bottom, this herding behaviour will have carried us far below fair value (which it has in every crash in recorded history). remember, to set the stage for the next bull market up, pretty much everyone has to give up on stocks. did NASDAQ ~1700 do that? i don't think so. but NASDAQ 700 likely would.

and who will be standing on the sidelines with billions and billions in cash? why warren buffett of course! everyone is once again calling him crazy. we'll just see about that.

what we saw start in march 2003 was pure irrationality in what i believe will ultimately prove to be the most severe and longest lasting bear market in US history. this makes sense to me because we just had the biggest financial bubble in US history, if not world history. all thanks to alan greenspan (look up a chart of M3 or MZM from 1970 to 2004 and notice what happened right around 1996 when everything went crazy... yup... hyperbolic money growth... easy money... greenspan is our own john law).

i think in this next part of the cycle the market could easily fall harder and faster than anyone now thinks is possible. the reason for this is that the prices we have today are the result of irrational herding behavior. nobody i know is holding shares of a tech stock with PE > 30 because they're in it for the long term. do YOU know anyone who isn't playing musical chairs? yup. they're all in it because they think *this time* they won't be the greater fool. the highly speculative nature of the NASDAQ becomes clear when you realize that the average share of amazon is now being held a matter of a few weeks before it is sold again. everyone is happy with that when the price is heading up. but if we just peaked in terms of herding and now supply exceeds demand...

the only question in my mind is "how long till someone blinks?" correct me if i'm wrong, but i think sornette predicts the breakdown will begin in fairly short order (before end of summer and definitely before end of 2004).
posted by muppetboy at 2:07 AM on May 15, 2004

"But...he is a Professor of Geophysics?????? That is so cool!"

yes. he believes that catastrophic events like earthquakes, and catastrophic failures like ruptures in gas tanks have a lot in common with stock market crashes in terms of the mathematics.

a catastrophic earthquake is essentially a small earthquake that doesn't stop shaking. in the same way, a stock market crash is an unusual string of independently unremarkable price drops. it's the positive or negative feedback that gets going which causes the failure/break.
posted by muppetboy at 2:17 AM on May 15, 2004

I've just been reading this book too, and I'm not convinced this guy knows what he is talking about. Those log-periodic graphs? He fits seven parameters to get them. Of course he is going to be able to get some nice-looking fits. And didn't he say somewhere (book not handy right now) that 40% of bubbles don't end in crashes? So even a good fit to this 7-parameter monster formula is not a reliable indicator of a crash.

But if you look at the market today, it doesn't look like a bubble at all -- or the collapse of a bubble.

He may have a point about short-term correlation of price changes increasing during crashes, although his emphasis on "outliers" makes me nervous. To me, an outlier is a symptom of ignorance about the underlying distribution, and an indicator that any statistical conclusions you are about to draw are completely meaningless. They are things to be eliminated, but he glories in them.

But...he is a Professor of Geophysics?????? That is so cool!

Except that it leads him to include long passages of physics, that go into way too much detail about a supposed analogy with finance. Some of this would be welcome, but he does it so much, he raises suspicions that he needed to pad the book out to a reasonable size.
posted by anewc2 at 4:51 AM on May 15, 2004

It is just data mining, at least at this stage.

In order to convince others that you made a discovery, you have to test the hypothesis on a very large set of data. If those events are alike, most (not all) of the anti-bubbles should display the same characteristics. They only present data for recent SP500. In finance there are cases where papers analyzing 20 years of data are suspected of data mining – e.g. Fama & French.

Similarity with Nikkei is just technical analysis, two sets of data that look the same; it is not a proof. On the arXiv.org website they have papers that document other instances of log-periodic components, but again, the evidence is not universal; it is a bit here, another bit over there.

If you go back to the Feb 2004 prediction, you will see there is (was) a high probability of US anti-bubble ending. Then, they changed the measurement unit (from USD to GBP) and the anti-bubble is still there. Instead of focusing on explaining the phenomenon, they modify the data to suit the theory!

Do not get me wrong, I like the theory, and I think it will be used more often in the future. However, I do not like how the evidence supporting the theory is presented!


muppetboy: did you have a look over the older predictions? Also, you do understand the prediction is conditional: only if the behavior continues, which, based on their own method, it is not very likely.

anewc2: for stock market data, there are no outliers, the data is real. If you go with Gaussian distributions you will tend to disregard extreme values, and thus, they are outliers. However, when studying large events, the focus is on “outliers” - soon you realize that, although large events (catastrophes) are present, there is not enough data to calibrate the model.
posted by MzB at 7:01 AM on May 15, 2004

"But if you look at the market today, it doesn't look like a bubble at all -- or the collapse of a bubble."

no, that's EXACTLY what it looks like.

"although his emphasis on "outliers" makes me nervous. To me, an outlier is a symptom of ignorance about the underlying distribution, and an indicator that any statistical conclusions you are about to draw are completely meaningless. They are things to be eliminated, but he glories in them."

while i agree the curve fitting is contrived (i think that was the point... to see what log-periodic curves fit the data), i thought the part on outliers was the most solid part of the book. it's clear that they don't and can't fit the patterns which are predicted by existing market models that work under normal conditions. based on random walk / EMH, these outliers are not exponentially impossible! so the conclusion he draws, which i think is really quite solid, is that the outliers are caused by something other than normal market activity. certainly outliers are to be eliminated if you don't or can't understand what produced them. and if there were lots of outliers in the rest of the model, you might have a point... but the normal models fit the rest of the data quite nicely. the outliers only show up at the extreme end... which indicates something fishy going on with existing models. i honestly don't see the problem here.
posted by muppetboy at 9:18 AM on May 15, 2004

heh. by "not expontentially impossible!" i mean "exponentially impossible!" ;-)
posted by muppetboy at 9:19 AM on May 15, 2004

From an article written in December, 2002:

Sornette and Zhou predict the Standard & Poor's 500 (currently above 900) will begin dropping by the second quarter of 2003 and will fall to approximately 700 in the first half of 2004.

The S&P is currently at 1100.
posted by Turtle at 9:26 AM on May 15, 2004

"muppetboy: did you have a look over the older predictions? Also, you do understand the prediction is conditional: only if the behavior continues, which, based on their own method, it is not very likely."

i agree. the predictive power is quite weak, although the log curve fitting is demonstrated for many other markets in the book and so it becomes more interesting. also, i think the log periods in the run-ups to bubble land are even more obvious than those in the break-downs (lot of good that would do you now).

it is not demonstrated how to determine if the herding will continue and the change to GBP admittedly shows how weak the original predictions were. OTOH, the reason given for switching to a different view (distorting actions of the Fed) make some sense in this very unusual financial climate.

btw, i think the author is pretty straightforward about what this all means (he explicitly says that these predictions are merely probabilistic and not deterministic... but what more could you really hope for?).

none of these issues damage the overall investigation, in my opinion. it's a new set of ideas in a branch of mathematics which is in its infancy.
posted by muppetboy at 9:33 AM on May 15, 2004

"Sornette and Zhou predict the Standard & Poor's 500 (currently above 900) will begin dropping by the second quarter of 2003 and will fall to approximately 700 in the first half of 2004.

The S&P is currently at 1100."

true, but you can subtract ~20% from that because 2004 dollars are worth less than 2002 dollars. so in constant dollars, the prediction holds (which is what this GBP conversion is all about). in other words, the argument is that the decline is being masked by a parallel decline in the dollar.
posted by muppetboy at 9:40 AM on May 15, 2004

i'd like to point out that if you took into account the massive bear market in the dollar, that the rising cost of fuel is greatly mitigated.

what we're all experiencing here is not primarily a problem with oil markets, but rather the beginning of hyperinflation at about the rate of the dollar decline (15-20% per year). in the end, the dollar will bottom out. but until then, expect soaring prices in EVERYTHING as companies are forced to knuckle under to pricing pressures and pass costs along to consumers. anyone for oil at $50/barrel?

basically, when you get down to brass tacks, a bunch of bankers have taken everyone's money again. your old dollars are worth $0.75 now, if not less.
posted by muppetboy at 9:49 AM on May 15, 2004

your old dollars are worth $0.75 now, if not less.

Assuming that most of us have any old dollars, aren't you? Still, a most interesting discussion.
posted by billsaysthis at 8:51 PM on May 15, 2004

OK. In the part with the outliers, he almost convinces me too that there is something special going on during extreme events. But not quite. I remember what I have read in Mandelbrot about the folly of trying to analyze price changes as if they obeyed different probability distributions at different times. A strong argument IMHO against what Sornette is doing here. If Sornette is going to use a Mandelbrot image on his dust jacket, and work in a field where Mandelbrot has been a major contributor, then he really ought to acknowledge that Mandelbrot disagrees with him on a significant philosophical point.

"But if you look at the market today, it doesn't look like a bubble at all -- or the collapse of a bubble."

no, that's EXACTLY what it looks like.

Huh? Markets climbing ever higher? Unbridled optimism? Price rises feeding on themselves, unresponsive to fundamentals? That's a bubble. The opposite -- a crash. In each case the market is inward-looking. But these days everybody is outward-looking and responsive to non-technical information.
posted by anewc2 at 4:43 AM on May 17, 2004

i'd be interested in reading mandelbrot's argument. if he's talking about overall averages on different time scales though, he's missed the boat. what sornette is point out is that the *clustering* of up/down movements is not statistically in-sync with random walk. that doesn't imply that random walk is not accurate statistically. in other words, the two things are compatible. you just can't see what sornette is talking about if you used fixed time intervals and average out the clustering. that's what i got out of it, anyway...

on the markets climbing higher, i have no idea what you mean by "outward looking" or "inward looking". but what i see is a gigantic bear market that has been in a temporary correction since march 2003. that is about to end soon. and precisely because prices have been rising (although adjusted by the bear market in the dollar, they really aren't up quite so much in "absolute" terms) and have not responded to fundamentals (stocks, especially in the NASDAQ, are still massively overvalued... unless you believe the "new era" is still upon us and valuation doesn't matter)
posted by muppetboy at 6:14 PM on May 18, 2004

i think on the markets climbing bit we're just looking at the same thing on two different time scales. you see a new bull market that is just over a year old. i see a weaking rally in a bear market that is sure to last several more years. if it doesn't that will be a first in market history.
posted by muppetboy at 6:26 PM on May 18, 2004