Dynamic Linear Modelling
July 30, 2010 8:04 AM Subscribe
It has applications in Economics, Biology, Pharmaceuticals, and is rooted in State Space Modeling, which with Kalman Filtering (paper, breakdown [warning: long]) was used in the Apollo program. Dynamic Linear Models are gaining in popularity. There exists an R package, and both a short doc and a really great (read: worth buying) book (sorry, not a download, but here's chapter 2) by Giovanni Petris, Sonia Petrone, and Patrizia Campagnoli with its own little website.
Kalman filtering is neat. I was building a drone and one of the things I found out, not having dealt with sensors and such, is that the vibration from the plane along with just general turbulence produces crazy numbers on the sensors. There's a couple of ways to smooth the results, but I just had a stupid grin on my face once I applied a Kalman filter and it magically worked.
I've purposely avoided looking into how the Kalman filters work (though I have a pretty good idea), simply because I like having some mystery and wonderment in my life.
posted by geoff. at 8:57 AM on July 30, 2010
I've purposely avoided looking into how the Kalman filters work (though I have a pretty good idea), simply because I like having some mystery and wonderment in my life.
posted by geoff. at 8:57 AM on July 30, 2010
Admittedly, I don't understand what these models are used for in economics, but I'm worried they will be the latest fad in risk analysis, or some such nonsense.
I'm sure they are very useful in engineering or science fields, but applying them to economics, brings out the Taleb in me.
posted by KaizenSoze at 9:18 AM on July 30, 2010 [2 favorites]
I'm sure they are very useful in engineering or science fields, but applying them to economics, brings out the Taleb in me.
posted by KaizenSoze at 9:18 AM on July 30, 2010 [2 favorites]
There's a pretty good video available from NBER regarding econometrics and the Kalman: here. (direct link to video and slides.)
Also, the OpenCV project has a decent Kalman implementation, last I checked.
KaizenSoze: What are your concerns wrt the Kalman in econ?
posted by rider at 9:58 AM on July 30, 2010
Also, the OpenCV project has a decent Kalman implementation, last I checked.
KaizenSoze: What are your concerns wrt the Kalman in econ?
posted by rider at 9:58 AM on July 30, 2010
Rider, Kalman filtering assumes you have a dynamic system that obeys equations of motion. How good are the equations of motion for an economic system? I honestly don't know.
posted by LastOfHisKind at 10:41 AM on July 30, 2010
posted by LastOfHisKind at 10:41 AM on July 30, 2010
Ah Kalman Filters, this is taking me back !
There seemed to be a flurry of conferences in the mid 90's about the application of Kalman Filters to finance, but you don't really hear much about it these days. A good Econometrics course at most Business Schools will touch upon the topic, perhaps do some modelling with it (GDP seemed to be especially popular for some reason, at least when I was in Business School) but everyone moves pretty quickly to GARCH modelling.
Kalman Filters don't seem to cope well in environments where there are sharp changes in volatility. I suspect for some of the other uses (robotics, spaceflight), volatility isn't that much of a problem, at least not the sharp changes we see in the most financial markets (NB not all assets trade with such clusters of volatility, but the asset classes I work mostly with - Equities & Fixed Income, a few commodities most definitely do).
GARCH (Generalised Autoregressive Conditional Heteroscdastisity) models, first proposed by by Bollerslev in 1986 building upon Engle's 1982 work, by contrast, are designed specifically to properly capture the characteristics of such data generating processes. They can model both periods of constant volatility (aka homoscedasticity, or "same" volatility)as well as periods where the underlying data generating process exhibits sharp changes in volatility, conditional (i.e., depending upon) volatility in prior periods. Heteroscedasticity, literally "different" volatility or precisely the behaviour we see in most financial markets.
A GARCH model will have two components; one a "traditional" ARMA (auto regressive moving average) formula, and the second an equation describing the data processes dynamically changing volatility.
Its this second equation that gives a GARCH model superior predictive power as it allows the model to deal with volatility that can and does change across the horizon, but the first is superbly flexible as well; pretty much any representation that suits can be defined and used, with the second term modelling the dynamic volatility.
In equities many folks fit AGARCH, or asymmetric GARCH models as these allow us to represent the asymmetric nature of stock returns (i.e., they can go down a hell of a lot faster than they can go up).
I should probably point out that having most folks use the same model might cause problems on its own, but this shouldn't be as big a problem as its sounds, primarily due to the extreme flexility of the technique.
I took a quick look in database of academic journals before crafting this comment, and it seems that while there is just as much research going on into the application of Kalman Filters as there is to GARCH modelling, most of it is non financial in nature, mostly industrial and other apps.
So if folks in finance are doing stuff with this technique, they're largely keeping it pretty quiet. In any case, if anyone has more current information about what folks are up to I've love to hear it (thanks rider)
Great post by the way, many thanks for lots-o-links!
posted by Mutant at 10:41 AM on July 30, 2010 [5 favorites]
There seemed to be a flurry of conferences in the mid 90's about the application of Kalman Filters to finance, but you don't really hear much about it these days. A good Econometrics course at most Business Schools will touch upon the topic, perhaps do some modelling with it (GDP seemed to be especially popular for some reason, at least when I was in Business School) but everyone moves pretty quickly to GARCH modelling.
Kalman Filters don't seem to cope well in environments where there are sharp changes in volatility. I suspect for some of the other uses (robotics, spaceflight), volatility isn't that much of a problem, at least not the sharp changes we see in the most financial markets (NB not all assets trade with such clusters of volatility, but the asset classes I work mostly with - Equities & Fixed Income, a few commodities most definitely do).
GARCH (Generalised Autoregressive Conditional Heteroscdastisity) models, first proposed by by Bollerslev in 1986 building upon Engle's 1982 work, by contrast, are designed specifically to properly capture the characteristics of such data generating processes. They can model both periods of constant volatility (aka homoscedasticity, or "same" volatility)as well as periods where the underlying data generating process exhibits sharp changes in volatility, conditional (i.e., depending upon) volatility in prior periods. Heteroscedasticity, literally "different" volatility or precisely the behaviour we see in most financial markets.
A GARCH model will have two components; one a "traditional" ARMA (auto regressive moving average) formula, and the second an equation describing the data processes dynamically changing volatility.
Its this second equation that gives a GARCH model superior predictive power as it allows the model to deal with volatility that can and does change across the horizon, but the first is superbly flexible as well; pretty much any representation that suits can be defined and used, with the second term modelling the dynamic volatility.
In equities many folks fit AGARCH, or asymmetric GARCH models as these allow us to represent the asymmetric nature of stock returns (i.e., they can go down a hell of a lot faster than they can go up).
I should probably point out that having most folks use the same model might cause problems on its own, but this shouldn't be as big a problem as its sounds, primarily due to the extreme flexility of the technique.
I took a quick look in database of academic journals before crafting this comment, and it seems that while there is just as much research going on into the application of Kalman Filters as there is to GARCH modelling, most of it is non financial in nature, mostly industrial and other apps.
So if folks in finance are doing stuff with this technique, they're largely keeping it pretty quiet. In any case, if anyone has more current information about what folks are up to I've love to hear it (thanks rider)
Great post by the way, many thanks for lots-o-links!
posted by Mutant at 10:41 AM on July 30, 2010 [5 favorites]
Heh: "Posterior distribution."
posted by ZenMasterThis at 10:43 AM on July 30, 2010
posted by ZenMasterThis at 10:43 AM on July 30, 2010
Mutant:
I know nothing about financial modelling, but from the way you're describing it, financial systems are volatile and have annoying complex dependencies on the past. Linear Kalman Filters don't work well in these situations.
The reason why they still see a lot of use in industry, or say, robotics, is because linear markov models work quite well in well modelled/managed physical systems. If you have a smoothly varying system which doesn't have long range dependencies on its past, Kalman Filters can be great. And they're analytically computationally tractable, which is the real key. So even now Linear Kalman filters still are the backbone of a lot of filtering... from what I've seen its the first thing engineers reach for before deciding to use more complex methods.
In my experience, when people in these fields need more complex methods, they drop the linear evolution assumption, but retain the Markov assumption... a lot of research these days is on sampling methods to solve these kinds of problems.
This Linear Dynamic Modelling stuff is cool. It's seems to be roughly the opposite of what I just laid out in that the systems still evolve linearly, but (roughly speaking) the Markov assumption is dropped.
So I guess could be quite good for systems like financial markets where the evolution of the system is extremely time dependent, both in the changing nature of the evolution and its dependence on the past.
posted by Alex404 at 12:42 PM on July 30, 2010
I know nothing about financial modelling, but from the way you're describing it, financial systems are volatile and have annoying complex dependencies on the past. Linear Kalman Filters don't work well in these situations.
The reason why they still see a lot of use in industry, or say, robotics, is because linear markov models work quite well in well modelled/managed physical systems. If you have a smoothly varying system which doesn't have long range dependencies on its past, Kalman Filters can be great. And they're analytically computationally tractable, which is the real key. So even now Linear Kalman filters still are the backbone of a lot of filtering... from what I've seen its the first thing engineers reach for before deciding to use more complex methods.
In my experience, when people in these fields need more complex methods, they drop the linear evolution assumption, but retain the Markov assumption... a lot of research these days is on sampling methods to solve these kinds of problems.
This Linear Dynamic Modelling stuff is cool. It's seems to be roughly the opposite of what I just laid out in that the systems still evolve linearly, but (roughly speaking) the Markov assumption is dropped.
So I guess could be quite good for systems like financial markets where the evolution of the system is extremely time dependent, both in the changing nature of the evolution and its dependence on the past.
posted by Alex404 at 12:42 PM on July 30, 2010
I like this post (who doesn't like the KF?), but should echo Mutant by saying these models definitely are not gaining in popularity in economics/finance. Most of the research today is concentrated on nonlinear/nongaussian state space models with parameter instability, jumps, and drifts.
Markov-models, too, are well known in economics. There is a 10 year old textbook by Kim and Nelson which describes how they're used in state space setting. Jim Hamilton also did a lot of work on the topic about 15 years ago.
Mutant: I am not a finance person, but it seems to me that a lot people today are estimating jump diffusion processes for asset prices. These things are nonlinear/nongaussian, so particle filtering is used to integrate unobservables. Complicated (and awesome!) MCMC is used in estimation. Nick Polson (at Chicago Booth) and coauthors have a bunch of recent survey papers / handbook chapters.
posted by diftb at 2:52 PM on July 30, 2010
Markov-models, too, are well known in economics. There is a 10 year old textbook by Kim and Nelson which describes how they're used in state space setting. Jim Hamilton also did a lot of work on the topic about 15 years ago.
Mutant: I am not a finance person, but it seems to me that a lot people today are estimating jump diffusion processes for asset prices. These things are nonlinear/nongaussian, so particle filtering is used to integrate unobservables. Complicated (and awesome!) MCMC is used in estimation. Nick Polson (at Chicago Booth) and coauthors have a bunch of recent survey papers / handbook chapters.
posted by diftb at 2:52 PM on July 30, 2010
Well, I think a lot of people jump straight to Kalman filters when they could use a simpler complementary-filter approach. But Kalman filters are awesome. I enjoyed the historical details from this NASA memo TM-86847.
Kalman filters work if the noise inputs are close-ish to additive white Gaussian noise (AWGN) and the system is linear or (in the case of extended or unscented Kalman filters) not too badly nonlinear. I think economic modeling fails both those criteria, but then, it sometimes seems like economic modeling is all about making ludicrous simplifying assumptions in hopes of getting lucky.
Anyway, this is all a derail I guess. I must go and read the stuff you've linked to.
posted by hattifattener at 3:31 PM on July 30, 2010
Kalman filters work if the noise inputs are close-ish to additive white Gaussian noise (AWGN) and the system is linear or (in the case of extended or unscented Kalman filters) not too badly nonlinear. I think economic modeling fails both those criteria, but then, it sometimes seems like economic modeling is all about making ludicrous simplifying assumptions in hopes of getting lucky.
Anyway, this is all a derail I guess. I must go and read the stuff you've linked to.
posted by hattifattener at 3:31 PM on July 30, 2010
financial systems are volatile and have annoying complex dependencies on the past
The real question is how much if any dependence on the past your economic measurement has.
posted by phrontist at 8:19 PM on July 30, 2010
The real question is how much if any dependence on the past your economic measurement has.
posted by phrontist at 8:19 PM on July 30, 2010
As noted, Kalman Filters are magic*.
* When applied properly, to well-suited systems.
But magic!
posted by IAmBroom at 2:13 AM on July 31, 2010
* When applied properly, to well-suited systems.
But magic!
posted by IAmBroom at 2:13 AM on July 31, 2010
Kalman filter is also used in multiphoton fluorescence microscopy.
Beautiful examples here and here
posted by volpe at 2:54 PM on July 31, 2010
Beautiful examples here and here
posted by volpe at 2:54 PM on July 31, 2010
« Older A portal into the past | Put Your Nook Back in Its Crannie Newer »
This thread has been archived and is closed to new comments
posted by nj_subgenius at 8:37 AM on July 30, 2010 [2 favorites]