FiveThirtyEight rips apart Standard and Poor's ratings

August 10, 2011 11:14 AM

Nate Silver: Why Standard & Poor’s Ratings Are Substandard and Porous
posted by East Manitoba Regional Junior Kabaddi Champion '94 (46 comments total)

Substandard and porous, hey-o.
posted by box at 11:18 AM on August 10, 2011

Oh, Sn&P!
posted by rouftop at 11:19 AM on August 10, 2011 [32 favorites]

Say what you will about his title, Nate Silver brings the analysis as well as anyone.
posted by ged at 11:22 AM on August 10, 2011 [1 favorite]

"By comparison, simply looking at a country’s ratio of net debt to G.D.P. would have been a better predictor of default" than the S&P rating. Ha.

Once upon a time, as a young lad, I had an internship working for Unnamed Asset Management Firm. I was a technology person, so I didn't really know much about finance. So one day a young trader there (I'm sure in his thirties, but it was a long time ago, who knows) offers me his advice. "Just buy the index funds. No matter what they tell you, NOONE is going to beat the index funds in the long run".

I'm sure applying this sort of lesson to rating agencies leaves a lot behind in the analogy, nevertheless, I am unsurprised.
posted by Phredward at 11:27 AM on August 10, 2011

Felix Salmon:

Tyler Cowen warned about this:

As a simple rule of thumb, if at this point, in response to this news, a commentator attacks the ratings agencies for their previous mistakes and stupid, corrupt behavior, it’s a sign the commentator is trying to muddy the broader issues.

Yes, the ratings agencies were in large part responsible for the financial crisis. But their mistake there was having too many triple-A ratings. If you were looking for a sign that they’d learned their lessons, it would be that they were downgrading triple-A borrowers before crisis hit. And also that they didn’t place overmuch stock in official models. Whatever else S&P is doing here, it isn’t repeating its mistakes of the subprime bubble.

A lot of this piece appears to take the tack that S&P's credit ratings are not efficient prices. Are they supposed to be?
posted by grobstein at 11:28 AM on August 10, 2011

Comment from Keith Wheelock who created Moody's international credit ratings in the 70s
posted by East Manitoba Regional Junior Kabaddi Champion '94 at 11:29 AM on August 10, 2011 [4 favorites]

"In fact, the evidence from the past five years suggests that it may be worthwhile to adopt a contrarian investing strategy that specifically bets against S.&P.’s ratings."

Awesome
posted by spaceviking at 11:34 AM on August 10, 2011 [2 favorites]

One of the more interesting bits, to me, is how S&P is so likely to do a second downgrade after a first. So, as Nate says, "A country that is downgraded from AAA to AA is riskier, in S.&P.’s view, than one that was just upgraded from A to AA".

I vaguely Meta-note, saying FiveThrityEight "rips apart" the ratings in the title isn't really useful. Rhetric like that, which Nate Silver probably doesn't agree with, doesn't help people either read nor discuss it. (It's also a peeve of mine after luminaries such as Jon Stewart and some other bloggers brought it up a bit ago.)
posted by skynxnex at 11:41 AM on August 10, 2011 [1 favorite]

Every time I turn around these days, it seems like I see someone like this:

Big, nameless, faceless group with a shitload of power who contribute nothing but take their cut from the middle, while feeling absolutely no need to explain what they're doing or justifying their existence in any way. Oracles, soothsayers and charlatans. It would be hilarious if weren't so fucking serious and sad.

(Nate Silver always rocks, though.)
posted by Benny Andajetz at 11:43 AM on August 10, 2011 [1 favorite]

This is interesting, but I'm getting a sort of nagging feeling that he's cherry-picking stuff with the benefit of hindsight. Like, it's obvious now that Ireland's in trouble, and looking back it seems like it should have been obvious then, but who spotted it then? Did anyone? (Not to mention, part of the problem in Ireland was that the government unconditionally guaranteed all deposits in Irish bank accounts, and took on a huge amount of bank debt. Which probably couldn't have been foreseen in 2006).

The bigger question though is: if we assume that the rating agencies are useless, or corrupt, or whatever, what replaces them? Individual investors aren't positioned to research every product they buy, to determine its risk. Seems like you need someone doing that. If it's not the big 3 CRAs, who is it?

On preview: Benny Andajetz: the rating agencies publish quite a lot of information about their methodologies, for what it's worth.
posted by Infinite Jest at 11:48 AM on August 10, 2011 [2 favorites]

The ratings agencies were key enablers of the financial meltdown in the first place. Have any of those fuckers been subject to criminal indictment?
posted by exogenous at 11:50 AM on August 10, 2011

BTW Frank Portnoy's book Fiasco (although it describes his experience in the 1990s rather than the mortgage-backed mess we face today) vividly relates how banks were able to obtain AAA ratings on shitting securities.
posted by exogenous at 11:57 AM on August 10, 2011 [1 favorite]

On preview: Benny Andajetz: the rating agencies publish quite a lot of information about their methodologies, for what it's worth.

They do, but Mr. Silver is saying that it's probably so much hokum. Their information lags the market and reacts sloooowly. Two very bad traits for an organization whose job is to predict risk. Silver also says he believes they factor politics at least as heavily as economic variables, which amounts to reading tea leaves in my book.
posted by Benny Andajetz at 11:59 AM on August 10, 2011

This is interesting, but I'm getting a sort of nagging feeling that he's cherry-picking stuff with the benefit of hindsight. Like, it's obvious now that Ireland's in trouble, and looking back it seems like it should have been obvious then, but who spotted it then? Did anyone?

That's actually one of the points: whether or not anyone else did, S&P certainly didn't. And what is the point of a bond-rating agency if they cannot, with some reasonable amount of accuracy, predict when a country will get "in trouble" in a way that makes it more likely to default on its debt?

The bigger question though is: if we assume that the rating agencies are useless, or corrupt, or whatever, what replaces them? Individual investors aren't positioned to research every product they buy, to determine its risk. Seems like you need someone doing that. If it's not the big 3 CRAs, who is it?

And the bigger answer is: no one. There is no one. If you are a retail investor / retirement saver, there is no one to help you. You are screwed. Your future is in the hands of the investment banking "Masters of the Universe" and you have no tools to even assess what kind of a job they are doing until the money is already gone. Just because you wish there was someone doing the job the CRAs are supposed to do, does not make it happen.
posted by rkent at 11:59 AM on August 10, 2011 [4 favorites]

Also, I know it's tilting at windmills, but the basic purpose of the markets is to raise capital. It's not supposed to be a savings plan or a casino.
posted by Benny Andajetz at 12:06 PM on August 10, 2011

Just because you wish there was someone doing the job the CRAs are supposed to do, does not make it happen.

Exactly.
posted by enn at 12:19 PM on August 10, 2011

Having lost a substantial (and I mean substantial) sum in 2008 by investing in equities that the grand mavens like S&P said were rock solid, I have little respect for their prognosticative analytical powers now.

I have come to the stark, but too-late, realization that the market is basically an insider's game, with the ratings agencies profiting off of whatever ripples may emanate from the big fish as they thrash around in the pool. Retail investors are left basically just trying to skim whatever froth they can off the top of all that activity.

Seriously, S&P can go fuck themselves.
posted by darkstar at 12:27 PM on August 10, 2011 [2 favorites]

The bigger question though is: if we assume that the rating agencies are useless, or corrupt, or whatever, what replaces them? Individual investors aren't positioned to research every product they buy, to determine its risk. Seems like you need someone doing that. If it's not the big 3 CRAs, who is it?

Individual investors just need a vague idea of what kind of entity the bond issuer is, and should diversify across that broad category with the assumption that some of them could default. As an individual investor I don't need someone to tell me if American Express is an AAA company and Waste Management is a AA+ company or whatever, even though plenty of analysts publish completely useless buy/hold/sell recommendations, because I just invest in an index fund or use a similar strategy to not have to care about any given company's outlook. Similarly I don't need someone to try to predict whether or not France is going to be financially solvent in 5 years, and it would be stupid of me to put all of my eggs in one basket and assume that France is going to be fine just because they are rated AAA no matter who is doing the rating. The current bond prices reflect the market's current idea of how much they are worth, and the current CDS prices reflect how likely the market thinks a default is. That's about all you can do, nobody is going to be able to predict the future for you to tell you anything you can't get from looking at the numbers that are already out there.
posted by burnmp3s at 12:35 PM on August 10, 2011

Government rules written around what ratings agencies say give way too much power to these institutions.
posted by Gregamell at 12:46 PM on August 10, 2011 [2 favorites]

S&P and other ratings agencies made some serious errors regarding mortgage-backed securities, but I think some of Nate Silver's criticism here misses the mark. Take Ireland as an example. He seems to fault S&P for Ireland's AAA rating in 2006, when credit default swaps now show substantial default risk for the country.

The problem is that S&P ratings are not designed to predict future CDS prices, they're designed to rank order default risk. Here (PDF) is more detail on S&P's methodology. Note that they calculate the default risk over one-year, three-year, and five-year horizons; that is, through 2007, 2009, and 2011 for Ireland. In other words, Ireland was correctly called by S&P according to what S&P is actually trying to do; that is, predict default risk*. S&P said the country wouldn't default in five years, and it didn't. Current CDS prices (which, for a standard five year CDS, would be concerned with defaults post-2011) have fuck-all to do with what S&P was trying to accomplish with its rating in 2006.

Nor is it obviously problematic that S&P lags the market. CDS markets, particularly in less-frequently traded securities, may be highly volatile. It wouldn't be desirable for ratings to bounce back and forth with every market gyration. If, for example, an investor wants to hold (or is required to hold) only BBB or higher-rated bonds, he won't have to suddenly sell off bonds in response to a short-term market movement, as long as ratings agencies don't change their outlook on the bond. The market price for CDS conveys useful information (and may be more useful for some purposes, since it better incorporates expectations of expected recovery rates, whereas as I understand it, S&P is simply trying to predict default as a binary outcome), but isn't the end-all be-all of bond risk.

* It's also not completely sensible to use individual failures to condemn (or credit) the system, since it's designed to rank-order risk. Otherwise, the ratings would have to be just "risk-free" and "risky," which would be more or less useless. Rarely, but sometimes, a AA+ bond will default, and that's OK, so long as they default less frequently than AA, and AA- is riskier yet, etc.
posted by dsfan at 1:01 PM on August 10, 2011 [1 favorite]

I like Nate Silver, but I think there is something very wrong going on here.

S&P had rated the US as AAA going into the debt limit negotiations. It was AAA because s&P like everyone else on the planet Earth had assumed that congress and the President would do what they've always don in the past 20 years, bite the bullet and cut a deal. Hell, even Newt Gingrich eventually cut a deal with Clinton.

Their failure to do so this time around was a surprise of course, but it also suggests that there may be a new trend in place in which Congress from now on takes an extreme, analytically indefensible position on raising taxes. And that is why the rating got downgraded. The increased likelihood of default is tied to new information from this last round of budget negotiations.

So went Nate Silver goes back to "five years ago..." it is unreasonable. Why five years? Why not 10, or 2? The S&P rating is a measure of present expectations about the future, but not an attempt to predict the future. The S&P rating says "Today we think the risk is this." Two weeks from now it may be different.

The only thing that changed before and after was a re-adjustment of everyone's (S&P's) assumptions about Congress. All the economic data is basically the same. So their assessment of risk based on politics is valid.
posted by Pastabagel at 1:02 PM on August 10, 2011 [4 favorites]

This is not an experts view, but an amateurs observation: it seems that economics and finance are not so much science as art. As a consequence, it seems to be foolish to trust those who claim to be able to deal with these fields as science.
On the other hand, it should give us reason to look at those individuals who are good at predictions and analysis. Krugmann is one of those. I know a few others, but they claim they would loose their jobs if they told the truth. [rolls eyes]. Most probably, many people in the financial sector know exactly what is going on, but somehow don't feel compelled to tell us.
posted by mumimor at 1:04 PM on August 10, 2011 [1 favorite]

S&P's ratings is more about the expected staying power of the Tea Party. In other words, they don't go away and force this game of chicken each time the limit has to go up.
posted by SirOmega at 1:26 PM on August 10, 2011

It was AAA because s&P like everyone else on the planet Earth had assumed that congress and the President would do what they've always don in the past 20 years, bite the bullet and cut a deal.

Er, that's pretty much what happened, isn't it?
posted by dersins at 1:28 PM on August 10, 2011 [2 favorites]

I'm not anywhere near knowledgeable enough about ratings agencies to know if there's truth to what Silver is saying, but the overwhelming dismissive reaction of the American press and government to this downgrade at first glance has the appearance of wounded pride and sour grapes.

Am I way off?
posted by Hoopo at 1:30 PM on August 10, 2011

Dersins:
No, that was not what happened.
posted by mumimor at 1:39 PM on August 10, 2011

I disagree; I think that is what happened! A deal was indeed struck and default did not occur.
posted by East Manitoba Regional Junior Kabaddi Champion '94 at 1:43 PM on August 10, 2011

Denial is the first step.
posted by Bovine Love at 1:44 PM on August 10, 2011 [1 favorite]

A deal was struck, and a lot of important issues were put off until later, and anyway, the most important issues weren't even on the board.
If I were an investor, I'd run away screaming
posted by mumimor at 1:46 PM on August 10, 2011

I mean when Obama says stuff like "no matter what some agency may say, we've always been and always will be a AAA country", he comes off sounding like the coach of a really shitty Little League team.
posted by Hoopo at 1:55 PM on August 10, 2011

And S&P didn't cut Fannie Me and Freddie Mac, which were almost-mortally wounded by the Real Estate fiasco and totally dependent on constant infusions from the US Treasury and the Fed, down from AAA until AFTER the U.S.A.

Total lack of credibility.
posted by oneswellfoop at 2:38 PM on August 10, 2011 [1 favorite]

Fannie MAE. Me never had a AAA rating.
posted by oneswellfoop at 2:39 PM on August 10, 2011

I'm not terribly inclined to trust the rating agencies, who seem to have extraordinary power without much cause. On the other hand, I find it sort of interesting that before the debt ceiling deal and the downgrade, everyone in the media was screaming, "We have to make a deal, or they'll downgrade us, and that will be the apocalypse!" And now, after the downgrade, they're screaming, "Actually, these guys are idiots and a downgrade doesn't mean anything because they're stupid stupidheads."

I just wonder: if we don't get downgraded, do we see all these articles about how horrible the ratings agencies are? Was anyone talking about how silly they think S&P is before? It seems to me that, before, much of the media was propping up the ratings agencies as foundational and integral to the overall fiscal climate.
posted by Errant at 2:41 PM on August 10, 2011

[awesome singing pirate]
OOOOOOHHHH
Who works in a office down on Waters Street?
Standardand Poorspants
Substandard, inefficient and porous is he
Standardand Poorspants
If financial nonsense be something you wish
Standardand Poorspants
Then drop like a stock and manipulate Congress
Standardand Poorspants
READY, kids?
Standardand Poorspants
Standardand Poorspants
Standardannnddddd Poorspannnttttsss!!!!
[/awesome singing pirate]
posted by Chipmazing at 3:08 PM on August 10, 2011 [3 favorites]

everyone in the media was screaming, "We have to make a deal, or they'll downgrade us, and that will be the apocalypse!"

I seem to recall that it wasn't necessarily the downgrade, but the default, which would have precipitated the Rapture.
posted by MikeKD at 3:09 PM on August 10, 2011

Sure, but wasn't part of the worry over default an ensuing downgrade? Maybe that's sort of a chicken and egg question.
posted by Errant at 3:17 PM on August 10, 2011

Rather than attack S&P's credibility for downgrading the US credit rating - I have yet to see an economic analysis that refutes the underlying issue that the US Government is not doing enough get to to a more sound financial position.

Under the most likely economic scenario as forecasted by the Congressional Budget Office - it's expected that public dept will be 100% of GDP by 2021. Yikes! That is going to put an enormous strain on capital markets as government spending needs elbows out the ablility for private company and citizens to raise and borrows money.

So instead of shooting the messenger with the S&P - perhaps this will spark a constructive debate in America about how to tackle this issue. Here in Canada - I live in British Columbia which has traditionally been a "have" province - ie we contribute more into the Federal system than we take out. When economic conditions around commodity prices hit the skids and we became a "have not" province for a few years - there was a fierce public discourse about how this could have happened and what could be done to get out of that situation.
posted by helmutdog at 4:22 PM on August 10, 2011 [1 favorite]

Under the most likely economic scenario as forecasted by the Congressional Budget Office - it's expected that public dept will be 100% of GDP by 2021. Yikes! That is going to put an enormous strain on capital markets as government spending needs elbows out the ablility for private company and citizens to raise and borrows money.

I think "crowding out" is more of a bugaboo than a real problem, especially with low employment. Government borrowing actually tends to lower interest rates, and there is no "finite" amount of money for private borrowers to borrow.

I also tend to agree with Paul Krugman that low GDP growth leads to higher debt more than the other way around. We need to spend and borrow wisely, but to phrase it like the TeaBaggers "We don't have a spending problem, we have a demand/jobs/revenue problem. It is much,much,much,much harder to save your way to prosperity than it is to grow your way to prosperity.

Increase revenue (stop the wars and let the Bush tax cuts expire), create jobs and demand ( stimulus programs, works programs, payroll tax incentives, etc.) and a good chunk of the problem goes away.
posted by Benny Andajetz at 4:44 PM on August 10, 2011

Does anyone know why there isn't something better than the ratings agencies?

Why can't you construct an objective, statistically based rating of the probability of default?

Given that there is now a massive amount of data and computing power out there surely it would make sense to have something better.

Or does anyone know why this hasn't happened already?
posted by sien at 4:51 PM on August 10, 2011

I think "crowding out" is more of a bugaboo than a real problem, especially with low employment. Government borrowing actually tends to lower interest rates, and there is no "finite" amount of money for private borrowers to borrow.

How so? I ask this out of genuine curiousity and discourse. If there is "infinite" money to borrow - isn't that just increasing the money supply and fueling inflation (or worse yet, stagflation)? Eventually business will have to borrow capital to expand and hire people - if they are competing with the government for money - doesn't that cause issues? BTW - I agree that it may not be a problem right now (the Globe and Mail had an excellent piece on Corporate cash hoarding given the current economic uncertainty) - but won't it be a drag on the economy in the future?
posted by helmutdog at 5:03 PM on August 10, 2011

How so? I ask this out of genuine curiousity and discourse. If there is "infinite" money to borrow - isn't that just increasing the money supply and fueling inflation (or worse yet, stagflation)? Eventually business will have to borrow capital to expand and hire people - if they are competing with the government for money - doesn't that cause issues? BTW - I agree that it may not be a problem right now (the Globe and Mail had an excellent piece on Corporate cash hoarding given the current economic uncertainty) - but won't it be a drag on the economy in the future?

It can turn into a serious problem is the economy is really heated and there is full employment.

But,until then, private borrowers aren't subject to some given amount of money. Banks will lend to any creditworthy borrower, subject to their capitalization. See "endogenous money".
posted by Benny Andajetz at 5:12 PM on August 10, 2011

Have any of any of these revenue forecasts 10 years out ever been remotely accurate?
posted by humanfont at 5:37 PM on August 10, 2011

who seem to have extraordinary power without much cause.

In reality, no. They are used as pre and post-hoc justifications by governments for driving neoliberal agendas and other unpopular actions; by private investment companies to provide a layer of deniability and justificaton for their buying/selling decisions, and by finance companies in many large companies to justify/explain their level of debt/risk.

But nothing they say comes before the horse hasn't bolted, but has actually lived a full and happy life, and is currently being used to hold wallpaper to the plasterboard. They are largely treated accordingly.

It's in the interest of the above parties to ascribe more power to ratings agencies than they possess because when the shit hits the fan they are a wonderful foil. The (relativelythose) paltry sums the agencies charge for this layer of plausible deniability is well worth it for those parties, but - despite the cant - no one is actually making finance/policy decisions based on what ratings agencies say. That would be a mug's game.
posted by smoke at 7:18 PM on August 10, 2011

People are making policy and finance decision based on what the ratings agencies say.

In addition, many institutional investors, including money market funds and pensions, are required to hold only AAA-rated securities. If the U.S. government is downgraded, those funds may be forced to dump billions worth of U.S. paper.

It may well be a mugs game, but it is required.

There is an interesting podcast on this from the Australian ABC.
posted by sien at 8:18 PM on August 10, 2011

I can't recall where I read it, but over the weekend I was seeing articles suggesting that the rules in place for most institutional investors put U.S. treasuries in a separate category, since they are/were assumed safe and a downgrade was historically unthinkable? (and thus they won't be forced to sell no matter future downgrades?)
posted by nobody at 1:04 PM on August 11, 2011

S&P Faces Justice Department Inquiry Over Mortgage Security Ratings. The timing of the news makes the investigation smell retaliatory, but it had reportedly begun before the downgrade.
posted by exogenous at 7:58 AM on August 18, 2011