Underestimating the Fog

May 27, 2005 10:01 AM

(As any Mets geek might say when talking to Mike & the Mad Dog: First time [MeFi] poster, long time reader)
Underestimating the Fog...No, not crochety ol' McNamara's take on the situation in Iraq. Rather, it's an astonishing (if only partial) recanting [.pdf] by the patron saint of statheads, seamheads, and rotogeeks everywhere, Bill James. Like his namesake, James is a radical empiricist, the Jedi master who defined sabermetrics (his coinage) as "the search for objective knowledge about baseball." Over the past several decades, James' influence has been enormous. After Michael Lewis famously detailed the saber-success of Billy Beane's Oakland A's, Sabermetric-leaning analysts have made their way into the scouting ranks and GM's offices of a growing number of major league ballclubs. From the halls of academia [.pdf] to newspapers and Cable personalities, even the NFL and NBA are on board!
posted by ericbop (22 comments total)

and stop hating my freedoms.
posted by jsavimbi at 10:13 AM on May 27, 2005

MeTa.
posted by Specklet at 10:27 AM on May 27, 2005

Lord... the shit is going to hit the fan now. Hang in the ericbop.
posted by Witty at 10:30 AM on May 27, 2005

An interesting post with alot of thought behind it. Lots of support and I like that you even got in a William James reference too.

I read or heard somewhere recently that one of Einstein's theorems was wrong but we still have nuclear (sorry, Mr. President, nuku-lar!) weapons so I'm not quite sure how important James' disclosure is. Then again, the A's suck this year so maybe there's something to it.
posted by fenriq at 10:31 AM on May 27, 2005

fenriq-- I think Beane realized a while back that 2005 would be a rough patch. This interview with him is fascinating, IMO.
posted by Kwantsar at 10:40 AM on May 27, 2005

I liked this thread a lot. Thanks for putting in the effort of typing it out for us baseball stats addicts.
posted by dios at 10:51 AM on May 27, 2005

What's all the fuss about??

If you're going to alter my post, could you at least link to it with a "more inside" link so that the point I was trying to raise (you know, for, like, people to have a thoughtful discussion about)
isn't completely vaporized?

And "jock post?!?" I guess it takes one to know one. Taking the "phor" out of "meta" since 1999...
posted by ericbop at 11:19 AM on May 27, 2005

ericbop, your beef belongs in MeTa, where Specklet advised you look.

Rookie.
posted by Kwantsar at 11:23 AM on May 27, 2005

Great post! I love baseball stats and this is a good roundup of the 'stat invasion' that started in baseball and is spreading to other sports.

I've always really enjoyed sabermetrics, but I still am not 100% convinced that they should be used to run baseball teams. They should be part of the decision making process, but what makes baseball great is that despite all the stats, people and players still surprise you year in and year out. Which, it seems that James is directly addressing in the PDF-- the difference between transient and persisent phenomena.
posted by chaz at 11:32 AM on May 27, 2005

Kwantsar, yeah, trading away your best two pitchers will tend to do overturn the apple cart. Yes, they were going anyway but it just gets tiring watching the A's develop ace talent and then let it go away to other teams (though I must say I'm quite pleased that the O's have Miquel now).
posted by fenriq at 11:36 AM on May 27, 2005

When I first read this article, I found it to be a big waste of time, and I haven't changed my mind. The article's thesis can be be summed up by the old phrase "You can't prove a negative." Which, of course, is true. We can't prove that there aren't "clutch hitters" in the major leagues; we can only prove that we can't prove that there are "clutch hitters" and that if there are, we can't identify anybody who shows this skill with any consistency. Almost every analyst I've read over the past twenty years has understood the difference. But James is treating this concept as if it's some new discovery.
posted by gspira at 11:41 AM on May 27, 2005

Great post, worth it for the James pdf alone. Not only does the man have a brilliant ability to rethink stats and how they work (or don't work), he has the guts to realize and admit when he's blown it.

We ran astray because we have been assuming that random data is proof of nothingness, when in reality random data proves nothing... Cramer argued, "I did an analysis which should have identified clutch hitters, if clutch hitting exists. I got random data; therefore, clutch hitters don't exist."... Random data proves nothing—and it cannot be used as proof of nothingness. Why? Because whenever you do a study, if your study completely fails, you will get random data. Therefore, when you get random data, all you may conclude is that your study has failed.

Thanks, ericbop. A promising first at-bat!

gspira: I don't think that's all he's saying.
posted by languagehat at 1:44 PM on May 27, 2005

Why can't you calculate a player's batting average over all his at-bats vs his clutch at bats (for some definition of clutch)?

Are there truly no players in baseball whose clutch batting average is persistenly different than their non-clutch at-bats?
posted by event at 2:23 PM on May 27, 2005

Very interesting post, thanks! I don't follow baseball stats but do try for the NFL. I didn't know much about sabermetrics, and am very intrigued by it and am curious about what other inroads this might make into other sports - or what effect James' "taking it back" will have.
posted by livii at 2:34 PM on May 27, 2005

Bill James, and his protege, Billy Beane, are the snake oil salesmen of baseball. They take credit for a team's success, but no blame for it's failure. Beane's insistence that the manager is not integral has certainly cost his team chances at the World Series, as Art Howe and Ken Macha have pulled one bonehead move after another. James insisted that a closer is not important and decreed bullpen by committee, but when the BoSox floundered, they went out and got a closer and won the World Series. Beane and the Oakland A's owe their success in the regular season to the good scouting, or good luck (probably a bit of both), of acquiring Zito, Mulder, and Hudson. Two of those guys are now gone (and the third to go soon). Now, they helplessly sink like a stone to the bottom of their division. What exactly should we give him credit for?
posted by a_day_late at 5:21 PM on May 27, 2005

fenriq writes "An interesting post with alot of thought behind it. Lots of support and I like that you even got in a William James reference too.

"I read or heard somewhere recently that one of Einstein's theorems was wrong but we still have nuclear (sorry, Mr. President, nuku-lar!) weapons so I'm not quite sure how important James' disclosure is. Then again, the A's suck this year so maybe there's something to it."

You know, I haven't yet read the post's links because I don't have a lot of time right now, but I want to point something out that I find hysterical...

Without context, the above statement is completely impossible to make heads or tales of. Einstein? Nuclear weapons? William James? BASEBALL?!

Not meant as a criticism, btw. I'm just pointing out something that made me laugh.
posted by shmegegge at 6:28 PM on May 27, 2005

James is right that teams don't need a closer; the White Sox are demonstrating that right now. Somehow no one objects when it's Ozzie Guillen saying that his team doesn't need a closer. But James' main point - and one that he's 100% right on - is actually that it's idiotic to use your best relief pitcher in the 9th inning with a 3 run lead vs. the 7,8, and 9 hitters just because it's a save situation and let your mediocre middle relief guy handle the bases loaded one out 1 run lead facing the clean-up guy situation in the 7th or 8th. The save is the most damaging statistic ever created for baseball, because it's actually changed the way relief pitchers are used, and thus the best relief aces expect to be used in situations in which saves are awarded -instead of the situations in which they can do the team the most good.

Event - Yes, you can calculate every player's production in "clutch" at-bats. Some categories include batting with runners in scoring positions, and batting in late inning pressure situations.

The problem is that these statistics appear to have no real predictive value. If you take the ten hitters with the best clutch differential over the past three years (say, batting with runners in scoring position minus batting in all other situations), they aren't anymore likely to have good clutch differentials over the next few seasons than the players with the worst clutch differentials.

Now, a few hitters still end up with better career averages in the clutch than in other at-bats - Paul Molitor was one, I think - but no more than you would expect to find if there was no clutch effect. You'll find similar results if you test whether hitters hit better on days of the month that are prime numbers.

Now, this does not prove that clutch hitting doesn't exist. You can't prove a negative, and it's kind of hard to define clutch anyway. But it does seem to indicate that we can't identify clutch hitters with any degree of certainty.
posted by gspira at 7:26 PM on May 27, 2005

gspira: Now, this does not prove that clutch hitting doesn't exist. You can't prove a negative....

But it would only take the existence of a single batter, over the entire history of baseball, who consistently (er... persistently) has a better batting average in clutch situations to prove the positive. No?
posted by event at 7:45 PM on May 27, 2005

No. As I said, there are such players. But not any more than one would expect if the distribution was by chance. There are also players who, over their careers, have hit better on days of the month that are prime numbers, but that's not proof of a skill to hit better on those dates.
posted by gspira at 12:45 AM on May 28, 2005

James is right that teams don't need a closer; the White Sox are demonstrating that right now.
The baseball season does not end before Memorial Day weekend. Perhaps you can provide some recent example with long term success (including success in the post season).

The save is the most damaging statistic ever created for baseball, because it's actually changed the way relief pitchers are used
Managers don't give a fart about saves. They use their pitchers to get the most wins for their team. The idea they are managing to better the pitcher's stats is laughable. Otherwise, I have no idea how this stat has "damaged baseball."

Not being snarky, BTW, just not at all convinced by your arguments.
posted by a_day_late at 5:10 AM on May 28, 2005

The idea that managers don't manage in a way about saves is ridiculous. Most closers are carefully managed so that, unless they need work, they won't pitch in any situation at all. Anytime there's a 3 run lead in the ninth, the closer will be brought in. A 4 run lead, and the closer won't be brought in. Unless, of course, the middle relief pitcher makes things slightly worse and the game becomes a save situation, in which case the closer is immediately brought in so he had pad his stats. I see this every day in many mlb games. The only similar situation in baseball is that sometimes managers will try and strech a sytarter who's having a poor start but has a lead through 5 inning (never 6) just so he can't get the win. But that happens pretty rarely, as opposed to thousands of times a year.

Of course, the big egos of players signed to big contracts come into play elsewhere in baseball, limiting what the manager can do. But most of the time the damage is minimal, since with everday players it just means that they'll stay in the middle of the lineup even if they're struggling and should be moved. In reality, lineups don't matter very much; if the same players are involved, the difference between the best and worst lineups managers will put out there are only a couple of wins a year.

If you don't think the save has changed baseball, why do you think the usage of relief pitchers has changed so much over the past 50 years. Until the save statistic was invented, there was no such a thing as a closer, and no one cared who finished up games. Baseball survived for 100 years without anyone counting saves, and there has been absolutely no change in the percentage of games won when a team has the lead after the 6th, 7th, 8th, and 9th, despite the huge shift in the way extra spots on the roster have shifted from being used for pinch hitters to being used for relief pitchers.

If you watch postseason games, you see much more intelligent usage of closers. Mariano Rivera is often brought in way earlier than the 9th inning in postseason games, because no one cares how many postseason saves Rivera gets; they only care about winning games. The postseason rarely comes up in arbirtration hearings when high totals of saves are the way for relief pitchers to win high salaries. So a good closer actually ends up being more valuable in the postseason (depth, of course, is generally less valuable in the postseason than the regular season in a number of ways) because he's used more like a relief ace should be. Check out Rivera's ration of decisions to saves in the regular season and his ratio of decisions to saves in the offsesaon; you'll find an unbelievable massive difference.

I obviously can't name many teams in the recent past that have succeeded without closer because few have tried, and the few that have tried are usually bad teams that couldn't afford one. (Nevertheless, if I had 4 hours, I'm sure I could find 10 teams over the last 7 years who did just as well without a closer as teams with one did) Thus someone usually gets assigned the role of the closer, even if the someone is someone radom. Derrick Turnbow and Brandon Lyon have been the closers of their teams so far this year. Who was Danny Kolb before last year? But again, the main point was never that you don't want an outstanding bullpen pitcher; the point is that you want to use him in key situations, and those situations are not the same as closing situations.

For most of baseball history, until about 1980, every single team survived without a closer, and lots of teams did just fine. The current closer role was defined by Tony LaRussa with Dennis Eckersley, and maybe it was the best way to use Eck; I don't know. But it sure isn't ideal for winning baseball.

As for the 2003 Red Sox, their problem at the beginning if the season wasn't that they didn't have a closer, but that everybody in their bullpen stank. They were blowing games long before a closer would have even come in.
posted by gspira at 3:27 PM on May 28, 2005

gspira, We're just going to have to disagree on principle. I don't see a connection between your beliefs and actual facts. For example, you say:

If you don't think the save has changed baseball, why do you think the usage of relief pitchers has changed so much over the past 50 years.

Because this change has brought success to teams that employed it, and was therefore copied by other teams.

If you watch postseason games, you see much more intelligent usage of closers. Mariano Rivera is often brought in way earlier than the 9th inning in postseason games, because no one cares how many postseason saves Rivera gets; they only care about winning games.

Why do you draw that conclusion? Of course Rivera will be used in a different fashion in the post season. In the regular season, you don't want to wear his arm out, and if you lose the game, you live to play another day. The post season is important and will end very soon, and he will have all winter to rest.

As for the 2003 Red Sox, their problem at the beginning if the season wasn't that they didn't have a closer, but that everybody in their bullpen stank. They were blowing games long before a closer would have even come in.

IMO, the bullpen is a team in itself and the closer is the anchor to the bullpen. When you have a dominant closer, it makes the other relievers better. Many set up pitchers have credited much of their success to the extra confidence they derive from having a dominant closer backing them up. One example is Tanyon Sturtz who has risen from mediocrity to being a very solid set up man (in large part due to Rivera). Sturtz has stated as much. There is a psychological aspect to the game too, you know. I doubt James and Beane understand that.

Here is my problem with James and Beane. They pass themselves off as geniuses but look at the results: If you win because of Billy Ball, you should win whether Mulder and Hudson are there or not. If you play a certain way because of percentages, but your team (which has dominant starting pitching) can't get past the first round of play offs year after year, your system is responsible for those failures too. If you don't need a closer, but then go out and get Faulk, and ride on his back to the World Series (which the BoSox did last year), err, your theory didn't work to begin with, so don't take credit for the success. Where are these guys' successes are in the real world???
posted by a_day_late at 6:10 PM on May 28, 2005