Skill-Luck Continuum

November 20, 2012 11:15 AM

"We have little trouble recognizing that a chess grandmaster’s victory over a novice is skill, as well as assuming that Paul the octopus’s ability to predict World Cup games is due to chance. But what about everything else?" [Luck and Skill Untangled: The Science of Success]
posted by vidur (16 comments total)

That was a great read!

Bowling and darts would have to be games of pure skill then, right?
posted by Renoroc at 11:29 AM on November 20, 2012

Paul the Octopus's victory over the grandmaster was due to being able to move 8 pieces at once.
posted by Egg Shen at 11:29 AM on November 20, 2012 [3 favorites]

Also, his canny insistence that the match be played on his home turf.
posted by gilrain at 11:33 AM on November 20, 2012 [2 favorites]

Bowling and darts would have to be games of pure skill then, right?

There is certainly randomness in bowling, if we define randomness as variables that operate at the system level and can't be controlled by players, and where there is randomness there is luck involved. We could probably say that bowling and darts are exercising in overcoming/improving luck through the application of skill. It looks like Mauboussin did a statistical analysis to rank the sports, and a similar analysis could be done with bowling match outcomes and bowling scores. I would guess that bowling fits somewhere on the "skill" end of basketball - sometimes mediocre players will have a 'hot streak' and do better than predicted, which Mauboussin would atribute to luck.
posted by muddgirl at 12:05 PM on November 20, 2012 [1 favorite]

Reminds me a bit of when I played "Tournament style" Super Smash Brothers Melee/Brawl with a gaming group. (no items, limited to several maps, certain characters were banned) which turned it into a mostly skills based game, and also sucked the fun right out of it.
posted by hellojed at 12:12 PM on November 20, 2012

I postulate that skill vs. skill in competitive sports can often be indistinguishable from luck.
Baseball is a perfect example. The author ranks it lower than football (soccer) in skill. But the skill involved in matching a skilled pitcher with a skilled hitter is extremely complex. There are batting average statistics for each hitter in at least nine different strike zones, and these vary with different pitchers and different pitches, e.g., fastball vs. change up, vs. curve ball. The batter also knows the pitcher's most hittable pitches, the most likely zones, the percentage of fastballs vs. curve balls, and the pitch he is looking to hit for each count and each runner-on-base scenario. The fielders also know the positions to play for each hitter's statistics. Then you may have a base runner who changes the pitches with the possibility of a steal--the pitcher normally can't throw a curve ball with a runner stealing second. The results can be pretty random, yet incredible knowledge and skill may be involved, with the managers signalling the players from the bench. Soccer isn't even in the same ball park. I heard a baseball analyst say statistics prove that the introduction of closing pitchers hasn't changed the game, so they are basically unnecessary and overpaid. But that is only because you have closers vs. closers. Put closers back in the old days, and there would be no contest.
posted by weapons-grade pandemonium at 12:35 PM on November 20, 2012

That's actually addressed near the end:

Absolutely. I think the critical distinction is between absolute and relative performance. In field after field, we have seen absolute performance improve. For example, in sports that measure performance using a clock—including swimming, running, and crew—athletes today are much faster than they were in the past and will continue to improve up to the point of human physiological limits....

But where there’s competition, it’s not absolute performance we care about but relative performance. This point can be confusing. For example, the analysis shows that baseball has a lot of randomness, which doesn’t seem to square with the fact that hitting a 95-mile-an-hour fastball is one of the hardest things to do in any sport. Naturally, there is tremendous skill in hitting a fastball, just as there is tremendous skill in throwing a fastball. The key is that as pitchers and hitters improve, they improve in rough lockstep, offsetting one another. The absolute improvement is obscured by the relative parity.

This leads to one of the points that I think is most counter to intuition. As skill increases, it tends to become more uniform across the population. Provided that the contribution of luck remains stable, you get a case where increases in skill lead to luck being a bigger contributor to outcomes. That’s the paradox of skill. So it’s closely related to the Red Queen effect.

posted by muddgirl at 12:42 PM on November 20, 2012

An old race car driver once told me that skill is luck you can trust.
posted by Phyllis Harmonic at 1:20 PM on November 20, 2012 [1 favorite]

The author ranks it lower than football (soccer) in skill. But the skill involved in matching a skilled pitcher with a skilled hitter is extremely complex.

Note that the rankings are not how much skill is required in playing the sport, but rather how big the impact of skill is relative to the impact of luck on the outcome. If it's all skill, the more skilled player will always win. If it's all luck, the all players will win with equal frequency (e.g., calling a flipped coin), regardless of how demanding the skills needed to play the game. Because luck plays a bigger role in baseball than in basketball does not mean that baseball requires less skill to play. One could even argue that baseball takes more skill than basketball, but the outcomes of those skills are much more influenced by luck. Certainly, hitting a pitch by a major league pitcher is one of the most difficult feats in sports (almost up there with scoring a goal in soccer), but it only occurs about one in three at bats for the best hitters and not predictably at all. A hit itself often depends on inches (off the foul line, off the glove of the fielder, past the fence0 or microsecond (between the arrival of the runner and the ball), and even the most skilled player doesn't control the flight of the ball or their running speed to that degree.
posted by Mental Wimp at 2:24 PM on November 20, 2012

Note that the rankings are not how much skill is required in playing the sport, but rather how big the impact of skill is relative to the impact of luck on the outcome. If it's all skill, the more skilled player will always win. If it's all luck, the(n) all players will win with equal frequency..

My point was that if you match two virtually identically skilled players or teams, they will also tend to win with equal frequency. So equal skill with very little luck is indistinguishable from all luck, from a statistical viewpoint. Skill vs. luck recognition in competitive sport is often extremely subjective. And with equally matched players, it may not be luck that determines the winner so much as variations in skill, e.g., Nadal plays better on clay. You can't reverse engineer the win/loss statistics to determine the skill level in competitive sports. You have to watch the match.
posted by weapons-grade pandemonium at 3:38 PM on November 20, 2012

I don't know the details of this particular analysis, but I worked on a similar exercise with college football. I wouldn't determine the skill/luck balance in an entire sport just by looking at one match (say, Federer/Nadal at the 2008 Wimbleton). I would take all the data from an entire season, or even several seasons, and analyze player/player or team/team matches both iteratively (who beat who beat who) and through time (in how many matches has John Isner beat Federer or Djokovic? What's the expected value if it's completely random? What's the expected value if it's completely skilled?)

So equal skill with very little luck is indistinguishable from all luck, from a statistical viewpoint.

Not if we're looking at the results of an entire sport's worth of outcomes. The purpose of Mauboussin's chart isn't to give fans excuses for bad outcomes of any particular game.
posted by muddgirl at 4:05 PM on November 20, 2012

Rereading your comment, another point is that when two people have equal skill, luck is a much greater contributing factor to who wins than two people with disparate skill. If Federer and I went one set, his winning is 100% attributable to our difference in skill. When Federer and Djokovic go head-to-head, since they have even skill levels, the outcome of the match is more attributable to luck. So yes, equal skill in a single match is functionally equivalent to all luck.
posted by muddgirl at 4:13 PM on November 20, 2012

So equal skill with very little luck is indistinguishable from all luck, from a statistical viewpoint.

Not if we're looking at the results of an entire sport's worth of outcomes

If you're looking at the result of an entire sport's worth of outcomes, you're no longer looking at equal skill, are you?
I don't dispute that good skill beats bad skill. Nobody disputes that.
posted by weapons-grade pandemonium at 4:31 PM on November 20, 2012

If you're looking at the result of an entire sport's worth of outcomes, you're no longer looking at equal skill, are you?

I am if all the teams in a sport have roughly equal skill levels.
posted by muddgirl at 4:48 PM on November 20, 2012

a chess grandmaster’s victory over a novice is skill

Skill: Proficiency, facility, or dexterity that is acquired or developed through training or experience.

Some of what a grandmaster does is skill. But the ability of many of them to play n>10 simultaneous games blindfolded suggests that the luck of genetics is involved. (If you consider something akin to autism luck.)

Anyone knows this who's spent too many quarters to find out that they will never be a pinball wizard. Or spent years bowling and never broke 200. You either got IT or you ain't. And you can't develop IT if'n you ain't got IT.
posted by Twang at 4:58 PM on November 20, 2012

So equal skill with very little luck is indistinguishable from all luck, from a statistical viewpoint.

Not if we're looking at the results of an entire sport's worth of outcomes.

If you're looking at the result of an entire sport's worth of outcomes, you're no longer looking at equal skill, are you?

I am if all the teams in a sport have roughly equal skill levels.

*Facepalm*
posted by weapons-grade pandemonium at 6:53 PM on November 20, 2012