Hockey Analytics: U.S. Olympic Team's Best Goalies Aren't In Sochi

Ok, I'm going to stop them right here because the headline makes no sense. The US Olympic Team's best goalies will be in Sochi because they'll be the only US Olympic Teams goalies there. The headline should say "US Olympic Team's Best Options for Goalies Aren't in Sochi".

Now that's I've satiated my inner grammar Nazi, let's get on with the show.

The Olympics are just around the corner and there’s no shortage of prognosticating and second guessing. Pundits have argued ad nauseam. What’s the magic formula? Experience? Speed? Heart? Size? Defense? Team chemistry? Ability to play on the larger international ice?

No mention of "ability to play hockey" so you know we're off to a good start.

I’m not going to talk about any of those things. I’m going to talk about something near and dear to my heart from my days as a professional poker player: game theory.

So the secret formula to winning at the Olympics is game theory? Oh this gunna be good.

Before you go scrambling to turn the page, terrified that if you read any further you might lapse into a boredom coma from which you may never emerge, hear me out.

I'm not sure what's worse; this terrible self-deprecating quote which might have well been "Sorry I write such boring articles. Aren't I just the worst? Lol numbers!", or the quote about they're just guys talking about stats and hockey over a few beers from their first installment.

From here our author proceeds to give us the wrong definition for what game theory is, as well as misunderstanding how the Olympics work.

This matters because the rules of the Olympic tournament are different from the rules of an NHL season in one critical respect. When it counts, the Olympics are single elimination. That one change should make a big difference in how an Olympic team is built.

The medal round in the Olympics is a single elimination tournament, but before that teams are seeded base on their divisional play. Do well enough in the 3 prelim games, and your team earns a bye (essentially a free win and a day's rest) as well as the easiest route to the gold medal game. A route that's at least 3 knockout games long, and as long as 4.

(The stuff about how the Americans aren't as good as the other teams because the Vegas odds makers say so it beyond stupid. You know it. I know it. Hell even the author probably knows it.)

Here's where the author's reasoning goes off the rails.

...you want to maximize the probability of an unlikely result. You want to increase what economists and poker players call variance. Picking reliable, consistently good players isn’t the way to make that happen.

(Note: This is intended to be read as: "...you want to maximize the probability of an unlikely positive result.")

Maximizing variance makes sense for a game like poker where you can only win a hand, and not losing is not an option. Having a card you need turned over by the dealer is an act of pure chance. Maximizing that chance by playing hands with a greater chance of winning based upon the revealed cards is a smart movie. For example, the best starting hand is two Aces because the chances that nothing is flopped that will help your opponents is greater than the chance that it will. On the other side, the statistically worse hand to play is 7-2 offsuit because that starting hand has the lowest chance of getting help from the revealed cards in order to beat any other possible two-card combination.

Goaltending in hockey is not like a single hand of poker and variance is not a trait to be desired. To increase a team's chance of winning a hockey game through goaltending, it's best to minimize the goaltending risk of losing it rather than increase the goaltending chance of winning it. Case in point, no team can win a game solely by not allowing a goal against; they must also score.

As an example let's create three hypothetical goaltenders: Goalie X, Goalie Y, and Goalie Toskala.

Goalie X stands on his head 50% of the time, is average 0% of the time, and shits the bed 50% of the time.

Goalie Y stands on his head 33% of the time, is average 33% of the time, and shits the bed 33% of the time.

Goalie Toskala stands on his head 25% of the time, is average 25% of the time, and shits the bed 50% of the time.

Goalie X has the most variance and the best chance of winning outright. Goalie Y has the least amount of variance, but is consistent. Goalie Toskala is dogshit.

If a goalie stands on his head, the team has an 80% chance to win. (Remember, they still need to score goals.) If a goalie is average, the team has a 50% chance to win. If the goalie shits the bed, the team has a 10% chance to win. So based on these goalies, what is the team's chance to win in any given game with one of the three goalies? (Again this is all just hypothetical with numbers pulled from my butt.)

Team Win% 0.80 0.50 0.10 Good Avg Bad Win%
Goalie X 0.50 0.00 0.50 x 0.40 0.00 0.05 0.45
Goalie Y 0.33 0.33 0.33 x 0.26 0.17 0.03 0.46
Goalie Toskala 0.25 0.25 0.50 x 0.20 0.13 0.05 0.38

Were you playing poker, you'd want Goalie X every time because you can only win in that game. But in hockey, you'd want Goalie Y because, while he doesn't have the extreme high end, his consistency allows his team to win more games of various scenarios. Said another way, Goalie Y not loses the game more than Goalie X wins the game. Nobody wants Goalie Toskala or his man purse.

Consistency at a high level is far more desirable than variance of a high level.

We came up with five stats (Note: their chart only has four stats.) that measure the propensity of each goalie to have that huge, difference-making game: shutout percentage, games allowing one goal or fewer and percentage of games with a save percentage of .940 and .950 or better.

Tracking games based on goals allowed is stupid because it shot volume plays a huge role in determining goals against. For example, two goalie each have identical .920 save percentages. The one who sees 20 shots a night will have fewer games with 2+ goals against than the goalie who sees 30 shots a night. Same thing goes for shutouts. The only goalie stat that matters is save percentage and if anyone tells you differently, they are dead wrong.

What I did was expand on their sv% data for the five goalies listed: Jonathan Quick, Jimmy Howard, Ryan Miller, Ben Bishop, and Cory Schneider, and gave sv% numbers for full games played or games started (no relief appearances with less than 30 minutes played) during the time frame they've established (January 1, 2013 thru December 31, 2013) all the way down to .900 and below. You can find the data at the following spreadsheet:

Team USA Goalies SV% Data

...for one game (or the three it takes to win gold), as the significant underdog, the Americans should want Bishop or Schneider between the pipes.

Based on the last calendar year, yes they should, but not for the reasons the Department of Hockey Analytics say they should. While Schneider and Bishop do have the largest percentage of games with a a high sv%, it's their low percentage of games with a low sv% and low variance of sv% between games (StDev of 0.060 and 0.05 each versus 0.1 for Quick and 0.07 for Howard), that actually makes them the correct choice for Team USA along with Ryan Miller (StDev of 0.05 and the next best sv%).

To further this, look at Steve Mason, a goalie who everyone agrees is highly inconsistent. Mason has a higher percentage of games over a .940 sv% (38.1%) than Howard (32.81%) or Miller (32.84%), but also a higher percentage of games under a .900 sv% (38.1%) than either Howard (32.81%) or Miler (29.85%).

Based on the DoHA assessment, were Mason an American, you'd be remiss to pick Howard or Miller above him because he can more frequently win you more games", but as I've shown that comes at the cost of the increased likelihood to outright lose you games.

So what are my conclusions?

a) Judging goalies on a 50-60 game sample size as the DoHA has is a stupid exercise.

b) That conclusion 'a' aside, they were looking at the wrong data when they made their own conclusions.

c) When selecting a goalie, it's best to pick one who doesn't lose rather than picking one who wins.

d) I would have picked Miller, Quick, and Bishop.

Update From Person Actually Paid By NHL Teams To Do Analysis