Rebutting Phil Birnbaum On Shot Quality

Phil Birnbaum, a respected sabermetrician known primarily for his work on baseball analytics, published a post arguing that the 2012-13 Toronto Maple Leafs have in fact discovered a way to dramatically increase team shooting percentage and that this is directly related to their low shot totals. This runs contrary to the usual statistical argument that the team's SH% had a significant element of luck and that it would likely have regressed somewhat if the season had been played for the standard 82 games rather than 48. Phil's argument has garnered a lot of attention and respect from people who would generally dismiss an argument about shot quality out of hand.

You should read Phil's post to fully understand what he's arguing, but I'll provide what I think is a fair summary of his position: teams that are leading have a higher SH% than teams that are tied or trailing. Teams that are leading also get beaten in terms of Corsi (shot attempts) because of that defensive shell. So a team could, as it were, play as though they were "up by 2" all of the time, which would result in their Corsi being low but their SH% being high. Birnbaum argues that this is in fact what happenned with the Leafs last season.

[EDIT: See the comments below, as Phil says that this is not an entirely accurate summary of his position.]

I'm not a big believer in shot quality at the team level, as you may already know. But I'm not going to argue about shot quality today. I'm also not a big believer in the ability of 48 game sample sizes to settle team-level statistical debates, but I'm going to try and stay away from that angle too. Instead I'm going to assume that the central premise of Phil's argument is true: that a team could play in a defensive shell that lowered their Corsi but increased their SH% dramatically. Even if that's true, I think you still need to demonstrate that last year's Leafs team did in fact deploy that strategy with that result. Phil doesn't do that (not that I can see, anyway) but I think I can pretty conclusively argue that the Leafs didn't work that way.

A Brief History Of Randy Carlyle

Phil's argument centers implicitly on Randy Carlyle; if the Leafs are employing this strategy then it must be because Carlyle has developed it and taught it to the team. If that were true, we would expect to see the effect also present in Carlyle's time in Anaheim. But as I showed in this post, we don't see it at all. Here's the relevant chart (updated to reflect end of season numbers for last year):

These statistics are all for Anaheim and they cover Randy Carlyle's last 3.5 seasons and Bruce Boudreau's first 1.5 (hence C and B for the split season). As I said in the linked post above, it's pretty difficult from this chart to draw the conclusion that Carlyle had any particular coaching technique in Anaheim that boosted SH%, since Anaheim shot just as well under Boudreau. While the Ducks did shoot about a half a percent higher than league average under Carlyle, I don't think you could conclude that it was because of a strategy linked to low Corsi. Boudreau's Ducks have not only had noticeably higher puck possession metrics, they've had shooting that was every bit as good.

A more likely explanation for the Ducks high SH% in recent years is that the top of the lineup has been filled with guys like Selanne, Ryan, and Perry who shoot a high volume of shots and have high individual shooting percentages. And it's worth noting that even with talents like those guys, the Ducks' long run shooting percentage is still only about a half a point above league average, nowhere near Toronto's 3 full points above league average.

An alternate explanation could be that Randy Carlyle developed some sort of new strategy that he employed in Toronto that he didn't deploy in Anaheim. But if that were true I would want to know what it was and how it worked. Carlyle's teams in Anaheim and Toronto have been low puck possession teams for several years. In one of those years the team shot way above league average. If Carlyle's new strategy elevates shooting percentages by playing "up by 2" and reducing Corsi, why has his teams' Corsi remained so similar while the SH% skyrocketed? Does that suggest a coach who developed an exciting new strategy that boosts SH% while reducing Corsi? Or does it suggest a coach whose teams have been bad possession teams for a while who just happenned to have some good luck?

Did The Leafs Play "Up By 2" When They Weren't?

Birnbaum's argument relies on the idea that the Leafs essentially adopted an "up by 2" strategy regardless of whether or not they were actually up by 2 or more goals. If that were true, we would expect that score effects on their possession metrics would be less pronounced than other teams. Here are the Leafs' Fenwick numbers for various score situations last year (in brackets is the difference from their score tied numbers):

Up 2 - 39.8 (-5.1)
Up 1 - 41.4 (-3.5)
Tied - 44.9 (0)
Down 1 - 47.3 (+2.4)
Down 2 - 51.8 (+6.9)

And here are the league averages in those situations, as taken from Phil's article:

Up 2 - 45.1 (-5.1)
Up 1 - 46.0 (-4)
Tied - 50.0 (0)
Down 1 - 54.1 (+4.1)
Down 2 - 57.0 (+7.0)

So the pattern for the Leafs is very much what you would expect for an average NHL team. This does not suggest a team playing "up by 2" when they aren't. It suggests a team that alters strategies in reaction to score effects the same way a league average team does but just happens to not be very good at controlling possession.

The evidence is pretty strongly against Phil Birnbaum's shot quality-based argument, I think. Carlyle's coaching history suggests that he is a coach whose style produces low possession numbers without a clear benefit to SH% and the situational Fenwick numbers suggest a team whose score-related strategies are pretty similar to what an average NHL team pursues. The 2012-13 Leafs didn't discover the shot quality unicorn. Not because of Randy Carlyle, anyway.