Tyson Barrie fails my eye test, but does it matter?

Originally posted while the season was underway, enjoy this Retro May look at a player who no one seems to agree on.

My eye test despises Tyson Barrie. Whenever I watch the Leafs, it’s a guarantee that he will, at some point, do something that makes me sigh. He takes too many shots on the power play*, his defensive play is horrific, and he routinely ‘activates’ himself too early or in inopportune situations (for example, with three forwards in below the faceoff dots) which leads to the Leafs scrambling to defend an odd-man rush the other way.

* One of my pet peeves from last year was people arguing the Leafs power play ‘needed’ a threat from the point. In my opinion, the Leafs first unit power play has four better options than a point shot — the opportunity cost of such a shot is quite large as a result.

And yet, when you take a step back and look at the stats, Barrie looks ... mostly fine? Depending on the time-span that you look at, he might even appear to be pretty good! I trust my eye test about as much as I would trust Brad Marchand in a shootout. I try and defer to the stats, because for most people, eye tests suck. We get clouded by our biases, and by our memories. I (among many others) was a vocal proponent of Jake Gardiner for exactly this reason. He had memorable failures, but many quiet successes that more than made up for them. Is Barrie the same way?

The Stats

The first thing we should do is level-set and get an idea of what the stats have said about Barrie this year, and across his career. In this article, I’m going to focus heavily on play driving — that is, how well does Barrie (or any player) do at ensuring that his team gets the lion’s share of the shots, expected goals, and goals when he’s on the ice. Points are of secondary importance to me, as I find they’re a crude measure of the above, especially for defencemen, but we will also take a look at Barrie’s individual stats.

In fact, let’s start there.

Individual Stats

The Leafs season can be pretty neatly divided into the Babcock and Keefe regimes, but to start, we’ll consider the season as a whole. The table below shows Barrie’s individual stats history (at 5v5) across his career.

Tyson Barrie Individual Stats (via NaturalStatTrick)

Barrie’s individual numbers are fairly similar to his numbers over the past few seasons. His goal rate is down from years prior, largely due to a notable drop in shooting percentage that sees him dip below his expected goal rate. One of Barrie’s calling cards over his career has been his ability to outshoot expected goals, which is quite rare from a defenseman. He hasn’t done so this year, which could very easily be the natural variance that occurs in shooting percentage on a year to year basis. His expected goal rate is still quite strong (ranking within the top 30 of defensemen league-wide), though not as strong as it was in his last year as a member of the Avalanche. He’s made up for it with a strong assist rate, which results in his overall point rate still being strong — he ranks 35th among defenseman in 5v5 scoring rate this year.

On the whole, there can be little complaint about his point production this season, especially given his poor shooting percentage, which is not completely within his control.

What happens if we split the season into Babcock and Keefe’s regimes, and look at the same stats?

Tyson Barrie under Different Coaches

There’s a pretty notable difference. While Barrie’s shot rate has remained similar (or decreased, if we look at shot attempts) under Keefe, the quality of chances he’s gotten has vastly improved, as has his point rate. It’s clear that Keefe’s system has Barrie more involved offensively than he was under Babcock. You probably didn’t need these stats to tell you that, but it’s reassuring that they reflect what we’d all intuitively think in this case.

Based on this alone, it’s tempting to say that Keefe has ‘fixed’ Barrie, and that Barrie was neutered under Babcock. Certainly, his individual stats have improved under Keefe, but that’s not the true test. Ultimately, the Leafs don’t exist to make Tyson Barrie get points and shots and feel good about himself. They exist to win hockey games, and you do that by outshooting, outchancing, and outscoring the opposition. So we now turn our attention to that.

On-ice Stats

Truthfully, this has always been one of the more tenuous aspects of Barrie’s game. Two of the best measures we have for assessing a player’s ability to impact his team’s chances of outscoring their opposition are Isolated Threat, and Regularized Adjusted Plus Minus (RAPM).

In short, both attempt to isolate the impact of every player on a target on-ice statistic. Isolated Threat isolates a player’s impact on team-level Threat (a very simple location only expected goals model). So a player who ranks highly in Isolated Threat would be a player whose team tends to generate a lot more expected goals than their opponents when said player is on the ice (and vice-versa for a player who ranks lowly in Isolated Threat). RAPM, on the other hand, is calculated for a variety of target on-ice statistics, including goals for (and against), expected goals for (and against), and shot attempts for (and against).

Barrie has never been a particularly strong play driver by either of these measures.

The above suggests that Barrie was last an above average play driver in 2014-2015. In every year since, he’s been both a negative on offense (the upper part of the figure) and a negative on defense (lower part of the figure; positive numbers are a bad thing for defense).

This has a slightly different conclusion to Isolated Threat. It says that Barrie was above average offensively from 2015/2016 - 2018/2019* (in terms of driving goal rate and shot rate, though not expected goal rate), but he is so catastrophic defensively that it outweighs that offense.

It’s worth noting that other (simpler and cruder) measures of play driving are kinder to Barrie over the course of his career. Simply looking at CF% Rel, xG% Rel, or GF% Rel would paint Barrie as a play driver in each of those metrics over his career. However, these measures do not purport to attempt to adjust for teammates, competition, and zone usage the way Isolated Threat and RAPM do. While the latter are not perfect, I think they are significantly better than simple relative stats, and both seemingly attribute more of the on-ice success that Barrie has been a part of to his teammates rather than to Barrie himself.

With that background out of the way, we can turn our attention to this season. As the Isolated Threat image above shows, Barrie has not been considered a good play driver this year either, at least not on the whole.

If we look at RAPM for this season, Barrie grades out as slightly below average in terms of his goal and expected goal impact, and above average in terms of his shot attempt impact. Either way, it’s nothing outstanding.**

* This is the largest sample that Evolving-Hockey allows one to run to aggregate RAPM

** There are some technical reasons why Isolated Threat and RAPM differ in their opinion of Barrie (and indeed, other players). It’s not worthy of getting into in this article — the important thing is that both feel he’s unimpressive at play driving.

But again, let’s dig into it a little more. Unfortunately, we can’t get RAPM and Isolated Threat for partial seasons, so there’s no way to split up the Keefe and Babcock regime’s in these stats. However, we can return to simple on-ice stats for now and get a rough sense of how they’ve changed under the two coaches.

Tyson Barrie On-ice Stats Across Coaches

Once again, we can see an immense difference. While the CF% is about the same, everything else has shifted about 180 degrees. His xGF% is around 56% under Keefe, which is stunning. His GF% is even better, and while that degree of overperformance is unsustainable, the true on-ice results have been so good that it almost doesn’t matter. From everything we’ve seen, Barrie has been both good and lucky under Keefe, and he was bad and unlucky under Babcock. The luck aspect doesn’t change the fact that the seemingly more sustainable elements of his performance have genuinely improved. Notably, Barrie’s zone usage under Keefe is not much more favourable than the usage under Babcock.

So Barrie has amazing on-ice numbers under Keefe and amazing point production (to the extent that we care about it). He’s playing real minutes too, so it seems there’s nothing really to discuss. If any other player on any other team had a 56% xG% and great point results, I’d be falling over myself to fantasy trade him onto our roster.

This relates back to the point I led off with. I don’t trust my eye test, and yet, even under Keefe, my eye test gets very heavily annoyed by Barrie. Am I just that bad at assessing hockey talent? Realistically, yes, but I’d like to think that I’m not completely out to lunch here. And I’d also like to think that Barrie’s track record of middling on-ice numbers (when adjusted for usage and context) play a part here. It doesn’t seem incredibly likely that he’s suddenly become a superstar in this regard. To me, it’s more likely a combination of playing within a system that is getting better results as a whole, improved play from Barrie, and increased time with the Leafs’ best players. But as of now, it’s hard to argue against the idea that Barrie’s results justify his process — as annoying as my eyes find them.

Still, there are a couple key questions that remain about Barrie. The first is whether he will be able to continue to be successful to the same degree under Keefe. His usage likely won’t change much, but as mentioned, he has never really been a play driver extraordinaire. 25 games of solid on-ice numbers in an offensive role on a team with elite forwards hasn’t convinced me that this tiger has changed its stripes.

The second is how scaleable he is. We can quibble over whether Babcock used him appropriately (at 5v5, I find it hard to find a compelling argument that Barrie was particularly misused, to be honest). At the same time, he bears some responsibility for his poor play under Babcock. We can’t simply throw it away and chalk it up to a bad fit — that doesn’t really hold up to scrutiny. More generally, if he has to be used in an extremely particular way in order to coax good results out of him, that has a knock on effect for the rest of your team. You deprive other players of those same opportunities, and force them to play in a role that may not be great for them, or your team. Conversely, if he’s so system dependent, is it him that is doing the heavy lifting, or the system. Even under Keefe, his relative stats are good, but not great.

So where does that leave us? Honestly, with more questions than answers. Hockey stats often try and divorce a player’s value from the context in which they play to get a sense of how good they are in an absolute sense. This is both a worthy goal, and something that is very challenging, even with players who seem to be pretty plug-and-play from role to role and system to system. With Barrie, I think the latter part is emphasised, as his usage and context seem to dictate a large part of his value this season. So far, Sheldon Keefe has found a role that allows the Leafs to get good results with him on the ice, even though it’s unclear to me how much of an impact Barrie himself has in that. Given that he’s a pending unrestricted free agent with no chance of re-signing here, that may be all we need.

Acknowledgements

Stats are courtesy of Natural Stat Trick and Evolving-Hockey. They are current as of January 17th, 2020.