Someone posted some analysis of Frederik Andersen that used an interesting method to decide he’s just not good. They took his worst games, averaged them out, and said ta-da, here is the proof.

I think that’s really not how one goes about things. We wouldn’t accept taking the cream of his crop of games as proof a goalie is good, so why would one give the reverse the time of day? One shouldn’t. And yet, it had that nice mic-drop quality that every good Tweet has. It was also factually accurate, or I assume so, I didn’t run the numbers. Fact checking that sort of hockey analysis is as rewarding as fact checking any other style over substance communication.

What’s more interesting is why this substance-free, stylish dunking on the Leafs starter resonated with people. Clearly, many fans have decided that Andersen is not pulling his weight, and the all of our suffering as the Leafs drop right back out of the playoff spot they’d climbed into, are on his head.

A Meaningful Statistic

At times like these, I avoid looking at save %, and instead, I turn to a much more meaningful stat: Goals Against Average.

Goals Against Average is a perception stat when applied to a goalie. When looked at for a whole team, you can get an inkling of how a team plays, particularly if you rank it against other teams. But for an individual goalie, nothing beats GAA to judge why people feel the way they feel.

The eye-test of a goalie, the fannish conversation about him, is just GAA without the number. “Sometimes you just need to make a save” means the GAA is too high to be comfortable. “Stood on his head” means the GAA is low. And absolutely none of it really takes into account in a meaningful way the context the goalie is playing in and the team’s overall contributions to goals against.

Andersen’s GAA is currently 2.89, which is the worst he’s ever had in his career in league play. He’s had a few national team tournaments that were worse, but never a season of hockey. Well, no wonder everyone thinks he’s bad! Oh, his GAA in 2017-2018 was 2.81, and last year it was 2.77, and in both of those seasons, at the points at which he had played a few games in a row that weren’t all that great, he was considered to be bad by varying numbers of people. More in 2017-2018, less last year. But now the GAA in game after game is deeply uncomfortable, and the culprit is obvious.

Maybe we need something a little more vigorous than GAA, tinged with recency bias, to figure out if Andersen is bad or has just been bad some of the time.

A New Way to Judge Appearances

Over at Raw Charce, Alan AKA loserpoints, came up with a modern definition of Quality Starts that is quite simple, and we can use that to see how often Andersen is part of the problem or part of the solution.

Putting numbers to two common goalie terms: Steals and quality starts

Alan’s definition of Quality Starts seeks to improve on the old one, as he explains:

The current definition, which is available at Hockey Reference, comes from Rob Vollman who now works for the Los Angeles Kings. It awards a quality start to goalies who post a save percentage at least as high as the league average for that season or if they allow two goals or less and post a save percentage above that of a replacement level goalie.

This was a reasonable definition in 2009 when Vollman created it but we have data available now that wasn’t then. This new data allows for improvements. Nick Mercandante moved in this direction in 2016 when he created Above Average Appearances, which incorporated some information on shot danger by giving goalies credit for games where they posted above average save percentages against low, medium, and high danger shots.

Alan chose to simplify this right down by using expected goals, and he decided a Quality Start is any game where the goalie’s saves above expected is a positive number. So zero or more.

Expected Goals, remember, is simply shots weighted for quality by location, type, etc. This weighting produces a probability that the given shot will become a goal against an imaginary league average goalie.

On a game-by-game basis, Expected Goals data gets less accurate due to errors from the scorers that wash out in larger amounts of data, and it is neither a perfect prediction of every single shot’s quality, nor can it truly tell you what the goaltender should have done in a game. It is however, a hell of a lot better than going with your gut, your eyetest or GAA.

For another perspective on relying on Expected Goals at the game level, check out Back to Excited where they talk about how seriously to take the stats from a blowout that was over in the first period (no matter who is doing the blowing out).

Back to Excited Episode 89 - Goalie Troubles

Alan plotted all of this season’s goalies when he posted his article (not very long ago), so I can just cheat and find Andersen on his chart and tell you his Quality Start % was somewhere around 55% and there are 13 goalies better than him, some of them backups with limited minutes. None of them have more starts than he does, so there’s your ah-ha moment if you’re a believer in judging goalie rest from a seat on your sofa with a collection of carefully curated anecdata and certainty that it is a direct cause of win/loss records.

Of course, only two goalies have more starts than Andersen, and one of them has a near identical QS% to Andersen, and that of course is Connor Hellebuyck, noted bad goalie. The other is Carey Price, who actually has been something like bad and had a QS% when this article was published right about 50%.

Also, note that this year’s average is under 50%, and that goes along with a general truth about this hockey season: goals are up, goalies are dejected. It’s tough out there for everyone but goal scorers. Given that, we should expect Andersen (and every other goalie) to be a little worse than their career averages might lead us to expect.

Speaking of career averages, Alan also included the last five seasons, where Andersen is sitting with only six goalies with a higher percentage of Quality Starts. And he’s right about at 58%, so a little better than this year so far.

No one is suggesting this is a repeatable skill or even a great way to measure a goalie. But is is a very interesting way to understand how a goalie is perceived.  And as little as a few weeks ago, Andersen looked very like himself, accounting for a league-wide uptick in goal scoring.

Let’s have a look at Andersen’s season to date, so we can understand how his games stack up as quality starts now that he’s been terrorizing with horribleness.

Andersen has gotten a little less great by the number of dots in the top half of the chart. His worst game was in October against Tampa, but the knot of yuck just above it is from December and January. In fact, 10 of his 11 worst games were in those two months.

Let’s have a look at this by order of game:

What have you done lately, Freddie? Not much that’s good. Now, for a reality check. Let’s look at last year (note the scale stops at -4 on this one):

Slumps happen, and maybe we need to change the meme to Winter Freddie. If we’d had this conversation at this time last year, we’d be looking at fairly similar sets of results.

By the end of this year, who knows what this season will look like because all of his results this year fall within his range of previous results.

The sad fact is that every honest discussion of goalies always ends up here. Their performance varies, and they are who they are, and who they are is not consistent game-by-game, no matter how much you want them to steal wins for you when the team looks like an AHL squad who just spent two hours at the buffet before the game.

Alan also covers this concept. He defines Steals, which is even more for fun than Quality Starts, as:

Again, we can turn to expected goals models to solve this for us. And maybe even more easily than we might expect at first. If a quality start is any time the goalie’s goals saved above expected is at least zero, then I suggest a steal is any time the goalie’s goals saved above expected is greater than the final goal differential in the game. This is a similar definition to the one used by Cole [Anderson] in his work.

Andersen is not a game stealer. He is well below the average line both in this season and over the last five years.

This might be a surprise because the feeling about Andersen is that he’s the only reason the Leafs ever win. And lately he’s the one, the only reason they lose. But that’s not really true at all. Andersen, like most successful starters with multiple years above average, is just that — a guy who can play 40+ games a year and average out above water.

If that sounds like a tepid description, then let’s just reality check how rare that is. Forget Quality Starts and steals, let’s just zero in on Goals Saved Above Expected over the long haul. Big data, in big amounts, where the errors are washed out, the variance recedes and it’s the most accurate measure we have of a goalie’s career performance.

Big Data

I already posted this back in November:

Naming Names: Who are the elite, the average and the replacement-level goalies?

Just using this year, plus the past four years, the current top 10 by unrated GSAX looks like this:

  • John Gibson
  • Corey Crawford (where have I seen him, lately?)
  • Braden Holtby
  • Sergei Bobrovsky
  • Ben Bishop
  • Antti Raanta
  • Frederik Andersen
  • Henrik Lundqvist
  • Thomas Greiss
  • Jonathan Quick/

And the top 10 by Fenwick Save% above Expected (minimum 1,000 shots faced):

  • John Gibson
  • Antti Raanta
  • Corey Crawford
  • (I’m so sorry) Alexander Georgiev
  • Jordan Binnington
  • Ben Bishop
  • Sergei Bobrovsky
  • Braden Holtby
  • Philipp Grubauer
  • Curtis McElhinney (really)/

Andersen is 14th. And this is something we should understand intuitively. He plays a lot, and the more you play, the more your raw unrated stats overstate your ability. Unless you’re so good, you’re John Gibson that is. Lundqvist also slips down the rankings to about where Andersen is by this measure.

Lately, Though…

So is Andersen really bad? His current season GSAX is not great right now, and he is 25th for goalies who play a lot, and the unrated number is -10.33. Last year at this time, he was at +10 and was second in the NHL to John Gibson’s amazing 24.84. People were (wrongly) talking Vezina about Andersen. Something interesting is that this year’s best goalie by this measure to date: Ben Bishop, has a measly 7.83. It is hard to make a save out there.

But that was then, and this year the Leafs need to sit him because rest is magic, or trade for a new starter, or basically just tell him he’s made us feel bad, and if he loved us, he’d make a damn save! Because Andersen is now bad and will be bad forever!

Well, no. This is the classic mistaking “thing you’ve done lately” for “thing you are”. Andersen is who he has always been. He’s not quite elite, but he’s so good, and so close to the consistency most goalies never achieve, that he’s going to keep his team in with a chance most of the time. But sometimes he plays better for a stretch and worse for a stretch, because that’s just how it is.

Given his historical record, a history that’s very long and reliable, the most likely thing to happen is that he comes out of this slump like he did last year. Maybe don’t look at last spring, though. Or do. Because there is no guarantees in life or with goalies, and no, Andersen has not been atrocious all year long.

But the myth that Andersen picks up the team and carries them should die. The Leafs must outscore their own weaknesses. That’s who they are. And sometimes the weakness is in net. And usually it’s the skaters in front of the net. On the terrible nights when it’s both, the burden needs to be taken up by the most expensive top six in hockey. They have to play every game, all three periods, like they’re in a tough playoff race. Because they are.

Note: All data is from Evolving Hockey in this post, and the linked posts.

Analysis