So last December I - mistakenly - asserted that Roberto Luongo was a solid bet for the Leafs because as goalies age they don't show appreciable decline. My mistake was ignoring survivorship bias in my analysis - and I was called to task for it a couple of times, most succinctly in this analysis by Eric T. at nhlnumbers.com that displayed how by tracking specific groups of goalies the decline in SV% becomes quite appreciable. So obviously - goalies DO decline as they age. (Although as Tom Tango points out in this posting he tossed up yesterday - we also have to consider that young goalies can be selected due to posting lucky seasons in smaller samples earlier in their careers)

This has come up again recently in a few discussions. First this excellent posting by Tyler Dellow about the Corey Crawford contract raised the concern around paying goalies for solid performance in small samples (he also factors in some solid explanations around player value, marginal wins, and replacement level - please take a look)

Then yesterday the discussion around the Jonathan Bernier trade and the competition for the starting goaltending job in Toronto he faces against incumbent James Reimer. Just a sampling of quotes from yesterday's Top 25 Under 25 posting that ranked Bernier 5th in the comments section:

"Bizarre as it seems Players actually get better as they accumulate pro experience. Why is the idea that professionals improve with experience so dearly and deeply lost amongst the multitudes here?"

by Bowerplay on Sep 4, 2013 | 7:54 PM

or perhaps this trolling attempt?

"You’ve predominantly only talked about Bernier in the context of a trade analysis, but only grudgingly pay him his dues as a prospect with great upside."

by Thundering Silence on Sep 4, 2013 | 11:40 AM

Hmm... so is this actually true? Do 25 year old goalies that post a .922 SV% in the previous season actually tend to age like a fine wine?

Well - that's the funny thing - we can say that based on observed probability to date? The opposite would appear to be the case. On average - young goalies peak in performance around 24-26 and then decline. Let's look at how we can tell.

Yesterday on Twitter our friendly neighbourhood Frag made the following suggestion that I hadn't even really looked into with all the goaltending data I'd accumulated.

Once the suggestion was made it was fairly easy to follow up on. Eric T. promptly posted this image of the theoretical decline in a 21 year old goalie's Even Strength SV% if he started with a .920. The idea here is we're averaging out the change in even strength save percentage of ALL goalies by age to get the shifts you observe in the graph.

I was busy teaching a class at the time this part of the conversation was going on, but once I got home I decided to apply this method to the full season save percentage data set for all goalies since 1984 with 60+ career NHL games played. Just to be clear, this is a massive data set including 2292 goalie seasons worth of data. By restricting it to just the 60 career gp group, we're still referencing over 200 different goaltenders' careers, so this is quite comprehensive.

To explain the methodology - what I've done here is determine the average change in save percentage by standard deviations from the seasonal mean for each age group. I then converted those changes into shifts in 2012-13 save percentage terms (to normalize for era) by multiplying the results by the standard deviation in last year's save percentage.

Once we have the average shifts in SV% by age, I've applied those to a 21 year old keeper who starts with a .920 SV% (our hypothetical case). Here's the graph:

Theoreticalsv_drop_medium

So obviously this decline is not automatic for all goalies, but on average, this is the change we observe as a goaltender goes from 21 to 43.

Peak performance on average will occur at the age of 23, and roughly maintain until the age of 26. Beyond that a slow and steady drop off occurs for the average goalie - and obviously the starting point is relevant here. A goalie who is posting a .920 at 21, will theoretically be above average if they're posting better than NHL average SV% by the time they're 29 or 30. Beyond 30 we're into the area where most goalies have dropped below average and are beginning to filter their way out of the NHL.

If you're of the opinion that it's likely that Jonathan Bernier's seasonal performance last year is likely to continue to improve long term, the data we have above indicates that it's actually particularly unlikely. The majority of goalies see a decline past 26 - how do we know this? Because those steps down are all the AVERAGE changes in seasonal SV% by age.

It seems to happen for both Even Strength and Total SV%, so at this point, I'm not sure we should assume that it's "likely" that just because Jonathan Bernier is a highly touted "prospect" at the age of 25 with only 62 NHL games to his name and a career SV% of .912 - that he's going to drastically improve in the future.

The caveat in all this is - James Reimer is likely to decline also. The difference is that Reimer's career SV% numbers are higher, he's played more games, he has a higher Even Strength SV% over the last 4 years, and if he DOES decline, he's got further to go before he's a less than viable commodity. Personally I think if the Leafs pick Bernier over Reimer based on even one year of decent play, they're likely to regret it a bit long term. But that's just me.

*Update*

I ran the same analysis for the goaltenders who have played since 1984 and have 200+ career NHL games in an attempt to control for the inflationary impacts of players who are flashes in the pan early (due to lucky hot streaks) that burn out of the NHL within a few seasons.  The results are interesting, and still support the theory that on average goaltending development peaks around 23 and maintains in top form until around 26.  From that point on the average NHL netminder is in slow decline and stops being around NHL average at 33-34.

This graph starts with a theoretical 19 year old that has a .905 SV% in 2012-13 terms - and yes the data was adjusted for era effects.

Theoreticalsv_change905_medium

Now - this is hopefully enough to help people realize that generally speaking, goalies with longer careers do NOT continue to develop well past the age of 26.  While some certainly do improve later, that is not the norm, and the average here would indicate most decline consistently beyond that age.

Analysis