It’s always a tricky bit of business to judge NHL coaches. The standard method in the league itself as practiced by GMs is to fire coaches when their own chair in the executive suite starts to feel like the seat in their Lexus after the seat heater got left on too long. And the standard method in the league for ownership to use to fire GMs is to chomp their metaphorical cigars call it a results-based business and fire them when the win/loss record tips to losses for too long.

The problem with all of that is the illusion of results-oriented judgement is just another way for men making more and more money out of a losing NHL team to shout, “It’s not my fault!” in a way that their own bosses/shareholders/personal willingness to lose money can be placated with. Eventually that stops working too, and the real problems need to be fixed.

When Sheldon Keefe was given the keys to the Maple Leafs, the next thing that happened was Kyle Dubas traded for a decent backup goalie and then another defenceman after the acquisition of Jake Muzzin less than a year before Keefe took over. Those three changes, and a recent offseason of player signings as well, make it difficult when looking back to sort out what was down to the coach, and what the changes on the team.

I call this sort of analysis post hoc, ergo propter hockey, and there is a huge appetite for explaining a largely unexplainable game by pointing at whatever thing has changed recently and asserting with total certainty that that’s the cause. That plus too much faith in the standings leads us all measuring with a faulty ruler. What does an NHL coach really do? How much do the affect the game? Is it all about motivating the players, group dynamics or what?

I don’t know, so I’m asking other people.

You could say that a bad coach can ruin a good team, but a great coach can’t make a bad team good. Do you think that’s true, and to what extent can a coach boost a good team to greatness?

Fulemin: All coaches are judged by expectations, and expectations are set by roster quality. We look at the names on the team first and estimate how good it ought to be, and if the actual results diverge from that, we attribute it to coaching greatness or lack thereof. This sounds like I’m just being glib here, but I really mean it: the Jack Adams is awarded on this basis every year. My suspicion is that the vast majority of NHL coaches don’t have a huge impact in either direction, except in really emotional circumstances where the team has given up or when it’s excited to play for somebody new. The biggest impact, I think, is that a well-suited coach can get a grinder team to play aggressive defence and that’s probably the best way to get a lot out of a grinder team. Shorter version: however good you think the New York Islanders are compared to their results under Barry Trotz ought to be your answer for maximum coaching impact.

Arvind: As usual, I’m going to be annoying and equivocate. No coach could turn the staff of PPP into a competitive NHL hockey team. However, I do buy that there are coaches who can get more than others would out of a given NHL-quality roster, depending on the skills of that roster (and obviously, the opposite holds too). The degree to which this occurs is tricky, and hard to analyze. The quintessential ‘coaching up’ example in recent NHL history is Gerard Gallant with the Vegas Golden Knights. It’s hard to allocate precise amounts of credit for their unexpected success between Gallant coaching up a roster of players who were not thought of as particularly great, and the players themselves being better than we initially thought.

Similarly, a coach can take a strong roster and push them to greatness. He’s a persona non grata at this point, but Mike Babcock took a strong 2007/2008 Red Wings roster and contributed at least to some degree to one of the most dominant Stanley Cup champions in the post-2005 lockout. Again, it’s hard to know the degrees. Was Babcock elevating those players, or were they just better than we thought? The one robust mathematical attempt I’ve seen to isolate coaching impact on 5v5 play is Micah McCurdy’s work, where over multiple year periods, coaches top out at providing an estimated 3-4% boost to the xG differential of a team. Obviously, this ignores one of the more obvious ways coaching staffs can improve a team, which is through special teams strategy.

Katya: I don’t think I’d trust an answer that didn’t equivocate on this to some extent. I think that coaches are kind of like goalies. They aren’t really part of the team, but they affect what the team does, and they often get the credit or blame for outcomes where it’s not warranted. The Vezina used to be a GAA award, straight up, so they are lauded on how good the skaters are as well. And like a goalie the extent to which they can be bad is infinite, but the range of good ones is very tight, hard to identify and predict and subject to a lot of mythologizing.

Also, I do seriously wonder if for regular season results, the person creating and maintaining the power play might be the single most import coach there is. Imagine if Montreal had a good power play, for example.

Brigstew: I actually wrote out a completely different answer, realized I didn’t like it at all and then re-wrote this. I guess I think of it as if a given roster of players on a team has a theoretical maximum potential for how good they can be. That will take into account the system they play, their motivation to try their best at any given time, and the development of them as players (mostly for younger guys). There’s other stuff too, like health, but none of those have to do with coaching. Let’s say that this theoretical potential starts at 100%. I do think that great coaches can get a team to that 100% for most of the time. Good coaches can get them close, to varying degrees. Bad coaches sink them. That’s more philosophical I suppose, but I guess TL;DR I don’t think a great coach can make a bad team good. He can only make them as good as the best they can be.

Hardev: This article is long so I’ll be short. I think a good coach can mask deficiencies caused through roster building. If a team has poor shooters, work to increase volume. If a team doesn’t have the ability to do either, limit chances against. If they’re the 2014 Rangers, don’t make them look like a turtle with an ass the size of Henrik Lundqvist.

Is it player performance, motivation, group dynamics or is it systems, personnel choices and player usage that matters?

Fulemin: I think motivation—ability to get buy-in, specifically—is most of it. Motivation is a more complex thing than just cracking the whip until the players play right, to be clear. It’s getting the players to follow the system effectively and instinctively, which there are many roads to. I think an NHL system effectively executed is going to be better than one executed half-heartedly, even if the latter is theoretically better. This is probably a variation on my feeling that most coaches don’t make that much difference because it’s very hard to be a brilliant motivator game in, game out over a long stretch; you see a bounce when a new coach busts in partly because the previous guy was likely fired in the depths of a PDO slump and partly because the team gets a new lease on life and chance to prove themselves. But sustaining that commitment longer term is tough.

Arvind: All of the above, but to different degrees with different teams and personalities. The way I view it is that a coach is essentially the manager of a small business. The content of what they’re trying to implement matters, but so does the messenger and the way the message comes across. Some coaches are not cut out for the talents and personalities of certain teams, even if they have great ideas systemically.

Katya: Okay the coach as boss is an interesting metaphor because we’ve ported the sports concept of teams and team dynamics into workplaces, often where it doesn’t belong. But sometimes, in some workplaces, you just need to get out of people’s way. I think of Jon Cooper here, and this is partly lore, because none of us have played for the Tampa Bay Lightning, but he has managed, most of the time, to handle a team playing at their peak for a very long time, with boring meaningless regular seasons and then a need to switch to playoff mode very well. And I don’t say that because they won last year, but because of the long period they’ve been in striking distance.

On player usage, I think most coaching choices are largely non-factors, but dumb ones can sink a team more than smart ones will really help. Players will play as they play. I actually think the worst thing a coach can do is expect players to do things they are not capable of or that cannot lead to success. That’s when what seems like motivation becomes demoralizing. Biggest example of that I can think of is Patrick Roy (who is popular with most players) who created a system guaranteed to lose.

Brigstew: Motivation I think is something that coaches have the most impact on. A coach is essentially a player’s boss, or manager. I think we all know from experience in our working lives that a good manager can have you working at your best, and a bad manager saps your motivation to try as hard as you could. Usage and personnel choices I think do matter as well, especially since it can impact motivation. Deciding to play Hyman higher in the lineup makes sense because he is genuinely great at making a line work, and that’s more important for your top lines. Deciding to stack all your stars on one line, leaving the rest of them to be essentially third/fourth lines, is not a good coaching decision, Sheldon.

Hardev: They’re all tied together. Performance ties to systems (within the bounds of randomness), motivation ties to usage, and group dynamics ties to personnel choices.

Does Sheldon Keefe make the Leafs better (not better than the previous coach, but better than they’d be with some imaginary average-quality coach)?

Fulemin: I haven’t seen a convincing case that he makes them either much better or much worse. I resolutely want to avoid any more Babcock discussion for the rest of my life, and I’m sorry to ignore the parenthetical, but since he’s the only other coach this version of the Leafs has had: the team looks like a somewhat possession-heavier version of about the same thing it did under Babcock. Which is progress, I suppose. If Keefe has a mark in his favour I think it’s that he genuinely is very flexible and gives lots of chances to different combinations, which theoretically should pay off when come playoff time he’s found the best answers with his given personnel...but it didn’t seem to help against Columbus and he also routinely tries things that I think are silly. So we’ll see.

Arvind: I think so, though not by an enormous amount. It’s hard to criticize the W/L ratio in Keefe’s time as a coach. The numbers are solid, if not elite, and they seem to match up with a roster that I’d consider between the 5th and 10th best in the league. The elite power play helps a lot, and while that may not strictly be Keefe’s doing, I’m crediting it to him since it’s his coaching staff.

Katya: I’m really, really not sure about this one. I think his system is complex, and okay, does anyone remember this?

If you just let the Leafs play as they play, would you get the moment when they go from the Farandole to grooving to Bowie? Or would you get chaos? Is Count Adhemar scowling away there every hockey coach ever? And is he, when it comes to hockey, right to scowl?

Something happened to the Leafs, and they went from this team that a Dallas Stars fan once called puck-a-doodle-doo that was like Patrick Roy’s misguided system only good, and it was fun, and I miss it. I want to see that with Muzzin and Brodie there to guard the back door. None of this is an answer, but that’s how I feel. There’s been three games already this season where I turned the sound off and played Iggy Pop’s Chairman of the Bored instead. Oh, and the Talking Heads’ Road to Nowhere. And it’s not a case of defensive success is boring to watch because the Leafs have not pulled up the red carpet that invites the opposition to cruise down the slot.

Brigstew: Thus far I am of the opinion that Keefe is an okay coach. He has some interesting ideas to manage the best and worst of the team. I don’t think he is a great coach by any means. I don’t think he has unlocked the utmost potential of the roster, but at least to his credit I do think he is good at having them motivated... at least compared to before.

Hardev: He is/has been the average-quality coach. Some things have been good, others bad, on the whole medium. He’s not going to sink this team or carry it to the promised land. That comes down to the things we talked about at the end of last playoffs.

What one thing would you like to see the coaching staff do that they don’t?

Fulemin: If they could convince the team to stop turtling when they get a multi-goal lead, that would be great. I know this is the lament of many fans of many teams and it may just be a fact of life given that they’re not a great in-zone defensive team, but the Leafs bleed chances against at an absolutely incredible rate once they’re up two. Look!

Arvind: I’m speaking with a lot of ignorance here, but for the sake of preserving Matthews and Marner somewhat, I’d like to get a little closer to equalizing the time between the first and second lines. The John Tavares / William Nylander duo has good overall numbers, even if they pale in comparison (especially offensively) to Matthews / Marner. They’ve received justified criticism, especially within the lens of how they match up to the good 5v5 teams in the division (a notable chunk of Tavares’ and Nylander’s strong overall 5v5 results come from games where they wipe the floor with Ottawa and Vancouver).

It makes us a worse team in the short-term to play Matthews / Marner less, but in a pretty compressed season, lowering the burden on them in lower leverage environments could be useful.

Katya: Shooot! I’m not sure if this is coached or if it’s something about this division, but they’ve now gone so far from the all point shots bad times to endless searches for the perfect high-danger chance. The only teams that have done that over a lot of time are the Wild and the Canes and that hasn’t worked out so great.

Brigstew: I know we all hated Babcock for his extreme lack of roster flexibility, and we wanted him to try out certain things more often. I feel like Keefe is the polar opposite. He’ll try tons of weird shit, seemingly just like throwing a bunch of stuff at a wall and seeing what sticks. But I wish he’d be smarter with it, and not go so bonkers. Trying out a 7D lineup and then saying he almost immediately regretted it (for good reason) does not instill faith in me that he will make well thought out decisions and experiments. And it’s one thing to say he only does it in the regular season, but I happen to remember what he did in game 5 against Columbus which he also just did against Calgary and I don’t have faith that he can reign in his impulsivity when it’s really fucking important to do something you know will work as well as it can.

Hardev: This is a nitpick but I feel Nylander has been underused at 5v5. Play him more with Tavares, maybe sometimes with Matthews? On the power play we’ve seen them give teams different looks, why not do that at 5v5? Mix the top-four up around on occasion, maybe you’ll find a better look against a specific opponent, or at the very least it keeps everyone on their toes.

What one thing would you like to see them stop doing?

Fulemin: Running a super line to start games. Tavares-Matthews-Marner is something you do in the last ten minutes when you plan to play that trio for six of them. Don’t do it in the first period.

Arvind: Playing Nylander at LW. I know he did it last year with success, but it’s been more dogmatic this year, to my eye. I feel like it neuters his puck carrying and passing too much, and in particular, it makes it harder for him and Tavares to pass to one another, as they have to make and receive passes between each other on their backhands.

Katya: Give up the dream of a shutdown line with Alex Kerfoot as the centre and Ilya Mikheyev on it. The idea seems to be to lighten the load on Tavares/Nylander, and as Arvind tells you above, it’s not working. Somewhere in the soul searching after the defeat at the hands of the Blue Jackets, the Leafs decided they needed to pull back from Kyle Dubas’ lean in to offence. I wonder if they pulled too far back?

Brigstew: Messing around with the third line. This is something that’s had a lot of experimentation and nothing so far has seemed to really work. Having a lot of injuries to most of your third/fourth line depth hasn’t helped, but man I’d like to see them get to a point where they can have a better impact. They’re not really shut down-able, but neither are they big offensive threats for putting up points. Pick a path, find a solution that works the best out of all the others, and stick with it until Dubas can find someone better to fill the hole.

Hardev: Stop giving up goals. Boom, fixed the team! That was easy.