Forecasting James Reimer and Jonathan Bernier

If you’re a sports analyst, one of the main problems is confronting the issue of survivorship bias.

In sports, results drive most of the business, even though the results from the previous year don’t always impact the results in the next year. Very few teams across sports are willing to be patient and stick to a process that will lead them into making good bets. “Buy low” is generally accepted in the stock market but not in sports. The Toronto Maple Leafs this offseason have bought high on Tyler Bozak and David Clarkson, rather than bought low on Clarke MacArthur, Mikhail Grabovski and Nikolai Kulemin.

Say you have two players with a “true talent” of 20 goals per season. The first player got some bad puck luck and scored just 15, while the second got some puck puck, a few extra chances, and scored 25. More often than not, when choosing between the two players, a team will go for the 25-goal scorer, even though the 15-goal scorer would be cheaper and he’s just as likely to produce at the same level in the future.

Despite scoring just 14 goals to David Clarkson’s 45 over the last two seasons, I firmly believe that Nikolai Kulemin is just as good as the slightly older, much more expensive unrestricted free agent the Maple Leafs picked up. Unfortunately, there is no way to test the theory unless you have thousands of multiple universes, and half of them play Kulemin in Clarkson’s spot in the lineup and the other half play Clarkson in that spot. We’d have to average out the totals of the universe to see who was right or wrong. For better or for worse, we have just the one universe, and its randomness, though teams make worse decisions because of it, makes sports more exciting.

I can sit here and say that I’d take Kulemin over Clarkson, but I’m not under a lot of pressure to bring notable players to Toronto and sell tickets and jerseys and convince Vinny from Woodbridge that the team is doing its best to perform on the ice. Working on the margins doesn’t necessarily work for a team like Toronto where every move is scrutinized in hindsight. I prefer to work with foresight.

Which brings us to Jonathan Bernier.

Bernier is an interesting case because he blends “buying low” with “buying high”. The Leafs spent quite a bit on Bernier: a pick, a depth winger, a backup goaltender and $500k in salary cap space, plus the contract they gave him, which works out to a $2.9-million cap hit.

That’s the price you pay for a former first round pick when you need to sell to your fanbase that you’re looking to fix any lingering goaltender issues.

But then there’s the “buying low” aspect. Bernier never made it as a starter in the NHL, at least not yet. Among active goalies, he is 61st in minutes played, behind noted gems such as Al Montoya, Jeff Drouin-Deslauriers, and — most notably — James Reimer. His career save percentage of .912 and even strength save percentage of .916 are not particularly impressive numbers. The average backup goaltender would have made 3.4 fewer saves over Bernier’s career than Bernier did, and when you consider it’s been seven years since he was the 11th overall pick, his NHL resume is none too impressive.

Comparatively, James Reimer has been the Leafs’ starter for three separate half seasons. Making his debut in 2011, Reimer dragged the Leafs out of the depths of the Eastern Conference standings with a .921 save percentage and a 20-10-5 record. On New Year’s Eve 2010, the Leafs were 13-19-4 and in 13th place. The next day, Reimer took over as the de-facto starter, and at the end of the season, Toronto was 37-34-11 and in 10th spot. Concussion and injuries took him over in the 2011-2012 season, but he responded with a .924 save percentage in 2013 and led the Maple Leafs to their first playoff appearance since Ed Belfour was the Leafs goalie. Belfour has not only retired, but he’s also got himself a Hall of Fame plaque.

Compare the two career records:

  EV Shots Faced EV SV% Average backup EV SV% Saves over Average Backup
James Reimer 2585 0.9253 0.9145 28.0
Jonathan Bernier 1230 0.9171 0.9143 3.4

With more shots faced, and performing better relative to the average backup (by combining the even strength save rates of goaltenders that were not in the Top 30 in shots faced in any given year), Reimer appears to be the leader.

But should we be entirely convinced he’ll have the better career? Think about regression to the mean:

In statistics, regression toward (or tothe mean is the phenomenon that if a variable is extreme on its first measurement, it will tend to be closer to the average on its second measurement—and, paradoxically, if it is extreme on its second measurement, it will tend to have been closer to the average on its first. To avoid making wrong inferences, regression toward the mean must be considered when designing scientific experiments and interpreting data.

Reimer possibly represents an extreme case. Since he’s performed well at even strength since coming to the NHL, I’d have him as my opening day starter.

That said, can we use comparable players to see whether Bernier or Reimer will perform better over the next three seasons? Well, we can try.

The first step was downloading all of the goaltender data (I looked at even strength only, since I’m a believer in total save percentages being influenced by a strong penalty kill) and sorting it per season, determining the save percentage of an average backup, and filling in birthdates for players as necessary. While the NHL season officially listed a 24-year-old as a player that turned 24 during the hockey season, for simplicity I used the junior hockey style, marking down a 24-year-old as a player that turned 24 during 2012.

With help of the delightful Rob Pettapiece in helping me figure out Microsoft Excel’s Pivot Tables, I was able to sort career-to-date statistics of goaltenders at age 24, and how they did beyond that. Survivorship bias is, unfortunately, huge when evaluating goaltenders. If Reimer were the first round pick in 2006 and Bernier were the third round pick, I would have no doubt that Bernier would not even be on the Maple Leafs’ radar.

It’s difficult to find comparable players, however, particularly in Reimer’s case. Limiting my goaltenders that were born between 1974 and 1985 so I would have full data prior to 1998 when the NHL began tabulating EV save statistics, and so that I could cover three seasons for goalies that were 24 three years ago, there are not a lot of goalies who came close to Reimer’s 28.0 Saves above Average Backup (SAR). He would have been 10th on our list.

Let’s define the comparables to include all players within .005 of save percentage at even strength up to age 24, and see how they combine to perform at ages 25, 26 and 27:

  n Avg. EV Shots Faced Combined EV SV% Average backup EV SV% EV SV%+ (Era Adjusted) Saves over Average Backup
Reimer Comparables 24 1506 0.9191 0.9125 0.9065 9.9
Bernier Comparables 27 1650 0.9186 0.9132 0.9053 8.8

There are 24 goalies age 24 and before belonging in Reimer’s sample, and 27 in Bernier’s which were culled to 18 and 22, respectively. Of those goalies that continued, they performed at a similar level. The Reimer group’s overall save percentage should be considered slightly higher since they tended to play in eras when even strength save percentages were slightly lower, but overall, the difference is pretty minimal, especially when you consider that certain goalies who (perhaps unluckily) played themselves out of a job in the Bernier group may have had more success in their future years if given the chance.

This is not a perfect way of doing it at all, but there are no real “good” ways of forecasting goaltenders. They get into the league, they try to make saves, and if they don’t make enough of them they don’t last and never get the chance to regress back to the mean.

So perhaps looking at Jonathan Bernier’s save percentage and criticizing the move isn’t the right way to go. Bernier is probably a pretty good goalie. Where the move deserves criticism is that the Leafs gave up a goalie with a similar track record in EV SV% (.9162 to .9171) to bring over a goalie that was a year younger and has made just 439 more even strength saves at the NHL level. Bernier also costs $2,787,500.00 more when you figure in the salary cap retention of the deal, plus a depth winger and a pick. 

In other words, if you’re going to buy low on a goaltender with a spotty career record under the belief that he’ll regress towards the mean, you may as well actually buy low on him.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  • Killawatt

    I was a bit shocked by the cash the Leafs gave to Bernier.
    Not so much because of the actual salary cap $$, because my interest doesn’t lie in cap management, but more so for what it says to Bernier, and Reimer, and everyone else.

    Bernier is the Leafs #1 goalie.

    That’s some pressure.

  • jasken

    Bernier will be Toskala all over again or the leafs. Reimer is getting traded like Rask and the only regression to mean we will see is leaf goaltending back toward its 880 SV%.

    Let’s fire Nonis before it is too late!

  • jasken

    This situation reminds me of the Kyle Turris trade. (although goalies are wildly more difficult to predict)
    Similar assets given up by Ottawa in Runblad and a 2nd for a player who was a high pick but never really produced in strong Phoenix system.
    Like the Turris deal, this is about a players ceiling. Instead of looking at past performance, Nonis is taking a gamble on Bernier’s ceiling. Personally, I like the move. I think the risk vs reward is worth the assets lost to L.A.

  • jasken

    Bernier’s cost was to high, however I think Bernier battling with Reims for #1 is going to be very beneficial. People are looking too much at the dollar he signed for Reimer’s contract is up next season. It gives leverage so Reimer cant ask for more then his worth. I dont see paying a goalie 7 mil/yr without a 35 win season is just plain messed up.

  • In my opinion Bernier is the more talented goaltender with a higher potential than Reimer.

    Bernier is much more fluid in his movements, has a much better glove hand, controls rebounds better, and handles the puck better than Reimer.

    Several years from now Bernier will be a top 3 NHL goaltender, Reimer will be what he his now, an average NHL tender.

    • The problem is that you probably don’t watch much hockey.

      How can you possibly say that Bernier “looks” better than Reimer considering he has half the number of NHL starts and shots faced, plus considering he plays irregularly in the late timezone.

      Basically, this comment is full of BS, and Toronto Sun-like, and whoever wrote it ought to be smarter.

      • jasken


        Just for the record before I get started – I do watch hockey… a lot.

        I don’t necessarily agree with mrj that Bernier is the more talented goalie – I think they’re both talented and have great up side. But he isn’t completely off base.

        Bernier does have better puck handling skills and rebound control. I don’t even think that’s a matter of opinion. His glove hand is arguably better as well. But puck handling skills; specifically, are a major upgrade over Reimer. A goalie who can stop the puck behind the net and deliver a great first pass is a huge asset. Helps to eliminate those 4-5 minute shifts in your own end (a la Orrsy).

        You know that quote from Mr. Miyagi in Karate Kid “best way to block a punch is not to be there in the first place”. It’s kind of like that.

        The other thing I’ll take exception to is the fact Reimer has played twice the number of games and faced twice the number of shots. Well… of course he has! He was playing behind a guy who’s been widely considered the best goalie on the planet over the last three years… top 3 at the very least. If Reimer and Bernier traded places I’m sure those numbers would be reversed. To me… this is inconsequential.

        Reimer obviously has his strengths over Bernier as well…

        To say one is better than the other at this point in their careers is hugely presumptuous. I, for one, like having both. Having two young goalies with upside is always better than having one. Improves your chances of hitting a home run.

        …. and just to see if Cam can take it as well as he can dish it out: when do you ever watch the hockey game – aren’t you too busy counting scoring chances?

    • jasken

      There have been plenty of players who never lived up to potential that dont mean much. Reimer is proven will Bernier push him maybe. I like to think these 2 would be good for each other, as the other side of this is disastrous.

  • jasken

    The only thing that will matter with these 2 goalies is who performs under Carlyle and who dont. The person who dont perform will find themselves backup.

    With Carlyle your not going find a losing goalie starting all the games, any more then a 30 pt player on top 2 lines. If either of goalies wants that starter job their gonna earn it, Grabovski is a prime example of how harsh Carlyle can be.