DS Errors Intriguing but Flawed Idea
I like to take in David Staples' Cult of Hockey blog 4-5 times a week; It's usually the closest I come to reading Edmonton MSM and he's pretty entertaining while being succinct. His Best of the Oilogosphere mentions are also a nice little summary for the average reader who doesn't quite have time to read all the great pieces by the Oilogosphere writers.
There's an idea he's been toying with for a while that I must say, has me intrigued. Staples claims he has watched (or rewatched) every Oilers game this season and assigned up to three 'errors' (such as those in baseball) for every even strength goal against. First of all, there's a lot I like about this idea.
+/- is definitely a flawed statistic in itself. It statistically punishes players for a goal against even if they did everything correctly, or it can reward a player even if they did nothing positive. A lot of times, it's not a true reflectance of how many mistakes or positives a player is making. On good teams players tend to have exaggerated pluses, and on bad teams players tend to have exaggerated minus'. Of course, my personal belief is that over the long term most statistical anomalies tend to even out, but certainly in the short term I think error counting could be a useful tool in determining the relative value of a player's on ice impact.
However, I can count a number of problems within the error system.
One of the first major issues I have with this system is it doesn't factor in errors that don't lead to goals. For example, let's say Sheldon Souray makes a bad pinch and it's a two on one the other way. Steve Staios then misplays the two on one leading to a very good chance that either; Garon makes a save or the shot goes wide.
In Staples' system, there is no error assigned there, even though it's just as legitimate a mistake as one that would have lead to a goal. Currently, the system over-penalizes players that have bad goalies or play against exceptional opponents. and over rewards players with good goalies who play poor opponents. (This assumes a better opponent is more likely to convert scoring chances). Why not assign errors to scoring chances period? This would help eliminate some of the bias.
Staples tries to simplify things by administering only even strength errors, which is I think a smart move, but he oversimplifies by only assigning errors that lead to goals. What about scoring chances or penalties? There are some penalties that are errors (such as an offensive zone trip) and there are some errors that cause penalties (a missed defensive cover by one player forces another player to take a penalty to negate the advantage). You may not have to count errors when on the penalty kill, but errors should at least be counted if it forces your team to take a penalty.
We should also be taking into account the magnitude of the error, or at least the quality of opponent. Let's say an attacking player loses the puck in the neutral zone to the defending team which leads to a 2-2 the other way. One of the defenders in the 2-2 loses his footing, falls, and then the attacking team scores. Which is the bigger error, the turnover in the neutral zone, or falling on a standard 2-2? In my opinion, one error is of a much larger magnitude and there should be some way of making a notification of such. We also should take into account the level of opponent against which the error is made. Obviously if Andrew Ference gets burned by Ilya Kovalchuk in the defending zone on a terrific deke or by pure physical strength, it should probably be weighted differently than if he gets burned by Donald Brashear. If I'm a player evaluator, I want to know the type of opponent situation in which the player is going to succeed or fail, but Staples' system completely ignores it.
I'd also like to make mention of anomalies within the error system, specifically, fluke plays. Sometimes a player can play a situation completely wrong and by pure fluke suceeds on the play. I have no idea how this can be accounted for in the system, maybe it will come out in the wash. Still, it's a concern.
Finally, I have a couple of nitpicks. Who is the person deciding if it's an error or not? That's going to be an extremely tricky wrinkle, as the correct play for any given situation is contendable depending on who is doing the evaluation. It's possible for a guy to do everything right and still get beat, and I'd be hard pressed in those situations to call it an 'error.'
Lastly, Staples explained that he assigns up to three errors, just as goals can be assigned three points (the goal and two assists). However, it's a bit arbitrary to assign only three purely due to the fact goals are awarded three points. Why not have the option of assigning up to 6 errors, one for every player on the ice. I know I've seen goals where 6 guys were at fault, so why would they not be assigned an error?
Staples' plan to assign errors to players is a good one, but I think some of the conclusions he's come to using his system are a bit suspect and I think if you applied some minor changes you'd get a lot clearer picture and a lot better evaluative tool.
There's an idea he's been toying with for a while that I must say, has me intrigued. Staples claims he has watched (or rewatched) every Oilers game this season and assigned up to three 'errors' (such as those in baseball) for every even strength goal against. First of all, there's a lot I like about this idea.
+/- is definitely a flawed statistic in itself. It statistically punishes players for a goal against even if they did everything correctly, or it can reward a player even if they did nothing positive. A lot of times, it's not a true reflectance of how many mistakes or positives a player is making. On good teams players tend to have exaggerated pluses, and on bad teams players tend to have exaggerated minus'. Of course, my personal belief is that over the long term most statistical anomalies tend to even out, but certainly in the short term I think error counting could be a useful tool in determining the relative value of a player's on ice impact.
However, I can count a number of problems within the error system.
One of the first major issues I have with this system is it doesn't factor in errors that don't lead to goals. For example, let's say Sheldon Souray makes a bad pinch and it's a two on one the other way. Steve Staios then misplays the two on one leading to a very good chance that either; Garon makes a save or the shot goes wide.
In Staples' system, there is no error assigned there, even though it's just as legitimate a mistake as one that would have lead to a goal. Currently, the system over-penalizes players that have bad goalies or play against exceptional opponents. and over rewards players with good goalies who play poor opponents. (This assumes a better opponent is more likely to convert scoring chances). Why not assign errors to scoring chances period? This would help eliminate some of the bias.
Staples tries to simplify things by administering only even strength errors, which is I think a smart move, but he oversimplifies by only assigning errors that lead to goals. What about scoring chances or penalties? There are some penalties that are errors (such as an offensive zone trip) and there are some errors that cause penalties (a missed defensive cover by one player forces another player to take a penalty to negate the advantage). You may not have to count errors when on the penalty kill, but errors should at least be counted if it forces your team to take a penalty.
We should also be taking into account the magnitude of the error, or at least the quality of opponent. Let's say an attacking player loses the puck in the neutral zone to the defending team which leads to a 2-2 the other way. One of the defenders in the 2-2 loses his footing, falls, and then the attacking team scores. Which is the bigger error, the turnover in the neutral zone, or falling on a standard 2-2? In my opinion, one error is of a much larger magnitude and there should be some way of making a notification of such. We also should take into account the level of opponent against which the error is made. Obviously if Andrew Ference gets burned by Ilya Kovalchuk in the defending zone on a terrific deke or by pure physical strength, it should probably be weighted differently than if he gets burned by Donald Brashear. If I'm a player evaluator, I want to know the type of opponent situation in which the player is going to succeed or fail, but Staples' system completely ignores it.
I'd also like to make mention of anomalies within the error system, specifically, fluke plays. Sometimes a player can play a situation completely wrong and by pure fluke suceeds on the play. I have no idea how this can be accounted for in the system, maybe it will come out in the wash. Still, it's a concern.
Finally, I have a couple of nitpicks. Who is the person deciding if it's an error or not? That's going to be an extremely tricky wrinkle, as the correct play for any given situation is contendable depending on who is doing the evaluation. It's possible for a guy to do everything right and still get beat, and I'd be hard pressed in those situations to call it an 'error.'
Lastly, Staples explained that he assigns up to three errors, just as goals can be assigned three points (the goal and two assists). However, it's a bit arbitrary to assign only three purely due to the fact goals are awarded three points. Why not have the option of assigning up to 6 errors, one for every player on the ice. I know I've seen goals where 6 guys were at fault, so why would they not be assigned an error?
Staples' plan to assign errors to players is a good one, but I think some of the conclusions he's come to using his system are a bit suspect and I think if you applied some minor changes you'd get a lot clearer picture and a lot better evaluative tool.
Posts
5 comments:
Thanks for writing about the error and thinking about the subject. If this stat is going to be useful, it's going to take a lot of discussion and fine-tuning.
Your main criticism is that the error doesn't properly single out players who make other terrible defensive miscues. But let me give you this to chew on . ..
Say there were no goal and assist stat right now, and I was saying, 'Hey, why don't we give a plus point to a player who last touches the puck before it goes in the net, and we'll call it a "goal"? And why don't we give two more plus points to players who touched the puck second last and third last before it goes in the net, and we'll call those "assists"?
'This way we'll know who are the players most responsible for those great moments when the puck goes in the net.'
Would your counter argument be: Well, goals and assists are fine, but what about other great plays where the puck almost goes in the net. This goal and assist thing doesn't recognize those plays? Or how about the defender who breaks up a play, then passes it up, and the forwards pass it around, and the defender doesn't get an assist because he was the fourth last person to touch the puck. He made a great play, but gets no assist. Is that fair?"
It's true, the "error" is only one stat, and it's not the be-all, end-all. Look at it as the "anti-point." Just as goals and assists rewards those who make plays that lead directly to goals, the errors punishes those who make plays that lead directly to goals against.
I would say my criticism isn't that error doesn't properly single out other defensive miscues, its that it doesn't give an accurate representation of the number of miscues made.
Think about it in these terms: every time a goaltender makes a defensive miscue, it's going to end up in the back of the net (pretty much anyway). So his error number is going to be invariably inflated. Whereas a forward's 'error' is only going to end up in the back of the net maybe 1/7 times or so (guessing, but the point is a lot less), and a defenceman is going to land somewhere in between. So when you come up with a conclusion like 'Roloson is most error prone Oiler' it's true in terms of how you define error, but it's not that helpful in determining who's making mistakes.
And I would agree that goals and assists don't reward good offensive plays all the time, but that's a different flaw.
I understand you're trying to show where mistakes were made on goals, I just don't think it paints a clear picture and with a few little tweaks it could. Just like +/- is helped by desjardins' +/- on/off, I think the error would be better served counting scoring chances and penalties as well.
That's just my opinion though.
Fair enough. I take your point. Like goals and assists, this stat would be only part of the equation.
You know, to properly assign an error takes about 10 minutes. I have to watch the replay of a goal 15-20 times. So it would be difficult, I'd asset, to assign errors on other plays, such as on all good scoring chances in a game. Whereas you might have five or six even strength goals, you would have 15 or 20 good scoring chances. That would be a lot of work. Now, I'm not saying someone shouldn't do this -- I volunteer you for the job! :)
Right now, of course, the NHL does collect a stat for turnovers and takeaways, which kind of addresses miscues that happen during the flow of a game. But this is a poorly understood stat, and not loved by many folks, and it doesn't address scoring chances, but all miscues during a game, I believe.
Quite frankly I'm impressed with the work you've put in already, and obviously my idea isn't really viable for the average person to keep track of. It's not like the average fan could keep track of hits, SOG without an professional counter so take what I say with a grain of salt.
I just think that if someone were going to take this idea to the next level, these would have to be the stipulations. Otherwise, like I say, it becomes misleading IMO.
Who is the person deciding if it's an error or not?
This is the real problem with the issue of counting "errors": they are a subjective value judgement. I tend to ignore or discount other stats of this kind - stuff like "hits" and "give-aways" suffer a similar fate in their collection in the NHL currently.
When it comes to stats, I think it's better to count discrete, identifiable events (goals, shots, time on ice) that are objectively measurable by some standard unit.
I have no problem with the concept of "errors" in the context of qualitative discussion of a player's performance during a game or after a game, but trying to make them into a legit counting number is fraught with peril.
Post a Comment