There's an idea he's been toying with for a while that I must say, has me intrigued. Staples claims he has watched (or rewatched) every Oilers game this season and assigned up to three 'errors' (such as those in baseball) for every even strength goal against. First of all, there's a lot I like about this idea.
+/- is definitely a flawed statistic in itself. It statistically punishes players for a goal against even if they did everything correctly, or it can reward a player even if they did nothing positive. A lot of times, it's not a true reflectance of how many mistakes or positives a player is making. On good teams players tend to have exaggerated pluses, and on bad teams players tend to have exaggerated minus'. Of course, my personal belief is that over the long term most statistical anomalies tend to even out, but certainly in the short term I think error counting could be a useful tool in determining the relative value of a player's on ice impact.
However, I can count a number of problems within the error system.
One of the first major issues I have with this system is it doesn't factor in errors that don't lead to goals. For example, let's say Sheldon Souray makes a bad pinch and it's a two on one the other way. Steve Staios then misplays the two on one leading to a very good chance that either; Garon makes a save or the shot goes wide.
In Staples' system, there is no error assigned there, even though it's just as legitimate a mistake as one that would have lead to a goal. Currently, the system over-penalizes players that have bad goalies or play against exceptional opponents. and over rewards players with good goalies who play poor opponents. (This assumes a better opponent is more likely to convert scoring chances). Why not assign errors to scoring chances period? This would help eliminate some of the bias.
Staples tries to simplify things by administering only even strength errors, which is I think a smart move, but he oversimplifies by only assigning errors that lead to goals. What about scoring chances or penalties? There are some penalties that are errors (such as an offensive zone trip) and there are some errors that cause penalties (a missed defensive cover by one player forces another player to take a penalty to negate the advantage). You may not have to count errors when on the penalty kill, but errors should at least be counted if it forces your team to take a penalty.
We should also be taking into account the magnitude of the error, or at least the quality of opponent. Let's say an attacking player loses the puck in the neutral zone to the defending team which leads to a 2-2 the other way. One of the defenders in the 2-2 loses his footing, falls, and then the attacking team scores. Which is the bigger error, the turnover in the neutral zone, or falling on a standard 2-2? In my opinion, one error is of a much larger magnitude and there should be some way of making a notification of such. We also should take into account the level of opponent against which the error is made. Obviously if Andrew Ference gets burned by Ilya Kovalchuk in the defending zone on a terrific deke or by pure physical strength, it should probably be weighted differently than if he gets burned by Donald Brashear. If I'm a player evaluator, I want to know the type of opponent situation in which the player is going to succeed or fail, but Staples' system completely ignores it.
I'd also like to make mention of anomalies within the error system, specifically, fluke plays. Sometimes a player can play a situation completely wrong and by pure fluke suceeds on the play. I have no idea how this can be accounted for in the system, maybe it will come out in the wash. Still, it's a concern.
Finally, I have a couple of nitpicks. Who is the person deciding if it's an error or not? That's going to be an extremely tricky wrinkle, as the correct play for any given situation is contendable depending on who is doing the evaluation. It's possible for a guy to do everything right and still get beat, and I'd be hard pressed in those situations to call it an 'error.'
Lastly, Staples explained that he assigns up to three errors, just as goals can be assigned three points (the goal and two assists). However, it's a bit arbitrary to assign only three purely due to the fact goals are awarded three points. Why not have the option of assigning up to 6 errors, one for every player on the ice. I know I've seen goals where 6 guys were at fault, so why would they not be assigned an error?
Staples' plan to assign errors to players is a good one, but I think some of the conclusions he's come to using his system are a bit suspect and I think if you applied some minor changes you'd get a lot clearer picture and a lot better evaluative tool.