Its no secret that the NBA is a stats driven league, the most popular of which are one-number player-evaluation metrics. While most agree that using only one number to evaluate an NBA player is foolhardy, that doesn't stop them (or me) from trying. Its a challenge, its intriguing, and most of all, its fun.
Therefore, I decided to make a quick post detailing "what" makes a one-number metric useful, and how to spot potential problems in popular metrics.
Accuracy
Explaining the past - Take the "metric-score" for each player in 2012, multiply it by their minutes played, and add them up by team. If the metric is accurate, the teams with the highest scores will be the teams who win the most games.
Predicting the future - The same as above, except you multiply the 2012 scores by the minutes played in 2013, and sum according to 2013 rosters. A high correlation between the team scores and winning percentages indicate metric accuracy.
*WARNING* A metric that "explains" without "predicting", or visa-versa, can't be considered extremely accurate.
Transparency
A good metric can't be a black-box. For instance, one may disagree with what Wins Produced (WP) says, but its very clear how to calculate Wins Produced, and what a player must do for a high WP score.
Conversely, one may agree with Plus/Minus metrics, but its unclear what specifically a player can do to increase his Plus/Minus, which could be detrimental to a coach making in-game adjustments.
Stability
NBA players perform at somewhat consistent levels from year to year. If a player's metric-score fluctuates wildly from one season to the next, then there could be a problem. If many/most players under a certain metric have their scores fluctuate wildly, then there's definitely a problem.
Stability indicates that the metric is measuring something, as opposed to picking up random noise.
Versatility
A good NBA metric can be broken up into smaller, but useful parts. I can use a component of XRAPM (D-XRAPM), to evaluate a player solely on defense. I can use the scoring component of Wins Produced (Net Points) to evaluate scoring prowess. Basically, a good one-number-metric is more then that one number. It can illuminate multiple facets of the game.
That's all I can think of off the top of my head. Apologies for the word-heavy post, with not a colorful table in sight.
Therefore, I decided to make a quick post detailing "what" makes a one-number metric useful, and how to spot potential problems in popular metrics.
Accuracy
Explaining the past - Take the "metric-score" for each player in 2012, multiply it by their minutes played, and add them up by team. If the metric is accurate, the teams with the highest scores will be the teams who win the most games.
Predicting the future - The same as above, except you multiply the 2012 scores by the minutes played in 2013, and sum according to 2013 rosters. A high correlation between the team scores and winning percentages indicate metric accuracy.
*WARNING* A metric that "explains" without "predicting", or visa-versa, can't be considered extremely accurate.
Transparency
A good metric can't be a black-box. For instance, one may disagree with what Wins Produced (WP) says, but its very clear how to calculate Wins Produced, and what a player must do for a high WP score.
Conversely, one may agree with Plus/Minus metrics, but its unclear what specifically a player can do to increase his Plus/Minus, which could be detrimental to a coach making in-game adjustments.
Stability
NBA players perform at somewhat consistent levels from year to year. If a player's metric-score fluctuates wildly from one season to the next, then there could be a problem. If many/most players under a certain metric have their scores fluctuate wildly, then there's definitely a problem.
Stability indicates that the metric is measuring something, as opposed to picking up random noise.
Versatility
A good NBA metric can be broken up into smaller, but useful parts. I can use a component of XRAPM (D-XRAPM), to evaluate a player solely on defense. I can use the scoring component of Wins Produced (Net Points) to evaluate scoring prowess. Basically, a good one-number-metric is more then that one number. It can illuminate multiple facets of the game.
That's all I can think of off the top of my head. Apologies for the word-heavy post, with not a colorful table in sight.