NCAAM teams
Seth Walder, ESPN Analytics 6y

Committee has right numbers, but will it use them correctly?

Men's College Basketball, Clemson Tigers

The NCAA selection committee's placement of Gonzaga and Clemson -- or lack thereof -- in its top 16 reveal on Sunday will be significant to more than just fans of the Bulldogs and Tigers.

The committee's opinion of those two will provide a clue about how new metrics such as Strength of Record, KenPom and BPI, which in January were added to the official team sheets provided to the committee, will be used.

The schools represent opposite ends of a debate that we have in college football all the time: best vs. most deserving. The Bulldogs are one of the best teams in Division I and rank 10th in BPI, accordingly, but in terms of wins and losses haven't achieved that standard and rank 22nd in strength of record. For Clemson, it's the opposite. The Tigers have achieved the sixth-most so far this season but are just the 18th-best team going forward.

What will the committee do with them? At this point, it's hard to say.

In the past, we found that strength of record already correlated substantially with the committee's selection and seeding choices.

When the NCAA made its team-sheet changes last month, it separated out the three résumé metrics (strength of record, KPI and RPI) from the three team-strength metrics (BPI, KenPom, Sagarin ratings). The sheet then averages the ranks of the three résumé metrics from the ranks of the three team strength metrics and then averaged all six ranks together. What we don't yet know is how the committee will weigh these two types of metrics.

Should a team like Gonzaga or Wichita State (BPI: 9, SOR: 33) be ranked at all, that would suggest the committee could be considering team-strength metrics such as BPI or Sagarin ratings more than it had in the past.

Regardless of how it weighs the new metrics, the key signal from the NCAA is that it is preparing to move away from RPI. That's a positive for anyone hoping for a more accurate bracket, and it will help the committee avoid seeding flubs such as in 2016 with Oregon, which received a No. 1 seed while ranked fourth in RPI but 10th in SOR and 28th in BPI. Regarding RPI this year, one team to watch is Rhode Island, which ranks fifth in the old statistic but 15th in strength of record and 26th in BPI.

It's worth noting that while any move away from RPI is a good one, the way the NCAA elected to display the new metrics on its team sheets still has its flaws. For starters, simply averaging the ranks from each list loses some detail in what the numbers are showing us (e.g., the difference between the fifth-ranked team and sixth-ranked team might be much larger than the difference between the 14th-ranked team and 15th-ranked team).

But even putting that aside, these metrics are telling us different things -- there's a reason why ESPN separates out forward-looking BPI and backward-looking strength of record, for example -- and so to squish them into one number can be misleading.

If the committee wants to replace RPI, it should not do so by averaging these six numbers together. Ultimately, it depends what the committee's objective is, but if the goal is to produce one number that signifies résumé while seeking input from a variety of models, a better option would be:

  1. Properly create an amalgam of KenPom, Sagarin and BPI by using their actual ratings and placing all three on the same scale.

  2. Use that new, combined team metric to be the basis for a singular strength of record metric (the combined team metric determines how good an opponent is, and therefore how impressive a win is or how detrimental a loss is).

All that being said: Even the incremental recent improvements to the team sheet significantly outweigh the drawbacks. There is better information available to committee members on their team sheets now than there was a year ago. Now we'll find out what they're going to do with it.

Ben Alamar and Paul Sabin contributed to this story.

^ Back to Top ^