Comparing the NCAA's 2015 ARPI and the 5 Iteration ARPI, 2009 BPs to the Committee's Actual Decisions

In the preceding post, I provided information about changes to the RPI that I've recommended to the Women's Soccer Committee.  The change is from the NCAA's current 2015 ARPI to what I call the 5 Iteration ARPI with the 2009 Bonus and Penalty regime.  As I discussed, the 5 Iteration ARPI, 2009 BPs system performs far better than the 2015 ARPI.

To add more information, I decided to compare the Committee's decisions on at large selections and seeds over the last 10 years to what the rankings would have been over those 10 years for the 2015 ARPI and for the 5 Iteration ARPI, 2009 BPs systems.  The purpose of this is to see which system's ratings match better with the Committee's actual decisions.

To do the comparison, I determined, for each rating system, the average rank of the teams the Committee gave at large selections, the average rank of the teams in the Top 60 (using the 2015 ARPI's ratings to determine the Top 60) to which the Committee denied at large selections, the average ranks of the teams to which the Committee gave #1, #2, #3, and #4 seeds respectively, and the average rank of the 16 teams to which the Committee seeds as a group.  The following table shows the results of the comparison:

Starting with the at large selections, the 5 Iteration ARPI, 2009 BPs ranked the teams to which the Committee gave selections 1.46 positions better than the 2015 ARPI.  What this means is that the Committee's at large selections matched better with the 5 Iteration version than with the NCAA's current version.

Moving on to the teams to which the Committee denied at large selections, the 5 Iteration version ranked those teams 3.73 positions more poorly than the 2015 ARPI.  Again, this means the Committee's decisions -- this time at large rejections -- matched better with the 5 Iteration version than with the NCAA's current version.

For seeds, for the #1 and #4 seeds, the Committee's decisions matched slightly better with the 5 Iteration version than with the NCAA's current version.  For the #2 and #3 seeds, on the other hand, the Committee's decisions matched slightly  better with the NCAA's current version.  When looking at seeds as a whole group, the Committee's decisions match very slightly better with the 5 Iteration version.

Looking particularly at the Committee's at large decisions, and assuming that those decisions in most cases were the right ones, the above numbers show that the 5 Iteration version's rankings come closer to being the right rankings than the current NCAA's version's rankings.  They certainly come closer to the decisions the Committee believes are the right decisions.

Proposal to Women's Soccer Committee for Changed RPI Formula

Recently, I gave the Women's Soccer Committee a proposal to change the RPI formula to address some of the current formula's major problems.  The proposal is to change to what I call the 5 Iteration Adjusted RPI using the 2009 Bonus and Penalty Points regime.  I'm not going to go into a detailed discussion of the proposed new formula here.  For details on the proposed new formula, you can go to the "RPI: Modified RPI?" page of the RPI for Division I Women's Soccer website.

As I show in detail at the "RPI: Modified RPI?" webpage, the 5 Iteration ARPI formula provides ratings that are at least as consistent with game results as the NCAA's ARPI versions.  More important:

• The 5 Iteration ARPI rates conferences more accurately in relation to each other (1) in terms of general fairness and (2) in relation to conference strength.  In fact, the 5 Iteration ARPI eliminates the NCAA ARPI's biases in relation to conference strength.

• The 5 Iteration ARPI rates the regional playing pools more accurately in relation to each other (1) in terms of general fairness and (2) in relation to region strength.  It doesn't eliminate the biases in relation to region strength of the NCAA's ARPI, but it significantly reduces the biases.

• The 5 Iteration ARPI, for practical purposes, eliminates the disconnect that the NCAA's ARPI versions have, between teams' ARPI ranks and their ranks as contributor to opponents' strengths of schedule.  Thus the 5 Iteration ARPI will eliminate the incentive and need of coaches of potential bubble teams to try to "game" the system in their scheduling of non-conference opponents.  Under the 5 Iteration ARPI, an opponent's value in terms of contribution to your strength of schedule will be roughly the same as its actual rank.  This is as distinguished from the NCAA's RPI versions, where an opponent's value in terms of contribution to your strength of schedule can be very different than its actual rank.

Again, I cover all of this in detail at the "RPI: Modified RPI?" page.

For those of you who are coaches that have followed my work on the RPI, if you agree that a change to the proposed new formula would be a good idea, it will be very helpful if you will let any contact you have on the Women's Soccer Committee know you think the Committee should take a careful look at my proposed change.  This could be particularly helpful as, driven by basketball, the NCAA is in the process of taking a careful look at the RPI as well as at other rating formulas.  Thus there is an opening for changes now that has not been there in the past.

Here are the current Committee members, with email addresses:

Karen Hancock Oklahoma State: karen.hancock@okstate.edu

Janet Oberle, Saint Louis: oberlejl@slu.edu

Janet Rayfield, Illinois: rayfield@illinois.edu

Mick D'Arcy, Central Connecticut: darcym@ccsu.edu

Foti Mellis, California: fmellis@berkeley.edu

John McElwain, Sun Belt Conf: mcelwain@sunbeltsports.org

Tony da Luz, Wake Forest: daluz@wfu.edu

Shawn Farrell, Seattle: farrells@seattleu.edu

Chad Miller, Western Carolina: millerc@email.wcu.edu

Stephanie Ransom, Georgia: sransom@sports.uga.edu