2025 ARTICLE 13: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 6, GEOGRAPHIC REGIONS IN RELATION TO NCAA RPI RANKS AND STRENGTH OF SCHEDULE RANKS

This article, for the geographic regions within which the teams from each state play most of their games, provides information similar to that provided for conferences in 2025 Article 12.  A map showing the four regions is at the RPI for Division I Women's Soccer RPI: Regional Issues page.

As you can see, when averaged across a region, the differences between average NCAA RPI ranks and average Strength of Schedule contributor ranks under the NCAA RPI formula are relatively small.  This makes sense, since each region has an array of strong and weak teams and conferences.  As a generalization, however, looking at the numbers for the regions' teams opponents, overall and on average teams from the West region are discriminated against due to the way the NCAA formula computes Strength of Schedule, the Middle region experiences no impact, and the North and South regions are benefitted by discrimination.

To be clear, there are teams and conferences from all of the regions that the NCAA RPI formula discriminates against and in favor of.  The numbers above simply show the net effect of the discrimination for each region.

A particular concern this year is a significant reduction in out-of-region competition, most likeky due to less funding being available for travel.  The following table shows the extent of the reduction looking at the nation as a whole:

As you can see, the number of out-of-region games will be reduced by 28.1% from what the number historically has been.

A break down of the numbers from the preceding table by region shows reductions in the number of out-of-region games as follows:

Middle  18.3%

North  28.5%

South  30.0%

West  31.7%

These reductions should be a concern for the Women's Soccer Committee.  The NCAA RPI already has a problem ranking teams dispersed among the conferences and across the regions within a single national system.  The reductions in out-of-region play are likely to make the problem worse. 

 

2025 ARTICLE 12: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 5, CONFERENCES IN RELATION TO NCAA RPI RANKS AND STRENGTH OF SCHEDULE RANKS

 In 2025 Pre-Season Predictions and Information, Parts 4 and 4B, for the individual teams I showed the relationship between predicted NCAA RPI ranks and Strength of Schedule Contribution ranks under the NCAA RPI formula, both for the individual teams and for their opponents.  In this article, I will show the same information, but for each conference.  This gives a good picture of how the NCAA RPI discriminates among conferences because of the defective way it calculates Strength of Schedule.

This table has the conferences in NCAA RPI rank order, based on the average rating of their teams.  See below the table for comments.

In the table, the first two green-highlighted columns on the left show, for each conference, the difference between its teams' average NCAA RPI rank and its teams' average Strength of Schedule contributor rank under the NCAA RPI formula.  As you read down the table from the strongest conferences at the top to the weakest at the bottom, you can see the clear pattern: For stronger conferences, the conference teams' Strength of Schedule contributor ranks are poorer than the teams' actual ranks say they should be; and for weaker conferences they are better than they should be.

The next two salmon-highlighted columns look at how this plays out for the conference teams' schedules.  The first of those columns shows the conferences' teams' opponents' average ranks and the second column shows those opponents' average ranks as Strength of Schedule contributors.  The pattern here is the same:  Stronger conferences' opponents' Strength of Schedule Contributor ranks are poorer than the opponents' actual ranks say they should be; and the opposite is true for the weaker conferences.

The next four columns break the numbers for the conference teams' schedules down into conference opponents (green-highlighted) and non-conference opponents (salmon-highlighted).  Given that in conference play, the conferences' teams are playing each other, it is no surprise that the contrasts between the conference opponents' NCAA RPI ranks and their ranks as Strength of Schedule contributors follow the same basic pattern.  For the non-conference opponents, where the individual teams have more control over their schedules, the pattern is similar but less extreme and with a little more variability.

It is important here to point out that coaches in top tier and most coaches in middle tier conferences are aware of these patterns and often take them into consideration in their non-conference scheduling.  They also are aware, however, that in the NCAA Tournament seeding and at large selection processes, good results against highly ranked opponents matter, including against highly ranked non-conference opponents.  Further, coaches of teams with NCAA Tournament aspirations often want to play at least some strong non-conference opponents.  This means that they sometimes decide to schedule opponents whose Strength of Schedule contributions are likely to be poorer than their RPI ranks say they should be, essentially deciding to take a potential RPI "hit" in exchange for the potential of a good result against a highly ranked opponent.

NOTE:  Being aware of the scheduling dilemma I just described, I designed my Balanced RPI, which is a modification of the NCAA RPI, with the specific objective of eliminating the difference between teams' ranks and their ranks as Strength of Schedule contributors.  Thus under the Balanced RPI, if a team has a rank of X, that also is either exactly or very close to exactly the team's rank as a Strength of Schedule contributor.  In other words, if the NCAA were to use the Balanced RPI, coaches no longer would have this scheduling dilemma.   (As an additional benefit, the RPI no longer would discriminate among conferences in relation to conference strength.)

2025 ARTICLE 11: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 4B, TEAMS' SCHEDULES IN RELATION TO OPPONENTS' NCAA RPI RANKS AND STRENGTH OF SCHEDULE RANKS

In Part 4, I discussed and showed the differences between teams' NCAA RPI ranks and their ranks as Strength of Schedule contributors under the NCAA RPI formula.  In this article, I will show predictions for how these differences will affect teams by the end of the 2025 season.

For each team, the following table shows its predicted:

Opponents' average NCAA RPI rank

Conference opponents' average NCAA RPI rank

Non-conference opponents' average NCAA RPI rank

Opponents' average rank as Strength of Schedule contributors under the NCAA RPI formula

Conference opponents' average rank as Strength of Schedule contributors under the NCAA RPI formula

Non-conference opponents' average rank as Strength of Schedule contributors under the NCAA RPI formula

These numbers allow you to see how the NCAA RPI rank versus Strength of Schedule contributor rank differences relate to:

1.  Teams' in-conference schedules, which teams basically can't control;

2.  Teams' non-conference schedules, which teams can control at least to some extent; and

3.  Teams' overall schedules.

If you review the table's numbers with a view to the strength of the teams' conferences, you will see that generally speaking the NCAA RPI formula understates the strengths of schedule of top tier conferences' teams, gets the strengths of schedule of middle tier conferences' teams about right, and overstates the strengths of schedule of bottom tier conferences' teams.  I've arranged the teams by conference so you can better see how this NCAA RPI defect affects teams by conference.  Scroll to the right, if necessary, to see the entire table.

NOTE: The differences in the Conference Opponents Average Rank column for teams from the same conference are primarily due to conference teams not playing full round robins.  The differences in the Non-Conference Opponents Average Rank column for teams from the same conference are due the different teams' non-conference scheduling strategies.

I'll use Baylor, from the Big 12, with a predicted NCAA RPI rank of #66, and Lamar, from the Southland, with a predicted NCAA RPI rank of #55, as examples.  I've chosen these teams because no team ranked poorer than #57 ever has gotten an at large position in the NCAA Tournament.  Thus Baylor is outside the historic at large candidate group and Lamar is within the candidate group.

Baylor (Big 12): 

Conference opponents' average NCAA RPI rank is 81 and conference opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 119.

Non-conference opponents' average NCAA RPI rank is 110 and non-conference opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 108.

Overall, opponents' average NCAA RPI rank is 93 and opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 115.

Thus Baylor's Strength of Schedule component of the NCAA RPI significantly discriminates against Baylor in relation to its conference schedule and only barely offsets that discrimination in relation to its non-conference schedule.  The overall result is that the Strength of Schedule component significantly discriminates against Baylor.

Lamar (Southland):

Conference opponents' average NCAA RPI rank is 212 and conference opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 185.

Non-conference opponents' average NCAA RPI rank is 153 and non-conference opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 184.

Overall, opponents' average NCAA RPI rank is 193 and opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 185.

Thus the Strength of Schedule component of the NCAA RPI significantly discriminates in favor of Lamar in relation to its conference schedule and offsets that discrimination some in relation to its non-conference schedule.  The overall effect, however, is that the Strength of Schedule component still discriminates in favor of Lamar.

Given that Baylor is outside but in the vicinity of the ranking area of teams that historically are candidates for NCAA Tournament at large selections and Lamar is only a little inside that ranking area, this demonstrates the importance of this NCAA RPI defect.  History suggests that Lamar, if not an Automatic Qualifier, would not get an at large selection.  For Baylor, however, being outside the historic candidate area, there is a question whether, if inside the candidate area and considered by the Committee, it might displace one of the "last in" at large teams.  In other words, this NCAA RPI defect may have negative NCAA Tournament at large selection consequences.  (And, by a similar analysis of seeding candidate groups, may have negative seeding consequences.)

The significance of this kind of example is reinforced if you consider Lamar's and Baylor's ranks using my Balanced RPI.  The Balanced RPI is a rating system that builds on the RPI, with modifications that fix the NCAA RPI's defective discrepancy between teams' NCAA RPI ranks and their ranks as Strength of Schedule contributors under the NCAA RPI formula.  The Balanced RPI's predicted rank for Lamar is 110, well outisde the NCAA Tournament at large selection candidate range.  For Baylor, its predicted rank is #57, in other words a candidate for at large selection.

2025 ARTICLE 10: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 4, TEAMS' NCAA RPI RANKS COMPARED TO THEIR RANKS AS STRENGTH OF SCHEDULE CONTRIBUTORS

The NCAA RPI has a major defect, which is the way in which it computes a team's strength of schedule.

As discussed on the RPI: Formula page at the RPI for Division I Women's Soccer website, the NCAA RPI has two main components:  a team's Winning Percentage and its Strength of Schedule.  Within the overall NCAA RPI formula, the effective weights of the two components are approximately 50% Winning Percentaqge and 50% Strength of Schedule.

Within the NCAA RPI formula, in turn, Strength of Schedule consists of two elements: the average of a team's opponents' winning percentages (OWP) and the average of a team's opponents' opponents' winning percentages (OOWP).  And, within Strength of Schedule, the effective weights of these two elements are 80% opponents' winning percentage and 20% opponents' opponents' winning percentage.  Thus for the NCAA RPI's Strength of Schedule component, a team's opponents' winning percentages matter a lot and against whom they achieved those winning percentages matters little.  This is a major defect.

In this and the next two parts of my Pre-Season Predictions and Information, using end-of-season predictions for the 2025 season, I will show how the NCAA RPI's strength of schedule defect plays out for teams (this Part 4), for conferences (Part 5), and for geographic regions (Part 6).

The following table shows, for each team, its predicted end-of-season NCAA RPI rank and its predicted rank as a strength of schedule contributor under the NCAA RPI formula.  In a good rating system, these ranks should be the same or, at least, very close to the same.  As the table shows, however, for the NCAA RPI formula, for many teams, the ranks are not close to the same.

Using some of the top teams in the alphabetical list as examples:

If Team A plays Air Force as an opponent, Team A will have played the NCAA RPI #232 ranked team.  When computing Team A's rating and rank, however, the NCAA RPI formula will give team A credit only for playing the #274 team.

On the other hand, if Team A plays Alabama State, Team A will have played the #340 team.  But when computing Team A's rating and rank, the NCAA RPI formula will give Team A credit for playing the #277 team.

Thus although the NCAA RPI ranks Air Force and Alabama State 108 rank positions apart, when considering each of their strengths for purposes of Team A's strength of schedule computation, the NCAA RPI treats Air Force and Alabama State as roughly equal.

You can scroll down the table and see how this NCAA RPI formula defect plays out for teams you are interested in,  I suggest you look, in particular, at teams in the middle to lower levels of top tier conferences and at teams in the upper levels of middle and bottom tier conferences.  For example:

Look at Alabama:  Its predicted NCAA RPI rank is #37.  But, its predicted rank as a strength of schedule is only #89.

Then look at Bowling Green:  Its predicted NCAA RPI rank is #86 but its predicted rank as a strength of schedule contributor is #26.

These kinds of differences have significant practical implications related to scheduling and the NCAA Tournament.  Teams' NCAA RPI ranks are a key factor in the Women's Soccer Committee's decisions on Tournament seeds and at large selections.  So, if a coach has NCAA Tournament aspirations, from strictly an NCAA RPI perspective, Bowling Green would be a significantly better opponent to play than Alabama.  This would be true for two reasons: (1)  Bowling Green probably is weaker than Alabama, so an easier game in which to get a good result; and (2) Bowling Green, as an opponent, will give the coach's team's NCAA RPI a better strength of schedule contribution than Alabama.

Thus when doing non-conference scheduling, coaches with NCAA Tournament aspirations or with other concerns about where their teams will finish in the NCAA RPI rankings must take this NCAA RPI formula defect into account.  In essence, they are in the position of having to learn how to "trick" the NCAA RPI through smart scheduling -- in the example, choosing Bowling Green rather than Alabama as an opponent. 

2025 ARTICLE 9: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 3, "PREDICTED" CONFERENCE REGULAR SEASON AND TOURNAMENT CHAMPIONS

Continuing with "predictions," using the "results likelihood" method described in 2025 Article 9, my system uses the same "3 points for a win and 1 for a tie" scoring that conferences use for their standings to create team standings within each conference.  It is worth noting that the results likelihoods take game locations into account and that a good number of conferences do not play full round robins.

Using the ACC as an example, here are what its "predicted" end-of-season standings look like:

Although Florida State's and North Carolina's points look the same, that is due to rounding.  Florida State's are slightly higher.  My interpretation of these standings is that it will be very close at the top of the conference among Florida State, North Carolinam Stanford, and Duke, with Virginia also in the mix.

Using the conference standings and the conference tournament formats (as published to date), my system next creates conference tournament brackets.  Then, since it is necessary to have winners and losers to fill out the entire tournament brackets, the system assigns as a game winner any team that has a win likelihood above 50%.  Where neither team has a win likelihood above 50%, the system treats the game as a tie.  For the tiebreaker, the advancing team is the one with the higher win likelihood.

This process results in the following conference regular season and conference tournament champions.  In most cases they are the same, but in two cases they are different.

2025 ARTICLE 8: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 2, "PREDICTED" END-OF-SEASON RANKS AND RATINGS

Once I have assigned pre-season strength ranks and ratings to teams, I combine those with teams' schedules to "predict" where teams will end up at the end of the season following completion of the conference tournaments.  The process to do this requires background work:

1.  For each game, I calculate the game-location-adjusted rating difference between the teams, using their assigned pre-season strength ratings as the base.  Since the assigned strength ratings are based on what the average historic NCAA RPI ratings are for those teams' ranks, my game location adjustments increase the home team's rating by 0.0085 and decrease the away team's rating by 0.0085, for an overall adjustment of 0..0170.  This is the value of home field advantage for the current version of the NCAA RPI with the "no overtime" rule in effect.

2.  For the location-adjusted rating difference between the teams, I calculate each team's expected win, loss, and tie likelihoods.  These likelihoods are based on a study of the location-adjusted rating differences and results of all games played since 2010 (excluding 2020).

[NOTE: For a detailed explanation of how I determine the game location adjustment and the win, loss, and tie likelihoods, go to the RPI for Division I Women's Soccer website's page RPI: Measuring the Correlation Between Teams' Ratings and Their Performance. 

3.  Rather than assigning the opponents in a game either a win, a loss, or a tie result, I assign each team its win, loss, and tie likelihoods since these will give a better picture of what a team's overall record will be given its entire schedule.  As an example, Colorado and Michigan State will play on August 14 at Colorado.  Their location-adjusted rating difference is 0.0333 in favor of Colorado.  For that rating difference, referring to a result probability table for the current NCAA RPI in a "no overtime" world, Colorado's result likelihoods are 53.6% win, 19.9% loss, 26.4% tie (which don't quite add up to 100% due to rounding).  Those numbers go into Colorado's win-loss-tie columns for NCAA RPI computation purposes.  Michigan State's win-loss percentages are the converse.  I assign these likelihoods for all teams' games, add up each team's percentages, and convert them from percentages to numbers .  Thus Colorado ends the season with 12.2 wins, 4.9 losses, and 4.9 ties.  These are the numbers I use for Colorado as its wins, losses, and ties when computing its NCAA RPI.

4.  I then compute all teams' NCAA RPI ratings and ranks.  It is important to understand that these are different than the teams' assigned pre-season strength ratings and ranks.  This is because the NCAA RPI does not measure team strength.  Rather, it measures team performance based on a combination of teams' winning percentages and their strengths of schedule (as measured by the NCAA RPI formula).  Thus two teams with identical strength ratings and ranks will end up with different NCAA RPI ratings and ranks if they have different winning percentages and/or different strengths of schedule.  Below are my computed end-of-season (including predicted conference tournaments) ratings and ranks for teams.  You can compare the ranks to the ones in the preceding post to see the differences between teams' assigned pre-season strength ranks and team' predicted end-of-season NCAA RPI ranks.  (NOTE: I have corrected these rankings since their initial publication to fix a programming error.  The changes are relatively minor.)

2025 ARTICLE 7: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 1, ASSIGNED PRE-SEASON RANKS AND RATINGS

INTRODUCTION

All the teams have published their schedules -- almost, with Akron yet to publish its non-conference schedule and Delaware State yet to publish, but unless those teams play each other, we can determine their schedules from what others have publlished.  So for practical purposes we have the all the schedules.  This makes it possible to do pre-season predictions of where teams will end up at the end of the regular season, including conference tournaments.

Pre-season predictions involve a lot of assumptions that may or may not prove correct.  Also, my prediction method depends entirely on teams' rank histories.  So don't take the pre-season predictions too seriously.  On the other hand, this series of articles will have educatational value, particularly about how the NCAA RPI works and about the importance of scheduling.  So I recommend not getting preoccupied with the details of the predictions but instead watch for what you can learn about the NCAA RPI and its interaction with teams' schedules.

ASSIGNED PRE-SEASON RANKS AND RATINGS

The first step in my process, which is the subject of this article, is to assign pre-season ratings and ranks to teams.  In essence, this is predicting teams' strength.

There are a lot of ways to predict team strength, some complex and some simple.  I predict strength using only teams' rank histories, without reference to changing player and coaching personnel.  There are others who do predictions using those kinds of detailed information about this year's teams -- conference coaches for their own conferences and in the past, but not currently, stats superstar Chris Henderson.  In the past, their predictions have been better than mine, but only slightly better.  So you can consider my predictions as somewhat crude but close to as good as you can get when making data-based predictions for the entire cast of teams.

When using only teams' rank histories, my analyses show that the best predictor of where teams will end up next year is the average of their last 7 years' ranks using my Balanced RPI, which is a modified version of the NCAA RPI that fixes its major defects.  There is a problem, however, with that predictor.  If a team has a major outlier year -- a much higher or lower rank than is typical -- using a 7-year average can significantly mis-rate that team.  On the other hand, if I use the median rating over the last 7 years, it avoids that problem.  It is a little less accurate as a predictor for where teams will end up next year, but not by much.  So for my predictor, I use teams' median Balanced RPI ranks over the last 7 years.

Once I have assigned teams' ranks, I then assign NCAA RPI ratings to the teams.  To do this, I have determined the historic average rating of teams at each rank level.  When I do this, however, I have to take into account the recent NCAA "no overtime" rule change and the 2024 NCAA RPI formula changes.  So when I determine historic average ratings, I use what past ratings would have been if the "no overtime" rule and 2024 NCAA RPI formula had been in effect, using the years 2010 to the present (but excluding Covid-affected 2020).

This produces the following "strength" ranks and ratings for teams.  You will note that no team has a #1 assigned rank and some teams have the same assigned rank.  This is because I am using teams' 7-year median ranks.  (Scroll to the right to see additional teams.)

2025 ARTICLE 6: THE KP INDEX (KPI): THE COMMITTEE'S MISTAKE AND THE BIG 10'S OWN GOAL

 From Report of the Women's Soccer Committee December 9, 2024 and January 29, 2025 Meetings:

"KPI, and other ranking systems.  The Committee felt the KPI should be used more in selections as a valued tool in the process.  The Committee reviewed the Massey Ratings and decided not to request it as an additional criterion at this point."

Thus, as it has done for the past two years, the Committee intends to supplement the NCAA RPI with the KPI as a system for ranking teams.  It intends to use the KPI more, however, than it has in the past.  This is a change from a year earlier, when the Committee reported that it had found the KPI not useful and proposed use of Massey beginning in 2026.

The Committee for a while has pushed for use of more than just the NCAA RPI for team ratings and rankings.  The Committee two years ago finally received approval to use the KPI.  After it received that approval, I asked a Committee member where the decision to use the KPI, as the particular system approved for use, came from.  The member advised me it did not come from the Committee, so far as he recalled.  My assumption, therefore, is it came from the NCAA staff.

What Is the KPI? 

The KPI is a product of Kevin Pauga, the Associate Athletic Director for Strategic Initiatives and Conference Planning at Michigan State.  It appears the KPI is something he produces outside his formal work for Michigan State.  Pauga also is regarded as a scheduling expert, employed by the Big 10 and other conferences to produce conference schedules for their teams.  His scheduling system considers many parameters.  I do not know if it considers the relationship between teams' schedules and their NCAA RPI rankings.

According to a New York Times article dated March 26, 2015, Kevin Pauga started using the KPI for college basketball ratings and rankings in 2013.  According to the article, a KPI rating is a

"number that, essentially, correlates to how valuable a team’s wins are versus how damaging its losses are. The ratings run from negative-one (bad) to plus-one (good), and combine variables that might change over the course of a season. His goal is to quantify so-called good wins and bad losses, and to assess how teams compare with one another."

And, according to an NCAA release dated March 5, 2025, for NCAA men's basketball:

"The Kevin Pauga Index metric ranks team resumes by assigning a value to each game played. The best win possible is worth about +1.0, the worst loss about -1.0, and a virtual tie at 0.0.  Adjustments are made to each game's value based on location of the game, opponent quality and percentage of total points scored. Game values are added together and divided by games played to determine a team's KPI ranking."

Beyond these descriptions, it appears the KPI is proprietary, so I do not know exactly what its formulas are.

Is the KPI a Good Rating System for Division I Women's Soccer?

I don't know how well the KPI functions as a rating system for other NCAA sports.  For Division I women's soccer, however, it is a poor rating system.  It has the same defects as the NCAA RPI.

In 2025 Article 2, I explained my system for grading Division I women's soccer rating systems and did a detailed review of how the NCAA RPI performs as a rating system.  As I showed:

"Based on the ability of schedulers to "trick" the NCAA RPI and on its conference- and region-based discrimination as compared to what the Balanced RPI shows is achievable, the NCAA RPI continues to get a failing grade as a rating system for Division I women's soccer."

The following review of the KPI is similar to the review of the NCAA RPI in 2025 Article 2.  I will not go through the same detailed explanation of the grading system here as I gave in Article 2, so you might want to review Article 2 before proceeding further.

Ability of the System to Rate Teams from a Conference Fairly in Relation to Teams from Other Conferences

This table has the conferences arranged in order from those with the best KPI average rating from 2017 through 2024 at the top and those with the poorest at the bottom.  (KPI ratings are available only for years since 2017.)

In the table:

The Conference NonConference Actual Winning Percentage column shows the conference's actual winning percentage against non-conference opponents.  In calculating Winning Percentage, I use the NCAA RPI Winning Percentage formula in effect as of 2024.

The Conference NonConference Likelihood Winning Percentage column shows the conference's expected winning percentage against non-conference opponents, based on the differences between opponents' KPI ratings as adjusted for home field advantage.  The expected winning percentage for each game is determined using a Result Probability Table for the KPI.  The table comes from an analysis of the location-adjusted rating differences and the results of all games played since 2017.  This method for determining expected winning percentages is highly precise when applied to large numbers of games as shown by the following table for the KPI:

 

In this table, the Total Win, Tie, and Loss Likelihood columns show the expected wins, losses, and ties by the higher rated team, after adjustment for home field advantage, for all games since 2017.  The columns show these in absolute numbers and as a percentage of all games played.  The Total Actual Wins, Ties, and Losses columns show similar numbers, but based on the actual results.  As you can see, out of the more than 22,000 games played, the difference between the expected wins and actual wins is 11 games, between the expected ties and actual ties is  6 games, and between the expected losses and actual losses is 4 games.  In other words, as I stated, the Result Probability Table method is a highly precise way of determining expected winning percentages for the KPI.

Returning to the larger table above, the Conference NonConference Actual Less Likely Winning Percentage column shows the difference between the conference teams' actual winning percentages in non-conference games and their expected winning percentages based on their games' location-adjusted KPI rating differences.  A positive difference means the teams' actual winning percentages are better than expected based on the KPI and a negative difference means the teams' actual winning percentages are poorer. 

This chart is based on the conferences table.  In the chart, conferences are arranged in order of those with the highest average KPI ratings on the left to those with the poorest ratings on the right.  The vertical axis is for the difference between a conference's actual and its KPI-expected winning percentage.  As you can see from the chart, stronger conferences (on the left) tend to perform better than their KPI ratings say they should and weaker conferences (on the right) tend to perform more poorly.  In other words, the KPI underrates teams from stronger conferences and overrates teams from weaker conferences.  This is the same pattern the NCAA RPI has.  The downward sloping straight line is a trend line showing the pattern of the data; and the formula on the chart is a formula whose use can tell what the data indicate the actual v expected difference should be for any particular conference average KPI rating.

This table draws from the data underlying the chart and from the chart itself, as well as from similar data and charts for the NCAA RPI and the Balanced RPI.  It thus compares the KPI to the NCAA RPI and also includes the Balanced RPI (which is similar to what the Massey rating system would show), to show what a good rating system can do.

In the first two color coded columns, the "Spread" column shows the performance percentage difference between the conference that most outperforms what the KPI says its performance should be and the conference that most underperforms.  The "Under and Over" column shows the total amounts by which all conferences either outperform or underperform.  Both of these columns are measures of rating system fairness in relation to teams from the different conferences.  As you can see, the KPI and the NCAA RPI both do a poor job, as compared tto what a good rating system can do.

The color coded column on the right comes from the chart and shows the rating system pattern in relation to conference strength, using the chart's trend line formula.  It shows the extent of discrimination in relation to conference strength.  As the column shows, the KPI and NCAA RPI have similar discrimination based on conference strength, whereas a good system as exemplified by the Balanced RPI has virtually no discrimination.

Ability of the System to Rate Teams from a Geographic Region Fairly in Relation to Teams from Other Geographic Regions

The following table and chart are similar to those above for conferences, but are for geographic regions:

The table and chart for geographic regions show an overall pattern similar to that for conferences.  The data point pattern, however, is up and down enough that I do not have a high level of confidence in the relationship the chart shows between regions' average ratings and their actual performance as compared to their KPI-expected performance.  This is indicated by the R squared value I have included in the chart below the trend line formula.  The R squared value is a measure of how well the data fit the trend line, with an R squared value of 1 being a perfect fit and of 0 being no fit.  The R squared value on the chart indicates a mediocre fit.

Here is another table-chart set.  They is based on the relationship between the proportion of games in each region that are ties, as a measure of the amount of parity within the region.

This chart shows a similar patter in relation to region average KPI rating, but with a significantly better R squared value.

Altogether, the tables and charts for regions result in the following comparison table:

As the first two highlighted columns show, the KPI and the NCAA RPI have relatively similar levels of discrimination among conferences, in terms of general fairness.  And as the Balanced RPI row shows, this could avoided by using a better rating system.  And, as the two color coded columns to the right show, to the extent that the KPI discriminates in relation to region parity or region strength, its discimination is similar to that of the NCAA RPI.  Again, the Balanced RPI row shows this could be avoided by using a better rating system.

Ability of the System to Produce Ratings That Will Match Overall Game Results

This table simply shows the extent to which the higher rated team, after adjustment for home field advantage, wins, ties, and loses for the particular rating systems.  Thus it is a gross measure of how well game results correlate with the systems' ratings.  Since the NCAA RPI and Balanced RPI numbers are based on the No Overtime rule having been in effect for all seasons, whereas the KPI numbers for 2017 through 2021 use overtime games' results, the best comparison is in the highlighted column on the right, which disregards tie games.  As a point of reference, a difference of 0.1% represents a difference of roughly 3 games per year.  As you can see, the KPI gets the correct result about 15 times fewer per year (0.5%) than the NCAA RPI and about 39 times fewer (1.3%) than the Balanced RPI.

This is similar to the previous table but limited to games involving at least one team in the system's Top 60.  Again, the best comparison is from the color coded column on the right.  Here, a difference of 0.1% represents one game per year.  Again, the KPI performs more poorly than the NCAA RPI and than the Balanced RPI.

Conclusion About the KPI as a Rating System

As the above information shows, the KPI has the same problems as the NCAA RPI, with higher levels of discrimination among conferences and geographic regions than can be achieved by a good rating system.

So why would the NCAA staff agree to use of the KPI, if it is the NCAA staff that selected the KPI as an additional system as I suspect?  And why would the Women's Soccer Committee double down on the KPI as it intends to do this coming year?  The answer is simple: Because the KPI has the same problems as the NCAA RPI.  It will not rock the NCAA RPI boat, but rather will make the NCAA RPI look legitimate.  Whereas if the Committee were to use Massey, it would have significant differences from the NCAA RPI and thus would expose the NCAA RPI as a poor rating system.  Indeed, I suspect that the staff and Committee looked at Massey ratings as compared to the NCAA RPI, saw that there were significant differences, and did not want to have to deal with the fallout those differences exposed.  I do not know this, I could be wrong, but it is the best explanation I can come up with for the Committee's reversal of its position on the KPI and Massey over the last year.

The Big 10's Own Goal

Finally, why do I say the Big 10, in all of this, has committed an "own goal" blunder?  Remember, Ken Pauga is an Associate Athletic Director at the Big 10's Michigan State.  If you look at the first table above, you will see that the Big 10 is the second most discriminated against conference by the KPI, actually winning 4.1% more games per year against non-conference opponents than the KPI ratings say they should win.

The following table breaks the numbers down by Big 10 team, based on the teams in the conference before absorption of the Pac 12 teams in 2024:

As you can see from the column on the right, the KPI discriminates against 11 teams (including Pauga's Michigan State) and in favor of 3.  If I were to add the four teams absorbed from the Pac 12 in 2024, there would be 3 more discriminated against and 1 more in favor of.

Thus so far as Division I women's soccer is concerned, the Women's Soccer Committee now will be using, apparently with added emphasis, a rating system that comes from a Big 10 school administrator yet discrimates against the Big 10 as a whole and even against his own school.  When it comes to the "last in" and "first out" for NCAA Tournament at large selections, that looks like an "own goal" to me.

2025 ARTICLE 4: THE NCAA RPI AND ITS EFFECT ON NCAA TOURNAMENT SEEDS

In  2025 Article 3, I showed how powerful the NCAA RPI is in the NCAA Tournament at large participant selection process.  In this article, I extend that discussion to show the power of the NCAA RPI in the NCAA Tournament seeding process.  I also, at the end, will summarize the data in Article 3 and this article.

This article assumes that you have reviewed Article 3, particularly for an explanation of the "Standards" and "Standards Plus Tiebreaker" systems I use as ways to show the relationship between the season's game results data, the factors the Women's Soccer Committee considers when forming the NCAA Tournament bracket, and the Committee's actual decisions.

#1 Seeds

The following table shows the 7 most "powerful" factors when it comes to the Committee's selection of #1 seeds.  I measure a factor's power, for example in relation to #1 seeds, by considering how many of the factor's seeds would match the Committee's seeds if the NCAA simply awarded seeds based on that factor.

As you can see, for #1 seeds, the NCAA RPI, by itself, can "pick," on average, between 3 and 4 #1 seeds per year.  Further, all of the most powerful factors include the NCAA RPI.

The following table shows the NCAA RPI number of teams picked by year and also includes the number when applying the Standards and Standards Plus Tiebreaker systems.  Since the NCAA RPI is the most powerful single factor for #1 seeds, that is the Tiebreaker factor in the Standards Plus Tiebreaker system for #1 seeds.

The Standards system (the three white columns) is useful for getting a picture of how the Committee might go through the #1 seed selection process.  The "In" Selections column is teams that are clear #1 seeds based on past history.  The Open Positions column is the number of additional teams the Committee needs to pick to fill out the seed group.  The Candidates for Remaining Open Positions column is the number of teams the Committee has to choose from to fill the open positions -- these are teams that have no aspects of their profiles that say "yes, a team with this profile always has gotten a #1 seed" and also none that say "no, a team with this profile never has gotten a #1 seed."  Thus in 2007, as an example, according to the Standards system 3 #1 seeds were clear, with 1 open position and 2 candidates to fill the position.  This happens to match the Median situation over the years: 3 clear #1 seeds, leaving 1 open position to be filled from 2 candidates (see the Median row at the bottom of the table).

The Standards With Tiebreaker system takes the open position(s) candidates and ranks them using the Tiebreaker, with the best-ranked candidate(s) filling the open position(s).  Since the NCAA RPI is the best factor at picking #1 seeds, that is the tiebreaker for the #1 seed Standards Plus Tiebreaker system.  As you can see from the column on the right, the Standards Plus Tiebreaker system picks match 97.1% of the Committee's picks -- in other words all but 2 of the 68 #1 seeds over the years.

#2 Seeds

As you can see, again the most powerful factor is teams' RPIs.  In the table, CO stands for Common Opponents and HTH for Head to Head, referring to the NCAA factors of results against common opponents and head to head results.

The RPI, on its own, would fill correctly 64.7% of all #2 seeds.  This is considerably fewer than the percentage for #1 seeds.  It means that in filling #2 seeds, the Committee is significantly influenced by the RPI but also by other factors.

As with the #1 seeds, the Standards system fills about 3 positions per year, leaving 1 position to be filled from 2 candidates.  The Tiebreaker again is teams' RPIs.  The Standards Plus Tiebreaker system correctly fills 92.6% of the #2 seeds, having missed 5 of the 68 #2 seeds over the years.

#3 Seeds

The RPI continues to be the most powerful factor.

You can see that the Committee's #3 seed picks are less predictable, for all the systems.

#4 Seeds

The RPI remains the most powerful factor.

As you can see, the #4 seeds show a continuing decline in the systems' ability to predict seeds.

#1 to #4 Seeds as a Group

It will help to see how the systems match up with the Committee's #1 through #4 seed decisions, when considering teams as a group without reference to which of those seeds they get.

As the tables show, the RPI itself and the Standards Plus Tiebreaker system do a quite good job of matching with the Committee's decisions on which 16 teams will share the #1 through #4 seeds, with the Standards Plus Tiebreaker missing a median of only 1 of the 16 positions per year.

#5 to #8 Seeds as a Group

With the NCAA having had #5 through #8 seed pods only since 2022, there are not enough data to do meaningful breakdowns like the above for each of those pods.  The following tables show data for the four pods as a group, although in my opinion there still are not enough data to draw firm conclusions.

It is important to note that this table suggests that the RPI, for the #5 through #8 seeds, by itself is not the most powerful decision factor.  Rather, the paired NCAA RPI Rank and Top 50 Results Score factor is the most powerful.  Significantly, this relates to the information in 2025 Report 3, which shows that the paired NCAA RPI Rank and Top 50 Results Rank factor is the most powerful for at large selections.

As this table shows, the Committee's #5 through #8 seeds as a group appear to be pretty data driven, with the Standards Plus Tiebreaker system being a quite good indicator of which teams will be in the group.  It will take more years' data, however, to reach firmer conclusions on this.

Summary

The following table summarizes the above tables as well as the At Large tables from 2025 Article 3:

The RPI is the most powerful factor the Committee considers when it comes to #1 and #2 seeds and which teams will be in the #1 through #4 seed group as a whole.  From the larger seed group, which teams will get #3 and #4 seeds appears to be somewhat data driven but also appears to be somewhat random.

For the #5 through 8 seeds and particularly for at large selections, the Standards system plus the paired RPI and Top 50 Results factors as a tiebreakers give a good match to the Committee's decisions.  Since the Top 50 Results scoring system relates to teams' RPI ranks, the RPI plays a significant role in both halves of the paired factor.

From an overall perspective, the RPI appears to be a critical factor in all of the "close" decisions the Committee must make.

2025 ARTICLE 3: THE NCAA RPI AND ITS EFFECT ON NCAA TOURNAMENT AT LARGE SELECTIONS

In my first two 2025 articles, I showed the NCAA RPI's defects and how they cause the RPI to discriminate against teams from some conferences and regions and in favor of teams from others.  In this post, I discuss how determinative the NCAA RPI is in the NCAA Tournament at large participant selection process.

Suppose the NCAA were to award at large NCAA Tournament positions to teams based strictly on their NCAA RPI ranks.  How much difference would it make from having the Women's Soccer Committee make the at large selections?  In other words, how many changes would there be from the Committee's awards?  Before reading further, as a test of your own sense of the NCAA RPI's importance in the at large selection process, write down what you think the average number of changes would be per year if the NCAA simply made at large selections based on teams' NCAA RPI ranks.  Later in this article, you'll be able to compare your guess to the actual number.

NCAA Tournament At Large Selection Factors

The NCAA requires the Committee to consider certain factors when making its NCAA Tournament at large selections.  As those of you who follow my work know, I have converted those factors into a series of individual factors and also have paired them to create an additional series of paired factors in which each individual factor has a 50% weight.  Altogether, this produces a series of 118 factors.  Some of the NCAA's individual factors have numerical scoring systems -- for example, the NCAA RPI and NCAA RPI Ranks -- and some do not -- for example, Head to Head Results.  For those factors that do not have NCAA-created scoring systems, I have created scoring systems.

It is possible, by comparing the teams to which the Committee has given at large positions to teams' scores for a factor, to see how close the match-up is between the Committee's at large selections and the factor scores.  The following table shows the factors that best match the Committee's at large selections over the 17 years from 2007 through 2024 (excluding Covid-affected 2020):

As you can see, the Committee's at large selections match teams' NCAA RPI ranks 92.6% of the time.  The Committee has "overruled" the NCAA RPI 42 times (568-526) over the 17 year data period.  The following table shows how the Committee overrules have played out over the years:

As the table shows, over the years, the Committee's selections have differed from the NCAA RPI ranks by from 1 to 4 positions.  The average difference has been 2.47 positions per year.  (This is the answer to the question at the top of this article.)  The median has been 2.  A way to think about this is that on average all the Committee's work has resulted in a change of only 2 to 3 teams per year from what the at large selections would have been if the NCAA RPI made the selections.  This suggests that no matter what the Committee members may think, the NCAA RPI mostly controls the at large selection process, with the Committee's work making differences only at the fringes.

NCAA Tournament At Large Factor Standards

For each of the factors, it is possible to use the 17 years' data to identify what I call "yes" --or "In" -- and "no" -- or "Out" -- standards.  For an "In" standard, any team that has done better than that standard over the 17 year data period always has gotten an at large selection.  Conversely, any team that has done more poorly than an "Out" standard never has gotten an at large selection.  The following table shows the standards for the NCAA RPI Ratings and NCAA RPI Ranks:

In the table, the At Large column has the "In" standards and the No At Large column the "Out" standards.  Thus teams with NCAA RPI ratings better than 0.6045 always have gotten at large selections and teams with ratings poorer than 0.5654 never have gotten selections.  Likewise teams with NCAA RPI ranks better than 27 always have gotten at large selections and teams with ranks poorer than 57 never have.

It is possible to match up teams' end-of-season scores for all of the factors with the factor standards to produce a table like the following one for the 2024 season.  There is an explanation following the table.

The table includes all NCAA RPI Top 57 teams that were not conference champion Automatic Qualifiers.  It is limited to the Top 57 since no team ranked poorer than #57 has ever gotten an at large selection.

NCAA Seed or Selection: This column shows the Committee decision for each team:

 1, 2, 3, and 4 are for 1 through 4 seeds

4.5, 4.6, 4.7, and 4.8 are for 5 through 8 seeds

6 is for unseeded teams that got at large selections

7 is for unseeded teams that did not get at large selections

8 is for teams disqualified from at large selection due to winning percentages belows 0.500

Green is for at large selections and red is for not getting at large selections.

NCAA RPI Rank for Formation:  This is teams' NCAA RPI ranks.

At Large Status Based on Standards:  This column is based on the two grey columns on the right.  The first grey column shows the number of at large "In" factor standards a team has met and the second shows the number of at large "Out" standards for the team.  In 2024, the Tournament had 34 at large openings.  Counting down the "In" and "Out" columns, there were 31 teams that met at least 1 "In" standard and 0 "Out" standards.  In the At Large Status Based on Standards column, these teams are marked "In" and color coded green.  This means the standards identified 31 teams to get at large selections, leaving 3 additional openings to fill.  Counting down further, there were 5 teams that met 0 "In" and 0 "Out" standards.  This means those teams could not be definitively ruled 'In" but also could not be definitively ruled "Out."   This means those 5 teams should be "Candidates" for the 3 remaining openings.  They are marked "Candidate" and color coded yellow.  And counting down further are teams that met 0 "In" standards and at least 1 "No" standard.  This means the standards identified those teams as not getting at large selections.  They are marked "Out" and color coded red.

Supplementing the NCAA Tournament At Large Factor Standards with a Tiebreaker

If you look at the first table in this article, you will see that the NCAA RPI Rank and Top 50 Results Rank paired factor is the best individual indicator of which teams will get at large selections. After applying the factor standards method described above, it is possible to use Candidate teams' scores for this factor as a "tiebreaker" to decide which of those teams should fill any remaining at large openings.

The following table adds use of this tiebreaker to the factor standards method for the 2024 season:

In the table, the "NCAA RPI Rank and Top 50 Results Rank As At Large Tiebreaker" column shows teams' scores for that factor,  The lower the score, the better.  In the At Large Status Based on Standards and Tiebreaker Combined column, the "In" green cells are for teams that get at large selections based on the Standards plus those teams from the Candidates that get at large selections based on the Tiebreaker.  If you compare these to the actual NCAA Seeds or Selections on the left, you will see that the Standards and Tiebreaker Combined at large selections match the actual selections for all but 1 at large position.

In relation to the power of the RPI in directing Committee decisions, it is important to note that the Tiebreaker is based on teams' NCAA RPI Ranks and their ranks based on their Top 50 Results.  Teams' Top 50 Results scores come from a scoring system I developed based on my observations of Committee decisions.  The scoring system awards points based on good results -- wins and ties -- against opponents ranked in the NCAA RPI Top 50, with the awards depending on the ranks of the opponents and heavily slanted towards good results against very highly ranked opponents.  Since the Top 50 Results scores are based on opponents' NCAA RPI ranks, even this part of the Tiebreaker is NCAA RPI dependent.

The following table adds to the preceding tables an At Large Status Based on NCAA RPI Rank column to give an overall picture of how the different "selection methods" compare -- the Committee method, the NCAA RPI rank method, and the Standards and Tiebreaker Combined method.

 Summary Data

The following table shows a summary of the data for each year:

 In this table, I find the color coded information at the bottom in the High, Low, Average, and Median rows most informative.  The information in the green column shows the difference between what the Committee has decided on at large selections over the years as compared to what the decisions would have been if the NCAA simply used the NCAA RPI.  The information in the salmon column shows what the difference would have been -- about 1 1/3 positions per year, with a median of 1 -- if the NCAA used a more refined method than the NCAA RPI, but one stilll very heavily influenced by the NCAA RPI.

Altogether, the numbers suggest that the NCAA RPI exerts an almost determinative influence on which teams get NCAA Tournament at large positions.  This does not mean the Committee members think that is the case, they may believe that they are able to value other factors as much as or even more than the NCAA RPI.  But whatever the individual members think, the numbers suggest that the Committee as a whole is largely under the thumb of the NCAA RPI.

Given the fundamental flaws of the NCAA RPI, as discussed in 2025 Articles 1 and 2, the near-determinative power of the NCAA RPI in the NCAA Tournament at large selection process is particularly disturbing.