Tuesday, September 9, 2025

2025 ARTICLE 17: RPI REPORT AFTER WEEK 4 GAMES

Below are my weekly tables showing predicted end-of-season rankings for teams, conferences, and regions based on the actual results of games played through Week 4 of the season and predicted results of games not yet played.  We are about a third of the way into the season, so the numbers remain pretty speculative.  Please note that the tables include NCAA RPI, my Balanced RPI, and Massey rankings.  As soon as KPI rankings are available, most likely next week, they also will be in the tables.

For those with a serious interest in the numbers, I sugest you download the 2025 RPI Report After Week 4 Excel workbook, which should be easier to use than the tables below.

As I have written previously, in the tables, it is worthwhile to look at the differences between teams' NCAA RPI ranks and their ranks as Strength of Schedule contributors, both for the teams themselves and for their opponents.  If you spend time looking through these differences on the Teams, Conferences, and Regions tables, you should be able to see the NCAA RPI's discriminatory patterns.

TEAMS


CONFERENCES


REGIONS



Tuesday, September 2, 2025

2025 ARTICLE 16: RPI REPORT AFTER WEEK 3 GAMES

Below are my weekly tables showing predicted end-of-season rankings for teams, conferences, and regions based on the actual results of games played through Week 3 of the season (including Labor Day, September 1) and predicted results of games not yet played.  Since we still are not very far into the season, the numbers remain pretty speculative.  Please note that the tables include NCAA RPI, my Balanced RPI, and Massey rankings.  Later in the season when KPI rankings are available, they also will be in the tables.

For those with a serious interest in the numbers, I sugest you download the 2025 RPI Report After Week 3 Excel workbook, which should be easier to use than the tables below.

Something I am watching very closely is the percentages, by region, of in-region games that are ties.  I watch this because, as discussed previously, the NCAA RPI formula punishes regions with high levels of parity, one measure of which is the percentage of in-region games that are ties.  This is a problem the Committee made a little worse last year when it changed the value of ties in the RPI formula's calculation of a team's winning percentage from half a win to a third of a win.  You can find the percentage of in-region ties for each region on the Regions page at the far right of the table.  So far this year, it looks like the West and Middle regions will have extraordinarily high levels of in-region ties, significantly higher than for the North and South regions.

The other things to look at on the tables, at this point, are the differences between teams' NCAA RPI ranks and their ranks as Strength of Schedule contributors, both for the teams themselves and for their opponents.  If you spend time looking through these differences on the Teams and Conference pages, you should be able to see for yourself the NCAA RPI's discriminatory patterns.

TEAMS



CONFERENCES


REGIONS



Tuesday, August 26, 2025

2025 ARTICLE 15: RPI REPORT AFTER WEEK 2 GAMES

In this week's RPI Report, I want to point out two sets of data that the Women's Soccer Committee and coaches should be concerned about.  Following that discussion, I'll post the regular weekly team, conference, and region tables.  For those who want to go directly to the weekly tables, here is a link to this week's Excel workbook 2025 RPI Report After Week 2.  (For those seriously interested in the current RPI Report information, I recommend downloading the workbook in an Excel format rather than using the weekly tables as reproduced below.)

Issue of Concern #1: Distribution of In-Region Ties, by Region



This table shows, for each of the four regional playing pools, the percentage of in-region games that are ties.  The percentage is based on actual ties to date and tie likelihoods for games not yet played.  The percentages in the table may be a little higher than they will be at the end of the season, but the placement of the regions in the table is consistent with what the placement has been historically.

Here is the problem the Committee and coaches should be concerned about:

As I have demonstrated elsewhere, the NCAA RPI historically has discriminated against teams from the West region, meaning that in games against teams from other regions, the West region teams on average have performed better than their ratings say they should have performed, i.e., are underrated.  At the other end of the spectrum, the South region teams on average have performed more poorly than their ratings say they should have performed, i.e., are overrated.  This doesn't mean all teams from the regions are under- or overrated, rather that on average teams from the regions are under- or overrated.

This year, the Committee made an NCAA RPI formula change that relates to this:  In the computation of a team's Winning Percentage (half the effective weight of a team's NCAA RPI rating), the Committee reduced the value of a tie from half of a win to one-third of a win.  In other words, it devalued ties.  Since it is reasonable to assume that regions with higher parity will have a higher proportion of ties, the effect of the change is to devalue the ratings of teams from regions with a high level of parity.

Also as I have demonstrated elsewhere, the West region historically has had the highest level of parity, followed by the Middle, then the North, and then the South.  The historic proportion of in-region games that are ties is consistent with this, as are the proportions for this year in the above table.

Thus the effect of the change likely will be to devalue the ratings of teams from the West, followed by the Middle, to the benefit of teams from the North and especially from the South.  This means that with the NCAA RPI already discriminating against teams from the West, the Committee's change has made the discrimination even worse.

The extent to which the Committee has made the change worse is not large, as indicated by the following table:


This table summarizes how the rating systems perform at rating teams from a region in relation to teams from the other regions.  It draws from the following data:

1.  For each region, its teams' actual winning percentages in games against teams from the other regions.

2.  For each region, its teams' expected winning percentages against teams from other regions based on teams' ratings as adjusted for home field advantage. [NOTE: Expected winning percentages are based on the rating differences between opponents as adjusted for home field advantage.  The expected winning percentages are calculated using result probability tables derived from the results of all games played since 2010 and are highly reliable when applied across large numbers of games.]

3.  The difference, for each region, between its actual winning percentage and its expected wining percentage.  Teams from regions with higher actual winning percentages are outperforming their ratings, in other words are underrated (discriminated against), whereas teams from regions with lower winning percentages are overrated.

In the table, the High column shows the actual v expected winning percentage difference for the region whose teams most outperform their ratings when playing teams from other regions -- the West region.  The Low column shows the difference for the region whose teams most underperform their ratings -- the South region.  The Spread column shows the difference between the High and Low, which is a measure of the extent of the rating system's discrimination among regions.  The Over and Under Total column shows the amount, for all four regions, by which the rating system misses a perfect match (no difference) between actual and expected winning percentage, which is another measure of the system's performance.

As you can see from the table's comparison of the two systems, although the difference between the systems' performance is small, the Committee's 2024 change increased the discrimination among regions.

 So, what was the Committee's rationale for the change?  It gave two reasons: (1) Valuing ties as 1/3 of a win matches how conferences compute in-conference standings and how leagues compute standings in the larger soccer world; and (2) The Division 1 men have made the change.

Neither rationale holds water.  (1) For conferences and leagues, for in-conference and in-league standings, there is no issue of whether different playing pools have different levels of parity.  Thus how they count ties, when determining standings, is irrelevant to how to count ties in a national rating system for teams playing games in different geographic regions.  (2) That the men have made the change should be a reason ... Why?  That does not demonstrate in any way that it is a good change for the women.  The Committee should make its own decision based on how the change will affect the NCAA RPI as a rating system for the women's teams, not let the Men's Soccer Committee make the decision for them.

The question the Committee should have considered is whether the change makes the NCAA RPI a better system for rating teams properly in relation to each other when the teams are distributed around the nation and tend to play in different regional playing pools.  They should have asked the NCAA statistics staff about this, and the staff should have advised the Committee that if the geographic playing pools have different levels of parity, then the change will punish teams from the pools with high parity and benefit stronger teams from the pools with low parity.  Did the Committee ask the staff about this and did the staff give them this answer?  I don't know, but I doubt it.

Issue of Concern #2: Proportions of Out-of_Region Games

As I showed above, the NCAA RPI discriminates against some regions and in favor of others.  At one time, I thought this was due exclusively to differences in region strength and there not being a high enough proportion of out-of-region games for the system to work properly on a national basis.  As it turns out, however, both Massey's and my Balanced RPI rating systems show only a small amount of discrimination in relation to region strength, and much less than the NCAA RPI.  In other words, there are enough out-of-region games for those systems to avoid all but a little discrimination in relation to region strength.  So the NCAA RPI's problem has not been driven mainly by there not having been enough out-of-region games (rather, as it has turned out, it is driven by how the NCAA RPI computes Strength of Schedule).

On the other hand, it is indisputable that there have to be "enough" out-of-region games for any rating system to work on a national basis.  As the following tables show, no doubt due to the changing college sports economic landscape, this year there is a substantial reduction in the proportion of out-of-region games from what the proportion has been in the past (based on teams' published 2025 schedules, adjusted to take canceled games to date into account):


This represents a 28.0% across the board reduction in the proportion out-of-region games.  For the Middle region, the reduction is 18.0%, for the North 28.5%, for the South 30.2%, and for the West 31.7%.  Given these reductions, a major question is whether they will significantly impair the ability of the NCAA RPI -- or any other rating system -- to properly rate teams on a national basis.  This is a question the Committee and coaches should be thinking about.

Weekly Team, Conference, and Region Tables

The following tables are based on the actual results of games played through Sunday, August 24, and predicted result likelihoods for games not yet played.

TEAMS



CONFERENCES



REGIONS




Tuesday, August 19, 2025

2025 ARTICLE 14: RPI REPORT AFTER WEEK 1 GAMES

In 2025 Article 7 and 2025 Article 8, I described how I assign pre-season NCAA RPI ratings and ranks to teams and then, assuming those ratings and ranks represent true team strength, apply them to teams' schedules to generate predicted end-of-season NCAA RPI ratings and ranks.  Once I have done that, at the end of each week of the season I replace that week's predicted results with games' actual results.  Then, using those actual results combined with predicted results for the balance of the season, I generate new predicted end-of-season NCAA RPI ratings and ranks.  After completing week 5 of the season, I will switch from using assigned pre-season NCAA RPI ratings and ranks as the basis for predicting future results to using the then actual NCAA RPI ratings and ranks as the basis.

Using this process, the predicted end-of-season NCAA RPI ratings and ranks are very speculative at the beginning of the season.  However, as each week passes, they become progressively closer to what the actual end-of-season ratings and ranks will be.  By the last few weeks of the season, they become helpful when trying to figure out what results teams need in their remaining games in order to get particular NCAA Tournament seeds or at large selections.

Today's report shows where things are with Week 1's actual results incorporated into the end-of-season predictions.  The report has a page for teams, for conferences, and for geographic playing pool regions.  You can download the report as an Excel workbook with this link: 2025 Week 1 RPI Report.  The same information also is set out in tables below, but I recommend downloading the workbook as it likely will be easier to use.  (If using the tables below, scroll to the right to see additional columns.)

This year, an emphasis in these reports is on showing why the NCAA RPI, because of how it measures the opponents' strengths of schedule that it incorporates into its formula, discriminates against or in favor of particular teams, conferences, and regions.

TEAMS

This page shows, for each team:

Team name

Geographic playing pool region

Conference

If the team is predicted to be its conference's NCAA Tournament automatic qualifier (AQ)

If the team is predicted to be disqualified from an NCAA Tournament at large selection due to having more losses than wins (1)

Team's 

NCAA RPI rank (based on past history, a key factor in selecting teams that will be in the NCAA Tournament #1 through #4 seed pods)

rank as a strength of schedule contributor to opponents under the NCAA RPI formula

Opponents'

average NCAA RPI rank 

average rank as strength of schedule contributors under the NCAA RPI formula

Conference opponents' 

average NCAA RPI rank 

average rank as strength of schedule contributors under the NCAA RPI formula

[NOTE: Teams have relatively little control over this part of their schedules.] 

Non-Conference opponents' 

average NCAA RPI rank

average rank as strength of schedule contributorsl under the NCAA RPI formula

[NOTE: Teams control this part of their schedules, to some extent.  Geographic factors such as travel expenses, available opponents, and other factors can be limiting considerations.]

NCAA RPI Top 50 Results Score

NCAA RPI Top 50 Results Rank (based on past history, a key factor in NCAA Tournament at large selections and in selecting teams that will be in the #5 through #8 seed pods)

Similar rank and strength of schedule contributor rank numbers under the Balanced RPI

KPI rank if available

Massey rank


 

CONFERENCES

This page shows, for each conference:

Conference name

Conference's NCAA RPI rank

Teams' 

average NCAA RPI rank 

average rank as strength of schedule contributors under the NCAA RPI formula 

 Opponents' 

average NCAA RPI rank 

average rank as strength of schedule contributors under the NCAA RPI formula

Conference opponents' 

average NCAA RPI rank 

average rank as strength of schedule contributors under the NCAA RPI formula

Non-Conference opponents' 

average NCAA RPI rank 

average rank as strength of schedule contributorsl under the NCAA RPI formula

Conference's Non-Conference RPI rank 

Similar rank and strength of schedule contributor rank numbers under the Balanced RPI

KPI rank if available

Massey rank


 

REGIONS

This page shows, for each region:

Region name

Number of teams in region 

Region's NCAA RPI rank

Teams' 

average NCAA RPI rank 

average rank as strength of schedule contributors under the NCAA RPI formula 

Opponents' 

average NCAA RPI rank 

average rank as strength of schedule contributors under the NCAA RPI formula

Region opponents' 

average NCAA RPI rank 

average rank as strength of schedule contributors under the NCAA RPI formula

(NOTE: Due to budget limitations, teams may be compelled to play all or most of their non-conference games against opponents from their own geographic regions.] 

Non-Region opponents' 

average NCAA RPI rank 

average rank as strength of schedule contributorsl under the NCAA RPI formula

Similar rank and strength of schedule contributor rank numbers under the Balanced RPI

KPI rank if available

Massey rank

Regions' proportions of games played against teams from each region (NOTE: This years, the numbers of out-of-region games are down about 30% from past patterns.  This may result in a significant degradation of the NCAA RPI's already impaired ability to properly rate teams from a region in relation to teams from other regions.)

Proportion of in-region games that are ties (as a measure of in-region parity) (NOTE: The NCAA RPI, because of how it measures Strength of Schedule, on average discriminates against teams from regions with higher region parity.)


 

Friday, August 1, 2025

2025 ARTICLE 13: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 6, GEOGRAPHIC REGIONS IN RELATION TO NCAA RPI RANKS AND STRENGTH OF SCHEDULE RANKS

This article, for the geographic regions within which the teams from each state play most of their games, provides information similar to that provided for conferences in 2025 Article 12.  A map showing the four regions is at the RPI for Division I Women's Soccer RPI: Regional Issues page.


As you can see, when averaged across a region, the differences between average NCAA RPI ranks and average Strength of Schedule contributor ranks under the NCAA RPI formula are relatively small.  This makes sense, since each region has an array of strong and weak teams and conferences.  As a generalization, however, looking at the numbers for the regions' teams opponents, overall and on average teams from the West region are discriminated against due to the way the NCAA formula computes Strength of Schedule, the Middle region experiences no impact, and the North and South regions are benefitted by discrimination.

To be clear, there are teams and conferences from all of the regions that the NCAA RPI formula discriminates against and in favor of.  The numbers above simply show the net effect of the discrimination for each region.

A particular concern this year is a significant reduction in out-of-region competition, most likeky due to less funding being available for travel.  The following table shows the extent of the reduction looking at the nation as a whole:


As you can see, the number of out-of-region games will be reduced by 28.1% from what the number historically has been.

A break down of the numbers from the preceding table by region shows reductions in the number of out-of-region games as follows:

Middle  18.3%

North  28.5%

South  30.0%

West  31.7%

These reductions should be a concern for the Women's Soccer Committee.  The NCAA RPI already has a problem ranking teams dispersed among the conferences and across the regions within a single national system.  The reductions in out-of-region play are likely to make the problem worse. 

 


2025 ARTICLE 12: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 5, CONFERENCES IN RELATION TO NCAA RPI RANKS AND STRENGTH OF SCHEDULE RANKS

 In 2025 Pre-Season Predictions and Information, Parts 4 and 4B, for the individual teams I showed the relationship between predicted NCAA RPI ranks and Strength of Schedule Contribution ranks under the NCAA RPI formula, both for the individual teams and for their opponents.  In this article, I will show the same information, but for each conference.  This gives a good picture of how the NCAA RPI discriminates among conferences because of the defective way it calculates Strength of Schedule.

This table has the conferences in NCAA RPI rank order, based on the average rating of their teams.  See below the table for comments.


In the table, the first two green-highlighted columns on the left show, for each conference, the difference between its teams' average NCAA RPI rank and its teams' average Strength of Schedule contributor rank under the NCAA RPI formula.  As you read down the table from the strongest conferences at the top to the weakest at the bottom, you can see the clear pattern: For stronger conferences, the conference teams' Strength of Schedule contributor ranks are poorer than the teams' actual ranks say they should be; and for weaker conferences they are better than they should be.

The next two salmon-highlighted columns look at how this plays out for the conference teams' schedules.  The first of those columns shows the conferences' teams' opponents' average ranks and the second column shows those opponents' average ranks as Strength of Schedule contributors.  The pattern here is the same:  Stronger conferences' opponents' Strength of Schedule Contributor ranks are poorer than the opponents' actual ranks say they should be; and the opposite is true for the weaker conferences.

The next four columns break the numbers for the conference teams' schedules down into conference opponents (green-highlighted) and non-conference opponents (salmon-highlighted).  Given that in conference play, the conferences' teams are playing each other, it is no surprise that the contrasts between the conference opponents' NCAA RPI ranks and their ranks as Strength of Schedule contributors follow the same basic pattern.  For the non-conference opponents, where the individual teams have more control over their schedules, the pattern is similar but less extreme and with a little more variability.

It is important here to point out that coaches in top tier and most coaches in middle tier conferences are aware of these patterns and often take them into consideration in their non-conference scheduling.  They also are aware, however, that in the NCAA Tournament seeding and at large selection processes, good results against highly ranked opponents matter, including against highly ranked non-conference opponents.  Further, coaches of teams with NCAA Tournament aspirations often want to play at least some strong non-conference opponents.  This means that they sometimes decide to schedule opponents whose Strength of Schedule contributions are likely to be poorer than their RPI ranks say they should be, essentially deciding to take a potential RPI "hit" in exchange for the potential of a good result against a highly ranked opponent.

NOTE:  Being aware of the scheduling dilemma I just described, I designed my Balanced RPI, which is a modification of the NCAA RPI, with the specific objective of eliminating the difference between teams' ranks and their ranks as Strength of Schedule contributors.  Thus under the Balanced RPI, if a team has a rank of X, that also is either exactly or very close to exactly the team's rank as a Strength of Schedule contributor.  In other words, if the NCAA were to use the Balanced RPI, coaches no longer would have this scheduling dilemma.   (As an additional benefit, the RPI no longer would discriminate among conferences in relation to conference strength.)

Thursday, July 31, 2025

2025 ARTICLE 11: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 4B, TEAMS' SCHEDULES IN RELATION TO OPPONENTS' NCAA RPI RANKS AND STRENGTH OF SCHEDULE RANKS

In Part 4, I discussed and showed the differences between teams' NCAA RPI ranks and their ranks as Strength of Schedule contributors under the NCAA RPI formula.  In this article, I will show predictions for how these differences will affect teams by the end of the 2025 season.

For each team, the following table shows its predicted:

Opponents' average NCAA RPI rank

Conference opponents' average NCAA RPI rank

Non-conference opponents' average NCAA RPI rank

Opponents' average rank as Strength of Schedule contributors under the NCAA RPI formula

Conference opponents' average rank as Strength of Schedule contributors under the NCAA RPI formula

Non-conference opponents' average rank as Strength of Schedule contributors under the NCAA RPI formula

These numbers allow you to see how the NCAA RPI rank versus Strength of Schedule contributor rank differences relate to:

1.  Teams' in-conference schedules, which teams basically can't control;

2.  Teams' non-conference schedules, which teams can control at least to some extent; and

3.  Teams' overall schedules.

If you review the table's numbers with a view to the strength of the teams' conferences, you will see that generally speaking the NCAA RPI formula understates the strengths of schedule of top tier conferences' teams, gets the strengths of schedule of middle tier conferences' teams about right, and overstates the strengths of schedule of bottom tier conferences' teams.  I've arranged the teams by conference so you can better see how this NCAA RPI defect affects teams by conference.  Scroll to the right, if necessary, to see the entire table.

NOTE: The differences in the Conference Opponents Average Rank column for teams from the same conference are primarily due to conference teams not playing full round robins.  The differences in the Non-Conference Opponents Average Rank column for teams from the same conference are due the different teams' non-conference scheduling strategies.

I'll use Baylor, from the Big 12, with a predicted NCAA RPI rank of #66, and Lamar, from the Southland, with a predicted NCAA RPI rank of #55, as examples.  I've chosen these teams because no team ranked poorer than #57 ever has gotten an at large position in the NCAA Tournament.  Thus Baylor is outside the historic at large candidate group and Lamar is within the candidate group.

Baylor (Big 12): 

Conference opponents' average NCAA RPI rank is 81 and conference opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 119.

Non-conference opponents' average NCAA RPI rank is 110 and non-conference opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 108.

Overall, opponents' average NCAA RPI rank is 93 and opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 115.

Thus Baylor's Strength of Schedule component of the NCAA RPI significantly discriminates against Baylor in relation to its conference schedule and only barely offsets that discrimination in relation to its non-conference schedule.  The overall result is that the Strength of Schedule component significantly discriminates against Baylor.

Lamar (Southland):

Conference opponents' average NCAA RPI rank is 212 and conference opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 185.

Non-conference opponents' average NCAA RPI rank is 153 and non-conference opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 184.

Overall, opponents' average NCAA RPI rank is 193 and opponents' average Strength of Schedule contributor rank under the NCAA RPI formula is 185.

Thus the Strength of Schedule component of the NCAA RPI significantly discriminates in favor of Lamar in relation to its conference schedule and offsets that discrimination some in relation to its non-conference schedule.  The overall effect, however, is that the Strength of Schedule component still discriminates in favor of Lamar.

Given that Baylor is outside but in the vicinity of the ranking area of teams that historically are candidates for NCAA Tournament at large selections and Lamar is only a little inside that ranking area, this demonstrates the importance of this NCAA RPI defect.  History suggests that Lamar, if not an Automatic Qualifier, would not get an at large selection.  For Baylor, however, being outside the historic candidate area, there is a question whether, if inside the candidate area and considered by the Committee, it might displace one of the "last in" at large teams.  In other words, this NCAA RPI defect may have negative NCAA Tournament at large selection consequences.  (And, by a similar analysis of seeding candidate groups, may have negative seeding consequences.)

The significance of this kind of example is reinforced if you consider Lamar's and Baylor's ranks using my Balanced RPI.  The Balanced RPI is a rating system that builds on the RPI, with modifications that fix the NCAA RPI's defective discrepancy between teams' NCAA RPI ranks and their ranks as Strength of Schedule contributors under the NCAA RPI formula.  The Balanced RPI's predicted rank for Lamar is 110, well outisde the NCAA Tournament at large selection candidate range.  For Baylor, its predicted rank is #57, in other words a candidate for at large selection.





2025 ARTICLE 10: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 4, TEAMS' NCAA RPI RANKS COMPARED TO THEIR RANKS AS STRENGTH OF SCHEDULE CONTRIBUTORS

The NCAA RPI has a major defect, which is the way in which it computes a team's strength of schedule.

As discussed on the RPI: Formula page at the RPI for Division I Women's Soccer website, the NCAA RPI has two main components:  a team's Winning Percentage and its Strength of Schedule.  Within the overall NCAA RPI formula, the effective weights of the two components are approximately 50% Winning Percentaqge and 50% Strength of Schedule.

Within the NCAA RPI formula, in turn, Strength of Schedule consists of two elements: the average of a team's opponents' winning percentages (OWP) and the average of a team's opponents' opponents' winning percentages (OOWP).  And, within Strength of Schedule, the effective weights of these two elements are 80% opponents' winning percentage and 20% opponents' opponents' winning percentage.  Thus for the NCAA RPI's Strength of Schedule component, a team's opponents' winning percentages matter a lot and against whom they achieved those winning percentages matters little.  This is a major defect.

In this and the next two parts of my Pre-Season Predictions and Information, using end-of-season predictions for the 2025 season, I will show how the NCAA RPI's strength of schedule defect plays out for teams (this Part 4), for conferences (Part 5), and for geographic regions (Part 6).

The following table shows, for each team, its predicted end-of-season NCAA RPI rank and its predicted rank as a strength of schedule contributor under the NCAA RPI formula.  In a good rating system, these ranks should be the same or, at least, very close to the same.  As the table shows, however, for the NCAA RPI formula, for many teams, the ranks are not close to the same.

Using some of the top teams in the alphabetical list as examples:

If Team A plays Air Force as an opponent, Team A will have played the NCAA RPI #232 ranked team.  When computing Team A's rating and rank, however, the NCAA RPI formula will give team A credit only for playing the #274 team.

On the other hand, if Team A plays Alabama State, Team A will have played the #340 team.  But when computing Team A's rating and rank, the NCAA RPI formula will give Team A credit for playing the #277 team.

Thus although the NCAA RPI ranks Air Force and Alabama State 108 rank positions apart, when considering each of their strengths for purposes of Team A's strength of schedule computation, the NCAA RPI treats Air Force and Alabama State as roughly equal.

You can scroll down the table and see how this NCAA RPI formula defect plays out for teams you are interested in,  I suggest you look, in particular, at teams in the middle to lower levels of top tier conferences and at teams in the upper levels of middle and bottom tier conferences.  For example:

Look at Alabama:  Its predicted NCAA RPI rank is #37.  But, its predicted rank as a strength of schedule is only #89.

Then look at Bowling Green:  Its predicted NCAA RPI rank is #86 but its predicted rank as a strength of schedule contributor is #26.

These kinds of differences have significant practical implications related to scheduling and the NCAA Tournament.  Teams' NCAA RPI ranks are a key factor in the Women's Soccer Committee's decisions on Tournament seeds and at large selections.  So, if a coach has NCAA Tournament aspirations, from strictly an NCAA RPI perspective, Bowling Green would be a significantly better opponent to play than Alabama.  This would be true for two reasons: (1)  Bowling Green probably is weaker than Alabama, so an easier game in which to get a good result; and (2) Bowling Green, as an opponent, will give the coach's team's NCAA RPI a better strength of schedule contribution than Alabama.

Thus when doing non-conference scheduling, coaches with NCAA Tournament aspirations or with other concerns about where their teams will finish in the NCAA RPI rankings must take this NCAA RPI formula defect into account.  In essence, they are in the position of having to learn how to "trick" the NCAA RPI through smart scheduling -- in the example, choosing Bowling Green rather than Alabama as an opponent. 




Wednesday, July 30, 2025

2025 ARTICLE 9: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 3, "PREDICTED" CONFERENCE REGULAR SEASON AND TOURNAMENT CHAMPIONS

Continuing with "predictions," using the "results likelihood" method described in 2025 Article 9, my system uses the same "3 points for a win and 1 for a tie" scoring that conferences use for their standings to create team standings within each conference.  It is worth noting that the results likelihoods take game locations into account and that a good number of conferences do not play full round robins.

Using the ACC as an example, here are what its "predicted" end-of-season standings look like:


Although Florida State's and North Carolina's points look the same, that is due to rounding.  Florida State's are slightly higher.  My interpretation of these standings is that it will be very close at the top of the conference among Florida State, North Carolinam Stanford, and Duke, with Virginia also in the mix.

Using the conference standings and the conference tournament formats (as published to date), my system next creates conference tournament brackets.  Then, since it is necessary to have winners and losers to fill out the entire tournament brackets, the system assigns as a game winner any team that has a win likelihood above 50%.  Where neither team has a win likelihood above 50%, the system treats the game as a tie.  For the tiebreaker, the advancing team is the one with the higher win likelihood.

This process results in the following conference regular season and conference tournament champions.  In most cases they are the same, but in two cases they are different.




Monday, July 28, 2025

2025 ARTICLE 8: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 2, "PREDICTED" END-OF-SEASON RANKS AND RATINGS

Once I have assigned pre-season strength ranks and ratings to teams, I combine those with teams' schedules to "predict" where teams will end up at the end of the season following completion of the conference tournaments.  The process to do this requires background work:

1.  For each game, I calculate the game-location-adjusted rating difference between the teams, using their assigned pre-season strength ratings as the base.  Since the assigned strength ratings are based on what the average historic NCAA RPI ratings are for those teams' ranks, my game location adjustments increase the home team's rating by 0.0085 and decrease the away team's rating by 0.0085, for an overall adjustment of 0..0170.  This is the value of home field advantage for the current version of the NCAA RPI with the "no overtime" rule in effect.

2.  For the location-adjusted rating difference between the teams, I calculate each team's expected win, loss, and tie likelihoods.  These likelihoods are based on a study of the location-adjusted rating differences and results of all games played since 2010 (excluding 2020).

[NOTE: For a detailed explanation of how I determine the game location adjustment and the win, loss, and tie likelihoods, go to the RPI for Division I Women's Soccer website's page RPI: Measuring the Correlation Between Teams' Ratings and Their Performance

3.  Rather than assigning the opponents in a game either a win, a loss, or a tie result, I assign each team its win, loss, and tie likelihoods since these will give a better picture of what a team's overall record will be given its entire schedule.  As an example, Colorado and Michigan State will play on August 14 at Colorado.  Their location-adjusted rating difference is 0.0333 in favor of Colorado.  For that rating difference, referring to a result probability table for the current NCAA RPI in a "no overtime" world, Colorado's result likelihoods are 53.6% win, 19.9% loss, 26.4% tie (which don't quite add up to 100% due to rounding).  Those numbers go into Colorado's win-loss-tie columns for NCAA RPI computation purposes.  Michigan State's win-loss percentages are the converse.  I assign these likelihoods for all teams' games, add up each team's percentages, and convert them from percentages to numbers .  Thus Colorado ends the season with 12.2 wins, 4.9 losses, and 4.9 ties.  These are the numbers I use for Colorado as its wins, losses, and ties when computing its NCAA RPI.

4.  I then compute all teams' NCAA RPI ratings and ranks.  It is important to understand that these are different than the teams' assigned pre-season strength ratings and ranks.  This is because the NCAA RPI does not measure team strength.  Rather, it measures team performance based on a combination of teams' winning percentages and their strengths of schedule (as measured by the NCAA RPI formula).  Thus two teams with identical strength ratings and ranks will end up with different NCAA RPI ratings and ranks if they have different winning percentages and/or different strengths of schedule.  Below are my computed end-of-season (including predicted conference tournaments) ratings and ranks for teams.  You can compare the ranks to the ones in the preceding post to see the differences between teams' assigned pre-season strength ranks and team' predicted end-of-season NCAA RPI ranks.  (NOTE: I have corrected these rankings since their initial publication to fix a programming error.  The changes are relatively minor.)





2025 ARTICLE 7: 2025 PRE-SEASON PREDICTIONS AND INFORMATION, PART 1, ASSIGNED PRE-SEASON RANKS AND RATINGS

INTRODUCTION

All the teams have published their schedules -- almost, with Akron yet to publish its non-conference schedule and Delaware State yet to publish, but unless those teams play each other, we can determine their schedules from what others have publlished.  So for practical purposes we have the all the schedules.  This makes it possible to do pre-season predictions of where teams will end up at the end of the regular season, including conference tournaments.

Pre-season predictions involve a lot of assumptions that may or may not prove correct.  Also, my prediction method depends entirely on teams' rank histories.  So don't take the pre-season predictions too seriously.  On the other hand, this series of articles will have educatational value, particularly about how the NCAA RPI works and about the importance of scheduling.  So I recommend not getting preoccupied with the details of the predictions but instead watch for what you can learn about the NCAA RPI and its interaction with teams' schedules.

ASSIGNED PRE-SEASON RANKS AND RATINGS

The first step in my process, which is the subject of this article, is to assign pre-season ratings and ranks to teams.  In essence, this is predicting teams' strength.

There are a lot of ways to predict team strength, some complex and some simple.  I predict strength using only teams' rank histories, without reference to changing player and coaching personnel.  There are others who do predictions using those kinds of detailed information about this year's teams -- conference coaches for their own conferences and in the past, but not currently, stats superstar Chris Henderson.  In the past, their predictions have been better than mine, but only slightly better.  So you can consider my predictions as somewhat crude but close to as good as you can get when making data-based predictions for the entire cast of teams.

When using only teams' rank histories, my analyses show that the best predictor of where teams will end up next year is the average of their last 7 years' ranks using my Balanced RPI, which is a modified version of the NCAA RPI that fixes its major defects.  There is a problem, however, with that predictor.  If a team has a major outlier year -- a much higher or lower rank than is typical -- using a 7-year average can significantly mis-rate that team.  On the other hand, if I use the median rating over the last 7 years, it avoids that problem.  It is a little less accurate as a predictor for where teams will end up next year, but not by much.  So for my predictor, I use teams' median Balanced RPI ranks over the last 7 years.

Once I have assigned teams' ranks, I then assign NCAA RPI ratings to the teams.  To do this, I have determined the historic average rating of teams at each rank level.  When I do this, however, I have to take into account the recent NCAA "no overtime" rule change and the 2024 NCAA RPI formula changes.  So when I determine historic average ratings, I use what past ratings would have been if the "no overtime" rule and 2024 NCAA RPI formula had been in effect, using the years 2010 to the present (but excluding Covid-affected 2020).

This produces the following "strength" ranks and ratings for teams.  You will note that no team has a #1 assigned rank and some teams have the same assigned rank.  This is because I am using teams' 7-year median ranks.  (Scroll to the right to see additional teams.)





Monday, March 10, 2025

2025 ARTICLE 6: THE KP INDEX (KPI): THE COMMITTEE'S MISTAKE AND THE BIG 10'S OWN GOAL

 From Report of the Women's Soccer Committee December 9, 2024 and January 29, 2025 Meetings:

"KPI, and other ranking systems.  The Committee felt the KPI should be used more in selections as a valued tool in the process.  The Committee reviewed the Massey Ratings and decided not to request it as an additional criterion at this point."

Thus, as it has done for the past two years, the Committee intends to supplement the NCAA RPI with the KPI as a system for ranking teams.  It intends to use the KPI more, however, than it has in the past.  This is a change from a year earlier, when the Committee reported that it had found the KPI not useful and proposed use of Massey beginning in 2026.

The Committee for a while has pushed for use of more than just the NCAA RPI for team ratings and rankings.  The Committee two years ago finally received approval to use the KPI.  After it received that approval, I asked a Committee member where the decision to use the KPI, as the particular system approved for use, came from.  The member advised me it did not come from the Committee, so far as he recalled.  My assumption, therefore, is it came from the NCAA staff.

What Is the KPI? 

The KPI is a product of Kevin Pauga, the Associate Athletic Director for Strategic Initiatives and Conference Planning at Michigan State.  It appears the KPI is something he produces outside his formal work for Michigan State.  Pauga also is regarded as a scheduling expert, employed by the Big 10 and other conferences to produce conference schedules for their teams.  His scheduling system considers many parameters.  I do not know if it considers the relationship between teams' schedules and their NCAA RPI rankings.

According to a New York Times article dated March 26, 2015, Kevin Pauga started using the KPI for college basketball ratings and rankings in 2013.  According to the article, a KPI rating is a

"number that, essentially, correlates to how valuable a team’s wins are versus how damaging its losses are. The ratings run from negative-one (bad) to plus-one (good), and combine variables that might change over the course of a season. His goal is to quantify so-called good wins and bad losses, and to assess how teams compare with one another."

And, according to an NCAA release dated March 5, 2025, for NCAA men's basketball:

"The Kevin Pauga Index metric ranks team resumes by assigning a value to each game played. The best win possible is worth about +1.0, the worst loss about -1.0, and a virtual tie at 0.0.  Adjustments are made to each game's value based on location of the game, opponent quality and percentage of total points scored. Game values are added together and divided by games played to determine a team's KPI ranking."

Beyond these descriptions, it appears the KPI is proprietary, so I do not know exactly what its formulas are.

Is the KPI a Good Rating System for Division I Women's Soccer?

I don't know how well the KPI functions as a rating system for other NCAA sports.  For Division I women's soccer, however, it is a poor rating system.  It has the same defects as the NCAA RPI.

In 2025 Article 2, I explained my system for grading Division I women's soccer rating systems and did a detailed review of how the NCAA RPI performs as a rating system.  As I showed:

"Based on the ability of schedulers to "trick" the NCAA RPI and on its conference- and region-based discrimination as compared to what the Balanced RPI shows is achievable, the NCAA RPI continues to get a failing grade as a rating system for Division I women's soccer."

The following review of the KPI is similar to the review of the NCAA RPI in 2025 Article 2.  I will not go through the same detailed explanation of the grading system here as I gave in Article 2, so you might want to review Article 2 before proceeding further.

Ability of the System to Rate Teams from a Conference Fairly in Relation to Teams from Other Conferences



This table has the conferences arranged in order from those with the best KPI average rating from 2017 through 2024 at the top and those with the poorest at the bottom.  (KPI ratings are available only for years since 2017.)

In the table:

The Conference NonConference Actual Winning Percentage column shows the conference's actual winning percentage against non-conference opponents.  In calculating Winning Percentage, I use the NCAA RPI Winning Percentage formula in effect as of 2024.

The Conference NonConference Likelihood Winning Percentage column shows the conference's expected winning percentage against non-conference opponents, based on the differences between opponents' KPI ratings as adjusted for home field advantage.  The expected winning percentage for each game is determined using a Result Probability Table for the KPI.  The table comes from an analysis of the location-adjusted rating differences and the results of all games played since 2017.  This method for determining expected winning percentages is highly precise when applied to large numbers of games as shown by the following table for the KPI:

 

In this table, the Total Win, Tie, and Loss Likelihood columns show the expected wins, losses, and ties by the higher rated team, after adjustment for home field advantage, for all games since 2017.  The columns show these in absolute numbers and as a percentage of all games played.  The Total Actual Wins, Ties, and Losses columns show similar numbers, but based on the actual results.  As you can see, out of the more than 22,000 games played, the difference between the expected wins and actual wins is 11 games, between the expected ties and actual ties is  6 games, and between the expected losses and actual losses is 4 games.  In other words, as I stated, the Result Probability Table method is a highly precise way of determining expected winning percentages for the KPI.

Returning to the larger table above, the Conference NonConference Actual Less Likely Winning Percentage column shows the difference between the conference teams' actual winning percentages in non-conference games and their expected winning percentages based on their games' location-adjusted KPI rating differences.  A positive difference means the teams' actual winning percentages are better than expected based on the KPI and a negative difference means the teams' actual winning percentages are poorer. 


This chart is based on the conferences table.  In the chart, conferences are arranged in order of those with the highest average KPI ratings on the left to those with the poorest ratings on the right.  The vertical axis is for the difference between a conference's actual and its KPI-expected winning percentage.  As you can see from the chart, stronger conferences (on the left) tend to perform better than their KPI ratings say they should and weaker conferences (on the right) tend to perform more poorly.  In other words, the KPI underrates teams from stronger conferences and overrates teams from weaker conferences.  This is the same pattern the NCAA RPI has.  The downward sloping straight line is a trend line showing the pattern of the data; and the formula on the chart is a formula whose use can tell what the data indicate the actual v expected difference should be for any particular conference average KPI rating.


This table draws from the data underlying the chart and from the chart itself, as well as from similar data and charts for the NCAA RPI and the Balanced RPI.  It thus compares the KPI to the NCAA RPI and also includes the Balanced RPI (which is similar to what the Massey rating system would show), to show what a good rating system can do.

In the first two color coded columns, the "Spread" column shows the performance percentage difference between the conference that most outperforms what the KPI says its performance should be and the conference that most underperforms.  The "Under and Over" column shows the total amounts by which all conferences either outperform or underperform.  Both of these columns are measures of rating system fairness in relation to teams from the different conferences.  As you can see, the KPI and the NCAA RPI both do a poor job, as compared tto what a good rating system can do.

The color coded column on the right comes from the chart and shows the rating system pattern in relation to conference strength, using the chart's trend line formula.  It shows the extent of discrimination in relation to conference strength.  As the column shows, the KPI and NCAA RPI have similar discrimination based on conference strength, whereas a good system as exemplified by the Balanced RPI has virtually no discrimination.

Ability of the System to Rate Teams from a Geographic Region Fairly in Relation to Teams from Other Geographic Regions

The following table and chart are similar to those above for conferences, but are for geographic regions:





The table and chart for geographic regions show an overall pattern similar to that for conferences.  The data point pattern, however, is up and down enough that I do not have a high level of confidence in the relationship the chart shows between regions' average ratings and their actual performance as compared to their KPI-expected performance.  This is indicated by the R squared value I have included in the chart below the trend line formula.  The R squared value is a measure of how well the data fit the trend line, with an R squared value of 1 being a perfect fit and of 0 being no fit.  The R squared value on the chart indicates a mediocre fit.

Here is another table-chart set.  They is based on the relationship between the proportion of games in each region that are ties, as a measure of the amount of parity within the region.




This chart shows a similar patter in relation to region average KPI rating, but with a significantly better R squared value.

Altogether, the tables and charts for regions result in the following comparison table:



As the first two highlighted columns show, the KPI and the NCAA RPI have relatively similar levels of discrimination among conferences, in terms of general fairness.  And as the Balanced RPI row shows, this could avoided by using a better rating system.  And, as the two color coded columns to the right show, to the extent that the KPI discriminates in relation to region parity or region strength, its discimination is similar to that of the NCAA RPI.  Again, the Balanced RPI row shows this could be avoided by using a better rating system.

Ability of the System to Produce Ratings That Will Match Overall Game Results



This table simply shows the extent to which the higher rated team, after adjustment for home field advantage, wins, ties, and loses for the particular rating systems.  Thus it is a gross measure of how well game results correlate with the systems' ratings.  Since the NCAA RPI and Balanced RPI numbers are based on the No Overtime rule having been in effect for all seasons, whereas the KPI numbers for 2017 through 2021 use overtime games' results, the best comparison is in the highlighted column on the right, which disregards tie games.  As a point of reference, a difference of 0.1% represents a difference of roughly 3 games per year.  As you can see, the KPI gets the correct result about 15 times fewer per year (0.5%) than the NCAA RPI and about 39 times fewer (1.3%) than the Balanced RPI.


This is similar to the previous table but limited to games involving at least one team in the system's Top 60.  Again, the best comparison is from the color coded column on the right.  Here, a difference of 0.1% represents one game per year.  Again, the KPI performs more poorly than the NCAA RPI and than the Balanced RPI.

Conclusion About the KPI as a Rating System

As the above information shows, the KPI has the same problems as the NCAA RPI, with higher levels of discrimination among conferences and geographic regions than can be achieved by a good rating system.

So why would the NCAA staff agree to use of the KPI, if it is the NCAA staff that selected the KPI as an additional system as I suspect?  And why would the Women's Soccer Committee double down on the KPI as it intends to do this coming year?  The answer is simple: Because the KPI has the same problems as the NCAA RPI.  It will not rock the NCAA RPI boat, but rather will make the NCAA RPI look legitimate.  Whereas if the Committee were to use Massey, it would have significant differences from the NCAA RPI and thus would expose the NCAA RPI as a poor rating system.  Indeed, I suspect that the staff and Committee looked at Massey ratings as compared to the NCAA RPI, saw that there were significant differences, and did not want to have to deal with the fallout those differences exposed.  I do not know this, I could be wrong, but it is the best explanation I can come up with for the Committee's reversal of its position on the KPI and Massey over the last year.

The Big 10's Own Goal

Finally, why do I say the Big 10, in all of this, has committed an "own goal" blunder?  Remember, Ken Pauga is an Associate Athletic Director at the Big 10's Michigan State.  If you look at the first table above, you will see that the Big 10 is the second most discriminated against conference by the KPI, actually winning 4.1% more games per year against non-conference opponents than the KPI ratings say they should win.

The following table breaks the numbers down by Big 10 team, based on the teams in the conference before absorption of the Pac 12 teams in 2024:


As you can see from the column on the right, the KPI discriminates against 11 teams (including Pauga's Michigan State) and in favor of 3.  If I were to add the four teams absorbed from the Pac 12 in 2024, there would be 3 more discriminated against and 1 more in favor of.

Thus so far as Division I women's soccer is concerned, the Women's Soccer Committee now will be using, apparently with added emphasis, a rating system that comes from a Big 10 school administrator yet discrimates against the Big 10 as a whole and even against his own school.  When it comes to the "last in" and "first out" for NCAA Tournament at large selections, that looks like an "own goal" to me.