[Coming next: Simulated 2023 End-of-Season Ranks]
********
This is the first of three pre-season reports I will post. As we move through the season, I will do weekly reports similar to the second and third of the pre-season reports. This week, each report will have a detailed explanation of how I arrived at the information in the report. For later weeks, I will not repeat the explanations, so if you have questions about the future reports, please first refer back to this week’s explanations. If you still have questions, ask them in a comment and I will answer.
********
After each season, I produce pre-season team strength rankings for the next season. At the end of this post, I show my pre-season strength rankings for the 2023 season. I produce them using a process that I set up intentionally to eliminate subjective decisions on my part. This makes the process transparent: Anyone can know what the process is and decide how much, if anything, it is worth. Here are the steps I go through:
1. I have a data base of all teams’ end-of-season ranks since 2007. For years beginning with 2010, the ranks are what they would have been if the current no-overtime rule had been in effect. There are three sets of ranks:
a. Using the current NCAA RPI formula;
b. Using the Kenneth Massey rating system, since he is a respected producer of sports ratings and produces ratings and ranks for Division I women’s soccer; and
c. Using my Balanced RPI, which is an RPI modification that produces more accurate and non-discriminatory ratings and ranks than the current NCAA RPI.
I use the Massey ranks simply as a basis for testing the credibility of the current NCAA RPI and my Balanced RPI. The current NCAA RPI produces ranks that are significantly different than Massey’s. My Balanced RPI produces ranks that are quite similar to Massey’s, athough using an entirely different calculation method.
2. Predicting teams’ strength for the coming year is difficult, as teams’ strength varies a lot from one year to the next except near the very top and bottom of the rankings. The average rank standard deviation for teams is 50 rank positions. What this means is that roughly two-thirds of the time, you can expect the team’s rank to be within 50 rank positions of its average rank -- in other words, to fall within a 100 position range that has its average rank in the middle. Thus while predicting teams’ strength next year may be worthwhile and useful for some purposes, it is speculative, especially in the middle of the rankings.
There are various ways to predict strength for the coming year, and I have chosen one of them: using averages and/or trends based on historic ranks. This has limitations as a method, but does reasonably well when predicting the relative strength of teams within a conference, as compared to other methods for ranking them: In 2022, conference coach pre-season rankings of teams within their conference on average were within 2.44 positions of the teams’ actual end-of-season ranks. Chris Henderson’s ranks matched the coaches’ 2.44. His rankings are based on returning starters, talent depth, goalkeeper rankings, transfer rankings, his CoachRank metric, and recruiting rankings, with penalties for losing 6 or more starters from the prior year or for extremely poor in-conference results in the prior year. My ranks, when applied to conferences, were within 2.61, in other words were almost as good as theirs notwithstanding that my method is based only on historic ranks.
3. Within my system, I have a list of alternative methods I choose from to generate a team’s strength rank for next year. The methods are:
a. Use the team’s rank from last year
b. Use the team’s average rank over the last two years, three years, four years, five years, six years, seven years, eight years, or nine years.
c. Use the team’s rank trend over time to create a formula that produces a rank for next year, using trends over the last two years, three years, four years, five years, six years, seven years, eight years, or nine years.
d. Pair each method from a and b with each method from c, using the average of the two, such as the average of the rank from last year and next year’s trended rank based on the last two years’ ranks, the average rank from the last two years and next year’s trended rank based on the last two years’ ranks, and so on.
4. For each team, for each of the methods in step 3, I calculate how close the method would have come to predicting the team’s rank for each future year if I had used the method year-after-year in the past. I then select the method that would have come closest, on average, to predicting the team’s next year ranks. This is the method I use for that particular team.
As a matter of interest, the method that most commonly is the best predictor of the coming year’s rank is last year’s rank, occurring for 58 teams for the 2023 strength rankings. Using that or using only average ranks over some period of years accounts for 138 teams. Using only trends accounts for 25 teams. Using average rank over some period of years combined with trend over some period of years accounts for the remaining teams, about half of them.
This method does produce occasional odd results of seriously overranking or underranking a team:
Quinnipiac. A good example this year is Quinnipiac. As it turns out, the best method for predicting its rank is to base the prediction on its two year trend. Quinnipiac’s rank has trended much better over the last two years, due to the fact that it had recruited an excellent Irish striker. Using its rank trend over the last two years, my method says it will be ranked tied for #14 this coming season. The striker, however, has transferred to Penn State. Quinnipiac almost certainly would not have been #14 even with her (see Arkansas, below) and almost certainly is not going to be #14 this coming season without her. Nevertheless, I stick to the method I have described and simply know that Quinnipiac likely is a case of serious overranking.
Arkansas. For Arkansas, the best method for predicting its upcoming rank is to use its eight year trend. For the coming year, this puts it predicted as #1. The problem with this is that it has been in or near the Top 10 for the last three years. In that area, it becomes increasingly difficult to improve rank positions, so that a team such as Arkansas is not likely to improve as much as its long term trend suggests. Here again, I stick to the method but know that Arkansas is not likely to end up where its trend suggests.
5. At this point in my process, I have a projected strength rank for next year for each team. Generally, a team rank will not be a whole number but rather something like #144.34. I put the teams in strength rank order from the best rank to the poorest and then, with the teams in that order, assign them whole number ranks in order from #1 to the poorest.
6. I then assign ratings to the teams. To do this, I have calculated the historic average current NCAA RPI formula ratings for all of the rank levels from #1 to the poorest. I assign #1 the historic average rating for the #1 ranked teams, #2 the historic average rating for the #2 ranked teams, and so on. (I also have fill-in steps I use when new schools sponsor Division I teams.)
7. After going through all of the above steps, I have assigned to each team a pre-season strength rating and rank. Here are the ranks this process produces for 2023, first in Rank Order and then in Alphabetical Order:
Rank Order:
Alphabetical Order:
No comments:
Post a Comment