By Dave Lackford
As a moderator for warchant.com I see a lot of comments from subscribers claiming Rivals rankings are totally off and the only sites that can be relied upon for accurate recruit rankings are 24/7 and ESPN, and that Scout should be disbanded. Personally, I respect the evaluators form all four sites and see them as hard working individuals and I prefer to use the composite ranking of the four sites as an indicator of a team’s/player’s future success. I also bristle at halfcocked proclamations masked as assertions of fact. If I learned anything from my Stats and Methods courses at Bucknell University it was that the only way to defend a proposed theory is to show your conclusion was based upon empirical research, with all of its curvy features as well as its warts. Honestly, nobody that truly matters cares about your unwashed opinion. With this in mind, I sought out to see how Rivals rankings of the FSU 2016 recruiting class stacked up against the other three sites. While the ’16 FSU class is admittedly a small sample size considering these sites rank almost every D1 recruit in the country, I chose this group because the complaints I read about Rivals’ rankings (and Scouts’) were from FSU fans on a Rivals’ affiliate’s FSU message board in regard to their team’s class/individual player rankings.
According to ESPN, FSU finished with the #1 class in the nation. So of course, so it’s not surprising that FSU fans tend to think they have the best talent evaluators. Rivals and 24/7 had FSU finishing 2nd in the country behind Alabama. Scout had FSU ranked 3rd behind Alabama and Ohio State, so naturally, this particular group of FSU fans think Scout’s rankings are a “joke” in their purely objective assessment, of course.
Often, the perception of a site’s credibility is largely based upon where a particular recruit, who ends up becoming one of the fan bases’ favorites, is ranked. I remember when Rivals dropped Malik Henry from 5-star status last season and the Premium Recruiting Board lost its collective minds. The most peculiar of the reactionary comments that week was how Rivals had jumped the shark long ago and how subscribers only read Rivals for the in-depth pieces and inside recruiting information. This take seemed ill founded to me for a number of reasons. The most glaring flaw with that take was that Rivals was the last site to drop Henry. While ESPN had Henry raked the highest on the national scale at #17 overall, Rivals wasn’t far behind rating him #34 overall. Both sites regarded him as the 3rd best QB in the nation behind Shea Patterson (Ole Miss) and Jacob Eason (UGA). If Rivals’ ranking of Malik Henry was the impetus for so much vitriol toward their evaluators, it’s a poor reason for outrage. It makes even less sense to say 24/7 is far better at evaluating talent, based on Henry, when Rivals was closer to ESPN, while 24/7 (Henry was #53 on their board) was closer to Rivals rating than ESPN’s.
Something has to give here. How can Rivals subscribers think 24/7 is better than FSU when FSU rates a guy like Henry higher, when they are mad that Henry was dropped in Rivals’ rankings? So, I decided to examine how each site ranked each individual player. If ESPN and 24/7 are better than Rivals and Scout, per the discussion on the recruiting boards, then there should not be much variation between ESPN and 24/7’s rankings and Rivals and Scout should be similar.
I made a chart of every player FSU signed and how he was ranked by each site.
In order to compare the sites against one another, this chart will show how often they “agreed” on a player’s ranking. In order for sites to be in agreement, they must have a player ranked in the same general area overall, at their position, or share the same star ranking while the other two sites do not. To find a way to objectively compare the site’s rankings, I have constructed the following rubric:
For a site to agree on a player, they must have that player ranked within 20 spots nationally, 5 spots at his position, or two sites designate a player a certain number of stars, while the two other sites designate a different star status.
- A yellow or green highlight on a player’s ranking for a respective site means that all sites “agreed” on a player.
- If one site had a player ranked 100 spots above or below the closest site’s OVERALL ranking, that site’s ranking is considered an outlier, and is denoted on the chart by a red highlight.
- If data collection issues made it impossible to determine if sites agreed on a player that phenomenon will be denoted by a blue highlight.
- Kickers and Punters were not accounted for in this study.
The chart below illustrates which players the four sites agreed upon, and which sites agreed with one or their peers more often than the others.