You know the drill. You get up on Tuesday, look over the final scoreboard for your league, weep (smile) over your latest loss (win) that shows how stupid and ridiculous fantasy football is (how well qualified you are to be a general manager of an NFL team), and then set your sights on your upcoming matchup.
There are two main questions fantasy owners face every week: who to pick up on the waiver wire, and who to start. For both decisions, the projected point total of any given player is a key piece of information. Sure, we consider the matchup, our personal views of the player, and how they have played recently, but when it comes down to it I bet a whole lot of fantasy owners have trouble going against the projected point totals when it comes to making a decision.
But how useful are the projections? A few weeks back, one poor guy in our league thought he had a sure win going into the weekend. ESPN projections had him posting a ridiculous 177 points; he ended up with just 86! I’m sure this is a familiar theme to anyone trying to get a leg up on the competition, week in and week out. It got me thinking: how accurate are the projections? Which projection websites are the best? Do they always tend to overestimate point totals? A friend of mine believes that point totals are deliberately overestimated in order to excite people and get more people playing.
I then compared the weekly point projections to the actual point totals for each player for each week. To avoid skewing the data too much, I only considered what I call “fantasy relevant players”, players who were projected to, or did, score more than 5 fantasy points. Why this choice? I’m sure Dion Sims (TE, MIA) is a great guy (or maybe not, since he is on Miami) but no one outside of his immediate family cares about his projections or performance on Sunday .
To evaluate the different sites I used two common statistical measures: Mean Bias Error (MBE) and Mean Absolute Error (MAE). Follow the footnote for the gritty details . If you don’t care for the math, just know that closer to zero is better.
Keep in mind that statistical indicators measure probability; an MAE of 5 points does not mean that every single projection is going to be 5 points off. It means that the average projection is 5 points off. Some projections could be exactly right, while others could be 20 points off. That being said, over time you will be more likely to choose the right players each week if you use a projection site that has a lower error.
How good are the point projection sites? The answer, as those who have been repeatedly let down by projections have likely already concluded, is not that good. As shown in the table below, most point projection sites have an MAE between 5-6.5 points. Considering that 10-15 points in a week is a pretty good showing for a player, and that 20-30 points is standout, that amount of projection error is fairly large.
However, there are sites that do a better job than others. Bloomberg should clearly be avoided at all costs, which isn’t surprising given that all I know of Bloomberg is their “Market Minutes” on news radio stations. Given the crap shoot that is the stock market, do we think they would be any better at football? Fantasy Pros projections are the closest to actual performance, but this is based on only three weeks of data, so more time is needed to fully evaluate the site. The CBS projections are the best of the full season sites.
Contrary to my friend’s belief, most sites appear to slightly underestimate the point projections on a weekly basis. My guess is that it only feels like sites overestimate production, because we always remember the players that so emphatically let us down. When Calvin Johnston pulls in 28 points instead of his projected 15, we take this a sign of how smart we were to acquire Megatron in the first place and ignore the 13 point gap in projection.
Another factor is that Adrian Peterson getting 6 points when he was projected to score 18 in Week 11 is balanced out by Bobby Rainey (!?!?) raking up 34 points when he was projected to score 5. Projections for the top fantasy players are probably overly optimistic, but this is balanced by the weekly unexpected point production from players like Rainey, Michael Floyd, and Rashad Jennings.
I have also included two graphs below that show the weekly MBE and MAE over the course of the season. What is most interesting about these is that the various sites tend to follow the same patterns in weekly error. Some weeks are better than others, and these weeks tend to coincide across the various projection sites.
In general, the projections seem to be improving past the midway point of the season (Weeks 8-9). This could be because sites are learning to adjust their expectations of players based on this season’s performance , or it could just be random fluctuation. We will update the data later in the season, so be sure to check back.
The bottom line: take the projections with a hefty grain of salt, and try to use projection sites like Fantasy Pros and CBS that tend to do a better job. Or, take a page out of Back to the Future, Part II, and get a future version of yourself to bring back game summaries to your present self. Either way, good luck.
 Note that Fantasy Pros is composite projection of many other sites.
 Since I started this project just a few weeks ago I only have full season data from CBS, ESPN, and FFtoday.
 A one yard TD catch in Week 3 and a 4 yard catch in Week 8, if you are related to Dion Sims but don’t check the box scores.
 Mean Bias Error indicates the tendency for a predictor to over or underestimate compared to the actual value. A positive MBE means that the projections tend to overestimate points, while a negative MBE means the projections underestimate the points. Mean Absolute Error measures how close the projection is to the actual outcome; a MAE of 7 indicates that the point projection for an average player in a given week is 7 points different from the actual total for that player in that week.
 For example, ESPN projections have finally caught on that Trent Richardson isn’t a workhorse RB. They are only projecting 5 points this week after multiple weeks of projection 12+ points, despite the fact he has only exceeded 7 points twice this year.