I read some advice that I disagree with on a website regarding preparation for next year. What is the advice? It advises that we should average three sets of projections.
Here is some advice that I recently read:
Which brings us to: PLANNING YOUR DRAFT-DAY STRATEGY . Ideally, the best thing you can do is make a spreadsheet of your own that incorporates projections for the top 300-500 fantasy-worthy players (depending on the size of your league). You should pick projections from at least three, but no more than five, services that are well-known and trusted. Plug all the projections into your own spreadsheet and average them out. These numbers will serve as your baseline projections.
Frankly I think this is terrible advice. Your goal should be to master one set of projections, knowing its methodology and how it values players, so that you know when the projection is likely to be right and when it is likely to be wrong.
As an example, let’s take PECOTA. We know, for example, that PECOTA can undervalue starting pitchers because of its conservative methodolgy. So when you disagree with PECOTA on a pitcher that is where the value of the projection lies. You can generate a set of projections and know when to deviate from them because you know the methodology and its strengths and weaknesses. This is part of the reason why we believe that “value” should be ignored at the top and bottom of the auction player pyramid.
What do you do when you average a few projections? You water down the exact projections that give you inside knowledge. When you average three sets of projections that have differing methodologies what you do is take the outliers and dull their impact. This is exactly what you do not want to do.
If you know the methodology of a system and know where it should be deviated from, then you have valuable information. When you average it with others you water down the exact outliers! All projection systems are roughly 65-70% accurate, no matter their methodolgy. What matters is not the 65-70% that are accurate but the ones that are not accurate and determining why they are not accurate.
System one predicts, for example, that Juan Pierre will only get 10 steals this year. System two and system three both predict 40 steals. Which is more valuable to you? It should be clear that it is the 10 steal projection, since if it is correct it identifies one of the 30-35% of projections that are outliers.
What this advice does is take the most valuable projection and dulls it. Instead of that 10 steal projection we now have a projection of 30 steals, by averaging the three. This is the worst scenario, taking away all of the impact of the minority projection and at the same time undershooting the majority projection by 10 steals.
5 responses so far ↓
1 Mike Podhorzer // Jul 30, 2008 at 7:37 am
I know you know this Patrick, but consider this a follow-up…This is exactly what Baseball HQ espouses and I know Ron Shandler has written many articles on this, I think one of them being “The Myths of Projective Accuracy” or something.
Basically, the value in a projection is not a whether it is perfectly accurate, but if it differs from the consensus enough to either cause you to go the extra buck for the player, or cause you to avoid said player. By averaging projections, you can’t possibly benefit from this concept.
2 Neil // Jul 30, 2008 at 10:12 am
I used an “aggregation of projections/rankings” strategy in my league and disagree with the belief that the accurate projections are less valuable than the outliers. If I offered you a projection system that was 100% accurate, you would win your league each and every time. The goal should be to find a system that gets you as close to 100% as possible, and if that means aggregating three or five 60% systems together, that’s fine by me. The Juan Pierre example is a little extreme (40 steals vs. 10 steals), in real life, the projections are usually very tight and bringing them all into one package smooths things out. Also, aggregation was very helpful to me because I am in a salary cap league. Sporting News set the prices, so their projections were useless (a $40 player was projected to have $40 stats…that doesn’t really help identify “value”). But by comparing the Sporting News prices to CBS/Baseball Prospectus/ESPN projections, I was able to grab the players that Sporting News didn’t like (e.g. Uggla, Fukudome, Burrell) , but who were liked by all the other publications/Websites. It was very rare that Sporting News got their projection right and the other three publications flamed out (e.g. Swisher).
3 Patrick DiCaprio // Jul 30, 2008 at 11:17 am
Neil, the thing is no matter what projection you use it will only be 70% accurate at best.
I disagree that if you had a 100% accurate projection system you woudl win every year. One thing we harp on here is second level thinking and analysis. so you have to ask what would others do, even in theory, if your conditions were true. This is what game theory is all about.
If you had that projection so would everyone else and either there would be imitators or that projection would have a monopoly, but everyone would be 100% accurate no matter what. So even as a theoretical construct you wouldn’t win every time; in that instance then draft strategy would be all that matters.
As far as this :”in real life, the projections are usually very tight and bringing them all into one package smooths things out” that is where we will have to disagree because in my view smoothing out is exactly what you do not want to do. At anything other than lower levels of play the accuracy of projections is not terribly important because everyone has accurate projections.
I hope this helps to explain things and thanks for reading!
4 Jeff // Jul 30, 2008 at 1:58 pm
I am the author of the piece you are slamming unmercifully and I have a couple of issues with your characterization of my work.
First, although I may have failed to say so, the entire point of my system is to determine where the world values every fantasy-worthy player. As you have indicated, fantasy championships are won by identifying outliers and picking them accordingly. But it does you no good to pick them two rounds too soon - my system is intended to give a reasonable projection of where everyone will be picked so you can jump on them just a little early or get them late.
Quibble if you like, but I am not really that interested in what each individual projection system has to say. Earlier in my piece, I advocated reading every possible bit of information available on every fantasy-worthy player known to man so that you can go into every draft with a set of ideas about every player. The idea is to combine the multiple projections with your inner instinct. I know that some stat-heads might dislike that approach, but it has worked exceptionally well for me.
Pat - you say yourself that projections aren’t that important. I agree. That is why my system gauges where most players will or should be chosen. If you have a decent idea of where everyone will go and you come into the draft with targets of your own (based on your aggregate projections, knowledge, and instinct), then you can be well-prepared to maximize the value you get from every pick.
Besides, a regular Joe who walks into a draft with the best projections is still going to get outplayed by the guy who understands on a gut level how, why, and where every player gets chosen. I understand fully why you have a problem with my system, but in reality it is more intended for the average fantasy baseball player than it is for “experts”.
5 Neil // Jul 30, 2008 at 5:32 pm
What I meant was that if you and you alone had a projection system that was 100% accurate, you would win every year. I didn’t mean a system that was publicly available to everyone. If you knew how to tweak and weight several systems (or only use the strengths of each system) and could get your projection hit rate higher, that would eliminate a nice % of your wrong projections. BTW, great site, it’s my favorite!
Leave a Comment