THE GREAT MYTHS OF PROJECTIVE ACCURACY
by Ron ShandlerJanuary 2009
Ashley-Perry Statistical Axiom #3: Skill in manipulating numbers is a talent, not evidence of divine guidance.Ashley-Perry Statistical Axiom #5: The product of an arithmetical computation is the answer to an equation; it is not the solution to a problem.
Merkin's Maxim: When in doubt, predict that the present trend will continue.
The quest continues for the most accurate baseball forecasting system.
I've been publishing player projections for more than two decades. During that time, I have been made privy to the work of many fine analysts and many fine forecasting systems. But through all their fine efforts at attempting to predict the future, there have been certain constants. The core of every system has been comprised of pretty much the same elements:
- Players will perform within the framework of their past history and/or trends.
- Their skills will develop and decline according to age.
- Their statistics will be shaped by health, expected role and environment.
These are the elements that keep all projections within a range of believability. This is what prevents us from predicting a 40-HR season for Wily Tavares or 40 stolen bases for Adam Dunn. However, within this range of believability is a great black hole where precision seems to disappear. Yes, we know that Alex Rodriguez is a leading power hitter, but whether he is going to hit 40 HRs, or 45, or 35, or even 50, is a mystery.
You see, while all these systems are built upon the same basic elements, they are constrained by the same limitations. We are all still trying to project...
- a bunch of human beings
- each with their own individual skill sets
- each with their own individual rates of growth and decline
- each with different abilities to resist and recover from injury
- each limited to opportunities determined by other people
- and each generating a group of statistics largely affected by tons of external noise.
As much as we acknowledge these limitations intuitively, we continue to resist them because the game is so darned measurable. The problem is that we do have some success at predicting the future and that limited success whets our desire, luring us into believing that a better, more accurate system awaits just beyond the next revelation. So we work feverishly to try to find the missing link to success, creating vast, complex models that track obscure relationships, and attempt to bring us ever closer to perfection. But for many of us fine analysts, all that work only takes us deeper and deeper into the abyss.
Why? Because perfection is impossible and nobody seems to have a real clear vision of what success is.
Measuring success
Is reasonable predictive accuracy even an attainable goal? Most agree that, given external variables such as injuries, managerial decisions and the like, only about 65-70% of the player population is even marginally predictable in any given year. But even within that group, you cannot get two analysts to agree about what it means to be accurate.
In truth, the only completely accurate projection would be one that looks like this:
AB HR RBI SB BA OBA SLG OPS
=== === === === === ==== ==== ====
PROJ 500 25 95 15 .280 .330 .450 .780
ACT 500 25 95 15 .280 .330 .450 .780
Clearly, you would be overjoyed if all of our projections yielded perfect results. But it is impossible to be on target with all of these individual categories, each moving more or less independently for 180 days each baseball season.
An alternative might be to focus only on the most important statistical gauges. After all, each raw data category measures only an isolated element, and some stats like batting average are flawed. Perhaps a better measure of accuracy can be gleaned by using a gauge of overall talent, like OPS.
It sounds reasonable in theory. However, if I projected a player to have an OPS of about .783, for instance, he could post any of the following 2008 stat lines and my projection would still be considered a success:
AB HR RBI SB BA OBA SLG OPS
=== === === === === ==== ==== =====
A 556 25 77 8 .266 .327 .457 .78357
B 375 6 33 20 .293 .357 .427 .78346
C 319 12 49 2 .276 .344 .439 .78262
D 579 9 55 6 .316 .342 .440 .78260
I suppose, for pure scientists, these players are all comparable. With my .783 OPS projection, any of these results would have provided for a perfect success story. But I'd hardly think that, if I projected Adrian Beltre (A) to have Cristian Guzman's stats (D), you'd consider I was a heck of a prognosticator. Kaz Matsui (B) and Ryan Church (C) are hardly comparable either, even though OPS says that they are.
In addition, Matsui and Church should not be compared to Beltre and Guzman, given their differences in at-bats. But aggregate gauges like OPS make no distinction for playing time.
Despite the similarities using a gauge that measures aggregate performance, these are very different skill sets for most fantasy applications.
One way to resolve this issue might be to use a more fantasy-friendly gauge. Rotisserie dollar values can serve a dual purpose here. First, they measure only those categories that we are interested in. A second benefit is that they incorporate the importance of playing time - which OPS does not. And in fact...
AB R HR RBI SB BA 5x5
=== === === === === === ====
Beltre,A 556 74 25 77 8 .266 $12
Matsui,K 375 58 6 33 20 .293 $ 8
Church,R 319 54 12 49 2 .276 $ 3
Guzman,C 579 77 9 55 6 .316 $14
...now this group is no longer cut from the same cloth. But Rotisserie values still do not negate the underlying problem when comparing sets of numbers. Beltre is a nice $12 player, but $12 doesn't always buy you the same type of statistics:
AB R HR RBI SB BA 5x5
=== === === === === === ====
Beltre,A 556 74 25 77 8 .266 $12
Schumaker,S 540 87 8 46 8 .302 $12
Reynolds,M 539 87 28 97 11 .239 $12
Gomez,C 577 79 7 59 33 .258 $12
Thome,J 503 93 34 90 1 .245 $12
So, using dollar values doesn't work either. The last thing that your power-rich, speed- starved team would need is for me to project Carlos Gomez and you end up with Jim Thome. But $12 is $12, right?
With all these obstacles to using aggregate performance gauges, perhaps we need to refocus on projecting individual stat categories. Can this provide any better hope for defining prognosticating success?
Here is where it gets "personal."
If I were to project that Albert Pujols is going to hit 45 HRs this year and he only hits 44, you will probably accept that level of inaccuracy. But what if he hits only 43? Or 42? Or 40? Or 39? At what point do we cross that imaginary line where the projection is "officially" deemed a failure?
You might say "40." I might say, "Okay, so if Pujols has 39 HRs on the final day of the season, and he hits a long fly ball that Carlos Beltran makes an amazing over-the-wall leap to rob him of #40, has that one event been the difference between success and failure?" We have to draw the line somewhere, but there is always going to be a grey area where it can go either way. You might consider the grey area as representing "inaccuracy." Fair enough. But more important is the fact that the size of this grey area is different for everybody.
In early 2003, we asked this type of question in two online polls at BaseballHQ.com. Here were the results:
If I were to project 35 HRs for Hideki Matsui this year, what is the threshold of actual HRs at which you would perceive that my projection had failed?34 2% 32 3% 30 18% 28 31% 26 24% 24 14% 22 5% 20 3%If I were to project 15 wins for Tom Glavine this year, what is the threshold of actual wins at which you would perceive that my projection had failed?
14 4% 13 10% 12 33% 11 27% 10 17% 9 3% 8 2% 7 3%
There is no clear consensus in either poll. That's why this is "personal." Accuracy can only be assessed based on your own subjective tolerance for error.
But you might say, "Shandler, there has to be some type of benchmark I can use. There has to be some way to gauge accuracy."
I'm not so sure. There are some people who might consider a broad stroke approach to be sufficient, using a flat percentage benchmark across all categories. For instance, you might be satisfied if a projection was off by only 10% across-the-board. Doesn't that seem reasonable? But a casual "eyeball test" can be deceiving. To wit:
AB R H HR RBI SB BA
=== == === == === == ====
PROJ 550 79 169 29 113 13 .307
ACT 599 70 169 26 100 10 .282
At first glance, this looks like a pretty good projection, at least one that you wouldn't be too unhappy with had you expected to purchase that first set of stats. Our eyeball test says that his overall productivity was pretty much on target. In reality, every one of his statistics was mis-projected by more than 10%. Based on a perceived "acceptable" 10% tolerance, this projection was a complete failure. Of course, I could just loosen that tolerance, perhaps to 15% or 20%, which will boost our success rate, but the eyeball test will get much fuzzier.
Here is the above example with actual results within 15-20% of projection:
AB R H HR RBI SB BA
=== == === == === == ====
PROJ 550 79 169 29 113 13 .307
ACT 632 62 169 23 87 7 .267
My own eyeball test says that, while this projection was marginally in the ballpark, perhaps a 20% error is beyond the limits of my comfort level. But again, you might look at the above results and think these are perfectly fine within your own tolerance for error.
The irony with the above examples is that, despite the shortfalls in batting average, both projections nailed this player's hit total. All of which begets other questions...
If a projected slugging percentage is dead on, but the player hits 10 fewer HRs than expected (and likely, 20 more doubles), is that a success or a failure?
If a projection of hits and walks allowed by a pitcher is on the mark, but the bullpen and defense implodes, and inflates his ERA by a run, is that a success or a failure?
If a speedster's projected rate of stolen base success is perfect, but his team replaces the manager in May with one that doesn't run, and the player ends up with half as many SBs as expected, is that a success or a failure?
If a batter is traded from San Diego to Texas and all the touts project an increase in production, but he posts a statistical line exactly what would have been projected had he not been traded, is that a success or a failure?
If the projection for a bullpen closer's ERA, WHIP and peripheral numbers is perfect, but he saves 20 games instead of 40 because the GM decided to bring in a high-priced free agent at the trading deadline, is that success or a failure?
If I project a .272 batting average in 550 AB and the player only hits .249, is that a success or failure? Most will say "failure." But, wait a minute! The real difference is only two hits per month. That shortfall of 23 points in batting average is because a fielder might have made a spectacular play, or a screaming liner might have been hit right at someone, or a long shot to the outfield might have been held up by the wind... once every 14 games. Does that constitute "failure?"
When it comes down to it, perhaps the only thing we can really trust is our own personal opinion of what is acceptable and what is unacceptable. And often that comes down to our own needs. For instance, if you have a loaded bullpen, it doesn't matter whether your third closer puts up 40 saves or 30. When you are leading your league in home runs by 25, it doesn't matter whether Alex Rodriguez hits 54 HR or 35. And when all the aggregates wash out come October, the fact that your $20 Justin Verlander saw his ERA rise by more than a run will only affect your team's bottom line by 0.15 - in most leagues, a loss of maybe 2-3 points at worst.
Comparing different systems
It's tough enough to answer these questions when you are trying to measure the accuracy of a single set of projections. When you open things up and begin to look at multiple prognosticators, then there are even more issues to address.
The number of published projections that appear in print and online rises annually, and with them, expectations, questions and unbearable hype. Everyone wants to know - which system is the most accurate? Despite the trumpeting of often outlandish claims, they can't all be the most accurate. Numerous "objective" studies are published that attempt to determine the Prince of Prognosticating Prowess - with our projections often included in the exercise - but the same thing happens time and time again:
1. We never finish first.
2. The purveyor of the study always does.
Is that a wonder?
Peter "Ask Rotoman" Kreutzer has this take: "Someone who tries to sell you projections that are "much better" than any others is bullshitting you. The important thing for you as a consumer to understand is what system your prognosticator is using, what biases that introduces, and learn to make the necessary adjustments to incorporate risk evaluation into the process. Only then can you get the players who fit your league's rules best."
Ah, biases. The truth is, there is an inherent bias that exists in any comparative analysis that includes the author as one of its subjects (whether blatantly or via proxy). It's impossible to avoid. The reason is obvious: A tout is not going to publish such an analysis unless he can present himself in a favorable light. And the only way to do this is to instill some level of bias into the structure of the study.
Here are some of the ways this is done:
Selection of the study group: This refers to both the group of touts involved in the study as well as the group of players studied.
Most analyses contain perhaps 6-10 prognosticators, but there are easily at least 20 books, magazines and websites that published projections last year. In these limited studies, how do we know whether there were other touts not chosen that might have fared better or shifted the results?
When it comes to the players, there are often qualifiers such as: "We evaluated only those players who had a forecast provided by each of the seven projections systems." This means, the addition or omission of any of the seven prognosticators could change the composition of the players studied, and thus the results of the study.
As such, unless the study contains a representative sample of touts and defines a consistent pool of players regardless of who those touts are, it cannot be completely objective.
Selection of the study variables: We've already discussed the limitations inherent in choosing a study variable. However, those who conduct comparative analysis have to select something to compare. Will it be an overall aggregate gauge like OPS or Win Shares? Will it be a fantasy-relevant gauge like dollar values or fantasy points? Will it be a raw, traditional measure like ERA or batting average? And most important, how do we know that the measuring gauge chosen isn't one that just happens to yield the most favorable results?
As such, unless the study uses a viable test variable, it cannot be completely objective.
Selection of the study methodology: Even if a comparative analysis includes all relevant test subjects and somehow finds a study variable that makes sense, there is still a concern about how the study is conducted. Does it use a recognized, statistically valid methodology for validating or discounting variances? Or does it use a faulty system like the ranking methodology used by Elias to determine Type A, B or C free agents? Such a system -- which ironically is the basis for Rotisserie scoring -- distorts the truth because it can magnify tiny differences in the numbers and minimize huge variances.
As such, unless the study uses a proven methodology, it cannot be completely objective.
And bias immediately enters into the picture. You simply cannot trust the results.
The only legitimate, objective analysis that can filter out the biases is one that is conducted by an independent third party. But the challenge of conducting such a study is finding a level playing field that all participants can agree on. Given that different touts have different goals for their numbers, that playing field might not exist. And even if one should be found, there will undoubtedly be some participants reluctant to run the risk of finishing last, which could skew the results as well.
Other challenges to assessing projections
Ashley-Perry Statistical Axiom #4: Like other occult techniques of divination, the statistical method has a private jargon deliberately contrived to obscure its methods from non-practitioners.
As users of player projections, and in a hurry to make decisions, we want answers, and quickly. We want to find a trusted source, let them do all the heavy lifting, and then partake of the fruits of their labor. The truth is, the greater the perceived weight of that lifting, the greater the perceived credibility of the source. Only the small percentage of users who speak in that "private jargon" can validate the true credibility. The rest of us have to go on the faith that the existence of experts proficient in these 'occult techniques' is proof enough.
Well, so what? That's why we rely on experts in the first place, isn't it? What is the real problem here?
Complexity for complexity's sake
One of the growing themes that I've been writing about the past few years is the embracing of imprecision in our analyses. This seems counter-intuitive given the growth in our knowledge. But, the game is played by human beings affected by random, external variables; the thought that we can create complex systems to accurately measure these unpredictable creatures is really what is counter-intuitive.
And so, what ends up happening in this world of growing complexity and precision is that we obsess over hundredths of percentage points and treat minute variances as absolute gospel. To wit...
It has been shown that a simplistic forecasting system that averages the last few seasons with minor adjustments for age is nearly as good as any advanced system. The simple system is called "Marcel" (named after the monkey on the TV show Friends) because any chimp with an Excel spreadsheet can do it. The truth is, if 70% accuracy is the best that we can reasonably expect, Marcel alone gets us to about 65%. All of our advanced systems are fighting for occupation of that last 5%.
Still, those conducting comparative analyses will crow about one system beating another 68% to 67%. This is a level of precision that can often be rendered moot across the entire player pool by a handful of wind-blown home runs and a few seeing-eye singles. Still, there has to be a "winner," right?
But we forget such "hard" baseball facts such as:
- The difference between a .250 hitter and a .300 hitter is fewer than 5 hits per month.
- A true .290 hitter can bat .254 one year and .326 the next and still be within a statistically valid range for .290.
- A pitcher allowing 5 runs in 2 innings will see a different ERA impact than one allowing 8 runs in 5 innings, even though, for all intents and purposes, both got rocked.
Gall's Law: A complex system that works is invariably found to have evolved from a simple system that works.Occam's Razor: When you have two competing theories which make exactly the same predictions, the one that is simpler is preferred.
Those systems that try to impress us with their complexity as proof of their credibility may be no better than a room full of monkeys with spreadsheets. At minimum, they generate projections that are 'close enough' for our player evaluation purposes and yield draft results that are virtually indistinguishable from any simian-driven system.
Married to the model
It's one thing if the model has a name like Christie Brinkley, but quite another if a tout is so betrothed to his forecasting model that "it" becomes more important than the projections.
Whenever I hear a tout write, "Well, the model spit out these numbers, but I think it's being overly optimistic," I cringe. Well then, change the numbers! The mindset is that you have to cling to the model, for better or for worse, in order to legitimize it. The only way to change the numbers is to change the model.
On occasion, I will take a look at one of my projections and admit that I think it's wrong. Then I change the numbers. Because, in the end, is the goal to have the best model or to have the best projections?
The comfort zone
Given the variability in player performance, a "real world" forecast should not yield black or white results. Some touts accomplish this by providing forecast ranges, others by providing decile levels. But most end up committing to a single stat line to describe their expectations for the coming year.
In October, reality will be black or white. In March, it's all shades of grey. But it's far easier for fantasy leaguers to draft their teams from blacks and whites, so touts have to commit. Grey is out, even when a projection carries great uncertainty.
One of the best examples from March 2008 was Andruw Jones. This was a hitter coming off a 26-HR, .222 BA season after having gone 41-.262 and 51-.263 in the two years prior. The questions on everyone's mind... Was 2007 an aberration? Would he bounce back? If so, how far would he bounce back?
The typical forecast would never venture into uncharted, sub-25 HR, sub-.222 territory. A typical computer-generated projection would not find those levels anywhere in Jones' history and there was not enough evidence to support extending the current trend that was, in fact, pointing due south. Neither would a computer projection see a complete rebound to 40 HR, .265 levels because 2007 couldn't just be ignored.
So what did some of the top touts project for Andruw Jones last spring?
Tout AB HR RBI SB AVG ===================== === == === == ==== Baseball HQ 536 27 97 3 .248 Baseball Prospectus 538 31 99 8 .257 ESPN 588 36 115 5 .241 Fanball 556 32 97 5 .241 Fantasy Baseball Guide 572 39 117 5 .249 FantasyBaseball.com 559 35 111 5 .249 Bill James 558 35 104 5 .251 Rotowire 580 36 111 5 .252 Rotoworld 575 35 104 4 .254
When a tout doesn't know which of two disparate expectations are for real, they will typically just split the difference. That's what all these 30-plus, 100-plus, .250ish projections are doing. These are all essentially equidistant between 2006 and 2007. But this is problematic, particularly on the batting average side. In Jones' nine years as a full-timer prior to 2007, he batted less than .261 only once. Yet every tout above - including us - was willing to commit to a batting average between .241 and .257.
As a group, there is a strong tendency for all pundits to provide numbers that are more palatable than realistic. That's because committing to either far end of a range of expectation poses a high risk. Few touts will put their credibility on the line like that, even though we all know that those outliers are inevitable. The easy road is often just to find a more comfortable spot to commit to.
I like to call this the "comfort zone," a range bordered by the outer tolerances of public acceptability of a projection. In most cases, even if the evidence is outstanding, published pundits will not stray from within the zone. In the end, what you get is a potentially watered-down view of the future.
This is the safest place to be in case Andruw Jones repeats 2007, or rebounds to 2005-6. But "safe" and "accurate" are not the same. It was likely that either 2007 was an anomaly or 2005-6 was an anomaly. But nobody was willing to commit to avoiding either one. Hence, the projections came to rest at a comfortable place where either potential reality wouldn't be that far off.
Of course, the reality of 2008 made us all look silly.
The Hedge
The hedge is used to formally straddle the fence rather than commit to anything, and typically takes place in the player commentary. In that aspect, the hedge might be a good thing because it embraces the "greys."
However, some touts use the commentary as a hedge against the numbers they've committed to, and in doing so, can negatively impact your ability to assess a projection.
In Andruw Jones' case, the comfort zone indicated a partial rebound. For our group of touts, some of their commentaries support this, but some used the commentaries strictly as a hedge:
Fanball: While a return to his .263 career average is possible, it may be a stretch to think he'll ever return to his 45-50 HR levels. (Projection: 32 HRs)
Fantasy Baseball Guide: He's young for all his experience and he's likely to have more years like 2006 (41 HRs), at least. (Projection: 39 HRs)
Rotowire: He'll likely rebound from career lows but playing home games in a pitchers park won't help his hitting stats. Still, he's just two years removed from a 51 HR season and will be just 30 years old. He could be a bargain. (Projection: 36 HRs)
Rotoworld: Jones should rebound this year. 120 RBI figures to be out of reach but 40 HRs may not be. (Projection: 35 HRs)
Still, in a world where we have become obsessed with projective accuracy, grey is good. We hedged in our own commentary as well, though we happened to luck into a bullseye...
Baseball HQ: We didn't project it, but it's possible... With no apparent explanation for 2007, we have to consider it a valid data point on its own. That makes 2007 just a stepping stone down the trend to even greater depths. (Projection: 27 HRs)
Andruw Jones actually batted .158 with three home runs in 209 AB in 2008. While nobody came even remotely close to this, the least optimistic projection holds the most fantasy relevance. As noted in the Baseball Forecaster:
"The best projections are often the ones that are just far enough away from the field of expectation to alter decision-making. In other words, it doesn’t matter if I project Player X to bat .320 and he only bats .295; it matters that I projected .320 and everyone else projected .280.
"Or, perhaps we should evaluate projections based upon their intrinsic value. For instance, coming into 2008, would it have been more important for me to tell you that Adam Dunn was going to hit 40 HRs or that Juan Pierre would only get 290 at bats? By season’s end, the Dunn projection would have been dead-on accurate, but the Pierre projection — even though it was off by 85 AB — would have been far more valuable."
The Outliers
They say that the winners in any fantasy league are those who have the most outliers on their teams. There is an element of truth to this. You would expect that owners who rostered surprises like Cliff Lee and Edinson Volquez in 2008 fared well in the standings. A Baseball HQ poll bears this out:
For AL-only and NL-only leagues, in what place is the team that owns Cliff Lee or Edinson Volquez? Safely in 1st 12% 1st but could fall 13% 2nd or 3rd 24% 4th or 5th 21% Lower than 5th 30%
One in four Lee/Volquez owners was in first place! About half were no lower than 3rd, and seven in 10 were no lower than 5th! While these results are remarkable, the problem is, these types of performances are the most difficult to project. Still, the prognosticators who fare the best in this exercise should get their props, shouldn't they?
According to analyst John Burnson, the answer is no. He says: "The issue is not the success rate for one player, but the success rate for all players. No system is 100% reliable, and in trying to capture the outliers, you weaken the middle and thereby lose more predictive pull than you gain. At some level, everyone is an exception!"
Peter Kreutzer again: "Those projections that are outside the comfort zone, as Ron calls it, are flashy, but they're of little statistical use. What you want is to follow the predictor who gets the general flow (guys who improve, guys who fall off) more right than anyone else. If someone does that they'll make you money in almost any league."
Yes! That "general flow" is far more important than any pure accuracy level. And far more attainable. And perhaps, that is the study variable that makes the most sense.
Finding relevance
Berkeley's 17th Law: A great many problems do not have accurate answers, but do have approximate answers, from which sensible decisions can be made.
Maybe I'm a bit exasperated by this obsession with prognosticating accuracy because the Baseball Forecaster/HQ projections system is more prone to stray from the norm - by design - and thus potentially fare worse in any comparative analysis. The HQ system is not a computer that just spits out numbers. We don't spend our waking hours tinkering with algorithms so that we can minimize all the mean squared errors. Our model only spits out an objective baseline and then the process becomes hands-on and highly subjective.
From the Projections Notes page at BaseballHQ.com:
"Skills performance baselines are created for every player beginning each fall. The process starts as a 5-year statistical trend analysis and includes all relevant performance data, including major league equivalent minor league stats. The output from this process is a first-pass projection.
"Our computer model then generates a series of flags, highlighting relevant BPI data, such as high workload for pitchers, contact rate and PX levels trending in tandem, xERAs far apart from real ERAs, etc. These flags are examined for every player and subjective adjustments are made to all the baseline projections based on a series of "rules" that have been developed over time."
The end result of this system is not just a set of inert numbers. As mentioned earlier, the commentary that accompanies the numbers is just as vital a part of the "projection," if not more so. Think of it this way... The numbers provide a foundation for our expectations, the "play-by-play," if you will. The commentary, driven by all the BPIs and component skills analysis, provides the "color." Both, in tandem, create the complete picture.
Admittedly, a system with subjective elements tends to give classic sabermetricians fits. But that's okay because, at the end of the day we're still dealing with...
- a bunch of human beings
- each with their own individual skill sets
- each with their own individual rates of growth and decline
- each with different abilities to resist and recover from injury
- each limited to opportunities determined by other people
- and each generating a group of statistics largely affected by tons of external noise.
Now here's the kicker... In the end, my primary goal is not accuracy. My goal is to shape the draft day behavior of fantasy leaguers. For certain players with marked BPI levels or trends, we often publish projections that are not designed to reflect a "most likely case" but rather a "strong enough case to influence your decision-making." Sometimes there are reasons to stray beyond the comfort zone.
For instance, sometimes, when our projection says $27, it is intended solely to make you say $22 when the bidding stops at $21 (assuming the context of normal market conditions). If we had published a projection of $23 or $24, that's not enough of a psychological push for you to take that last leap of faith. We need a set of numbers that screams at you: "These BPIs could be HUGE! His upside could be far greater than any projection system would reasonably predict! It's worth the risk -- yes, SAY $22!"
And I want you to make these decisions with a minimum of hesitation. That lack of hesitation comes from a trust I try to build between us, from sound analysis and a 23-year track record that has been shown to work.
How can I play so loose with dollar values? Because they are entirely market-driven anyway. If you are convinced that David Ortiz is worth $26 and land him for $21, you will have overpaid if the rest of the league sees him as no more than a $17 player. Even if he is really worth $35. So my goal is to get you into the mode of playing off that volatility with the knowledge of where your profit opportunities really lie.
It works the same way in snake drafts. If I project a player to be a sixth-rounder and all the ADPs show that he's probably no better than a 9th-rounder, then you can probably feel safe in grabbing him near the tail end of Round 8... and likely still make a profit. However, if I project this player as an 8th-rounder, you'd be more inclined to wait, and possibly miss out on a huge profit opportunity.
And that answers the question, "For any player, what is the one piece of information that is far more important than the most accurate projection?" That information is how the other owners in your league value that player. If you know that, and have a sense of a player's potential, it doesn't matter a whit how accurate your projections are.
What's more, if 70% accuracy is the best we can reasonably ever expect, these dollar figures are almost irrelevant anyway. Research has shown that 65% of players will ultimately earn between +/- $5 of their projected value. So, if a player on your draft list is valued at $20 and you agonize when bidding hits $23, odds are about two chances in three that he could really earn anywhere from $15 to $25. That's a huge range.
In snake drafts, a projected 9th round player could produce anywhere between a 6th and 13th rounder and fall squarely within the range of 70% accuracy. Yes, 70% accuracy means that any player's value can vary by as much as seven rounds.
So our track record is not necessarily built on any given level of prognosticating accuracy. Our track record is built on a series of analytical tools and a decision-making process that has led to success in playing this game. And since your ultimate goal is to fare better in your fantasy competitions, I see this all as a justifiable means to an end.
We're not publishing deliberately inaccurate projections. We're just taking a potential reality from the outlying reaches of the comfort zone, based on strong underlying indicators, and engaging in a bit of behavior modification. If you are offended by the psychological implications, I apologize. If you now consider me a sabermetric hack, I've been called worse. But the users of this information seem to be winning their leagues so I'll accept the baggage that comes along with it.
It's all about winning. Reasonably accurate projections are important, but will only get you part of the way there. The rest is knowing what to do with the information, especially at the draft table. Even if you had a crystal ball and knew exactly what every player's statistics were going to be next year, you can still lose at this game.
How is that possible? Unless your league format provides for solo play with no roster restrictions, every player you select or buy has to be within the limits set by the other owners in your league. This game is not about what you know; it's about what you know within the context of what everyone else knows. Our system helps you navigate that context and those limits with more confidence.
We believe your goal is to win. As such, you should not worry if our analysis says that Albert Pujols is going to hit 39 HRs and another prognosticator says 42. Even if his projection is powered by the latest shiny, new computer model, by next October, the difference between his and ours may be three unexpected gusts of wind.
Baseball Variation of Harvard Law: Under the most rigorously observed conditions of skill, age, environment, statistical rules and other variables, a ballplayer will perform as he damn well pleases.
