HQ Radio Sep 8, 2023: Friday Full Edition w Gene McCaffreyForecaster's Toolbox: Basics
Click here to return to the Education page.
Click here to return to the Forecaster's Toolbox home page.
General Concepts | Batting | Pitching | Minor League Toolbox | Strategy Toolbox | Team Toolbox
What is Fanalytics?
Fanalytics is the scientific approach to fantasy baseball analysis. A contraction of “fantasy” and “analytics,” fanalytic gaming might be considered a mode of play that requires a more strategic and quantitative approach to player analysis and game decisions.
The three key elements of fanalytics are:
1. Performance analysis
2. Performance forecasting
3. Gaming analysis
For performance analysis, we tap into the vast knowledge of the sabermetric community. Founded by Bill James, this area of study provides objective and progressive new ways to assess skill. What we do in this book is called “component skills analysis.” We break down performance into its component parts, then reverse-engineer it back into the traditional measures with which we are more familiar.
Our forecasting methodology is one part science and one part art. We start with a computer-generated baseline for each player. We then make subjective adjustments based on a variety of factors, such as discrepancies in skills indicators and historical guidelines gleaned from more than 20 years of research. We don’t rely on a rigid model; our method forces us to get our hands dirty.
You might say that our brand of forecasting is more about finding logical journeys than blind destinations.
Gaming analysis is an integrated approach designed to help us win our fantasy leagues. It takes the knowledge gleaned from the first two elements and adds the strategic and tactical aspect of each specific fantasy game format.
Definitions
Leading Indicator: A statistical formula that can be used to project potential future performance.
Noise: Irrelevant or meaningless pieces of information that can distort the results of an analysis. In news, this is opinion or rumor that can invalidate valuable information. In forecasting, these are unimportant elements of statistical data that can artificially inflate or depress a set of numbers.
Situation Independent: Describing performance that is separate from the context of team, ballpark, or other outside variables. Strikeouts and walks, as they are unaffected by the performance of a batter’s team, are often considered situation independent stats. Conversely, RBIs are situation dependent because individual performance varies greatly by the performance of other batters on the team (you can’t drive in runs if there is nobody on base). Situation independent gauges are important for us to be able to isolate and judge performance on its own merits.
Soft Skills: BPIs with levels below established minimums for acceptable performance.
Surface Stats: Traditional gauges that the mainstream media uses to measure performance. Stats like batting average, wins, and ERA only touch the surface of a player’s skill and often distort the truth. To uncover a player’s true skill, you have to look at component skills statistics.
Component Skills Analysis
Familiar gauges like HR and ERA have long been used to measure skill. In fact, these gauges only measure the outcome of an individual event, or series of events. They represent statistical output. They are “surface stats.”
Raw skill is the talent beneath the stats, the individual elements of a player’s makeup. Players use these skills to create the individual events, or components, that we record using measures like HR and ERA. Our approach:
1. It’s not about batting average; it’s about seeing the ball and making contact. We target hitters based on elements such as their batting eye (walks to strikeouts ratio), how often they make contact and the type of contact they make. We then combine these components into an “expected batting average.” By comparing each hitter’s actual BA to how he should be performing, we can draw conclusions about the future.
2. It’s not about home runs; it’s about power. From the perspective of a round bat meeting a round ball, it may be only a fraction of an inch at the point of contact that makes the difference between a HR or a long foul ball. When a ball is hit safely, often it is only a few inches that separate a HR from a double. We tend to neglect these facts in our analyses, although the outcomes—the doubles, triples, long fly balls—may be no less a measure of that batter’s raw power skill. We must incorporate all these components to paint a complete picture.
3. It’s not about ERA; it’s about getting the ball over the plate and keeping it in the park. Forget ERA. You want to draft pitchers who walk few batters (Control), strike out many (Dominance) and succeed at both in tandem (Command). You also want pitchers who keep the ball on the ground (because home runs are bad). All of this translates into an “expected ERA” that you can use to compare to a pitcher’s actual performance.
4. It’s never about wins. For pitchers, winning ballgames is less about skill than it is about offensive support. As such, projecting wins is a very high-risk exercise and valuing hurlers based on their win history is dangerous. Target skill; wins will come.
5. It’s not about saves; it’s about opportunity first and skills second. While the highest-skilled pitchers have the best potential to succeed as closers, they still have to be given the ball with the game on the line in the 9th inning, and that is a decision left to others. Over the past 10 years, about 40% of relievers drafted for saves failed to hold the role for the entire season. The lesson: Don’t take chances on draft day. There will always be saves in the free agent pool.
Accounting for “luck”
Luck has been used as a catch-all term to describe random chance. When we use the term here, we’re talking about unexplained variances that shape the statistics. While these variances may be random, they are also often measurable and projectable. To get a better read on “luck,” we use formulas that capture the external variability of the data.
Through our research and the work of others, we have learned that when raw skill is separated from statistical output, what’s remaining is often unexplained variance. The aggregate totals of many of these variances, for all players, is often a constant. For instance, while a pitcher’s ERA might fluctuate, the rate at which his opposition’s batted balls fall for hits will tend towards 30%. Large variances can be expected to regress towards 30%.
Why is all this important? Analysts complain about the lack of predictability of many traditional statistical gauges. The reason they find it difficult is that they are trying to project performance using gauges that are loaded with external noise. Raw skills gauges are more pure and follow better defined trends during a player’s career. Then, as we get a better handle on the variances—explained and unexplained—we can construct a complete picture of what a player’s statistics really mean.
Baseball Forecasting
Forecasting in perspective
Forecasts. Projections. Predictions. Prognostications. The crystal ball aura of this process conceals the fact it is a process. We might define it as “the systematic process of determining likely end results.” At its core, it’s scientific.
However, the outcomes of forecasted events are what is most closely scrutinized, and are used to judge the success or failure of the forecast. That said, as long as the process is sound, the forecast has done the best job it can do. In the end, forecasting is about analysis, not prophecy.
Baseball performance forecasting is inherently a high-risk exercise with a very modest accuracy rate. This is because the process involves not only statistics, but also unscientific elements, from random chance to human volatility. And even from within the statistical aspect there are multiple elements that need to be evaluated, from skill to playing time to a host of external variables.
Every system is comprised of the same core elements:
• Players will tend to perform within the framework of past history and/or trends.
• Skills will develop and decline according to age.
• Statistics will be shaped by a player’s health, expected role and venue.
While all systems are built from these same elements, they also are constrained by the same limitations. We are all still trying to project a bunch of human beings, each one...
• with his own individual skill set
• with his own rate of growth and decline
• with his own ability to resist and recover from injury
• limited to opportunities determined by other people
• generating a group of statistics largely affected by external noise.
Research has shown that the best accuracy rate that can be attained by any system is about 70%. In fact, a simple system that uses three-year averages adjusted for age (“Marcel”) can attain a success rate of 65%. This means all the advanced systems are fighting for occupation of the remaining 5%.
But there is a bigger question… what exactly are we measuring? When we search for accuracy, what does that mean? In fact, any quest for accuracy is going to run into a brick wall of paradoxes:
• If a slugging average projection is dead on, but the player hits 10 fewer HRs than expected (and likely, 20 more doubles), is that a success or a failure?
• If a projection of hits and walks allowed by a pitcher is on the mark, but the bullpen and defense implodes, and inflates his ERA by a run, is that a success or a failure?
• If the projection of a speedster’s rate of stolen base success is perfect, but his team replaces the manager with one that doesn’t run, and the player ends up with half as many SBs as expected, is that a success or a failure?
• If a batter is traded to a hitters’ ballpark and all the touts project an increase in production, but he posts a statistical line exactly what would have been projected had he not been traded to that park, is that a success or a failure?
• If the projection for a bullpen closer’s ERA, WHIP and peripheral numbers is perfect, but he saves 20 games instead of 40 because the GM decided to bring in a high-priced free agent at the trading deadline, is that a success or a failure?
• If a player is projected to hit .272 in 550 AB and only hits .249, is that a success or failure? Most will say “failure.” But wait a minute! The real difference is only two hits per month. That shortfall of 23 points in batting average is because a fielder might have made a spectacular play, or a screaming liner might have been hit right at someone, or a long shot to the outfield might have been held up by the wind... once every 14 games. Does that constitute “failure”?
Even if we were to isolate a single statistic that measures “overall performance” and run our accuracy tests on it, the results will still be inconclusive.
According to OPS, these players are virtually identical:
BATTER HR RBI SB BA OBA SLG OPS Fowler,D 17 46 20 .250 .346 .411 .757 McCann,B 26 94 0 .232 .320 .437 .757 Walker,N 16 71 4 .269 .329 .427 .756
If I projected Fowler-caliber stats and ended up with Brian McCann’s numbers, I’d hardly call that an accurate projection, especially if my fantasy team was in dire need of steals.
According to Roto dollars, these players are also dead-on:
BATTER HR RBI Runs SB BA R$ Rodriguez,A 33 86 83 4 .250 $17 Beltre,A 18 83 83 1 .287 $17 Burns,B 5 42 70 26 .294 $17
It’s not so simple for someone to claim they have accurate projections. And so, it is best to focus on the bigger picture, especially when it comes to winning at fantasy baseball.
More on this: “The Great Myths of Projective Accuracy”
http://www.baseballhq.com/great-myths-projective-accuracy
Baseball Forecaster’s forecasting process
We are all about component skills. Our approach is to assemble these evaluators in such a way that they can be used to validate our observations, analyze their relevance and project a likely future direction.
In a perfect world, if a player’s raw skills improve, then so should his surface stats. If his skills decline, then his stats should follow as well. But, sometimes a player’s skill indicators increase while his surface stats decline. These variances may be due to a variety of factors.
Our forecasting process is based on the expectation that events tend to move towards universal order. Surface stats will eventually approach their skill levels. Unexplained variances will regress to a mean. And from this, we can identify players whose performance may potentially change.
For most of us, this process begins with the previous year’s numbers. Last season provides us with a point of reference, so it’s a natural way to begin the process of looking at the future. Component skills analysis allows us to validate those numbers. A batter with few HRs but a high linear weighted power level has a good probability of improving his future HR output. A pitcher whose ERA was poor while his command ratio was solid might be a good bet for ERA improvement.
Of course, these leading indicators do not always follow the rules. There are more shades of grey than blacks and whites. When indicators are in conflict—for instance, a pitcher who is displaying both a rising strikeout rate and a rising walk rate—then we have to find ways to sort out what these indicators might be saying.
It is often helpful to look at leading indicators in a hierarchy, of sorts. In fact, a hierarchy of the most important pitching base performance indicators might look like this: Command (k/bb), Dominance (k/9), Control (bb/9) and GB/FB rate. For batters, contact rate might top the list, followed by power, walk rate and speed.
Assimilating additional research
Once we’ve painted the statistical picture of a player’s potential, we then use additional criteria and research results to help us add some color to the analysis. These other criteria include the player’s health, age, changes in role, ballpark and a variety of other factors. We also use the research results described in the following pages. This research looks at things like traditional periods of peak performance and breakout profiles.
The final element of the process is assimilating the news into the forecast. This is the element that many fantasy leaguers tend to rely on most since it is the most accessible. However, it is also the element that provides the most noise. Players, management and the media have absolute control over what we are allowed to know. Factors such as hidden injuries, messy divorces and clubhouse unrest are routinely kept from us, while we are fed red herrings and media spam. We will never know the entire truth.
Quite often, all you are reading is just other people’s opinions... a manager who believes that a player has what it takes to be a regular or a team physician whose diagnosis is that a player is healthy enough to play. These words from experts have some element of truth, but cannot be wholly relied upon to provide an accurate expectation of future events. As such, it is often helpful to develop an appropriate cynicism for what you read.
For instance, if a player is struggling for no apparent reason and there are denials about health issues, don’t dismiss the possibility that an injury does exist. There are often motives for such news to be withheld from the public.
And so, as long as we do not know all the facts, we cannot dismiss the possibility that any one fact is true, no matter how often the media assures it, deplores it, or ignores it. Don’t believe everything you read; use your own judgment. If your observations conflict with what is being reported, that’s powerful insight that should not be ignored.
Also remember that nothing lasts forever in major league baseball. Reality is fluid. One decision begets a series of events that lead to other decisions. Any reported action can easily be reversed based on subsequent events. My favorite examples are announcements of a team’s new bullpen closer. Those are about the shortest realities known to man.
We need the media to provide us with context for our analyses, and the real news they provide is valuable intelligence. But separating the news from the noise is difficult. In most cases, the only thing you can trust is how that player actually performs.
Embracing imprecision
Precision in baseball prognosticating is a fool’s quest. There are far too many unexpected variables and noise that can render our projections useless. The truth is, the best we can ever hope for is to accurately forecast general tendencies and percentage plays.
However, even when you follow an 80% percentage play, for instance, you will still lose 20% of the time. That 20% is what skeptics use as justification to dismiss prognosticators; they conveniently ignore the more prevalent 80%. The paradox, of course, is that fantasy league titles are often won or lost by those exceptions. Still, long-term success dictates that you always chase the 80% and accept the fact that you will be wrong 20% of the time. Or, whatever that percentage play happens to be.
For fantasy purposes, playing the percentages can take on an even less precise spin. The best projections are often the ones that are just far enough away from the field of expectation to alter decision-making. In other words, it doesn’t matter if I project Player X to bat .320 and he only bats .295; it matters that I project .320 and everyone else projects .280. Those who follow my less-accurate projection will go the extra dollar to acquire him in their draft.
Or, perhaps we should evaluate the projections based upon their intrinsic value. For instance, coming into 2015, would it have been more important for me to tell you that Anthony Rizzo was going to hit 30 HRs or that Manny Machado would hit 25 HRs? By season’s end, the Rizzo projection would have been more accurate, but the Machado projection—even though it was off by 10 HRs—would have been far more valuable. The Machado projection might have persuaded you to go an extra buck on Draft Day, yielding far more profit.
And that has to be enough. Any tout who projects a player’s statistics dead-on will have just been lucky with his dart throws that day.
Perpetuity
Forecasting is not an exercise that produces a single set of numbers. It is dynamic, cyclical and ongoing. Conditions are constantly changing and we must react to those changes by adjusting our expectations. A pre-season projection is just a snapshot in time. Once the first batter steps to the plate on Opening Day, that projection has become obsolete. Its value is merely to provide a starting point, a baseline for what is about to occur.
During the season, if a projection appears to have been invalidated by current performance, the process continues. It is then that we need to ask... What went wrong? What conditions have changed? In fact, has anything changed? We need to analyze the situation and revise our expectation, if necessary. This process must be ongoing.
When good projections go bad
Although we’d like to think otherwise, we cannot predict the future. All we can do is provide a sound process for constructing a “most likely expectation for future performance.” If we’ve captured as much information as is available, used the best methodology and analyzed the results correctly, that’s the best we can do.
All we can control is the process. We simply can’t control outcomes.
However, one thing we can do is analyze the misses to see why they occurred. This is always a valuable exercise each year. It puts a proper focus on the variables that were out of our control as well as providing perspective on those players with whom we might have done a better job.
In general, we can organize these forecasting misses into several categories. To demonstrate, here are all the players whose 2015 Rotisserie earnings varied from projections by at least $10.
The performances that exceeded expectation
Development beyond the growth trend: These are young players for whom we knew there was skill. Some of them were prized prospects in the past who have taken their time ascending the growth curve. Others were a surprise only because their performance spike arrived sooner than anyone anticipated... Chris Archer, Nolan Arenado, Jake Arrieta, Charlie Blackmon, Xander Bogaerts, Kris Bryant, Lorenzo Cain, Gerrit Cole, Jacob deGrom, Bryce Harper, Ender Inciarte, Dallas Keuchel, Manny Machado, Carlos Martinez, David Peralta, A.J. Pollock, Danny Salazar.
Skilled players who just had big years: We knew these guys were good too; we just didn’t anticipate they’d be this good... Yoenis Cespedes, Nelson Cruz, Josh Donaldson, Zack Greinke, John Lackey.
Unexpected health: We knew this player had the goods; we just didn’t know whether he’d be healthy or would stay healthy all year... Jaime Garcia
Unexpected playing time: These players had the skills—and may have even displayed them at some time in the past—but had questionable playing time potential coming into this season. Some benefited from another player’s injury, a rookie who didn’t pan out or leveraged a short streak into a regular gig... Billy Burns, Carlos Correa, Matt Duffy, Gerardo Parra.
Unexpected return to form: These players had the skills, having displayed them at some point in the past. But those skills had been M.I.A. long enough that we began to doubt that they’d ever return; our projections model got tired of waiting. Or those previous skills displays were so inconsistent that projecting an “up year” would have been a shot in the dark; our projections model got tired of guessing. Yes, “once you display a skill, you own it” but still... Chris Davis, Kendrys Morales, Danny Valencia.
Unexpected role: This category is reserved for 2015’s surprise closers. There are always some every year, relievers who are on nearly nobody’s radar for front-line saves and are suddenly thrust into the role with great success (some did not clear the $10 hurdle but are worth mentioning anyway)... Santiago Casilla, Wade Davis, Jeurys Familia, Ken Giles, Jason Grilli, Andrew Miller, Roberto Osuna, A.J. Ramos, Carson Smith, Joakim Soria, Shawn Tolleson, Brad Ziegler.
Celebrate and claim we’re geniuses: How these players put up the numbers they did is a mystery, but fantasy owners will likely chalk it up to their own superior scouting skills as they count their winnings. The truth is, who knows? However, the odds of a comparable follow-up for these players—particularly those with soft peripherals—will be small:
Logan Forsythe came into 2015 as a barely draftable commodity. He had shown some small signs of productivity back in 2012 (.273 BA over 315 AB) but has looked up at a .225 BA in the two years since. He got off to a good start this year and benefited from some early injuries to pick up playing time, but nobody expected a 17-HR, .281 season.
If you found yourself with 31-year-old Chris Colabello on your roster, congratulations. Now you can feel safe that you’ve bucked the odds of getting struck by lightning. This 15-HR, .321 season was a nice surprise, but with a 41% hit rate, .272 xBA and a 94 xPX, this is not something you’ll ever see again.
Marco Estrada was not supposed to be this good. A flyball pitcher moving into the Rogers Centre was supposed to be a disaster waiting to happen. Instead, he posted the best numbers of his career. But beware: his 3.13 ERA belies a 4.61 xERA. He’s showing declining Dom, rising Ctl (and declining FpK rate), a low 22% hit rate and the highest flyball rate of his career. He’s still a disaster waiting to happen.
The performances that fell short of expectation
The DL denizens: These are players who got hurt, may not have returned fully healthy, or may have never been fully healthy (whether they’d admit it or not)... Matt Adams, Homer Bailey, Miguel Cabrera, Alex Cobb, Carl Crawford, Corey Dickerson, Sean Doolittle, Jacoby Ellsbury, Neftali Feliz, Doug Fister, Freddie Freeman, Yan Gomes, Carlos Gomez, Alex Gordon, Greg Holland, Matt Holliday, Hisashi Iwakuma, Jon Jay, Desmond Jennings, Jacob Lamb, Mat Latos, James Loney, Jed Lowrie, Jonathan Lucroy, Leonys Martin, Brandon McCarthy, Jake McGee, Devin Mesoraco, Justin Morneau, Michael Morse, Wil Myers, Steve Pearce, Dustin Pedroia, Hunter Pence, Yasiel Puig, Anthony Rendon, Alex Rios, Michael Saunders, Drew Smyly, Denard Span, George Springer, Giancarlo Stanton, Koji Uehara, Adam Wainwright, Jered Weaver, Jayson Werth, Matt Wieters, David Wright, Ryan Zimmerman.
(Some of these players seemed to be putting up sub-par numbers before they actually hit the DL. They may have been playing through the hurt before breaking down.)
Accelerated skills erosion: These are players who we knew were on the downside of their careers or had soft peripherals but we did not think they would plummet so quickly. In some cases, there were injuries involved, but all in all, 2015 might be the beginning of the end for some of these guys... Erick Aybar, Adrian Beltre, Michael Bourn, Rajai Davis, Alcides Escobar, Omar Infante, Adam LaRoche, Kyle Lohse, Yadier Molina, Brandon Moss, Mike Napoli, Aramis Ramirez, Jose Reyes, Fernando Rodney, Chase Utley.
Inflated expectations: Here are players who we really should not have expected much more than what they produced. Some had short or spotty track records, others had soft peripherals coming into 2015, and still others were inflated by media hype. Yes, for some of these, it was “What the heck was I thinking?” For others, we’ve almost come to expect players to ascend the growth curve faster these days. (You’re 23 and you haven’t broken out yet? What’s the problem??) The bottom line is that player performance trends simply don’t progress or regress in a straight line; still, the BPI trends were intriguing enough to take a leap of faith. We were wrong... Billy Butler, Robinson Cano, Andrew Cashner, Rusney Castillo, Starlin Castro, Lonnie Chisenhall, Michael Cuddyer, Ian Desmond, Jarrod Dyson, Scooter Gennett, Gio Gonzalez, Josh Harrison, Phil Hughes, Austin Jackson, Chris Johnson, Juan Lagares, Victor Martinez, Marcell Ozuna, Angel Pagan, Michael Pineda, Dalton Pompey, Hanley Ramirez, Wilson Ramos, Jimmy Rollins, Wilin Rosario, Pablo Sandoval, Daniel Santana, Jorge Soler, Steven Souza, Julio Teheran, Chris Tillman, Alex Wood, Eric Young, Jr.
Misplaced regression: Sometimes, we’re so bullish on a player that we ignore the potential for regression within the bounds of normal random variance. Gravity is a powerful force, for... Chris Carter, Johnny Cueto, Cole Hamels, Felix Hernandez, Chris Sale, James Shields, Joe Smith, Drew Storen, Stephen Strasburg, Jordan Zimmermann.
Unexpected loss of role: This category is usually composed of closers who lost their job, sometimes through no fault of their own... Dellin Betances, Brett Cecil, Steve Cishek, Jennry Mejia, Addison Reed, Sergio Romo, Drew Storen.
Throw our hands up and yell at the TV: These are the players for whom there is little explanation for what happened. We can speculate that they hid an injury, went off of PEDs, or just didn’t have their head on right in 2015. For some, it was just the turn of an unlucky card this year:
I’m not sure what to think about Troy Tulowitzki anymore. He was hitting well in the first half in Colorado, but his power was M.I.A. When he got dealt to Toronto, his batting average not unexpectedly cratered. And then a shoulder injury in September mercifully ended his season. He’ll end up with the most ABs since 2011. Is a sub-20 HR, .280 hitter what he’s become?
All sorts of problems dogged Anibal Sanchez but his skills remained intact for a good part of the season. He entered June with a 5.75 ERA but his xERA was nearly two runs better. He had a solid June and came out of the first half with a 94 BPV. The wheels came off in the second half and that could be injury-related as he was eventually shut down in August. I’d be in on him again in 2016.
With the exception of 2014, Jeff Samardzija has always seemed to underperform his skills set. This year, the skills tumbled as well. Perhaps the most notable – and surprising – shift was his ground ball to fly ball ratio, which plummeted from 1.61 to 0.98 and led to a spike in HRs. U.S. Cellular was partially to blame, but his road numbers weren’t great either.
About fantasy baseball touts
As a group, there is a strong tendency for all pundits to provide numbers that are publicly palatable, often at the expense of realism. That’s because committing to either end of the range of expectation poses a high risk. Few touts will put their credibility on the line like that, even though we all know that those outliers are inevitable. Among our projections, you will find few .350 hitters and 70-steal speedsters. Someone is going to post a sub-2.50 ERA next year, but damned if any of us will commit to that. So we take an easier road. We’ll hedge our numbers or split the difference between two equally possible outcomes.
In the world of prognosticating, this is called the comfort zone. This represents the outer tolerances for the public acceptability of a set of numbers. In most circumstances, even if the evidence is outstanding, prognosticators will not stray from within the comfort zone.
As for this book, occasionally we do commit to outlying numbers when we feel the data support it. But on the whole, most of the numbers here can be nearly as cowardly as everyone else’s. We get around this by providing “color” to the projections in the capsule commentaries. That is where you will find the players whose projection has the best potential to stray beyond the limits of the comfort zone.
As analyst John Burnson once wrote: “The issue is not the success rate for one player, but the success rate for all players. No system is 100% reliable, and in trying to capture the outliers, you weaken the middle and thereby lose more predictive pull than you gain. At some level, everyone is an exception!”
Formula for consistent success
Anyone can win a league in any given season. Winning once proves very little, especially in redraft leagues. True success has to be defined as the ability to win consistently. It is a feat in itself to reach the mountaintop, but the battle isn’t truly won unless you can stay atop that peak while others keep trying to knock you off.
What does it take to win that battle? We surveyed 12 of the most prolific fantasy champions in national experts league play. Here is how they rated six variables:
Percent Ranked 1-2 3-4 5-6 Score Better in-draft strategy/tactics 77% 15% 7% 5.00 Better sense of player value 46% 46% 7% 4.15 Better luck 46% 23% 31% 3.85 Better grasp of contextual 31% 38% 31% 3.62 elements that affect players Better in-season 31% 38% 31% 3.54 roster management Better player projections 12% 31% 54% 2.62
Validating Performance
Performance validation criteria
The following is a set of support variables that helps determine whether a player’s statistical output is an accurate reflection of his skills. From this we can validate or refute stats that vary from expectation, essentially asking, is this performance “fact or fluke?”
1. Age: Is the player at the stage of development when we might expect a change in performance?
2. Health: Is he coming off an injury, reconditioned and healthy for the first time in years, or a habitual resident of the disabled list?
3. Minor league performance: Has he shown the potential for greater things at some level of the minors? Or does his minor league history show a poor skill set that might indicate a lower ceiling?
4. Historical trends: Have his skill levels over time been on an upswing or downswing?
5. Component skills indicators: Looking beyond batting averages and ERAs, what do his support ratios look like?
6. Ballpark, team, league: Pitchers going to Colorado will see their ERA spike. Pitchers going to Oakland will see their ERA improve.
7. Team performance: Has a player’s performance been affected by overall team chemistry or the environment fostered by a winning or losing club?
8. Batting stance, pitching style/mastery: Has a change in performance been due to a mechanical adjustment?
9. Usage pattern, lineup position, role: Has a change in RBI opportunities been a result of moving further up or down in the batting order? Has pitching effectiveness been impacted by moving from the bullpen to the rotation?
10. Coaching effects: Has the coaching staff changed the way a player approaches his conditioning, or how he approaches the game itself?
11. Off-season activity: Has the player spent the winter frequenting workout rooms or banquet tables?
12. Personal factors: Has the player undergone a family crisis? Experienced spiritual rebirth? Given up red meat? Taken up testosterone?
Skills ownership
Once a player displays a skill, he owns it. That display could occur at any time—earlier in his career, back in the minors, or even in winter ball play. And while that skill may lie dormant after its initial display, the potential is always there for him to tap back into that skill at some point, barring injury or age. That dormant skill can reappear at any time given the right set of circumstances.
Caveats:
1. The initial display of skill must have occurred over an extended period of time. An isolated 1-hit shut-out in Single-A ball amidst a 5.00 ERA season is not enough. The shorter the display of skill in the past, the more likely it can be attributed to random chance. The longer the display, the more likely that any re-emergence is for real.
2. If a player has been suspected of using performance enhancing drugs at any time, all bets are off.
Corollaries:
1. Once a player displays a vulnerability or skills deficiency, he owns that as well. That vulnerability could be an old injury problem, an inability to hit breaking pitches, or just a tendency to go into prolonged slumps.
2. The probability of a player correcting a skills deficiency declines with each year that deficiency exists.
Normal Production Variance (Patrick Davitt)
Even if we have a perfectly accurate understanding of a player’s “normal” performance level, his actual performance can and does vary widely over any particular 150-game span—including the 150-game span we call “a season.” A .300 career hitter can perform in a range of .250-.350, a 40-HR hitter from 30-50, and a 3.70/1.15 pitcher from 2.60/0.95 to 6.00/1.55. And all of these results must be considered “normal.”
Contract year performance (Tom Mullooly)
There is a contention that players step up their game when they are playing for a contract. Research looked at contract year players and their performance during that year as compared to career levels. Of the batters and pitchers studied, 53% of the batters performed as if they were on a salary drive, while only 15% of the pitchers exhibited some level of contract year behavior.
How do players fare after signing a large contract (minimum $4M per year)? Research from 2005-2008 revealed that only 30% of pitchers and 22% of hitters exhibited an increase of more than 15% in BPV after signing a large deal either with their new team, or re-signing with the previous team. But nearly half of the pitchers (49%) and nearly half of the hitters (47%) saw a drop in BPV of more than 15% in the year after signing.
Risk management and reliability grades
Forecasts are constructed with the best data available, but there are factors that can impact the variability. One way we manage this risk is to assign each player Reliability Grades. The more certainty we see in a data set, the higher the reliability grades assigned to that player. The following variables are evaluated:
Health: Players with a history of staying healthy and off the DL are valuable to own. Unfortunately, while the ability to stay healthy can be considered skill, it is not very projectable. We can track the number of days spent on the disabled list and draw rough conclusions. The grades in the player boxes also include an adjustment for older players, who have a higher likelihood of getting hurt. That is the only forward-looking element of the grade.
“A” level players would have accumulated fewer than 30 days on the major league DL over the past five years. “F” grades go to those who’ve spent more than 120 days on the DL. Recent DL stays are given a heavier weight in the calculation.
Playing Time and Experience (PT/Exp): The greater the pool of MLB history to draw from, the greater our ability to construct a viable forecast. Length of service—and consistent service—is important. So players who bounce up and down from the majors to the minors are higher risk players. And rookies are all high risk.
For batters, we simply track plate appearances. Major league PAs have greater weight than minor league PAs. “A” level players would have averaged at least 550 major league PAs per year over the past three years. “F” graded players averaged fewer than 250 major league PA per year.
For pitchers, workload can be a double-edged sword. On one hand, small IP samples are deceptive in providing a read on a pitcher’s true potential. Even a consistent 65-inning reliever can be considered higher risk since it would take just one bad outing to skew an entire season’s work.
On the flipside, high workload levels also need to be monitored, especially in the formative years of a pitcher’s career. Exceeding those levels elevates the risk of injury, burnout, or breakdown. So, tracking workload must be done within a range of innings. The grades capture this.
Consistency: Consistent performers are easier to project and garner higher reliability grades. Players that mix mediocrity with occasional flashes of brilliance or badness generate higher risk projections. Even those who exhibit a consistent upward or downward trend cannot be considered truly consistent as we do not know whether those trends will continue. Typically, they don’t.
“A” level players are those whose runs created per game level (xERA for pitchers) has fluctuated by less than half a run during each of the past three years. “F” grades go to those whose RC/G or xERA has fluctuated by two runs or more.
Remember that these grades have nothing to do with quality of performance; they strictly refer to confidence in our expectations. So a grade of AAA for Kyle Lohse, for instance, only means that there is a high probability he will perform as poorly as we’ve projected.
Reliability and age
Peak batting reliability occurs at ages 29 and 30, followed by a minor decline for four years. So, to draft the most reliable batters, and maximize the odds of returning at least par value on your investments, you should target the age range of 28-34.
The most reliable age range for pitchers is 29-34. While we are forever looking for “sleepers” and hot prospects, it is very risky to draft any pitcher under 27 or over 35.
Evaluating Reliability (Bill Macey)
Fantasy baseball owners are like investors who are always looking for a good return. Calculating our expected return includes assessing the risk of our draft-day investment.
Managing risk leads to two kinds of valuation adjustments. We downgrade talented players we believe to be higher injury risks, who have a history of inconsistent performance, or whose playing time (PT) is less certain. But we upgrade players we deem more reliable with respect to health, consistency, PT, or all three.
When you head into an upcoming auction or draft, consider the following with regard to reliability:
• Reliability grades do help identify more stable investments: players with “B” grades in both Health and PT/Experience are more likely to return a higher percentage of their projected value.
• While top-end starting pitching may be more reliable than ever, the overall pool of pitchers is fraught with uncertainty and the position represents a less reliable investment than batters.
• There does not appear to be a significant market premium for reliability, at least according to the criteria measured by BaseballHQ.com.
• There are only two types of players: risky and riskier. So while it may be worth going the extra buck for a more reliable player, be warned that even the most reliable player can falter—don’t go overboard bidding up a AAA-rated player simply due to his Reliability grades.
Using 3-year trends as leading indicators (Ed DeCaria)
It is almost irresistibly tempting to look at three numbers moving in one direction and expect that the fourth will continue that progression. However, for both hitters and pitchers riding positive trends over any consecutive three-year period, not only do most players not continue their positive trend into a fourth year, their Year 4 performance usually regresses significantly. This is true for every metric tested (whether related to playing time, batting skills, pitching skills, running skills, luck indicators, or valuation). Negative trends show similar reversals, but tend to be more “sticky,” meaning that rebounds are neither as frequent nor as strong as positive trend regressions. Challenge any analysis that hints at a player’s demise coming off of a negative trend or that suggests an imminent breakout following a positive trend; more often than not, such predictions do not pan out.
Health Analysis
Disabled list statistics
Year #Players 3yr Avg DL Days 3yr Avg 2008 422 391 28,187 26,394 2009 408 411 26,252 27,654 2010 393 408 22,911 25,783 2011 422 408 25,610 24,924 2012 409 408 30,408 27,038 2013 442 419 29,551 28,523 2014 422 424 25,839 28,599 2015 454 439 28,982 28,124
D.L. days as a leading indicator (Bill Macey)
Players who are injured in one year are likely to be injured in a subsequent year:
% DL batters in Year 1 who are also DL in year 2 38%
Under age 30 36%
Age 30 and older 41%
% DL batters in Year 1 and 2 who are also DL in year 3 54%
% DL pitchers in Year 1 who are also DL in year 2 43%
Under age 30 45%
Age 30 and older 41%
% DL pitchers in Yr 1 and 2 who are also DL in year 3 41%Previously injured players also tend to spend a longer time on the DL. The average number of days on the DL was 51 days for batters and 73 days for pitchers. For the subset of these players who get hurt again the following year, the average number of days on the DL was 58 days for batters and 88 days for pitchers.
Spring training spin (Dave Adler)
Spring training sound bites raise expectations among fantasy leaguers, but how much of that “news” is really “noise”? Thanks to a summary listed at RotoAuthority.com, we were able to compile the stats for 2009. Verdict: Noise.
BATTERS No. IMPROVED DECLINED Weight change 30 33% 30% Fitness program 3 0% 67% Eye surgery 6 50% 33% Plans more SB 6 17% 33% PITCHERS No. IMPROVED DECLINED Weight change 18 44% 44% Fitness program 4 50% 50% Eye surgery 2 0% 50% New pitch 5 60% 40%
In-Season Analysis
April performance as a leading indicator
We isolated all players who earned at least $10 more or $10 less than we had projected in March. Then we looked at the April stats of these players to see if we could have picked out the $10 outliers after just one month.
Identifiable in April
Earned $10+ more than projected
BATTERS 39%
PITCHERS 44%
Earned -$10 less than projected
BATTERS 56%
PITCHERS 74%Nearly three out of every four pitchers who earned at least $10 less than projected also struggled in April. For all the other surprises—batters or pitchers—April was not a strong leading indicator. Another look:
Pct.
Batters who finished +$25 45%
Pitchers who finished +$20 44%
Batters who finished under $0 60%
Pitchers who finished under -$5 78%April surgers are less than a 50/50 proposition to maintain that level all season. Those who finished April at the bottom of the roto rankings were more likely to continue struggling, especially pitchers. In fact, of those pitchers who finished April with a value under -$10, 91% finished the season in the red. Holes are tough to dig out of.
The weight of early season numbers
Early season strugglers who surge later in the year get no respect because they have to live with the weight of their early numbers all season long. Conversely, quick starters who fade late get far more accolades than they deserve.
For instance, take Pablo Sandoval’s month-by-month batting average. The perception is that his .245 BA was a disappointment. Well, it was more than just a disappointment. His hot .312 start masked what was an even worse performance. From May 1 on, he batted just .232.
Month BA Cum BA April .312 .312 May .200 .251 June .298 .267 July .241 .260 August .205 .249 September .205 .245
Seasonal trends in hitting and pitching (Bob Berger)
A study of monthly trends in traditional statistical categories found:
• Batting average, HR/game and RBI/game rise from April through August, then fall in September/October.
• Stolen bases decline in July and August before rebounding in September.
• ERA worsens in July/August and improves in September.
• WHIP gets worse in July/August.
• K/9 rate improves all season.
The bold statement that hitters perform better in warmer weather seems to be true broadly.
Courtship period
Any time a player is put into a new situation, he enters into what we might call a courtship period. This period might occur when a player switches leagues, or switches teams. It could be the first few games when a minor leaguer is called up. It could occur when a reliever moves into the rotation, or when a lead-off hitter is moved to another spot in the lineup. There is a team-wide courtship period when a manager is replaced. Any external situation that could affect a player’s performance sets off a new decision point in evaluating that performance.
During this period, it is difficult to get a true read on how a player is going to ultimately perform. He is adjusting to the new situation. Things could be volatile during this time. For instance, a role change that doesn’t work could spur other moves. A rookie hurler might buy himself a few extra starts with a solid debut, even if he has questionable skills.
It is best not to make a decision on a player who is going through a courtship period. Wait until his stats stabilize. Don’t cut a struggling pitcher in his first few starts after a managerial change. Don’t pick up a hitter who smacks a pair of HRs in his first game after having been traded. Unless, of course, talent and track record say otherwise.
Half-season fallacies
A popular exercise at the midpoint of each season is to analyze those players who are consistent first half to second half surgers or faders. There are several fallacies with this analytical approach.
1. Half-season consistency is rare. There are very few players who show consistent changes in performance from one half of the season to the other.
Research results from a three-year study conducted in the late-1990s: The test groups... batters with min. 300 AB full season, 150 AB first half, and pitchers with min. 100 IP full season, 50 IP first half. Of those groups (size noted):
3-year consistency in BATTERS (98) PITCHERS (42) 1 stat category 40% 57% 2 stat categories 18% 21% 3 stat categories 3% 5%
When the analysis was stretched to a fourth year, only 1% of all players showed consistency in even one category.
2. Analysts often use false indicators. Situational statistics provide us with tools that can be misused. Several sources offer up 3- and 5-year stats intended to paint a picture of a long-term performance. Some analysts look at a player’s half-season swing over that multi-year period and conclude that he is demonstrating consistent performance.
The fallacy is that those multi-year scans may not show any consistency at all. They are not individual season performances but aggregate performances. A player whose 5-year batting average shows a 15-point rise in the 2nd half, for instance, may actually have experienced a BA decline in several of those years, a fact that might have been offset by a huge BA rise in one of the years.
3. It’s arbitrary. The season’s midpoint is an arbitrary delineator of performance swings. Some players are slow starters and might be more appropriately evaluated as pre-May 1 and post-May 1. Others bring their game up a notch with a pennant chase and might see a performance swing with August 15 as the cut-off. Each player has his own individual tendency, if, in fact, one exists at all. There’s nothing magical about mid-season as the break point, and certainly not over a multi-year period.
Half-season tendencies
Despite the above, it stands to reason logically that there might be some underlying tendencies on a more global scale, first half to second half. In fact, one would think that the player population as a whole might decline in performance as the season drones on. There are many variables that might contribute to a player wearing down—workload, weather, boredom—and the longer a player is on the field, the higher the likelihood that he is going to get hurt. A recent 5-year study uncovered the following tendencies:
Batting
Overall, batting skills held up pretty well, half to half. There was a 5% erosion of playing time, likely due, in part, to September roster expansion.
Power: First half power studs (20 HRs in 1H) saw a 10% drop-off in the second half. 34% of first half 20+ HR hitters hit 15 or fewer in the second half and only 27% were able to improve on their first half output.
Speed: Second half speed waned as well. About 26% of the 20+ SB speedsters stole at least 10 fewer bases in the second half. Only 26% increased their second half SB output at all.
Batting average: 60% of first half .300 hitters failed to hit .300 in the second half. Only 20% showed any second half improvement at all. As for 1H strugglers, managers tended to stick with their full-timers despite poor starts. Nearly one in five of the sub-.250 1H hitters managed to hit more than .300 in the second half.
Pitching
Overall, there was some slight erosion in innings and ERA despite marginal improvement in some peripherals.
ERA: For those who pitched at least 100 innings in the first half, ERAs rose an average of 0.40 runs in the 2H. Of those with first half ERAs less than 4.00, only 49% were able to maintain a sub-4.00 ERA in the second half.
Wins: Pitchers who won 18 or more games in a season tended to pitch more innings in the 2H and had slightly better peripherals.
Saves: Of those closers who saved 20 or more games in the first half, only 39% were able to post 20 or more saves in the 2H, and 26% posted fewer than 15 saves. Aggregate ERAs of these pitchers rose from 2.45 to 3.17, half to half.
Teams
Johnson Effect (Bryan Johnson): Teams whose actual won/loss record exceeds or falls short of their statistically projected record in one season will tend to revert to the level of their projection in the following season.
Law of Competitive Balance (Bill James): The level at which a team (or player) will address its problems is inversely related to its current level of success. Low performers will tend to make changes to improve; high performers will not. This law explains the existence of the Plexiglass and Whirlpool Principles.
Plexiglass Principle (Bill James): If a player or team improves markedly in one season, it will likely decline in the next. The opposite is true but not as often (because a poor performer gets fewer opportunities to rebound).
Whirlpool Principle (Bill James): All team and player performances are forcefully drawn to the center. For teams, that center is a .500 record. For players, it represents their career average level of performance.
Other Diamonds
The Fanalytic Fundamentals
1. This is not a game of accuracy or precision. It is a game of human beings and tendencies.
2. This is not a game of projections. It is a game of market value versus real value.
3. Draft skills, not stats. Draft skills, not roles.
4. A player’s ability to post acceptable stats despite lousy BPIs will eventually run out.
5. Once you display a skill, you own it.
6. Virtually every player is vulnerable to a month of aberrant performance. Or a year.
7. Exercise excruciating patience.
Aging Axioms
1. Age is the only variable for which we can project a rising trend with 100% accuracy. (Or, age never regresses.)
2. The aging process slows down for those who maintain a firm grasp on the strike zone. Plate patience and pitching command can preserve any waning skill they have left.
3. Negatives tend to snowball as you age.
Steve Avery List
Players who hang onto MLB rosters for six years searching for a skill level they only had for three.
Bylaws of Badness
1. Some players are better than an open roster spot, but not by much.
2. Some players have bad years because they are unlucky. Others have many bad years because they are bad... and lucky.
George Brett Path to Retirement
Get out while you’re still putting up good numbers and the public perception of you is favorable. Like Mike Mussina, Billy Wagner, Chipper Jones and Mariano Rivera.
Steve Carlton Path to Retirement
Hang around the majors long enough for your numbers to become so wretched that people begin to forget your past successes.
Classic cases include Jose Mesa, Doc Gooden, Nomar Garciaparra and of course, Steve Carlton. Recent players who have taken this path include Miguel Tejada, Travis Hafner, Jason Bay, Brian Roberts and Kevin Youkilis. Current players who could be on a similar course include Jimmy Rollins, Chase Utley and Dan Uggla.
Christie Brinkley Law of Statistical Analysis
Never get married to the model.
Employment Standards
1. If you are right-brain dominant, own a catcher’s mitt and are under 40, you will always be gainfully employed.
2. Some teams believe that it is better to employ a player with any experience because it has to be better than the devil they don’t know.
3. It’s not so good to go pffft in a contract year.
Laws of Prognosticating Perspective
• Berkeley’s 17th Law: A great many problems do not have accurate answers, but do have approximate answers, from which sensible decisions can be made.
• Ashley-Perry Statistical Axiom #4: A complex system that works is invariably found to have evolved from a simple system that works.
• Baseball Variation of Harvard Law: Under the most rigorously observed conditions of skill, age, environment, statistical rules and other variables, a ballplayer will perform as he damn well pleases.
Brad Fullmer List
Players whose leading indicators indicate upside potential, year after year, but consistently fail to reach that full potential. Players like Justin Smoak, Josh Rutledge, Brett Lawrie are on the list right now.
Good Luck Truism
Good luck is rare and everyone has more of it than you do. That’s the law.
The Gravity Principles
1. It is easier to be crappy than it is to be good.
2. All performance starts at zero, ends at zero and can drop to zero at any time.
3. The odds of a good performer slumping are far greater than the odds of a poor performer surging.
4. Once a player is in a slump, it takes several 3-for-5 days to get out of it. Once he is on a streak, it takes a single 0-for-4 day to begin the downward spiral.
Corollary: Once a player is in a slump, not only does it take several 3-for-5 days to get out of it, but he also has to get his name back on the lineup card.
5. Eventually all performance comes down to earth. It may take a week, or a month, or may not happen until he’s 45, but eventually it’s going to happen.
Health Homilies
1. Staying healthy is a skill (and “DL Days” should be a Rotisserie category).
2. A $40 player can get hurt just as easily as a $5 player but is eight times tougher to replace.
3. Chronically injured players never suddenly get healthy.
4. There are two kinds of pitchers: those that are hurt and those that are not hurt... yet.
5. Players with back problems are always worth $10 less.
6. “Opting out of surgery” usually means it’s coming anyway, just later.
The Health Hush
Players get hurt and potentially have a lot to lose, so there is an incentive for them to hide injuries. HIPAA laws restrict the disclosure of health information. Team doctors and trainers have been instructed not to talk with the media. So, when it comes to information on a player’s health status, we’re all pretty much in the dark.
Hidden Injury Progression
1. Player’s skills implode.
2. Team and player deny injury.
3. More unexplained struggles.
4. Injury revealed; surgery follows.
The Livan Level
The point when a player’s career Runs Above Replacement level has dropped so far below zero that he has effectively cancelled out any possible remaining future value. (Similarly, the Dontrelle Demarcation.)
The Momentum Maxims
1. A player will post a pattern of positive results until the day you add him to your roster.
2. Patterns of negative results are more likely to snowball than correct.
3. When an unstoppable force meets an immovable object, the wall always wins.
Paradoxes and Conundrums
1. Is a player’s improvement in performance from one year to the next a point in a growth trend, an isolated outlier or a complete anomaly?
2. A player can play through an injury, post rotten numbers and put his job at risk… or… he can admit that he can’t play through an injury, allow himself to be taken out of the lineup/rotation, and put his job at risk.
3. Did irregular playing time take its toll on the player’s performance or did poor performance force a reduction in his playing time?
4. Is a player only in the game versus right-handers because he has a true skills deficiency versus left-handers? Or is his poor performance versus left-handers because he’s never given a chance to face them?
5. The problem with stockpiling bench players in the hope that one pans out is that you end up evaluating performance using data sets that are too small to be reliable.
6. There are players who could give you 20 stolen bases if they got 400 AB. But if they got 400 AB, they would likely be on a bad team that wouldn’t let them steal.
Process-Outcome Matrix (Russo and Schoemaker)
| Good Outcome | Bad Outcome | |
| Good Process | Deserved Success | Bad Break |
| Bad Process | Dumb Luck | Poetic Justice |
Quack!
An exclamation in response to the educated speculation that a player has used performance enhancing drugs. While it is rare to have absolute proof, there is often enough information to suggest that, “if it looks like a duck and quacks like a duck, then odds are it’s a duck.”
Tenets of Optimal Timing
1. If a second half fader had put up his second half stats in the first half and his first half stats in the second half, then he probably wouldn’t even have had a second half.
2. Fast starters can often buy six months of playing time out of one month of productivity.
3. Poor 2nd halves don’t get recognized until it’s too late.
4. “Baseball is like this. Have one good year and you can fool them for five more, because for five more years they expect you to have another good one.” — Frankie Frisch
The Three True Outcomes
1. Strikeouts
2. Walks
3. Home runs
The Three True Handicaps
1. Has power but can’t make contact.
2. Has speed but can’t hit safely.
3. Has potential but is too old.
Zombie
A player who is indestructible, continuing to get work, year-after-year, no matter how dead his skills metrics are. Like Kevin Correia, Dan Johnson, Travis Ishikawa and Dan Uggla.
User login
Search HQ:
Podcasts
