(*) GM's OFFICE: Baselining our approach to projecting 2023

GM's Office

Ray Murphy

Dec 19 2022 1:06am

(*) GM's OFFICE: Baselining our approach to projecting 2023

Ray Murphy

Dec 19 2022

It's been a full decade now since Ron Shandler handed me the keys to the BaseballHQ projections system. I remember at the time being wildly excited: after years of gradually assuming various site-administration responsibilities from Ron, taking over the projections was "the big one." And, even though Ron had been preaching his gospel of "The myth of projective accuracy" for just about as long as I'd worked for him, I was excited to dive into the process and, despite Ron's preaching, figure out how to make our projections better.

Heck, I was so excited, I couldn't even imagine why Ron would ever want to relinquish his own control of the projections. This was the nerve center of the site, after all. More importantly it was cool.

Ha. Ha.

It only took one season with my hands in the projections for my wide-eyed optimism (ok, naivete) to be replaced with a level of jadedness that rivaled Ron's. I think I wrote to him near the end of that year and said something to the effect of "Now I understand why you were ready to hand this off to me. Thanks for nothing."

Again, that was 10 years ago. And it's only gotten harder since then. Sure, we've got more and better data, more metrics to measure player performance. That's helpful, though the task of separating signal from noise just gets harder as we get more noise thrown into the system. And where Ron used to complain 10-20 years ago about projections being rendered inaccurate because we didn't know what substances players were or were not putting in their bodies, this decade we have multiple instances of MLB surreptitiously manipulating the baseball, sometimes year-to-year and sometimes within seasons.

Entering 2023, MLB intends to really throw the kitchen sink at us, with a host of rule changes on tap that promise league-wide impact. Worse yet, as Ron pointed out in his introductory essay in the 2023 Baseball Forecaster, MLB isn't even using a controlled approach to introduce well-vetted individual changes over a long time period. Instead, they're rolling out a bunch of changes that may be decently understood individually, but are not at all understood as a collective. It is, as Ron labeled it, chaos theory in action.

Chaos, of course, is the enemy of projective accuracy.

In discussing this looming chaos (both with our subscribers, and other projection providers) at First Pitch Arizona last month, one theme did emerge. While not a substitute for accuracy, transparency is probably the next best thing. Whether or not we choose to modify our projections for these rules changes, it's critical that we make those decisions publicly, so that you know what positions we have taken. If you know what we have done, you are then empowered to make your own decisions on top of ours, to stake out your own positions about who will be impacted by these changes, and to what degree. (Of course, this isn't a new concept to us: we've always been committed to showing our work with our formulas, our playing time allocations, etc. But this was a useful reminder to me of how that same principle applies to this new problem.)

With our initial 2023 projections now live on the site, this seems like the right time to state our positions on these new rules vis a vis our projections; to declare how we intend to handle them over the coming months. Despite the "chaos theory" problem stated above, let's treat these changes as individual items for the moment:

Park factors

For as long as I've been involved in the process, we have used three-year park factors as inputs to our projections system. This makes intuitive sense, as 81 games in a ballpark per year are not necessarily enough of a sample size to conclusively determine how a ballpark will play. However, in the ever-changing MLB environment described above, three-year scans run the risk of becoming something of a Frankenstein concoction of data from wildly different eras. For instance, as our three-year scan rolls from 2019-21 to 2020-22 this year, the "Happy Fun Ball" HR-heavy 2019 season ages out of the data set, potentially causing significant changes in the data.

Perhaps the most important (documented) change in the game last year was the adoption of a universal baseball-storage environment at all 30 ballparks last year. (In 2021, only 10 parks utilized humidors.) The problem with that change is that it creates another Frankenstein effect as alluded to above: for 20 parks, a three-year park effect would be melding two years of non-humidor data with one year of humidor data. And the first "year" of that three-year scan is now 2020, which featured only 30 games per ballpark, all played in summer weather, with other screwy issues like no fans, seven-inning doubleheaders, and so on.

This leads us to our first projections adjustment: we are ditching our three-year park factors in favor of single-year park factors. Single-year factors may be more volatile, but at least they are based on a foundational assumption of all 30 parks using humidors, as they will be again in 2023.

We have updated our Park Factors chart to reflect these 2022 park factors that are baked into our projections. Here is a side-by-side comparison of how the 2019-21 chart versus the 2022-only chart:

2019-21: 2022-only:

I don't intend for you to be able to actually read the data above, but if you just absorb the color palette, heat-map style, you get a sense that the new chart is fairly similar in overall "temperature" to the old one: a similar level of deep blues and reds (the most extreme effects, +/- 10%), albeit distributed differently on the new chart. It's worth clicking through to the new chart and spending a few minutes absorbing it. For instance, did you know that the National League's least HR-friendly park last year was Chase Field in Arizona?

Status: done; changes reflected in the current on-site projections.

Unbalanced schedule

This was an easy one for us, thanks to a little secret that dates back to my taking over projections from Ron a decade ago. Once I learned how the projections system worked, I was surprised to find that there wasn't anything in the system that directly accounted for the unbalanced schedule. There are league-crossing mechanisms built in that account for effects on players changing leagues, but there wasn't anything that natively accounted for teams playing nearly half of their games within the same divisional environment.

Over the years, that became one of the common manual adjustments I would make when reviewing projections. Then, in the shortened 2020 season, when MLB adopted the short-season "pod" format (AL East teams only playing AL East and NL East teams, and so on), I finally updated the system to reflect this. And then I modified it for 2021-22 to accommodate the traditional "unbalanced, but not an outright pod" schedule.

This year, I ripped it out altogether, reverting to the "only league weightings" that we had been using through 2019. 60 intra-division games out of 162 doesn't seem significant enough to warrant any adjustments, especially for pitchers given the vagaries of how their starts might end up distributed by where their rotation turns fall. Even the league adjustment (that I've kept around for now) is really a relic of the no-DH-in-NL era, and perhaps will be retired in the future. (Did you know that more runs were scored, and more HR hit, in the NL last year than in the AL?)

Status: (un)done; changes reflected in the current on-site projections.

Bigger bases/pickoff throw limitations

There seems to be less consensus about how to handle changes to the base size and pickoff throw limitations will impact SBs. Well, that's not entirely true: we think we know the direction of the impact (more stolen bases), but don't know the distribution. Ron Shandler theorized in his Forecaster intro essay that there could be a population of speedsters who hover just under the 80% SB success threshold who are the likely beneficiaries of this change. But we don't know for sure: these changes were made in the minors last year, but the results were somewhat noisy across different leagues, and of course the quality of catcher throwing/pitchers holding runners on is different in the minors than in MLB.

So, this is the first case where we are choosing to take no action, at least for now. We'll reserve the right to make adjustments during spring training, as we start to see this new rule in action. (And we're mindful of our long-ago research on "the one spring training stat that matters", team-level SBA%.) It will be an action item for us to refresh that study before camps open, and be ready to apply it to spring stats this March.

Status: no current changes, to be revisited this March

Pitch clock

BaseballHQ researcher Andy Andres made a compelling case at First Pitch Arizona that the introduction of the pitch clock will be, to use some analytical jargon, a nothingburger.

His argument boils down to a few points:

the pitch clock has been in use in the minors for several years, so most younger pitchers have been exposed to it already
the vast majority of pitchers are either compliant with the clock already, or close enough to compliant that they only need a minor adjustment to get compliant.
the remaining population of veteran pitchers who are very much out of range of the clock right now is vanishingly small.

We could make adjustments for that third group; we have the data to identify them. Again, in his Forecaster intro (which obviously you should read!), Ron identified the primary offenders, and speculated that max-effort closers could be the population most likely to feel the impact, both in terms of reduced effectiveness and potentially more injuries. But then again, one of those primary offenders in the ranks of veteran closers is Kenley Jansen, who just got 2 years/$32M from the Red Sox. Surely, the Red Sox are aware of his pace-of-play issues, and apparently aren't concerned. So maybe we shouldn't be either.

Status: no current changes. (Feel free to de-value Jansen, et al, on your own.)

Defensive shift limitations

We're putting this one in a category of its own: there are clearly some quantifiable adjustments to be made here, but we have not made them yet. Well, that's not completely accurate: as Ron Shandler shared in last week's Baseball Forecaster release party livestream, there were some individual projections where Ron put his thumb on the book projection to reflect impact of the shift change. Max Kepler was one such case he cited, and that projection modification does carry over from the book to the site.

However, I believe there are some more systemic changes that should be made. Derek Carty presented findings on the potential effects of the shift ban at First Pitch Arizona. And I think there's an opportunity to apply his work to our projections data intelligently: using his league-wide findings but then adjusting for contact rate, ground ball rates, potentially even pull% and speed, we can probably create some different tiers or impact levels, and sort batters into those tiers and make BA adjustments accordingly.

Let's not forget, there's also a needed adjustment for pitchers here, and much the same approach can probably work, as we push h% higher, perhaps using GB% as a tiering mechanism.

Clearly, there's work to do here. But I needed to get these baseline projections posted before I could dive into that work. My intent is to get into that work before the end of the calendar year; we'll see how that goes.

Status: no current changes, but they're coming (hopefully soon).

Status: projections were updated for the shift on February 9th, with the process detailed in this follow-up article.

Follow @RayHQ