BaseballHQ.com

On Opening Day 2016, we rolled out our #newPQS scores throughout the site. It has been about four weeks since then, so we wanted to revisit the topic to address reader feedback and other aftereffects related to the methodology change. So we'll answer some common questions; call this our "PQS follow-up five”:

The changes to PQS are too drastic; it is too hard for a starter to earn a PQS-DOM now.
It is correct to say that DOMinant performances are now more rare. But earning a PQS-DOM is not now "too hard.” Rather, under the old methodology, earning a PQS-DOM was just way too easy — a full 50% of starts were classified as DOMinant! Or put another way, anything league average or better was considered DOMinant. Or put another-nother way, any start that wasn’t tagged as DOMinant was statistically very bad.

#newPQS restores the middle class (in the form of PQS-DECent starts: PQS-2s and PQS-3s). Just as importantly, it also brings the extremes (PQS-0s and PQS-5s) into sharper focus. It should be news-worthy (in fantasyland, anyway) when a pitcher throws a PQS-5. With 20-30 pitchers throwing most nights, #newPQS gives us a way to highlight the 2-4 that that were outright DOMinant and the 2-4 that were truly DISastrous.

Speaking of PQS-0s ... How can you possibly call 6IP 6H 2BB 4K 1HR A PQS-0 DISaster?!?
The original article explains this at some length, but this edge case is admittedly difficult to come to terms with since this same line used to be called a PQS-5 DOMinant start under oldPQS, so let’s review our calculation logic again here.

PQS’s value lies in its simplicity. To create such a simple metric, one must use very blunt statistical instruments. In the case of PQS, our statistical instrument is a stick; and the only thing that our stick is good for is to draw a line in the dirt on the pitcher’s mound. For each of our five metrics, we look to see on which side of the line a pitcher lands. PQS does not care if a pitcher lands really, really far away from the line or nearly touches it. It simply looks at the cleat marks, then decides: if the pitcher lands on this side, he earns a point; if he lands on that side, he doesn't.

We drew our new line at IP>6, H<IP, K>=5, K/BB>=3, and HR=0. It doesn’t matter how close pitchers finish to each of those figures. They either make it, or they fall short. When a pitcher falls just short on each metric, such as with a 6IP 6H 2BB 4K 1HR outing, there is no way for PQS to acknowledge how close the pitcher came to reaching the line each time. It just records miss, miss, miss, miss, miss, and awards a PQS-0.

Could we have used our stick to draw our line in a different place? Sure, we could have. But we didn’t just draw our line randomly; we specifically chose its location so that pitchers would, on balance, land on each side with equal frequency (or as equal as possible, anyway) on each metric.

Why do you award a PQS point for only 5Ks in 8 or 9 innings? That’s a below average K/9 rate.
There are a few ways we could have handled strikeouts as a PQS component. Under oldPQS, you had to first look at the pitcher’s innings pitched, subtract two, and then see if the starter racked up more or less than that number of strikeouts. Given how easy all of the other metrics were to “calculate," that always seemed a bit convoluted. Plus, it awarded a point to pitchers who struck out 3 batters in 5 IP, while withholding that point from pitchers who struck out 5 batters in >7IP (or 6 batters in >8 IP).

Once you decouple strikeouts from innings, 5 strikeouts emerges from the data as the perfect place to draw the line. And though striking out “only” 5 batters in >7 IP (or 6 in >8 IP) is technically a below average K/9 rate, on a per-start basis, 5 strikeouts is well above the league average of 4.3 K/start.

And also, now that we’ve removed the automatic PQS-0 rule for starts lasting less than 5IP, we can reward those pitchers who still managed to contribute a helpful strikeout performance despite their brief outing.

What’s the best way to use the newly-termed PQS-DECent starts alongside DOM and DIS starts?
PQS-DECent starts, as demonstrated in the original article, are now statistically league average starts in terms of both skills and surface-level performance. PQS-3s are a bit above average, and PQS-2s are a bit below average, but taken together, they track really close to league average, as we would expect given where we drew our #newPQS line.

Now, a pitcher with a 30%/40%/30% DOM/DEC/DIS split is a roughly league average pitcher. A pitcher who can manage a 30%/60%/10% split, despite being no more dominant, is actually an above-average pitcher, because he’s truly DECent 60% of the time (as opposed to under oldPQS where PQS-2 and PQS-3 results were very bad) and is DOMinant three times more frequently than he is DISastrous in his other starts.

Another interesting PQS split to consider is simply PQS+ vs. PQS- starts. That is, PQS-3 and above vs. PQS-2 and below. Any pitcher whose PQS+ % is higher than his PQS- % is probably a pretty good pitcher. You’d want to dive in further to make sure he doesn’t have 50% PQS-3s vs. 50% PQS-0s, but as a quick filter to separate targets from warnings, PQS+ vs. PQS- does the job. You could even consolidate those into a single number by subtracting PQS- from PQS+ and you’d quickly see the best pitchers (with positive differentials) float to the top and the worst (with negative differentials) sink to the bottom.

How do I use the HQ Pitcher Matchup Tool now? I don’t know what’s good or bad anymore!
The starting pitcher’s score in BaseballHQ.com’s Pitcher Matchup Toll is calculated by taking the starter’s average PQS earned and subtracting (five minus) his opponent’s average PQS allowed. The change to #newPQS has altered the inputs to this equation (on both sides). While that was expected, it did throw off the matchup score benchmarks that we use to guide our start-or-sit decisions. So, here we will provide new benchmarks for the SP Report.

In the following chart, you see pitchers' earned PQS scores in rows (down the left side) and opposing offenses' PQS allowed in columns (across the top). At each intersection, you see the resulting SP Score for that type of matchup.

The change to #newPQS has shifted the league averages from the thick outlined area (1.0-2.0) to the black fill area (0.0).

Under oldPQS, a typical starting pitcher's PQS average would be somewhere between 3.0 and 3.5. It mathematically followed that a typical offense would allow that typical opposing pitcher to earn a PQS somewhere between 3.0 and 3.5. The SP score formula doesn't care about these averages explicitly, it just calculates, for example, 3.5 minus 1.5 (where 1.5 = 5.0 minus opposition’s average PQS allowed of 3.5) and yields a score of 2.0. This is why under oldPQS you saw typical SP scores in the 1.0-2.0 range, which became the de facto benchmark.

Under #newPQS, a typical starting pitcher's PQS average is right around 2.5. Again, it mathematically follows that a typical offense would allow that typical opposing pitcher to earn a PQS right around 2.5. Our unsympathetic matchup score formula now just calculates 2.5 minus 2.5 and yields an average score of 0.0. This is now our new benchmark. Now, intuitively, positive is good and negative is bad. And really positive is really good and really negative is really bad.

Distribution-wise, about 40% of SP matchup scores fall between -0.5 and 0.5. About 25% of starts will be above 0.5, and 35% will be below -0.5. But the ones we really want to be on the lookout for are starts that score greater than 1.0 (about 10% of starts) or less than -1.0 (about 20% of starts). For a 1.00 SP Score, we'd expect a PQS-3 based on history, and with good performance by the pitcher and/or an off day by the offense, even higher. It’s an “opportunity for DOMinance.” For a -1.00 SP Score, we'd expect a PQS-2 based on history, and with a bad performance by the pitcher and/or a good day by the offense, even lower. It’s a “risk of DISaster.”

**Bonus Follow-Up Coverage!*** Which players look most different under #newPQS vs. oldPQS?
Since astute readers may note that PQS is really a six-point scale (0-1-2-3-4-5), we’ll address one final topic: Which players are particularly affected, for better or worse, by our PQS methodology change (which on average reduced the number of DOMinant starts by half and more evenly distributed the PQS-0s up to other levels by eliminating the auto-zero rule)?

On the negative side, there are pitchers whose DOM/DEC/DIS splits take a sharper dive than expected given the methodology change. Take a look at these five pitchers from 2015:

Tyson Ross looked elite under oldPQS with his 73/24/3 DOM/DEC/DIS split. But 40% of his DOMinant starts fell to DECent or DISaster starts under #newPQS. As a result his average PQS score fell from 3.88 (ranked 15th highest) to 2.85 (ranked 26th). Still solid, but not near elite.

James Shields had similar numbers to Ross but fell even more drastically under #newPQS. Why? Because while oldPQS classified 39% of his 2015 starts as PQS-5, #newPQS knocked that figure all the way down to 3%. Meaning just one PQS-5 outing. His game log shows there was always something that kept him from pure dominance — a home run, a few too many walks, getting pulled before entering the 7th inning … something.

Michael Wacha experienced more of a general downshift in his PQS scores. His DOMinant percentage fell from 70% under oldPQS to only 23% under new PQS, while his DECent percentage jumped from 17% to 53%. He still ranked a solid 43rd in average #newPQS score. 

Garrett Richards saw his DISaster percentage balloon from 9% to 31% of starts. His downfalls were his propensity to go exactly 6 IP, his evenly spread 20 HR allowed, and his below average command.

Ian Kennedy was very interesting because his PQS scores became drawn more toward DECency from both sides. His biggest downfall was his home runs allowed, which cost him a PQS point in over half of his starts. But he also benefited from removing the auto-zero rule, where he salvaged some abbreviated starts with at least 5 strikeouts.

On the positive side, there are pitchers whose DOM/DEC/DIS splits hold up much better than expected given the methodology change. Again, here are five pitchers from 2015 who fit this mold:

Chris Sale was amazing in 2015 now matter how you looked at him. His starts were typically so far above each component threshold that raising the bar didn’t affect him. He jumped from 9th to 4th in the PQS rankings as others around him were more impacted by the change.

Carlos Carrasco’s DOM/DEC/DIS splits hardly moved an inch with the change to #newPQS. 27% of his starts were still PQS-5s under #newPQS, and his PQS-0 percentage dropped from 23% to 3%, because in five of his sub-5IP starts he still managed to strike out five or more batters. As a result, he jumped from the 31st ranked starter to 11th according to average PQS score.

Stephen Strasburg, even moreso than Carrasco, benefited from removing the auto-zero rule, as he saw most of his PQS-0s erased by exhibiting at least some skill in each outing.

Trever Bauer held up fairly well with the PQS change, as his inconsistency put him clearly over both the old and #newPQS thresholds when he was “on” and clearly under when he was “off.” As a result, many other mid-tier pitchers simply dropped below him in the rankings.

Finally, Mat Latos had a forgettable season to be sure, but taken together his distribution of #newPQS game logs ended up mirroring the league average almost perfectly, as opposed to far worse than average splits under oldPQS.

---

Your continued feedback on #newPQS is welcome as we all get more and more accustomed to this change. Use the comments field below.

Click here to subscribe

  For more information about the terms used in this article, see our Glossary Primer.