"Uz at Your Own Peril"

Sorry if the tone of this piece is a little more surly than usual. It's mostly a case of being overly annoyed at the gold glove announcements.

I love Fangraphs.com. They provide a treasure trove of statistics that are often at the "bleeding edge" of current baseball study. They also don't clog the internet tubes with tons of resource sapping ads. But, one problem with having all these stats so readily available is that it's really, really easy for people to latch onto the latest, greatest Stat du Jour and then begin wielding these stats like broadswords to bludgeon the unsuspecting casual fan into blogdom oblivion. As great as this site, (and others, like bbref), may be, there is the ever-present reality that we may well all be marching forward with the certainty (and blind cluelessness) of Theodoric of York - Medievel Barber, who waxed poetically: "Why, just fifty years ago, they thought a disease like your daughter's was caused by demonic possession or witchcraft. But nowadays, through science, we know that Isabelle is suffering from an imbalance of bodily humors, perhaps caused by a toad or a small dwarf living in her stomach."

Alas, I am becoming convinced that UZR may well be the leeches and blood letting of SABRmetrics. It's often difficult to get a grasp on whether it is a problem with a system of personal bias that is at play. It's easy to believe that player X is a great (or horrid) defender, only to be told that the numbers say something very different. In regards to defense, this is especially important, as there is yet to be a defensive system that is predictive, nor one that is regarded widely as the Gold Standard. UZR, (Ultimate Zone Rating), has been one of the leading contenders for some time. But, the more I see, the less I like.

Without going into too much detail, (oh heck, I'm incapable of not going into too much detail - but I digress), the foundation block of UZR is ZR - or "Zone Rating". ZR is determined by people attending games, who chart out where balls in play go, and judge the catchability of these BIPs. While there is a tome of objective criteria for these people to use, in the end, there is "some" subjectivity involved. When I first found ZR, I was thrilled. It seemed to be a significant improvement over simple fielding percentage or range factor. But, over time, I noted that some cities seemed to have entire teams that were skewing good or bad, while the raw team stats didn't coincide with the ZR ratings. That was a red flag. But, if one has other alternative measures, one can at least potentially filter out egregious errors.

UZR is a much more rigorous attempt to fully quantify the value of each defender, breaking the runs saved (or surrendered), down by range, arm, double-play and error sub-UZR categories. Sounds really thorough. And I have no doubt the people putting this together are doing their best to find the truth. But, too many numbers fail a simple sanity check. UZR may have some uses - but as it stands today, I would say anyone putting much faith in it as a true judge of defensive value is getting a pig in a poke. Why?

Well, the most glaring concerns can be found comparing different data on the same pages. If you bring up all qualifying CFs for 2009, there are 18 guys ranked. Gutierrez leads with a UZR of 29.1, while Jacoby Elsbury is last with a -18.6. The distribution for CF looks okay, (9 better than average, 9 less), though the edges seem kind of extreme. But, the weirdness strikes in individual lines for individual players.

Gutierrez, according to Fangraphs had 420 "expected outs". The system adjusts, (I'm hoping), for innings played and chances, (high strikeout pitching staffs cut down on total chances, for example). Gutz played 1353 innings, one of the higher totals, but despite a nearly dead average pitching staff, (in terms of Ks), he was expected to make 24 more outs than the 2nd CF, (Granderson in 1384 innings), and almost 50 more outs than Matt Kemp in nearly identical innings. Seems kinda high, but for the sake of argument, let us accept (for the moment), that the "expected outs" (in the time played), is a reasonble number. Gutz actually made 445 Putouts. We won't add his assists, since the Arm factor is a separate portion of the UZR total, (though it is unclear as to whether the expected outs would include assists in its projection for OFs), 445 outs with 420 expected -- 25 better than expected. Yet, the Range portion of UZR has him at 29.3 RUNS saved.

If we look at Adam Jones, we see 341 expected outs, and 349 actual putouts. Jones' range portion under UZR is that he COST his team 11.7 runs!?! He made 8 more outs than expected, but COST his team 11.7 runs in the process. My word! If he had gotten 25 extra outs like Gutz, would he have cost his team 30?!? But, Jones wasn't all bad. He actually led all CF with a +7.3 ARM portion under UZR. He had 9 OF assists, (trailing Kemp-14; McCutchen-10; and Bourn-11). Somehow, McCutchen's arm ends up a -0.4 runs. Jones only played 116 games in Center, so a full season might have resulted in many more assists - (but McCutchen only played 108).

B.J. Upton had 375 expected outs, and had 375 putouts. What would YOU set his runs saved at? Well, UZR put it at 8.2. And that, coupled with his 3.4 arm rating, (off his 6 assists), made him the 2nd best CF in baseball with a total UZR of 11.0. Three LFs managed double-digit UZR scored better than Upton's, (Crawford-17.6; DeJesus-14.9; Rivera-12.7).

Of course, when you look at the LFs, only three actually managed to break the 300 PO barrier - Crawford, (#1 LF in baseball), along with Jason Bay, (next to last with a -13.0 UZR), and Ryan Braun (last with a -14.4). Of the 13 LFs that qualified, the 2nd and 3rd in ACTUAL plays ended up 12th and 13th in UZR. Jason Bay was also saddled with the embarrassment of leading all LFs in assists, (15), but ended up with a -0.7 arm rating. DeJesus for the Royals was next with 13 assists, but got a robust 6.7 arm rating.

Okay, a bad arm can generate lots of assists because everyone will run on you. But, Braun, dead last among LFs in UZR had 302 expected outs and 304 actual outs. Yet, he ends up with a -16.2 range factor, worst of everyone. There's a MASSIVE disconnect here. If you're getting exactly the number of outs that you're expected to get, *HOW* can you be costing your team 16 runs?

This is using two different advanced systems which utterly and completely contradict each other. Either (or both) could be wrong. But, what do you do? Throw both out? Cherry pick the one that matches your instinct on a given day? Me? I try and identify whether the system has a major philosophical flaw that defies logic. At this point, I believe UZR does.

When you sort ALL fielders by UZR, which is "supposed" to represent total runs saved, Gutierrez ends up at 29.1, a mile ahead of Longoria at 18.5. But, the entire list ORDER makes my head twitch. While we know little about individual defensive prowess, something we do know with utter certainty is that 2B, SS and CF make the most plays. They generate the most outs, (ignoring putouts by 1B which are a result of fielding plays made by the other infielders). For more than a century EVERY baseball manager, scout, player and pundit has understood you put your best defensive players at short, second and center *BECAUSE* those positions are where you save the most runs.

Meanwhile, third, left and right are on a lower tier of production. Fewer balls are hit to these areas, so these areas CANNOT prevent as many runs as you can with great fielders up the middle. So, what does UZR tell us? Of the top 10 run savers in baseball in 2009, you get 1-CF, 1-SS, (who only played 105 games), 4-3Bs, 3-LFs and a RF. The UZR standings are DOMINATED by players from lower traffic areas. Why? Because the UZR scores are adjusted against the COMPETITION at said position. The best fielders play CF. The WORST fielders get placed in LF. So, runs saved is computed against the other players at a single position. Beltre doesn't rank high because he's saving runs -- he ranks high because there are a bunch of other 3Bs who stink. Michael Young moves to 3B, EVERY other 3B "saves more runs". Say what?!?

While UZR might (and I do use that word tentatively) have some use in comparing players at a single position, it is uncategorically dreadful at attempting to compare players across different positions. BECAUSE managers put the best defenders in CF, the system prevent CFs from being generally highly regarded defensively. Carl Crawford "saves" lots of runs because Adam Dunn plays LF. Well, you let Adam Dunn play CF, and the number of runs saved by Jones and Hunter and Gutierrez is going to change -- and it SHOULDN'T. Adam Dunn playing LF or CF has *ZERO* impact on how many runs Carl Crawford or Gutierrez or Shoeless Joe Jackson saves with his glove.

Ichiro is considered an elite RF. The cream of the crop among RFs. He's golden. He saves (according to UZR) over his career, 11.0 runs per 150 games. You take that talent and speed and move him to a position with MORE chances, and what happens? His UZR/150 drops to 7.7. Say what? Did they plant Alfalfa in CF to slow him down?!? In 2007, Ichiro had 424 putouts in CF, and got a -1.7 range factor from UZR. That tied him with Granderson for the most POs in the AL.

From the most basic tenets of logic, if you take your best defensive player, (who is playing CF), and you move him to a position where he gets fewer opportunities, (like RF), his runs saved is NOT going to go up. Either UZR is completely and utterly busted, and it is in fact not even remotely counting actual runs saved -- or every player, coach, scout and pundit who has agreed for the last century that you need to put your best fielders up the middle is a moron.

I cannot accept a defensive rating system that routinely has the lower traffic defensive positions as the ones saving the most runs. Heck, the White Sox finished 2nd to the Ms in fewest runs allowed. They finished middle of the pack in team DER. UZR says there were only two worse teams in the AL, (the Twins and Royals). Look at the FIP for the WhiteSox and it says they should've allowed a 4.21 FIP, while they actually allowed a 4.16. It would seem the ChiSox defense SAVED 0.05 runs per game, (times 162) ... (carry the ...) ... about 8 runs over the season. UZR says the ChiSox team defense COST them 35.6 runs. UZR thinks the team that allowed the 2nd fewest runs in the AL had the 3rd WORST defense. I don't care what the math of the system is ... the system DOESN'T WORK!

(Except for actually measuring run prevention, UZR is rigorously tested?)

I'm sorry, but UZR fails miserably when a sanity check is made. Third basemen do NOT routinely dominate the run prevention landscape. They can't. If you put Willie Mays in his prime in CF for all 30 teams, what would happen? NO CF would save any runs at all. If that is the foundation for your system, then your system needs an enema.