Sandy's Pitch, UZ at your own peril Options

[Sandy - Raleigh]

Sorry if the tone of this piece is a little more surly than usual. It's mostly a case of being overly annoyed at the gold glove announcements.

"Uz at Your Own Peril"

I love Fangraphs.com. They provide a treasure trove of statistics that are often at the "bleeding edge" of current baseball study. They also don't clog the internet tubes with tons of resource sapping ads. But, one problem with having all these stats so readily available is that it's really, really easy for people to latch onto the latest, greatest Stat du Jour and then begin wielding these stats like broadswords to bludgeon the unsuspecting casual fan into blogdom oblivion. As great as this site, (and others, like bbref), may be, there is the ever-present reality that we may well all be marching forward with the certainty (and blind cluelessness) of Theodoric of York - Medievel Barber, who waxed poetically: "Why, just fifty years ago, they thought a disease like your daughter's was caused by demonic possession or witchcraft. But nowadays, through science, we know that Isabelle is suffering from an imbalance of bodily humors, perhaps caused by a toad or a small dwarf living in her stomach."

Alas, I am becoming convinced that UZR may well be the leeches and blood letting of SABRmetrics. It's often difficult to get a grasp on whether it is a problem with a system of personal bias that is at play. It's easy to believe that player X is a great (or horrid) defender, only to be told that the numbers say something very different. In regards to defense, this is especially important, as there is yet to be a defensive system that is predictive, nor one that is regarded widely as the Gold Standard. UZR, (Ultimate Zone Rating), has been one of the leading contenders for some time. But, the more I see, the less I like.

Without going into too much detail, (oh heck, I'm incapable of not going into too much detail - but I digress), the foundation block of UZR is ZR - or "Zone Rating". ZR is determined by people attending games, who chart out where balls in play go, and judge the catchability of these BIPs. While there is a tome of objective criteria for these people to use, in the end, there is "some" subjectivity involved. When I first found ZR, I was thrilled. It seemed to be a significant improvement over simple fielding percentage or range factor. But, over time, I noted that some cities seemed to have entire teams that were skewing good or bad, while the raw team stats didn't coincide with the ZR ratings. That was a red flag. But, if one has other alternative measures, one can at least potentially filter out egregious errors.

UZR is a much more rigorous attempt to fully quantify the value of each defender, breaking the runs saved (or surrendered), down by range, arm, double-play and error sub-UZR categories. Sounds really thorough. And I have no doubt the people putting this together are doing their best to find the truth. But, too many numbers fail a simple sanity check. UZR may have some uses - but as it stands today, I would say anyone putting much faith in it as a true judge of defensive value is getting a pig in a poke. Why?

Well, the most glaring concerns can be found comparing different data on the same pages. If you bring up all qualifying CFs for 2009, there are 18 guys ranked. Gutierrez leads with a UZR of 29.1, while Jacoby Elsbury is last with a -18.6. The distribution for CF looks okay, (9 better than average, 9 less), though the edges seem kind of extreme. But, the weirdness strikes in individual lines for individual players.

Gutierrez, according to Fangraphs had 420 "expected outs". The system adjusts, (I'm hoping), for innings played and chances, (high strikeout pitching staffs cut down on total chances, for example). Gutz played 1353 innings, one of the higher totals, but despite a nearly dead average pitching staff, (in terms of Ks), he was expected to make 24 more outs than the 2nd CF, (Granderson in 1384 innings), and almost 50 more outs than Matt Kemp in nearly identical innings. Seems kinda high, but for the sake of argument, let us accept (for the moment), that the "expected outs" (in the time played), is a reasonble number. Gutz actually made 445 Putouts. We won't add his assists, since the Arm factor is a separate portion of the UZR total, (though it is unclear as to whether the expected outs would include assists in its projection for OFs), 445 outs with 420 expected -- 25 better than expected. Yet, the Range portion of UZR has him at 29.3 RUNS saved.

If we look at Adam Jones, we see 341 expected outs, and 349 actual putouts. Jones' range portion under UZR is that he COST his team 11.7 runs!?! He made 8 more outs than expected, but COST his team 11.7 runs in the process. My word! If he had gotten 25 extra outs like Gutz, would he have cost his team 30?!? But, Jones wasn't all bad. He actually led all CF with a +7.3 ARM portion under UZR. He had 9 OF assists, (trailing Kemp-14; McCutchen-10; and Bourn-11). Somehow, McCutchen's arm ends up a -0.4 runs. Jones only played 116 games in Center, so a full season might have resulted in many more assists - (but McCutchen only played 108).

B.J. Upton had 375 expected outs, and had 375 putouts. What would YOU set his runs saved at? Well, UZR put it at 8.2. And that, coupled with his 3.4 arm rating, (off his 6 assists), made him the 2nd best CF in baseball with a total UZR of 11.0. Three LFs managed double-digit UZR scored better than Upton's, (Crawford-17.6; DeJesus-14.9; Rivera-12.7).

Of course, when you look at the LFs, only three actually managed to break the 300 PO barrier - Crawford, (#1 LF in baseball), along with Jason Bay, (next to last with a -13.0 UZR), and Ryan Braun (last with a -14.4). Of the 13 LFs that qualified, the 2nd and 3rd in ACTUAL plays ended up 12th and 13th in UZR. Jason Bay was also saddled with the embarrassment of leading all LFs in assists, (15), but ended up with a -0.7 arm rating. DeJesus for the Royals was next with 13 assists, but got a robust 6.7 arm rating.

Okay, a bad arm can generate lots of assists because everyone will run on you. But, Braun, dead last among LFs in UZR had 302 expected outs and 304 actual outs. Yet, he ends up with a -16.2 range factor, worst of everyone. There's a MASSIVE disconnect here. If you're getting exactly the number of outs that you're expected to get, *HOW* can you be costing your team 16 runs?

This is using two different advanced systems which utterly and completely contradict each other. Either (or both) could be wrong. But, what do you do? Throw both out? Cherry pick the one that matches your instinct on a given day? Me? I try and identify whether the system has a major philosophical flaw that defies logic. At this point, I believe UZR does.

When you sort ALL fielders by UZR, which is "supposed" to represent total runs saved, Gutierrez ends up at 29.1, a mile ahead of Longoria at 18.5. But, the entire list ORDER makes my head twitch. While we know little about individual defensive prowess, something we do know with utter certainty is that 2B, SS and CF make the most plays. They generate the most outs, (ignoring putouts by 1B which are a result of fielding plays made by the other infielders). For more than a century EVERY baseball manager, scout, player and pundit has understood you put your best defensive players at short, second and center *BECAUSE* those positions are where you save the most runs.

Meanwhile, third, left and right are on a lower tier of production. Fewer balls are hit to these areas, so these areas CANNOT prevent as many runs as you can with great fielders up the middle. So, what does UZR tell us? Of the top 10 run savers in baseball in 2009, you get 1-CF, 1-SS, (who only played 105 games), 4-3Bs, 3-LFs and a RF. The UZR standings are DOMINATED by players from lower traffic areas. Why? Because the UZR scores are adjusted against the COMPETITION at said position. The best fielders play CF. The WORST fielders get placed in LF. So, runs saved is computed against the other players at a single position. Beltre doesn't rank high because he's saving runs -- he ranks high because there are a bunch of other 3Bs who stink. Michael Young moves to 3B, EVERY other 3B "saves more runs". Say what?!?

While UZR might (and I do use that word tentatively) have some use in comparing players at a single position, it is uncategorically dreadful at attempting to compare players across different positions. BECAUSE managers put the best defenders in CF, the system prevent CFs from being generally highly regarded defensively. Carl Crawford "saves" lots of runs because Adam Dunn plays LF. Well, you let Adam Dunn play CF, and the number of runs saved by Jones and Hunter and Gutierrez is going to change -- and it SHOULDN'T. Adam Dunn playing LF or CF has *ZERO* impact on how many runs Carl Crawford or Gutierrez or Shoeless Joe Jackson saves with his glove.

Ichiro is considered an elite RF. The cream of the crop among RFs. He's golden. He saves (according to UZR) over his career, 11.0 runs per 150 games. You take that talent and speed and move him to a position with MORE chances, and what happens? His UZR/150 drops to 7.7. Say what? Did they plant Alfalfa in CF to slow him down?!? In 2007, Ichiro had 424 putouts in CF, and got a -1.7 range factor from UZR. That tied him with Granderson for the most POs in the AL.

From the most basic tenets of logic, if you take your best defensive player, (who is playing CF), and you move him to a position where he gets fewer opportunities, (like RF), his runs saved is NOT going to go up. Either UZR is completely and utterly busted, and it is in fact not even remotely counting actual runs saved -- or every player, coach, scout and pundit who has agreed for the last century that you need to put your best fielders up the middle is a moron.

I cannot accept a defensive rating system that routinely has the lower traffic defensive positions as the ones saving the most runs. Heck, the White Sox finished 2nd to the Ms in fewest runs allowed. They finished middle of the pack in team DER. UZR says there were only two worse teams in the AL, (the Twins and Royals). Look at the FIP for the WhiteSox and it says they should've allowed a 4.21 FIP, while they actually allowed a 4.16. It would seem the ChiSox defense SAVED 0.05 runs per game, (times 162) ... (carry the ...) ... about 8 runs over the season. UZR says the ChiSox team defense COST them 35.6 runs. UZR thinks the team that allowed the 2nd fewest runs in the AL had the 3rd WORST defense. I don't care what the math of the system is ... the system DOESN'T WORK!

(Except for actually measuring run prevention, UZR is rigorously tested?)

I'm sorry, but UZR fails miserably when a sanity check is made. Third basemen do NOT routinely dominate the run prevention landscape. They can't. If you put Willie Mays in his prime in CF for all 30 teams, what would happen? NO CF would save any runs at all. If that is the foundation for your system, then your system needs an enema.

[Sancho Panza]

Theodoric's test for witchcraft also seems somehow relevant to your critique. Any way you could post this at Fangraphs? ( One poster in a current UZR thread asks if the last ranked player is called a LUZR. )

[Pirata Morado]

Excellent as usual Sandy!

The best way to improve something is to question its validity.

You made a very good point about UZR, which is worth considering.

I've never liked a +/- system in which the comparison in order to be positive or negative only deals with certain players. Yes, I understand that since the stat is for defense, they consider the position of the player into account. I guess they should make a note that you can only compare players among the same position, a -1 UZR centerfielder might be better than a 5 UZR thirdbaseman. We have to keep that in mind.

That's the same reason I never liked OPS+ and ERA+, I understand its ease of understanding, many people see 100 and immediately understand that as average. Below 100 is below average and Above 100 is above average, easy to understand, great, but we lose information about where the average is.

A player with 30 HR might have been average in the 90's, but he sure wasn't average in the 20's, so when we use OPS+ we get a good idea of how he fared against his own competition, but we lose information about how he actually did!

A pitcher that had a 3.00 ERA was good, no matter what, we know he allowed 3 earned runs per nine innings. It might had been because his defense was awesome, it might have been for whatever reason, you like, but we know how many earned runs he allowed per nine! That same pitcher might get a 90 ERA+ only because he happened to pitch in the 60's in which many many pitchers were also good. Was he below average? He might had been, but was he a bad pitcher? I don't think so.

Similar thing happens with UZR and I agree with Sandy. We should change the system to something less "comparative" which is subjective to something more objective. Ichiro converts 110% of the expected putouts he gets. It doesn't matter if he plays RF or CF, that's good because he makes more plays than expected.

[Sandy - Raleigh]

Nice idea Pirata. Actually pretty easy to implement.

Take ZR for a position, divide it by the leaque average for that position - and you've got ZR+. Unfortunately, since my firewall blocks ESPN, (something to do with streaming video embedded in every page or something), I've quit using ESPN even at home. But I know they used to produce include the ZR figures for everyone.

This wouldn't be a bad tool to utilize, (assuming, of course, that ZR is good).

I am nearly always in favor of stat forms that are ready-to-use -- (you don't have to learn the scale).

[Pirata Morado]

Nice idea Pirata. Actually pretty easy to implement.

Take ZR for a position, divide it by the leaque average for that position - and you've got ZR+. Unfortunately, since my firewall blocks ESPN, (something to do with streaming video embedded in every page or something), I've quit using ESPN even at home. But I know they used to produce include the ZR figures for everyone.

This wouldn't be a bad tool to utilize, (assuming, of course, that ZR is good).

I am nearly always in favor of stat forms that are ready-to-use -- (you don't have to learn the scale).

Well, they certainly are useful, of course, but as I said, you lose some information using them. Yes ESPN gets also blocked here.

There also seems to be something intuitively wrong with the arm factor.

Several years ago I was thinking about a system that took many more things into account, I never formally proposed it, but it went something like this:

Catchers.

I don't understand much about Range Factor for catchers (are they good at getting up and making the bunt play?), but it seems like there are much more important things to consider a catcher's defensive skills, than range for them. We noticed it in the past World Series. Molina was great at blocking pitches, Posada wasn't, but Molina couldn't hit a thing while Posada was great, so it was really a dillema for Girardi.

I was proposing back then that we need a stat for blocks saved for Catchers, of course referenced to chances. We need to keep track of Wild Pitches saved by a catcher and also blocking Home Plate when a play is made at home. Those are 2 fundamental catching skills that never get considered in a stat per se. If a catcher saves a clear wild pitch then tally his WPS (wild pitch saves), if he doesn't, then consider it a WPnS (Wild Pitch not saved or a Passed Ball), I don't care what it is, he simply allowed a runner to advance. Then calculate his Wild Pitch Saving Efficiency like this: WPS / (WPS+WPnS), I'm certain good defensive catchers are going to have high WPS%. Similar for HPB (Home Plate Blocking).

1Bmen.

There are very good 1B who never get credited like they should. Fielding Average tells me nothing about them. We need to do something about their picks, if an infielder throws it in the dirt but the 1Bman picks it up, tally them as Picks Up the dirt. If the ball gets past him, don't give him an error, but give him a Pick Up Chance. Calculate a 1B pick up efficency as PK / PKCh (Pickups / Pickup chances). Many times a throw would go to the stands but the 1B saved it from going there. He doesn't get a putout, but gets a save.

For outfielders Range or UZR is almost everything, but assists is misleading sometimes. We need to tally the times a runner doesn't advance out of respect for the OF arm. Many times the leading assists players are usually the ones with low reputation, so they get mmore assists because they run a lot on them. We need to know how many chances of nailing runners they had.

This post has been edited by Pirata Morado: Nov 12 2009, 11:08 AM

[Sandy - Raleigh]

Agree on almost everything, P.

On arms, you identify the obvious problem -- number of actual assists slides based on opponent aggression. But, there's a trap. If a players REPUTATION is high, and nobody runs on him ... should he get credit for that? The knee jerk is yes. But, what if the rep is 10 years old? Catchers get credit for throwing out baserunners -- but when people stop running, you can't throw as many out. It's not JUST about percentage. The fact that the opposition has control over the choice of doing something complicates attempting to assess a given player's "credit".

But, the "situation" can also influence base-running choices. Baltimore had THREE OFs who scored off the charts in "Arm" UZR, (UZarm?). Well, Baltimore allowed more base runners, and had horrid pitching and defense. Did they not respect the arms ... or was it just the opposition being stupidly aggressive, because they were routinely ahead by 3-5 runs? THIS is a case where I could see WPA being the absolutely perfect tool to utilize for getting a clearer picture.

(And, of course - teams WILL run on a rookie arm until they get a good read on how dangerous the arm truly is. It is not unusual for a rookie to come up and dominate OF assists for 1/2 season, and then see the numbers change drastically thereafter).

[danduke]

Sandy,

Very interesting stuff. I honestly don't know much about UZR. It has been something of a black box to me for a while, mostly because I've never taken the time to really understand it. But your post prompted me to dig deeper. Here's a brief summary of UZR I typed up based on M. Lichtman's explanatory post from a longggg time ago - http://www.baseballthinkfactory.org/files/...n_2003-03-14_0/

# Range Factor - outs/game
# Zone Rating - % balls converted into outs within zone
# Ultimate Zone Rating - break a fielder's zone into subzones, track his productivity in each zone, and then COMPARE IT to how a baseline/average fielder with the same batted ball distribution would fare
* Does not address OF arm or IF ability to turn DP
* UZR Rate: a fraction of 1, just like ZR, based on a weighted average calculation
* UZR Runs: Runs saved or cost in comparison to mean performance of all fielders at that position, in that league, during that year
* Defensive complement to Pete Palmer's offensive linear weights

Your point about the disconnect between extra outs made and extra runs prevented is a strong one. So how could one player make 25 extra outs and save 29 runs, while one player makes 8 extra outs and costs his team 11 runs? I don't know the specifics, but based on what I've read so far, the logical explanation is that Franklin's unreached balls resulted in extra singles, while Adam's unreached balls resulted in extra 2B or 3B. That said, the differential seems EXTREME. Further, I would MUCH rather just have the UZR RATE data, which would be a neat alternative to DER. Fangraphs does NOT display the UZR Rate data.

I am tempted to think that relying on the results off balls not reached would skew the data. This is a much bigger question than can be addressed in one little blog post!

This post has been edited by danduke: Nov 12 2009, 01:57 PM

[Pirata Morado]

Agree on almost everything, P.

On arms, you identify the obvious problem -- number of actual assists slides based on opponent aggression. But, there's a trap. If a players REPUTATION is high, and nobody runs on him ... should he get credit for that? The knee jerk is yes. But, what if the rep is 10 years old? Catchers get credit for throwing out baserunners -- but when people stop running, you can't throw as many out. It's not JUST about percentage. The fact that the opposition has control over the choice of doing something complicates attempting to assess a given player's "credit".

But, the "situation" can also influence base-running choices. Baltimore had THREE OFs who scored off the charts in "Arm" UZR, (UZarm?). Well, Baltimore allowed more base runners, and had horrid pitching and defense. Did they not respect the arms ... or was it just the opposition being stupidly aggressive, because they were routinely ahead by 3-5 runs? THIS is a case where I could see WPA being the absolutely perfect tool to utilize for getting a clearer picture.

(And, of course - teams WILL run on a rookie arm until they get a good read on how dangerous the arm truly is. It is not unusual for a rookie to come up and dominate OF assists for 1/2 season, and then see the numbers change drastically thereafter).

Precisely I've taken into consideration that for some of my analyses. For example in my factor analysis I not only take into account the catcher's caught stealing efficiency but also something I call Stealing propensity, which isn't easy to derive. It is how many stealing chances there were (SB+CS) devided by the number of stealing opportunities (times a runner was on first base with second base unoccupied), which measures how much do teams "run on him". Happens that factor analysis joins those stats toghether into one factor which I call "Catcher's Arm", because they are inversely related. High CS% catchers get low Stealing Propoensity and viceversa.

Similar things could be done about outfielders, take both things into account, the efficency of nailing runners but also the propensity of running on them. There was once an attempt to measure this but from an offensive point of view, which is called "Bases Produced", a very good approach: Bases Produced I'm sure we can do a lot of neat things with it.

This post has been edited by Pirata Morado: Nov 12 2009, 02:26 PM

[danduke]

More detailed info about UZR, for anyone that feels like they're looking at a black box (like me):

*Entire field treated as 78 zones - http://www.retrosheet.org/location.htm - BUT UZR only uses 64 of these
---> For IF, popups and all line drives through IF are excluded
---> For OF, foul balls excluded
---> P and C excluded

*A computer tracks the following for each ZONE as the year progresses:
---> # BASE HITS in that zone
---> EXPECTED RUN VALUE of a hit in that zone using offensive linear weights
---> # OUTS recorded in zone FOR EACH FIELDING POSITION
---> Comp also TRACKS ERRORS BY POSITION, not zone
// differentiates REACH ON ERROR errors v. Non-ROE errors, e.g. Ichiro drops the ball v. Ichiro throws the ball to the backstop

*A comp tracks the following for each PLAYER AND ZONE as the year progresses:
---> # Hits in zone, by position
---> # Outs in zone, by position
---> TRACKS ERRORS BY PLAYER AND POSITION, but not by zone

*The "Tricky Part" - How Player X UZR Runs in Zone Y are calculated
---> Treat all ROE errors as outs (???)
---> Get OUT RATE in Zone Y: Outs/Total Chances, for all fielders
---> Get Player X's POSITIVE CONTRIBUTION in Zone Y: (1 - Out Rate) * Outs Made in Zone Y
// Multiply by 1- Out Rate because that's the contribution ABOVE WHAT'S EXPECTED
---> Get Player X's NEGATIVE CONTRIBUTION in Zone Y: (X's hits in Zone Y) * (League Average SHARE OF OUTS MADE for players at X's position) * ( - Out Rate)
// i.e. if Player X is a SS and SS's recorded 21% of outs in that Zone, X will only be held responsible for 21% of total hits allowed in that zone
// Multiply by negative out rate because that's the contribution BELOW WHAT'S EXPECTED
---> Add POSITIVE + NEGATIVE CONTRIBUTION
---> Multiply by average run value of a hit in that zone
---> BAM! UZR Runs

Add up the UZR Runs for each zone/position pair and you will have the player's total UZR Runs for the Range category.

This post has been edited by danduke: Nov 12 2009, 03:24 PM

[DrNaka]

Team DER is a more concrete stat for team defense.
But As Sandy pointed out UZR has its failures.
No wonder that the order of Team DER and Team UZR are different.

Any defensive stat should be consistent with team DER if you sum up all these individual stats to team.

[Sandy - Raleigh]

Any defensive stat should be consistent with team DER if you sum up all these individual stats to team.

Agree wholeheartedly. I'd be willing to accept some minor variation -- but if UZR says a team was dreadful, and DER says it was above average, then I'm sticking with DER.

If a team allows 90 fewer (or more) runs than the rest of the league, (with average pitching), but UZR says the defense was also average -- then either the pitching number or UZR number *HAS* to be off.

If the TEAM number for UZR doesn't land somewhere in the vicinity of the actual team totals for hits and runs allowed, then whatever UZR is measuring -- it ain't runs prevented or allowed.

[danduke]

But shouldn't a team's UZR RATE stat, weighted appropriately for each player and zone, sum up to be ~ DER? Obviously there are some sources of discrepancy - pitcher's fielding, infield line drives, etc. (I believe DER, like UZR, ignores OF foul balls). It seems to me that leaves the application of linear weights values to each zone as the most likely logical flaw, no?

This post has been edited by danduke: Nov 13 2009, 07:38 AM

[Sandy - Raleigh]

To my knowledge, DER includes every fielding out, (foul or not). I suppose you're correct in that a foul in play (not caught), is invisible to the system exactly as a foul in the stands would be. For DER, it's just another strike.

danduke, you're absolutely correct, that the DER and UZR stats "should" be roughly equivalent. Even if the numbers didn't match up precisely, at the very least, one would expect TEAM rank order to be close to identical, (except where differences are very small). Unfortunately, in too many cases, team UZR and DER don't just fail to match, they fail to match on a massive scale.

My single biggest complaint with UZR is that, though it might be useful for something, (I don't know exactly what at the moment), it *CLEARLY* does not measure runs saved in any way that is verifiable by any other statistical means. Yet, the #1 use for UZR today is to speak of Run Saved by individual players, as if it is getting this right.

[DrNaka]

DER of 2009 teams

1 Seattle Mariners 0.728
2 Los Angeles Dodgers 0.725
3 San Francisco Giants 0.721
4 Cincinnati Reds 0.717
5 Chicago Cubs 0.715
6 Texas Rangers 0.712
7 Tampa Bay Rays 0.711
8 New York Yankees 0.710
9 St. Louis Cardinals 0.708
10 New York Mets 0.708
11 Detroit Tigers 0.707
12 San Diego Padres 0.706
13 Chicago White Sox 0.706
14 Milwaukee Brewers 0.705
15 Philadelphia Phillies 0.704
16 Washington Nationals 0.704
17 Florida Marlins 0.702
18 Arizona Diamondbacks 0.702
19 Colorado Rockies 0.701
20 Atlanta Braves 0.701
21 Pittsburgh Pirates 0.700
22 Minnesota Twins 0.699
23 Los Angeles Angels 0.699
24 Oakland Athletics 0.698
25 Cleveland Indians 0.695
26 Baltimore Orioles 0.693
27 Kansas City Royals 0.693
28 Toronto Blue Jays 0.692
29 Boston Red Sox 0.691
30 Houston Astros 0.689

Team UZR of 2009
Team UZR DERranking
1. Mariners 85.5 1
2.Rays 69.5 7
3.Reds 52.6 4
4.Giants 51.2 3
5.Tigers 43.6 11
6.Rangers 32.5 6
7.Pirates 30.1 21
8.Phillies 27.9 15
9.Diamondbacks 21.6 18
10.Angels 13.3 23
11.Brewers 9.9 14
12.Athletics 5.2 24
13Dodgers -0.1 2
14Padres -13.4 12
15Rockies -14 19
16Red Sox -16.3 29
17Astros -17.7 30
18Cardinals -17.8 9
19Yankees -18.5 8
20Braves -18.8 20
21Cubs -19.8 5
22Marlins -20.5 17
23Orioles -23.6 26
24Nationals -26.7 16
25Blue Jays -33.3 28
26Indians -33.5 25
27White Sox -35.6 13
28Twins -37.3 22
29Mets -47.3 10
30Royals -49.9 27

I underlined teams which have more than 10 in difference for DER and team UZR.
So how aboiut these teams?


This post has been edited by DrNaka: Nov 13 2009, 09:37 AM

[Sandy - Raleigh]

Nice work, DrNaka!

Wow! A full third of teams are 10 slots or more away from their DER results?!? Been too long since I took probability, but that can't be too far removed from monkey throwing at a dart board accuracy. That the discrepence is THAT large astounds even me.

A quick glance, and it looks like the largest skew is 19 slots, (Mets).

DER has the Mets almost dead average in the NL. .691 compared .690 league average.
Mets runs allowed just below average (4.67 compared to avg. of 4.49).

But Mets pitching was pretty bad -- their HR rate was exactly league average 1.0 -- but their walk rate was 3.9 (league average of 3.5), and K-rate was only 6.5 (league avg. of 7.1).

So, only two teams had a worse K/BB ratio in the NL ... the Mets pitching was very bad. But, the Mets run allowance was close to league average, (just over 757 runs compared to average of 727). My take would be that to do that, your defense would have to be above average -- slightly above average -- to manage that.

DER has them just barely above average, while UZR has them as the worst defensive team *BY FAR* in the NL.

You tell me which system is broken.

[danduke]

I hate to beat a dead horse, but it would be informative to see how the team DER rankings compare to the team UZR RATE rankings (that are also on "the out of 100%" scale, and do not try to express runs saved/cost).

I am also interested in moving from the question "is UZR good?" to WHY that may (not) be the case. A helpful starting point would be to look for common threads between teams whose DER vastly outranked their UZR (Dodgers, Cardinals, Yankees, Cubs, White Sox, Mets) and vice versa (Pirates, Angels, Athletics, Red Sox, Astros).

[muddyfrogwater]

ZR is determined by people attending games, who chart out where balls in play go, and judge the catchability of these BIPs.

Is there any chance that whoever charts the balls in play may be somewhat biased weather intentional or not? The subconscious mind has the ability to fool the practical mind. For instance fan cheering may lead a person to believe that a play was more spectacular than it really was. When MLB launches it's first satellite to actually track the data then I'll buy into it. I'm sure it will happen one day. On perception alone I'd call Grady Sizemore the best center fielder in baseball. Not to mention we wouldn't have Gutz if it weren't for Grady.

[Sandy - Raleigh]

Is there any chance that whoever charts the balls in play may be somewhat biased weather intentional or not? The subconscious mind has the ability to fool the practical mind. For instance fan cheering may lead a person to believe that a play was more spectacular than it really was. When MLB launches it's first satellite to actually track the data then I'll buy into it. I'm sure it will happen one day. On perception alone I'd call Grady Sizemore the best center fielder in baseball. Not to mention we wouldn't have Gutz if it weren't for Grady.

I think this is not only possible, but likely. While I expect most, if not all, the people handling this job are doing the best they can to be impartial ... I think general emotional mindset can influence perceptions quite easily. If you have a team that is "expected" to be great, and they swoon, you might begin to see every play not made as one that somebody better would've made.

Conversely, if you were expected to be a bad team, and you get a shiny new CF, and your team is suddenly winning games with pitching and defense, maybe the people charting perceive the guy getting to WAY more balls than a normal guy would reach. (Sound like any team we know in 2009?).

For most stats, the results are completely objective ... a double is a double, and out is an out, with only a small subset being subjective, (was it a single and error, or a double with no error charged). But, if 95% of your plays are objective, even a slanted official scorer cannot have much of an impact on the final stats. If, however, your system is 100% subjective, (like an ump calling strikes), then the chance of bias influencing results is going to spike drastically.

If half of all games are judged by the same people, the chance of subjective bias impacting final numbers gets very, very large. I don't think there is any conspiracy, and I think the people are doing the best they can -- but I have often wanted to see exactly how badly the home and road ZR scores vary. While one would expect slightly higher ZR scores at home, (familiarity), I bet there are cases where the opposite is true - or where the home ZR advantage gets ridiculously large.

There's also a simple argument that road ZR alone is a better judge of actual skill, as everyone's road defense is going to have similar lack of familiarity potentially messing up results. Basically, the easiest way to remove park effects from any stat analysis is to look ONLY at road stats. That isn't the entire story ... but it's a really easy way to get a snapshot that doesn't hold the danger of a 50% bias.

[danduke]

I'm having a hard time understanding how this systemic bias favoring a Gutz would crop up. As best I can tell, the ONLY judgment the UZR scorekeeper makes is IN WHAT ZONE DID THE BALL LAND. Hence, it would not matter whether one is familiar with Player X or not.

[muddyfrogwater]

I'm having a hard time understanding how this systemic bias favoring a Gutz would crop up. As best I can tell, the ONLY judgment the UZR scorekeeper makes is IN WHAT ZONE DID THE BALL LAND. Hence, it would not matter whether one is familiar with Player X or not.

More important than where the ball lands; is where was the player positioned? How fast was the ball traveling? At what degree was the arc of the trajectory of the baseball? What route did the fielder take to the ball? What was the wind direction and speed? Did the ball have top spin or bottom spin when it left the bat? There are too many missing variables to make the data reliable. I believe scouting plays a better role when trying to determine a players fielding ability. While UZR may serve as a useful stat it can not be used to sum up a players defensive capabilities.