Sandy's Pitch, UZ at your own peril Options

[Sandy - Raleigh]

I'm having a hard time understanding how this systemic bias favoring a Gutz would crop up. As best I can tell, the ONLY judgment the UZR scorekeeper makes is IN WHAT ZONE DID THE BALL LAND. Hence, it would not matter whether one is familiar with Player X or not.

Maybe it isn't manual bias. Perhaps it is system bias. My initial assumption was based on observed major diversions from logic in particular cities.

It may be that because of park effects, wind patterns, park dimensions and layouts, that the actual predicted outs per zone SHOULD vary from arena to arena, but they don't. Based on your data above, (and thanks for providing the much greater level of detail), there HAVE to be human judgements in all of this.

You couldn't possibly have the same zone layout in Petco as Safeco as Fenway. It's physically impossible. Some PERSON is making choices on what to do with those slices of zones that don't match up from place to place.

And people make choices on whether a ball landed in zone 48 or 49. The kneejerk is to expect those to balance out. But, it is blatantly obvious from the ZR results that in "some cases" this is definitely not the case.

==========================

The Mets are the worst skew from UZR (16th of 16) compared to DER (7th of 16) within one league. In runs allowed they ended up 9th. In FIP, they ended up 13th. But, they weren't just 16th in UZR. They were more than double worse than all but one NL team. They get a -47.3 UZR, while the Nats end up at -26.7. This isn't just a little noise. This is an order of magnitude apart from every other team in the league.

The Nats, who actually allowed the most runs, (874 to the Mets 757), get a range UZ of -7.4, while the Mets end up at -33.8!?!

Even when the system seems to match reality in one area, it has an odd tendency to screw up somewhere else making the final look odd. Houston was last in the NL in DER, (.676), finishing just behind Arizona and the Nats at .680. Under UZR-range, the Astros end up worse than the Mets at -44.0 -- so worst DER and worst Range. We have a match. But, UZR then gives them back 13 in arm and 2.7 in DP and 10.7 in error. So, the aggregate ends up at only -17.7, ranking 9th in the NL.

Meanwhile, Arizona ranks 2nd BEST in UZR-range, but 2nd worst in DER. In truth, DER doesn't measure arm at all, and lumps in errors with range. But sorry -- you shouldn't see a #2 best #2 worst result when comparing two defensive systems. The Final DER results *DO* match up (roughly) with runs allowed. And when one takes into account the TTO numbers of the pitching staff -- DER and FIP do a pretty good job of combining to match realworld results. But, UZR doesn't in FAR too many cases and by far too wide a mark.

We can speculate forever about WHY UZR doesn't match up with real-world results. But, first we have to look at the data and admit that it doesn't.

[SABR Matt]

Several sources of bias.

1) Park effects...UZR tries to correct for this by comparing a player to the league average rate of balls caught in each zone of each park (in other words, that player and all of the other players who have ever caught balls in the same outfield in the same zone)...but there is likely to be major error in those attempts

2) FB vs. LD scoring. UZR accounts for the lower probability of catching a line drive than catching a fly ball. If the home scorer scores line drives more commonly than average, that team is going to look better in the outfield.

3) Bias in landing spot judgment...like it or not, eac ball in play has to be scored on where it is fielded...and I've watched the landing spot indicator on gameday and found it to be amazingly inaccurate sometimes.

Off the top of my head.

But I do want to caution you, Sandy, that DER != run prevention. It's a major part of it, but you're forgetting that not all .700 DERs are the same. Some teams get to 70% of balls in play but give more singles to stop the doubles and triples. Some teams get to 70% of BIP but do it by catching all the singles and giving up lots of doubles and triples. Some teams get to 70% of the BIP and are better at turning the DP. Some teams make costly errors, some teams make less costly ones.

[Sandy - Raleigh]

Several sources of bias.

1) Park effects...UZR tries to correct for this by comparing a player to the league average rate of balls caught in each zone of each park (in other words, that player and all of the other players who have ever caught balls in the same outfield in the same zone)...but there is likely to be major error in those attempts

2) FB vs. LD scoring. UZR accounts for the lower probability of catching a line drive than catching a fly ball. If the home scorer scores line drives more commonly than average, that team is going to look better in the outfield.

3) Bias in landing spot judgment...like it or not, eac ball in play has to be scored on where it is fielded...and I've watched the landing spot indicator on gameday and found it to be amazingly inaccurate sometimes.

Off the top of my head.

But I do want to caution you, Sandy, that DER != run prevention. It's a major part of it, but you're forgetting that not all .700 DERs are the same. Some teams get to 70% of balls in play but give more singles to stop the doubles and triples. Some teams get to 70% of BIP but do it by catching all the singles and giving up lots of doubles and triples. Some teams get to 70% of the BIP and are better at turning the DP. Some teams make costly errors, some teams make less costly ones.

I understand DER isn't everything. But there is a VERY strong correlation between DER and runs allowed. I'm perfectly willing to have someone show me that team X's .700 DER is better than team Y's .700 DER, because of additional bases allowed - or some other reason. There is also a very, very strong correlation between hits allowed and runs allowed ... even IGNORING HRs and Walks.

UZR wouldn't have to map "exactly" with DER for me to accept it as a valid tool. But, if you'll pardon the pun, it should at least be in the ballpark. Pirata did a Hits vs. Runs correlation, and IIRC a DER vs. Runs correlation. Hits vs. Runs was one of the strongest correlations of anything I've seen him run. DER wasn't quite as good, (I wouldn't expect it to be), but it was pretty easy to spot the transients with spikes in the TTO numbers as an easy to spot explanation for those cases where DER wasn't quite as cozy with runs allowed.

DER has many limitations. It's a team-only stat. It doesn't capture arm values. It doesn't capture BIP slugging. But everything about team-UZR that I see indicates that DER is drastically better ... so much so that the validity of the individual player rankings by UZR must be questioned.

To me, DER is just the simplest tool to bring the problems with UZR into the light. But, hey, ignore DER, go straight to runs allowed. The ChiSox at -35.6 UZR (12th in the AL), finished 2nd in the AL in fewest runs allowed, (right behind the Ms), allowing 732 total runs. FIP has them at 3rd best ... solid pitching, but the ERA was BETTER than the FIP. 3rd in FIP and 12th in UZR doesn't add up to 2nd in runs allowed.

[SABR Matt]

Oh I agree...DER is a good starting point for proving that UZR is likely garbage. I'm just cautioning against using it as the only defensive stat you look at. But you already knew that.

I think UZR is total crap...you already know that too...I think a team-driven top-down defensive analysis is the only accurate way to proceed.

[DrNaka]

Now having the list of DER and team UZR it will provide a list where you can find bargain SP or RP for 2010.
The bottom 5 in DER:

25 Cleveland Indians 0.695
26 Baltimore Orioles 0.693
27 Kansas City Royals 0.693
28 Toronto Blue Jays 0.692
29 Boston Red Sox 0.691
30 Houston Astros 0.689

The bottom 5 in UZR:
25Blue Jays -33.3 28
26Indians -33.5 25
27White Sox -35.6 13
28Twins -37.3 22
29Mets -47.3 10
30Royals -49.9 27

Find there pitchers on these teams with higher ERA but lower BB/9.
They will benefit from M's defence.

I think FIP is also a function of DER.

Look at Wash who's FIP climed when he left Mariners.
Some pitcher cannot throw strikes when they cannot believe in the defense in the back.

BTW I think there are bargain pitchers in the bottom 5 of DER and not UZR...

[SABR Matt]

This is a very common strategy, yes. Look for pitchers who were in front of bad defenses and turn them around...that's how St. Louis gets its rotation so cheaply.

[DrNaka]

This is a very common strategy, yes. Look for pitchers who were in front of bad defenses and turn them around...that's how St. Louis gets its rotation so cheaply.

Hmm St. Louis !

The WC St. Louis 2006 was

7th overall at DER at 0.712

Now in 2009 they have been 9th with 0.708.

UZR have them weaker.

[SABR Matt]

STL has been among the most consistently excellent defensive teams since the mid 90s. LaRussa has demonstrated an outstanding feel for pitching and defense and how they interact much like Cox in Atlanta.

[danduke]

And even LaRussa was experimental enough to try Skip Schumaker at 2B (Dustin Ackley cough cough)

[Taro]

I wouldn't go as far to say that UZR is "useless", but it does have its problems (mainly human judgement).

I used to be one of its biggest bashers, but the stat seems to have improved in the last couple years. Its possible that the record-keepers are improving with time.

[Gomez]

I've got to chime in since I cite UZR so much... this is a great post, Sandy. One of the main issues with UZR and other popular SABR stats is that we have no access to the specific methodology that determines the end quantity and allows outsiders to fact-check the data. We can go through game logs and quantify RBI totals, batting average, slugging, stolen bases, ERA... because all the numbers used to create those stats are freely available.

However, defensive numbers aren't readily quantifiable. They're based on data collected by proprietors like STATS Inc, whose workers subjectively mark on field charts where a ball landed, then a somewhat subjective set of criteria is used to determine whether a typical fielder at that position would have caught said ball, and then credits or penalizes the respective defender accordingly.

I mean... there's no way for an outsider to confirm or deny the stats given. We are powerless to do anything but take the stat proprietors at their word. Between that and the sometimes political nature of their discourse, plus curious and repeated insistences that outsiders ought to just trust them with a benefit of the doubt... I can totally understand even a basic skepticism for the validity of the stats, not to mention a rational calling into question of those stats.

That aside, any stat that calls Raul Ibañez a crappy LF for his entire career then suddenly calls him a decent LF once he joins the Phillies ought to be called into question.

Also, Sandy,

[DrNaka]

DER of 2009 teams

1 Seattle Mariners 0.728
2 Los Angeles Dodgers 0.725
3 San Francisco Giants 0.721
4 Cincinnati Reds 0.717
5 Chicago Cubs 0.715
6 Texas Rangers 0.712
7 Tampa Bay Rays 0.711
8 New York Yankees 0.710
9 St. Louis Cardinals 0.708
10 New York Mets 0.708
11 Detroit Tigers 0.707
12 San Diego Padres 0.706
13 Chicago White Sox 0.706
14 Milwaukee Brewers 0.705
15 Philadelphia Phillies 0.704
16 Washington Nationals 0.704
17 Florida Marlins 0.702
18 Arizona Diamondbacks 0.702
19 Colorado Rockies 0.701
20 Atlanta Braves 0.701
21 Pittsburgh Pirates 0.700
22 Minnesota Twins 0.699
23 Los Angeles Angels 0.699
24 Oakland Athletics 0.698
25 Cleveland Indians 0.695
26 Baltimore Orioles 0.693
27 Kansas City Royals 0.693
28 Toronto Blue Jays 0.692
29 Boston Red Sox 0.691
30 Houston Astros 0.689

Team UZR of 2009
Team UZR DERranking
1. Mariners 85.5 1
2.Rays 69.5 7
3.Reds 52.6 4
4.Giants 51.2 3
5.Tigers 43.6 11
6.Rangers 32.5 6
7.Pirates 30.1 21
8.Phillies 27.9 15
9.Diamondbacks 21.6 18
10.Angels 13.3 23
11.Brewers 9.9 14
12.Athletics 5.2 24
13Dodgers -0.1 2
14Padres -13.4 12
15Rockies -14 19
16Red Sox -16.3 29
17Astros -17.7 30
18Cardinals -17.8 9
19Yankees -18.5 8
20Braves -18.8 20
21Cubs -19.8 5
22Marlins -20.5 17
23Orioles -23.6 26
24Nationals -26.7 16
25Blue Jays -33.3 28
26Indians -33.5 25
27White Sox -35.6 13
28Twins -37.3 22
29Mets -47.3 10
30Royals -49.9 27

I underlined teams which have more than 10 in difference for DER and team UZR.
So how aboiut these teams?

B. League will be lights out pitcher with the 2010 M's.