Measuring the impact of a manager on a Major League Baseball team is a difficult thing to do at best. The closest approximation we can come up with is to compare the number of games a team wins to the number of games the team should have won.

Using Fangraphs data for the last four seasons, we compared total Wins Above Replacement (WAR) for each team to the number of wins each team had greater than 55. In theory, my six-week old daughter could manage a big league club to 55 wins, so we want to see how each team performed compared to the worst possible scenario (55 wins).

What we see is a close relationship between a team’s total WAR and the number of games a team wins. No surprise there. We are more interested in the trendline (black line above). This gives us a formula that will help us determine how many games a team should have won based on how the players performed.

EXPECTED WINS = (0.95 * WAR) + 48.2

Now let’s look at every team from this past season and see how their actual wins compares to the number of expected wins.

As you can see, the Rays 2009 win total was 12.6 fewer than what would be expected based on how the players performed this season. That was the worst mark in Major League Baseball this season.

The way we see it, there are two possible explanations. Either the Rays were the unluckiest team in baseball, or Joe Maddon did not get the most out of the talent he was working with [Ed. note: see comments for discussion on Strength of Schedule as a factor. In summary, it likely plays a role for some teams, but not necessarily for the Rays in 2009].

While we don’t want to completely discount the possibility of “luck” being a factor, there are a couple indicators that suggest the Rays were not that unlucky this season.

First is the team’s Pythagorean Win Percentage. Based on the number of runs scored and runs allowed, the Rays should have won 86 games this season. That is only 2 wins more than the actual number. This suggests the Rays were a little unlucky, but certainly not 13 wins unlucky.

Also consider that we are dealing with a 162-game schedule. How much of a factor can “luck” have over 162 games? We aren’t exactly talking about a small sample size. Maybe luck can swing a team’s record one way or the other by a few games, but can it cause a team to lose 13 more games than expected? We don’t know the answer to that question, but our gut says “No.”

Also, if the Rays were “unlucky,” then certainly this should be an aberration on the record of Maddon. Actually, turns out it is not.

In three of Maddon’s four seasons as manager of the Rays, the team’s win total underperformed the performance of the players by a wide margin, ranking at or near the bottom (complete data for 2006-2008 can be found after the jump). Even 2008, when the Rays won the AL East and Maddon was named American League Manager of the Year, the team still only won 2 more games than expected.

The counter-argument is that Maddon was not managing to win in 2006 and 2007 as the team was in a developmental stage. So maybe we should not hold Maddon accountable for 2006 and 2007.

Also, one could make a case that outside of his in-game decisions, the manager is somewhat responsible for getting the players to perform at a high level. The Rays WAR values suggest that the players are indeed playing at a high level. However, that level did not lead to the number of wins that would be expected, and that could be directly related to Maddon’s managerial decisions during the games.

We are not ready to say JoeMa has been a bad manager. You can’t be a bad manager and win a pennant in the same division as the Red Sox and Yankees. But according to this data, the team played like a 97-win team this season and yet they only won 84. Is that all Maddon’s fault? Not likely. But it is hard to imagine that any other manager would have done worse than 84 wins with this level of talent.

Likes(0)Dislikes(0)

1. Gus says:

Hey Cork, not every season can be a Picasso.

You've proved what I always suspected. His moves are very frustrating, particuarly with (a) the bullpen in really all 4 seasons, but particularly 2007 and again this year and (b) batting order and the use of Upton.

He is a very nice guy. I just wish he had (a) a strong pitching coach to manage the bullpen and (b) a "bad cop" in the dugout to offset his "good cop" routine.

I can't wait for the study proving my belief that Tom Foley is the worst third base coach in baseball.

Likes(0)Dislikes(0)
2. Beth says:

What we are looking at is how many wins to expect given the individual player stats. But part of what a manager does is create situations in which individual players can excel. We often will look at a player who is performing above or below his previous level and credit (or blame) managing and coaching.

In other words, a manager has multiple tasks, including player development; figuring out when and where to use players; and tactical gametime decisions like when to bunt or whether to pull your starter. This analysis asks us to separate those tasks in a pretty artificial way. Maybe it does tell us that Maddon is better in some aspects of his job (player development) than others (game time tactics). But that's different from drawing conclusions overall about whether someone is a good manager.

Likes(0)Dislikes(0)
• I usually try not to be this long-winded during posts, but I did address that subject towards the end...

"Also, one could make a case that outside of his in-game decisions, the manager is somewhat responsible for getting the players to perform at a high level. The Rays WAR values suggest that the players are indeed playing at a high level. However, that level did not lead to the number of wins that would be expected, and that could be directly related to Maddon's managerial decisions during the games."

so yes, I do try to make the distinction between his management of the players (very good?) and his management of the games (not so good?).

Likes(0)Dislikes(0)
• Gus says:

I think many Rays fans would agree with you here. Very good in all respects for this team until the lineup cards get filled out and the game starts. He needs help in his dugout with game management to save him from himself.

Likes(0)Dislikes(0)
3. Tone says:

He has been a bad gametime manager his whole career with the (Devil) Rays. Why they signed him to a 3 year deal is beyond me, but they seem to believe they are smarter than everyone so what can you do. The bullpen usage was absolute crap, it overshadowed all other mistakes. He broke their souls with way too many crushing losses. Then bringing out the same pitchers to repeat the nightmare over and over. Crushing. They used Kaz and Sonny WAY too long when they were obviously burnt toast. Water under the bridge. Baseball is total shit right now anyway.

Likes(0)Dislikes(0)
4. Justin H says:

I think he really struggles with the bullpen. He has his match-up mind set, but it rarely works for an entire season. Guys don't know a role and aren't prepared to pitch. It'd be better if the Rays had a closer already set up so when the 9th inning comes it isn't chaos. He's basically a bench coach like he was with the Angels, that's what he's good at. My best comparison is a great Offensive Coordinator in football who goes to be an NFL coach, but is terrible at it (Norv Turner).

Likes(0)Dislikes(0)
5. Don says:

Oh my God....I'm not a stats man...but glad to see some figs. that back up what I've been saying all along....
Look... its hard for fans to be critical of good ole Joe!!..but
He just does not get RESULTS from the talent available to him...his decision making is very weak for the position he holds...
He is much better suited for the position he held for 25 years....
Good ole Joe on the bench and the clubhouse...leave the heavy work to someone else!

Likes(0)Dislikes(0)
6. Wreckard says:

One big problem with this is that every team does not play every other team an equal amount. So even if simply summing a team’s WAR was indicative of a team’s talent level for a given season, some teams would be favored in this because they had an easier schedule.

Case in point: every team in the weak NL central overachieved last season by this measure. And the same thing in 2008 (only 1 team below 0). Meanwhile, on the other side of that coin - for 3 years in a row the majority of the teams in the AL East underachieved according to this measure. It would appear at a glance that division strength correlates negatively with this measure of Wins vs Expected, which is what you might expect.

Likes(0)Dislikes(0)
• I did consider strength of schedule but left it out because the post was already way too long. but two things: 1) every game within the division must have a winner and a loser. So you cant really have a situation where EVERY team in the division benefits or struggles because of strength of schedule. you would expect to see teams on both sides. Take the AL East. The Rays, O's and Jays are all low. That makes sense if strength of schedule is hurting them because each team faced Red Sox and Yankees 36-38 times. But then why dont we see the Yankees and Red Sox at the opposite end of the spectrum? We dont. Both have win totals right about where they would be expected based on WAR; 2) also, unbalanced schedule means a lot of games in division, but it is still less than half of a team's schedule. there might be a small impact, but there is no way the strength of schedule accounts for the entire difference between actual and expected wins. I dont know this for certain (obviously) but that is my hunch.

Likes(0)Dislikes(0)
• Wreckard says:

"But then why dont we see the Yankees and Red Sox at the opposite end of the spectrum?"

But we do. The Red Sox are >= 0 for every single season you charted here. The Yankees are right around the median every season except for 2006. And besides, the AL East schedule isn't as tough for the Yankees as it is for the Orioles, because the Yankees don't have to play the Yankees.

"unbalanced schedule means a lot of games in division, but it is still less than half of a team’s schedule. there might be a small impact, but there is no way the strength of schedule accounts for the entire difference between actual and expected wins."

I don't see how you can say this. What's the standard deviation here, 2-3 wins? That could easily be accounted for by strength of schedule alone.

That said, I agree that schedule isn't entirely responsible for this - there are probably a lot of factors here that could cause some of this variance. Another one that jumps out is park effects - WAR is a park-neutral statistic, but wins is not - so teams like the Twins, who are historically excellent at home due to their dome, are overacheivers every single season you charted here.

At any rate there are a lot of possible explanations for this data that don't have anything to do with the manager. And I say this as a neutral party, I'm no Rays fan so I have no horse in the "Joe Maddon Sucks" race.

Likes(0)Dislikes(0)
• Mike Summers says:

Strength of Schedule cant be a factor with the Rays. They were 40-32 in the division and they were a respectable 16-20 against arguably the 2 best teams the AL. The Rays actually had a BETTER winning percentage in the tough AL East than they did outside the division.

Likes(0)Dislikes(0)
• to add on to Mike's comments: you say the "yankees dont play the yankees." this is true, but the Rays had the second largest total WAR in baseball behind the Yankees. So while the Yankees dont play the Yankees, the Rays didnt have to play the Rays. If you add up the total WAR of the 4 divisional opponents for the Rays and the 4 divisional opponents for the Yankees, there is only a 6 WAR difference. A little advantage for the Yankees, but not much. And definitely not 19-wins advantage in strength of schedule. And the Rays have the exact same WAR as the Red Sox, so they have the exact same strength of schedule based on WAR, the exact same WAR and yet the Red Sox finished with 11 more wins. I'm not saying SOS isnt a factor with some of the teams, but it was not a factor with the Rays.

Likes(0)Dislikes(0)
• Wreckard says:

I guess I'm focusing less on the 2009 Rays specifically here and more on the methodology overall, which I think is unsound. I'm not saying all the variance here can be explained by SOS alone, just pointing out that the methodology doesn't take it into account and it could be creating a lot of noise here.

And the conclusion that all of the variance here can be solely ascribed to the manager seems pretty far-fetched to me. I think you're seeing a side effect of using WAR to do this, because WAR works hard to remove a lot of the things that affect a player's counting stats (the things that result in wins) such as park effects, opponent strength, etc. Wins and losses don't remove those things, so you're using stats with conflicting goals.

Likes(0)Dislikes(0)
• that is a very fair comment. and that is why I didn't make any grand claims beyond the Rays.

a team like the O's might be an extreme case. They probably are at a large disadvantage with schedule. However, with most teams, I am guessing they are at least in the ballpark of where they should be. Even if there was some noise from the schedule for some teams, I think it is safe to say that the teams at the extremes would still be at the extremes, they just might be in a different order.

Likes(0)Dislikes(0)
7. Sam A says:

I don't know about the worst manager in the game, but some areas of concern are:

The bullpen (over)management drove me nuts all season. On several occasions Maddon used 4-5 relievers to get through two innings or less. I wasn't surprised when the bullpen fell apart down the stretch.

Poor performance against mediocre teams and poor performance on getaway days. These two may be related, but it seemed like the players either weren't prepared for some games/series or weren't in the games (or both).

Likes(0)Dislikes(0)
8. John says:

It's also possible that WAR is a flawed stat because it attempts to measure something that cannot actually be measured. But that's a pet peeve of mine as it relates to all these newfangled stats.

Likes(0)Dislikes(0)
• batboy1222 says:

The difference between Pythagorean W-L and WAR is this: Pyth uses RS and RA to determine how well a team should have performed, while WAR includes defense, defense-independent pitching, and context-neutral hitting. It's more rigorous than Pyth, so it's more likely that we're missing something by using Pyth in this analysis than that WAR is inherently wrong.

Likes(0)Dislikes(0)
• yes, this is true. but in theory, WAR measures how the players performed. ultimately, that should determine how many runs were scored and how many runs were allowed. And of course it is RS and RA that determines wins and losses. In the case of the Rays, their RS and RA played likes an 86-win team. The actual performance of the players was closer to 96 wins. so the difference appears to be that the Rays production at the plate, in the field and on the mound did not translate on the scoreboard (RS and RA).

Likes(0)Dislikes(0)
9. dtpollitt says:

Scraggler from BleedCubbieBlue here.

I like the set-up, it looks like a regression equation from 4 years worth of correlational data (right?). Looks like it was a good rainy Sunday activity.

Any reason 4 years of data was selected? I'm curious.

Can we use individual-level performances to predict group-level wins? I myself don’t know if this is an level-appropriate analysis; perhaps something like a HLM approach would be easier, with players nested within teams.

I'm sure there are some questions about the value of WAR in and of itself, but that's why we use analyses like these, to learn more about WAR, too.

Conversely, and simply enough, can a manager be to blame for a loss? He doesn’t hit, field, or pitch. Perhaps it means for as good some players are, they are poor on other days.

Fun analysis, I liked it. Good job.

Dan

Likes(0)Dislikes(0)
• "regression equation from 4 years worth of correlational data" yep. I used four years because that represents Joe Maddon's tenure as manager for the Rays. I do have some issues with WAR, but in this case, it is a simple way to put batters and pitchers on a similar scale. And certainly the players are the biggest factor in wins and losses, but ultimately a manager decides how long a starter stays in or when to pinch hit or which relief pitchers to use. Interestingly, if you had asked me before hand, I would have guessed that a terrible manager might cost his team 7-10 wins a year while the best managers might be worth 7-10, with most of the managers falling somewhere in the -3 to 3 range (that is, minimal impact). If we assume there is some noise in this data (there is ALWAYS noise in baseball data, way too many variables to control) maybe managers do fall within that range.

Likes(0)Dislikes(0)
10. batboy1222 says:

While I admire the effort, it's really, really tough to quantify the effect that a manager has on its team. That can be said for any coach, since there are so many different variables affecting how players perform. What do you attribute to a manager or coach, and how much credit do you give them for winning and/or losing a game? It's tough to say for certain, but this is a nice effort at least.

I do think you discounted the possibility that the Rays were the victims of bad luck too quickly. You relied solely on Pythagorean W-L, which is a good measure, but is entirely dependent upon a team's Runs Scored and Runs Against totals - and these two totals can be affected by luck.

1. Defense. While our pitching wasn't as good this year as last, we did have the second best defense in the league (as measured by UZR). Last year, our defense helped our pitchers over-perform their FIPs, but that doesn't appear to be the case this year. Many of our pitchers were still unlucky on batted balls, which would make our Run Against measure higher.

2. As for Runs Scored, the Rays had a high-powered offense this year, but their scoring was ridiculously bunched. Some games we'd score 6+ runs, others we could barely scrape together 1 or 2. It was feast or famine, and if you look at the historical distribution of runs, teams that are excessively bunched on year tend to regress to more evenly spaced games the next year. So maybe our offense was high powered, but we possibly didn't gain as much value from our Runs Scored total as one would expect.

Likes(0)Dislikes(0)
• there is probably something to the feast or famine theory. This team played like a 95-win team from May-August. But in April and September they played like a 110-loss team. That is a HUGE difference.

Likes(0)Dislikes(0)
11. Peter says:

You can't pit under achievement on managers. When you truly dive into the question, the answer as to why teams under achieve is lost in a cloud of untested or untestable variables. One point, though. Part of the reason the Rays were under achievers in '06, '07, and '09 is the fact that their bullpen was awful. A good bullpen allows a team to over achieve.

Likes(0)Dislikes(0)
• First of all I completely agree that there are untestable/uncontrollable variables. Second of all, that a bullpen can impact pythag record is not news. I have long argued on this site that they out-performed their 2008 pythag bc of their bullpen. Specifically, you can look at the Rays MLB best record in 1-run games and deduce that a strong bullpen leads to more wins in close games than would be expected by chance.

But...I am not sure how that is relevent here since the performance of the bullpen is included in WAR and by all measures the bullpen was average or slightly better than average in 2009. So the bullpen cannot be the reason the Rays underperformed their WAR.

Likes(0)Dislikes(0)
• Peter says:

WAR treats relievers the same way it does starters. Strong bullpens can influence actual records beyond their WAR value.

Likes(0)Dislikes(0)
• You make it sound like relievers have more of an impact on W-L record than starters. I will concede, and have written many times that a bullpen will impact a team's record in close games. I don't think that is any big news. But 1-run games acount for less than 25% of a team's games. A starting pitcher impacts close games AND blowouts. Again, the difference between Pythag and actual can be explained in part by bullpen. But that should be less so when comparing WAR because the talents of the bullpen are included in the data. Might there be a slight bump from a really good or really bad bullpen? Maybe. But the Rays had an above-average bullpen this season. So you can't say their bullpen accounts for the 13-win difference.

Likes(0)Dislikes(0)