(Draft) Order in the Bullpen

Volatility is rampant throughout baseball, but is at its most visible in the bullpen and the player draft. Year to year, each can make or break a team and their evaluators, especially when you consider the high stakes involved with first round draft picks and the money and expectations that are involved with being a team’s first pick. With regards to the bullpen, the year-to-year stability and production of a player is tenuous at best, and it is rare to find a reliever that provides not only elite results – be it saves, holds, or another metric – but results that remain consistent across years.

It is also rare to have first round draft picks in the bullpen; these high-profile selections are typically slotted for players that will perform at higher impact positions than reliever for the foreseeable future, such as starting pitcher. However, it does happen; take for instance Kansas City Royals former starter Luke Hochevar:

The best part of the bullpen, of course, is that Luke Hochevar is on pace to throw 40 innings, all of them when losing by 2 or winning by 9.—
Rany Jazayerli (@jazayerli) April 11, 2013

As you can gather from Rany Jazayerli‘s tweet, the role Hochevar finds himself in (a multimillion mop up reliever) wasn’t the one envisioned when the Royals made him the first player picked in the 2006 amateur draft. Sometimes, the best laid plans and the information at hand end up not being enough to prevent a poor decision being made; sometimes, things simply don’t work out. Yet, the time, effort and (most importantly) money involved with the development of any player, let alone a first round draft pick, forces a team to make allowances and to give the player an opportunity to succeed in a different role.

Enter the bullpen.

The bullpen is a sanctuary from past failures in many respects. Failure to live up to your starting pitcher billing. Failure to learn a third pitch. Failure to get hitters out with a traditional release point. Whatever the case may be, it is a rare breed of reliever that has spent his entire career in the ‘pen and even rarer to be drafted in the first round – Drew Storen comes to mind as an example of the infrequent player drafted in the first round and projected as a career reliever.

Yet, first round talent is first round talent – and whether it’s to start the game or come in at a high leverage point in the later innings, talent will prevail. With this and Jazayerli’s Hochevar tweet in mind, I began to wonder: do bullpens display a draft round effect? Do teams who employ higher round draft picks in their relief corps enjoy more success than those who don’t?

As we set off to find out if this effect is legitimate, let’s get the details of the materials and methods used sorted out.

First, I used Baseball Reference to determine team bullpens – if they weren’t listed as a starting pitcher and hadn’t started a game thus far in 2013, I included them as a reliever.

Next, I went and collected their draft data – both round and pick. Now here’s where things get a little shaky; I excluded players who were not drafted, which means non drafted free agent signings, guys like Baltimore’s Darren O’Day, for example, are not part of the data set. Another type of player excluded are international free agent signings, so guys like Grant Balfour and Fernando Rodney won’t be part of this analysis.

For bullpen rankings, I used Fangraphs, and their positional power rankings articles for relievers, which you can find here.

I also grabbed each reliever’s career Fielding Independent Pitching (FIP) from their Fangraphs page as a measure of each pitcher’s ‘talent’; theoretically, the more talented a pitcher, the more readily he can determine outcomes, which would be reflected in a lower career FIP. While FIP isn’t perfect, it’s a nice analog for our purposes here.

OK, on to data!

This chart shows the average round of each team’s bullpen pitcher, as well as how many first rounders they have coming in relief, contrasted with their Fangraphs bullpen ranking. Overall, Colorado leads all of MLB with their drafted relievers, on average, drafted in the first round (actually, 1.25th round), and shares the lead with the Los Angeles Angels of Anaheim and Toronto Blue Jays for most bullpen first round draft picks, each with 3. Atlanta’s bullpen was ranked #1 by Fangraphs for 2013, with Houston’s coming in 30th place, with Atlanta doing so with no first round talent.

Team Bullpen By Number of First Round Draft Pick, Average Round Drafted, and Fangraphs Power Positional Ranking

Right away, we can see the volatility previously mentioned – no real smooth progressions or trends jump out at us. Exemplifying this volatility are 2 teams that have an average bullpen draft round of 20 – the New York Yankees and Milwaukee Brewers – yet New York’s ‘pen is ranked 3rd, while Milwaukee’s is 29th. You would assume ‘lesser’ talent, as judged by average draft round, would mean a lesser relief corps, yet New York’s staff flies in the face of that assumption.

Good stuff already; now let’s look at team average FIP:

Team Bullpen By Average FIP, Average Round Drafted, and Fangraphs Power Positional Ranking

Again, we don’t see much of a trend here. Aside from the #1 ranked team having the lowest team FIP amongst its drafted pitchers and the #30 team having the highest FIP, it’s a hodge podge in between. Let’s confirm with a regression.

Regressing FIP by average draft round gives us the following:

f(x) = -0.006x +3.906  and R² = 0.005

This means there is a slight negative trend between FIP and round drafted, which is peculiar; you would assume that as draft round increased, so would FIP, but this doesn’t seem to be the case. However, a coefficient of determination (R²) of .005 means there isn’t a terribly strong association between FIP and draft round. Doing the same exercise, but replacing average draft round with average player draft number gives us the same results – nothing. Strong correlations between data are normally accepted at R² greater than 0.8.

While you could do a regression between FIP and Fangraphs ranking, I don’t include it here, since the Fangraphs rankings included all members of a bullpen, drafted or otherwise, so our dataset doesn’t exactly match with what Fangraphs was working with when they made their rankings. However, you will end up with a very low R² and strength of association; not shocking, given the disparity between the datasets.

Let’s look at things in one more way – is there a significant difference in Fangraphs ranking between teams, grouped by number of first rounders?

Cutting to the chase, there isn’t.

Overall, we see that average draft round or position doesn’t appear to have much effect on a bullpen and being drafted in a low numbered round doesn’t necessarily equate to superior performance. Since non-drafted free agents weren’t included in the analysis, we can’t make too many inferences or come to too many rock solid conclusions about the results, unless we make a whopper of an assumption that the drafted and non-drafted groups are similar enough to assume what we learned from the drafted players can be applied to the non-drafted players. Yeah, let’s not do that. While not an exhaustive exercise, it does lend merit to the notion that so much of baseball is a crapshoot – #1 draft picks flame out, while guys who couldn’t even get drafted in a 50 round format are All-Stars. While the stats don’t necessarily lie in this instance, they certainly don’t appreciably explain or quantify all of the variables that go into determining the career success of those that are acquired by through the amateur draft, at least within the scope of interest I have presented.

In a game already defined by uncertainty, volatility reigns.