This year will be the fourth season that I’ve played Fantasy Baseball. I’m a baseball fan, and I have a pretty good knowledge of the most prominent players of the game. But it’s far from encyclopedic. I have at least a couple of co-workers who can walk into draft night with no real preparation at all, and end up schooling me. What I do have, though, is an analytic mind and a desire for rigor (albeit half-assed rigor). The last couple of years, I’ve been pulling down players stats from the season before, doing a little magic, and coming up with some sort of an order. But I’ve finished in sixth place and then in seventh (out of 12).

There are some complexities in fantasy baseball that make an accurate analysis difficult. As a manager, you need to draft at least one player at each position, a couple general “hitters”, a couple starting pitchers, and a couple relief pitchers (the exact numbers vary from league to league). But there are some positions (catcher and second base in particular) that have very concentrated hitting skills. There are probably three catchers who could be considered “great” hitters. Does the scarcity increase the value of those catchers? How much?

Not all of the stats that are tracked by fantasy leagues can be considered equal. Compare, for instance, RBI and stolen bases. Just about any player that gets drafted will have a minimum of 40 RBIs, but only about half a dozen players will finish with that many stolen bases and at least half will probably have less than 10. So any specific stolen base is more likely to make the difference, since teams' numbers in that stat are more likely to be low. Is a base-stealing average-hitter worth more than a slow heavy-hitter? How much?

And every time it’s your turn to draft a player, you can choose any available player at any position. If you take one of the great catchers early, you might not get a premiere starting pitcher. How can you be sure the tradeoff you made was wise? Hitters and pitchers are scored on completely different criteria, making comparing them even harder.

So how can we possibly take this kind of system and predict how much a player is worth? At the end of the day, what you want in a head-to-head league is “wins”, so the most useful end statistic would be one that you can use to determine “If I drafted Player A instead of Player B, I’d end up with (on average) this many more wins”. After spending some abortive attempts trying to come up with an algorithm, I realized it didn’t have to be that hard. I could throw computing power at it and run a Monte Carlo simulation.

So that’s what I did. I started with someone else’s player-by-player predictions for 2011 stats (the Marcel the Monkey Forecasting System, specifically). Then I wrote a script that would simulate a ten-team league over and over. I had to simplify the rules a little bit – figuring out what to do about benching players and trades and pitcher off-days was just too much. After each simulated season, I determined how many wins over exactly average each team was, and I credited every player on that team with that many wins. Over time, good players would more likely to be on good teams than bad ones, and their averages would be high. Less-good players' averages would be low.

After two million one million (see update below) seasons, I had some pretty firm numbers, and some surprising results.

Keep in mind the caveats: This was a ten-team five-by-five head-to-head league. The WOA column (“Wins Over Average”) is the most useful number, and it’s what the spreadsheet is sorted by. It’s scaled for a nine-week season (each team played each other team once), which obviously is not standard, but it’s directly proportional to a real 24-week season.

Multiply by 2.66, and you see that drafting Albert Pujols is worth almost 26 wins over the average player and 2.44 wins over the second-most-valuable player, Hanley Ramirez. After those first two infielders, there’s a lot of pitchers, which goes counter to a lot of the common advice out there. Which I like. Joe Mauer is the first catcher, at 27th (third round in a ten-team league).

Here’s the code and source files that I used, if you’re interested in giving it some tweaks of your own. I used the standard 5x5 stats, but you can change those pretty easily. It took about 90 minutes to run a million seasons on my not-state-of-the-art computer.

Update 11 Feb: This morning, I realized that these numbers were off, in particular those of pitchers. I had been miscalculating team ERA: just averaging all of the players together. The problem is that the true team ERA value is IP-weighted: Total team ER divided by total team IP. So it was over-valuing players with good ERA and few IP, like relief pitchers. The same was similarly true about WHIP and BA, so everyone who didn't play much was being given too much credit. I've updated the code and re-run a million seasons. There are a lot of starting pitchers rated highly now, 8 of the top 10 and thirteen of the top twenty. But actually, the top hundred players are exactly 50/50 batters and pitchers.

Update 14 Feb: I changed the spreadsheet so it includes Marcel's predicted stats for each player. This should make it easier to see why each player might be ranked where he is, plus make it easier to use on Draft Day. I updated the code linked to above, as well.

Update 22 Mar: A couple of last minute tweaks before my draft tonight. I couple weeks ago I added code to remove the "chaff" players (really, only the top couple hundred players will be drafted in a fantasy league, so I shouldn't be comparing Pujols against the 1000th-best batter). And a co-worker noticed David Ortiz missing from the list, which revealed a bug in the way I was dealing with players who play no positions. Also, I've removed hands-down first-round pitcher Adam Wainwright from the stats list.