Home > Draft, NFL > Selection Bias and Quarterbacks Drafted Between Rounds 3-6 of the NFL Draft

Selection Bias and Quarterbacks Drafted Between Rounds 3-6 of the NFL Draft

NEW ORLEANS - OCTOBER 24: Colt McCoy  of the Cleveland Browns throws a pass against the New Orleans Saints at the Louisiana Superdome on October 24, 2010 in New Orleans, Louisiana. (Photo by Chris Graythen/Getty Images)

I have a theory and it goes like this: There should be, on average, more than two college quarterbacks each year who enjoy NFL success at average or above average levels.  This doesn’t make a lot of intuitive sense though, considering how many scouting dollars and everything else goes into projecting rookie quarterbacks.  The simplest response to the evidence would be that my theory is wrong: on average, a draft doesn’t produce more than two quarterbacks capable of throwing 2,000+ passes in the NFL at a reasonable level of performance.

After all, since 2005, there have been 9 passers to achieve a success level that would justify a player’s selection in the NFL Draft (though not necessarily where a player was taken).  In order of the amount of success enjoyed per start, these nine players are: Aaron Rodgers (05), Matt Ryan (08), Joe Flacco (08), Chad Henne (08), Jay Cutler (06), Josh Freeman (09), Jason Campbell (05), Kyle Orton (05), Vince Young (06), and to a lesser non-qualifying extent: Kevin Kolb (07).  That’s an average of two guys per year (though 2009, at least), but just one guy drafted after the second round (Orton).

Throwing in Colt McCoy as the next possible franchise QB to come out of the middle rounds, we’re looking at numbers that suggest that only about 5% of players drafted in the middle rounds of the 2005-2010 NFL drafts become starters.  Making an argument that someone (Charlie Whitehurst, Matt Flynn, or Nate Davis) else in this group eventually emerges for someone in future years brings us up to a little over 7% of QBs drafted in rounds 3-6 giving more than just a backup.  That’s a number I feel comfortable working with.

If that number is accurate — that more than 90% of middle round picks offer little more than a backup — then my theory offered at the top of this article is simply wrong.  There’s no other way to put it.  But I have an alternative explanation: selection bias.

Let’s say that I’m right, and of all quarterbacks who are drafted later on in the middle rounds, one-third of them can play at the NFL level, and play measurably BETTER than the middle rounders who can’t (Bruce Gradkowski, Derek Anderson, Rusty Smith etc).  One-third, that’s 33%.  That’s no where near the 7% that recent history dictates can play.  But I’m going to argue that there may very well be 2 or 3 players out of every 7 middle rounders who would produce performance on par with the average player picked in the first two rounds — and thusly, would by definition have been underdrafted/undervalued by the draft process.  Selection bias, however, dictates that not all mid-round quarterbacks are ever going to play in a meaningful sample, since the majority of the opportunities are going to be given to first rounders, and even first round retread quarterbacks rank a bit higher on the opportunity chain than young mid-round quarterbacks.

My other point is that quality mid-round prospects have proven somewhat indistinguishable from other prospects who have a ceiling at the level of NFL backup.  As evidence, Matt Flynn may fit in either category at this point, and Matt Schaub was still somewhat unknown when acquired by the Texans.  It’s easy to predict right now that Flynn will not be the next Schaub, but much harder to do so on the day before the Texans acquired him.  They looked fairly similar.

If mid-round prospects are both indistinguishable and prone to being overlooked and mistreated by coaches due to selection bias, it makes sense to conclude that it cannot be accepted that players who get drafted in the mid-rounds and fail to qualify in terms of attempts cannot play.  If we shrink it to the top 9 mid-rounders in terms of attempts, we get three players with average or above average value out of the 9: Dan Orlovsky, Colt McCoy, and Kyle Orton.

We’ve also shrunk the sample too small to make any definitive conclusions.  But three successful players with 200+ attempts suggest against the unsuccessful players with the requisite attempts: (Andrew Walter, Brodie Croyle, Bruce Gradkowski, Charlie Frye, Trent Edwards, Derek Anderson).  No way to say if this sample is representative of the greater whole as it’s too small.  But out of the remaining 31 players, it’s not-unreasonable to think that theres ten quality quarterbacks hiding in a group of thirty.  I’d venture to try these ten over all the others:

  • Mike Kafka
  • Jonathon Crompton
  • Tony Pike
  • Nate Davis
  • Erik Ainge
  • Andre Woodson
  • Stephen McGee
  • Dennis Dixon
  • Josh Johnson
  • John Skelton

The problem, of course, is that there’s no compelling reason to express great confidence in any of these guys over all the others.  I left players off this list that didn’t and haven’t had an opportunity to prove they cannot play.  There’s an opportunity cost to playing any of these players.  Does a team not draft Ryan Mallett so Stephen McGee can get a shot instead?  I don’t know where the ‘yes’ line starts, but I imagine it’s pretty short.  For all the budgets spent on scouting, there’s not a whole lot of information coming out about these players after the draft unless you have a connection within your coaching staff.  And just because a team like Seattle gets interested enough to invest a pick into a guy like Charlie Whitehurst, there’s still a 2/3 chance of getting a guy who can’t play.  There’s not much certainty in the middle rounds, and that’s why teams don’t pursue these options often.

If because of the uncertainty, only 1 in 8 mid-round quarterbacks play enough to sport a near-conclusive amount of film, and there is hardly no relationship based on aptitude between the 1 in 8 that gets to be evaluated and the 7 in 8 who do not, then we can multiply the 1/8th chance of getting to play by the 1/3rd chance of being good enough to play, and find that — with no change in the behavior of NFL teams — only 1/24 mid-round prospects are going to enjoy franchise QB status, and even then, will likely have to go elsewhere to find it.  Which is very much consistent with what we have seen in the past six years.

The selection bias principle does not affirm or deny that 1/3 of all mid round prospects have what it takes to make it at the next level, but when you consider that the majority opinion of talent-evaluators would deny that QB talent can be found in that quantity, there would seem to be a market inefficiency siding with those who have faith in their ability to scout and give opportunity to late round quarterbacks.

Categories: Draft, NFL Tags: , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: