The Problem With Player Stats In The NFL, And A Project On QBs

I talk a lot about metrics and statistical analysis here in 12MR. Whether its its my quest for a better analysis metric for evaluating defenses, or my mathematical power rankings, or just my general evaluation of players, I prefer everything to be grounded in solid data-driven analysis as well as tape study.

This idea has taken off in baseball, and that’s why I got my start as a writer coving MLB. I’m a scientist, and my mathematics and analysis background allowed to me jump right in. The problem was that  I’ve always been a football guy. I didn’t belong in baseball, so here and I am, trying to do my analytical thing for a sport that is the still in the dark ages when it comes to developing useable metrics.

Unfortunately, a lot of football doesn’t show up on a stat sheet.

For example, most running plays end with a tackle by a LB. Too bad not all tackles are equal. A tackle made 8 yards downfield isn’t the same as as one made up near the line of scrimmage. If you just look at the number of tackles, you don’t get that information, and unfortunately, that’s all most people have access to.

We call those tackles close to the line of scrimmage “stops.” Bobby Wagner had 69 of them last season, which is a lot compared to his NFL peers. That particular piece of information is out there if you know where to look, but few know it’s importance and even fewer bother to use it.

And what else happened to make that tackle possible? What about the DT who commanded a double team of the C and G and still wasn’t pushed back? That meant the G wasn’t able to peel off and get up to the next level and block the LB who made the tackle. Where’s that on the stat sheet? Or how about the DE who got 2 yards up field and set the edge, making it impossible for the RB to bounce the run to the outside? Where’s the stat for that?

Those things are just as important as the tackle. If they don’t happen, then the stop doesn’t happen either. Football is a team game. Everyone must do their job, and do it well, or the play breaks down.

This is why we have to be careful using stats to evaluate individual players. It is also why I often use Pro Football Focus for quantitative player analysis, and not traditional stats. PFF are the only ones who evaluate the players for doing those not-so little things that make each play work. It’s also why NFL teams pay big money for their data.

For a data-driven guy like me, this is all very frustrating. I’d love to putting together mathematical models that allow us to determine a player’s impact on a game relative to other players at other positions. I could still do that, but I’d have to use PFF’s data to do so, and I don’t think they’d be happy about it.

Unfortunately, this means I’m stuck looking at team-wide data for the most part in order to generate something meaningful.

May 20, 2013; Seattle, WA, USA; Seattle Seahawks quarterback Russell Wilson (3) participates in organized team activities at the Virginia Mason Athletic Center Mandatory Credit: Joe Nicholson-USA TODAY Sports

There is one position though where we can do some statistical work and not have it be meaningless. I’m talking about QB. Currently, we have 2 measures that attempt to evaluate the position: passer rating and Total QBR.

Passer rating is archaic and fairly pointless. The different variables (TD, interceptions, completions, yards, etc.) are combined in a haphazard way that does not have any connection to points scored. It also doesn’t take into account some of the the thing that QBs do, like scrambling and avoiding sacks.

QBR, on the other hand is quite new. ESPN’s staff put it together in an attempt to fix some of the problems that the passer rating stat has. The problem is that they took it too far. Sacks seem to be the biggest factor in determining the rating, which is a bit silly.

The system is also not purely stats based, and it is hardly objective. Plus, the fact that they refuse to divulge the formula used only adds to the problem. I simply refuse to use Total QBR because ESPN refuses to tell us what actually goes into the metric.

That leaves us with almost nothing for a true QB rating system. After a ton of prodding by a few other analysts  I’m going aim to fix that.

Over the next year, I’m going to be working with some of the best numbers guys in the business to try and develop a QB rating metric that improves on the archaic passer rating, but doesn’t get into the stupidity that is QBR. It’s likely to be a comically frustrating voyage, so I’m going to share my progress here on 12MR.

At this point, I have no idea what this is going to look like, but i’m sure I’ll have more grey hair when it’s completed.