NFL Play-by-Play Data—Analyzed, Visualized, and Quizzified


There’s an updated, better version of this post available here: http://blog.statwing.com/dataset-ten-years-of-nfl-plays-analyzed-visualized-quizzified-downloadable/ .

It’s third-and-2 and you desperately need a first down. What do you do, run or pass?

A few months ago, Advanced NFL Stats released an
incredible dataset
containing information on every play of the 2012 NFL regular season. We structured that dataset and

uploaded it into Statwing for analysis.

Now you can test your coaching instincts against the data.

Early in the game, the score is tied. You have fourth-and-goal at the 2-yard line. What should you do?



Correct

Wrong. You should go for it.

Wrong. You should go for it.

About 1.5% of field goal attempts are unsuccessful at that range, so you’re virtually assured three points if you kick.
When teams go for it on fourth-and-goal from the 2, they get a touchdown 44% of the time.
So on average teams get 3.1 points when they go for it—roughly the same amount they’d expect if they kicked a field goal.

Touchdowns on 4th and Goal



Click the image to explore this analysis and play with a dataset consisting of the last ten years of plays from third-and-goal and fourth-and-goal from within the 10-yard line.

But that’s not all. If you’re stopped on fourth-and-goal, the opponent starts with terrible field position. You’ll even get a safety about 5% of the time.
By comparison, you can expect the opponent to start from the 23-yard line after a kickoff following a made field goal,
and they will even have a 0.5% chance of returning the kickoff for a touchdown

Yards to Endzone Immediately After Kickoff



Click the image to see the full NFL 2012 regular season dataset in Statwing. It contains all the analyses cited above.

Going for it and kicking a field goal both yield about 3 points on average, and the field position is much better if you go for it.
Despite this, coaches usually kick a field goal on fourth-and-goal.
In fact, during the last ten seasons coaches only went for the touchdown twice in this situation.

You’ve gotten out of answers correct so far.

It’s third-and-1. Which type of run is most likely to result in a first down?


Correct

Wrong. Side story: your author’s mom always yelled at the Chiefs not to go up the middle on third-and-short.
It saddens your author to find out that she was mostly likely leading the Chiefs astray.

Going up the gut just barely beats running around the end.

Run Direction vs. First Down



Click the image to see an even more detailed breakdown of running plays (e.g., off-center versus off-guard).

You’ve gotten out of answers correct so far.

On third-and-2, are you more likely to pick up a first down by running the ball or passing it?



Wrong

Wrong

Correct

We used Statwing to look at every third-and-2 in the 2012 NFL regular season dataset.
Teams picked up first downs a bit more than half of the time, regardless of whether they ran or passed.

Run vs. Pass for First



This difference is not statistically significant.


Click the image to explore this analysis and the rest of the dataset in Statwing.

In case you’re curious, here are the odds of picking up a first down on third-and-x, split by running versus passing:

Likelihood of Getting 1st Down, by Play Type


Runs are statistically significantly more effective than passes on third-and-1 and similarly effective on third-and-2, third-and-3, and third-and-4.
In 2012 passes were not statistically significantly more successful for third-and-5 through third-and-10.

As an aside, coaches tend to pass on third with more than a yard to go.

Proportion of 3rd Down Plays That Are Runs


Coaches very rarely run on third down with three or more yards to go.

You’ve gotten out of questions correct so far.

You need a two-point conversion. What kind of play should you call?



Correct

Wrong

Wrong

During the last ten years running has succeeded 61% of the time, versus 45% for passing.

This seems odd because we just found out that running wasn’t better than passing on third-and-2.
But the two-point conversion situation is different from a typical third-and-2.
For example, the defense isn’t spread out, so it’s hard for receivers to find gaps in the coverage.

Two-point Conversions - Running vs Passing



Click the image to see statistical data, explore this analysis, and play with the rest of the 2003–2012 two-point conversion dataset in Statwing.

This suggests that coaches should run more often than they currently do.

Coaches Tend to Pass for Two-point Conversions



Click the image to see the confidence intervals and play with the rest of the 2003–2012 two-point conversion dataset in Statwing.

You’ve gotten out of answers correct so far.

How depressing was it to be a Chiefs fan last year?



Indeed

Wrong-ish. This is a judgment call, but the author claims the expertise of having
specialized in this field for some time, and thus feels confident to assert that one does not get used to it.
Though please feel free to credit yourself as correct if you are a Chiefs fan or root for another team that

hasn’t won a playoff game since George Bush was President (the other George Bush)
.

No, not really. Last year’s Chiefs made the Royals look competitive.

Here’s how depressing it was. On an average defensive play in 2012, the Chiefs were losing by a touchdown. And that calculation includes the first quarter.

Chiefs Score Differential Before Defensive Play



This data allows you to precisely measure how seldom a Chiefs fan thinks, “Wow, I can’t believe we’re winning this game!”


Click the image to see more and explore the 2012 regular season dataset in Statwing.


Thanks for playing

You got out of answers correct.
Not ideal. But, hey, such is life.
Not ideal. But, hey, such is life.
Not ideal. But, hey, such is life.
That’ll do just fine.
That’ll do just fine.
Change your name to Harbaugh.

Tweet your quiz results




Discussion on Hacker News

A special thanks to David Laughlin, who wrote most of the copy for this post. David is available for freelance work at
davidclaughlin@gmail.com.

Notes


The original data from
Advanced NFL Stats is mostly free-text play descriptions,
which we interpreted into structured data using Excel.
The original data does have a few errors here and there, not all of which could be cleaned up.
Some plays are missing, and a very small number of plays have some inaccurate data.
But overall the dataset is comprehensive and accurate.
For example, the dataset only omits four of the roughly 1,000 field goals attempted in 2012.

You can download the structured data in Excel to see the specifics of how we calculated various fields from the original data:

  • 2012 regular season data (39MB).
    This data is excellent, though not perfect.
  • All data from 2003–2012 except the 2012 playoffs (101MB).
    Our formulas are only tailored to the 2012 season, so there are a few difficulties when applying our formulas to the multi-year dataset.
    For example, the older dataset doesn’t always distinguish quarterback scrambles from run plays.
    Still, the data for the other seasons seems similar in most respects.
  • Two-point conversions (200kB).
    A subset of the 2003–2012 dataset. It contains every two-point conversion during that time frame.
  • Third- or fourth-and-goal (4MB).
    A subset of the 2003–2012 dataset. It contains every third- or fourth-and-goal from inside the 10-yard line during that time frame.


We make the following assumptions throughout:

  • You are coaching the “average” team. Individual teams would vary, but the data we use is the average of all teams’ behaviors in whatever situation we are analyzing.
  • There are more than five minutes left in the half or game. Last minute plays have slightly different outcomes, so we exclude them.
  • Yardage from penalties committed during a play is included in the outcome.
  • The hypothetical coach does not always call the same type of play in the same situation.
    That is, the coach randomizes play calling enough to be unpredictable while still favoring the more advantageous plays.
    The very awesome Brian at Advanced NFL Stats does a great job of
    describing randomization and game theory.


Click through the image (or this link)
to see the analysis that yielded the 1.5% number.


In the last ten years, there have only been twenty-five instances where teams went for it on fourth-and-goal at the 2-yard line. It turns out, though, that going for the end
zone on third-and-short is pretty similarly successful to going for it on fourth-and-short, so we used both types of plays in this analysis. For example, the 44% figure
was calculated using both third- and fourth-and-2. Here, as with the rest of this fourth-and-2 analysis, we were inspired by a
2002 paper by David Romer. Romer is a notable Berkeley economist, and
his wife chaired the White House Council of Economic Advisors in 2009 and 2010.


This was barely statistically significant in the 2012 dataset,
so we ran the same analysis in the 2003–2012 dataset. There we found a very clearly statistically significant relationship.
The magnitude of the difference between the two running directions was only 6.5% in the ten-year dataset,
as opposed to 11% (with wide confidence intervals) in the 2012 dataset.


If objectives other than just picking up a first down are considered, there is actually some evidence that running is better. Brian at Advanced NFL Stats
does a great job of diving into that question,
though you might want to learn about the concept of
expected points before reading Brian’s analysis.


Brian at Advanced NFL Stats looked at this same question
with data from 2000 to 2007. He found that running was more likely to have resulted in a first down on third-and-2.
He brought more data to bear, but our confidence intervals were pretty narrow. This suggests that in 2012 running and passing were
more similar in their success on third-and-2 than during the years Brian looked at.

Sorry, comments are closed for this post.