Statwing is an easy-to-use data analysis tool. Click through the visualizations below to see analyses in Statwing (and to play around with the rest of the dataset).
It’s third-and-2 and you desperately need a first down. What do you do, run or pass?
A few months ago, Advanced NFL Stats released an incredible dataset containing information on every play of the 2012 NFL regular season. We structured that dataset and uploaded it into Statwing for analysis. Now you can test your coaching instincts against the data.
Early in the game, the score is tied. You have fourth-and-goal at the 2-yard line. What should you do?
Wrong. You should go for it.
Wrong. You should go for it.
About 1.5% of field goal attempts are unsuccessful at that range, so you’re virtually assured three points if you kick. When teams go for it on fourth-and-goal from the 2, they get a touchdown 44% of the time. So on average teams get 3.1 points when they go for it—roughly the same amount they’d expect if they kicked a field goal.
But that’s not all. If you’re stopped on fourth-and-goal, the opponent starts with terrible field position. You’ll even get a safety about 5% of the time. By comparison, you can expect the opponent to start from the 23-yard line after a kickoff following a made field goal, and they will even have a 0.5% chance of returning the kickoff for a touchdown
Going for it and kicking a field goal both yield about 3 points on average, and the field position is much better if you go for it. Despite this, coaches usually kick a field goal on fourth-and-goal. In fact, during the last ten seasons coaches only went for the touchdown twice in this situation.
You’ve gotten out of answers correct so far.
It’s third-and-1. Which type of run is most likely to result in a first down?
Wrong. Side story: your author’s mom always yelled at the Chiefs not to go up the middle on third-and-short. It saddens your author to find out that she was mostly likely leading the Chiefs astray.
On third-and-2, are you more likely to pick up a first down by running the ball or passing it?
In case you’re curious, here are the odds of picking up a first down on third-and-x, split by running versus passing: As an aside, coaches tend to pass on third with more than a yard to go.
You’ve gotten out of questions correct so far.
You need a two-point conversion. What kind of play should you call?
During the last ten years running has succeeded 61% of the time, versus 45% for passing.
This seems odd because we just found out that running wasn’t better than passing on third-and-2. But the two-point conversion situation is different from a typical third-and-2. For example, the defense isn’t spread out, so it’s hard for receivers to find gaps in the coverage.
This suggests that coaches should run more often than they currently do.
You’ve gotten out of answers correct so far.
How depressing was it to be a Chiefs fan last year?
Wrong-ish. This is a judgment call, but the author claims the expertise of having specialized in this field for some time, and thus feels confident to assert that one does not get used to it. Though please feel free to credit yourself as correct if you are a Chiefs fan or root for another team that hasn’t won a playoff game since George Bush was President (the other George Bush).
No, not really. Last year’s Chiefs made the Royals look competitive.
Here’s how depressing it was. On an average defensive play in 2012, the Chiefs were losing by a touchdown. And that calculation includes the first quarter.
Thanks for playing
You got out of answers correct. Not ideal. But, hey, such is life. Not ideal. But, hey, such is life. Not ideal. But, hey, such is life. That’ll do just fine. That’ll do just fine. Change your name to Harbaugh.Tweet your quiz results
A special thanks to David Laughlin, who wrote most of the copy for this post. David is available for freelance work at firstname.lastname@example.org.
The original data from Advanced NFL Stats is mostly free-text play descriptions, which we interpreted into structured data using Excel. The original data does have a few errors here and there, not all of which could be cleaned up. Some plays are missing, and a very small number of plays have some inaccurate data. But overall the dataset is comprehensive and accurate. For example, the dataset only omits four of the roughly 1,000 field goals attempted in 2012.
You can download the structured data in Excel to see the specifics of how we calculated various fields from the original data:
- 2012 regular season data (39MB). This data is excellent, though not perfect.
- All data from 2003–2012 except the 2012 playoffs (101MB). Our formulas are only tailored to the 2012 season, so there are a few difficulties when applying our formulas to the multi-year dataset. For example, the older dataset doesn’t always distinguish quarterback scrambles from run plays. Still, the data for the other seasons seems similar in most respects.
- Two-point conversions (200kB). A subset of the 2003–2012 dataset. It contains every two-point conversion during that time frame.
- Third- or fourth-and-goal (4MB). A subset of the 2003–2012 dataset. It contains every third- or fourth-and-goal from inside the 10-yard line during that time frame.
- You are coaching the “average” team. Individual teams would vary, but the data we use is the average of all teams’ behaviors in whatever situation we are analyzing.
- There are more than five minutes left in the half or game. Last minute plays have slightly different outcomes, so we exclude them.
- Yardage from penalties committed during a play is included in the outcome.
- The hypothetical coach does not always call the same type of play in the same situation. That is, the coach randomizes play calling enough to be unpredictable while still favoring the more advantageous plays. The very awesome Brian at Advanced NFL Stats does a great job of describing randomization and game theory.
Click through the image (or this link) to see the analysis that yielded the 1.5% number.
In the last ten years, there have only been twenty-five instances where teams went for it on fourth-and-goal at the 2-yard line. It turns out, though, that going for the end zone on third-and-short is pretty similarly successful to going for it on fourth-and-short, so we used both types of plays in this analysis. For example, the 44% figure was calculated using both third- and fourth-and-2. Here, as with the rest of this fourth-and-2 analysis, we were inspired by a 2002 paper by David Romer. Romer is a notable Berkeley economist, and his wife chaired the White House Council of Economic Advisors in 2009 and 2010.
This was barely statistically significant in the 2012 dataset, so we ran the same analysis in the 2003–2012 dataset. There we found a very clearly statistically significant relationship. The magnitude of the difference between the two running directions was only 6.5% in the ten-year dataset, as opposed to 11% (with wide confidence intervals) in the 2012 dataset.
If objectives other than just picking up a first down are considered, there is actually some evidence that running is better. Brian at Advanced NFL Stats does a great job of diving into that question, though you might want to learn about the concept of expected points before reading Brian’s analysis.
Brian at Advanced NFL Stats looked at this same question with data from 2000 to 2007. He found that running was more likely to have resulted in a first down on third-and-2. He brought more data to bear, but our confidence intervals were pretty narrow. This suggests that in 2012 running and passing were more similar in their success on third-and-2 than during the years Brian looked at.