The Jaguars seemed to pull defeat from the jaws of victory against the Bengals on Sunday, enough so that head coach Liam Coen had to answer for a particular fourth down play late in the game. 

With just under 4 minutes left and a 3-point lead, they chose to go for a 4th-and-5 from the Bengals’ 7 yard line instead of lining up for a chip-shot field goal. They failed to convert, and the Bengals proceeded to march 92 yards down the field and score the winning touchdown with barely any time left on the clock.

Asked about it afterward, Coen said, “In all analytics, in all data, it’s 100% a go.” And in response to that, I watched a TV personality question that statement, saying you have to feel out the situation.

But honestly, we might not even need to bring analytics into play. 

Normally, there’s a lot of Monday-morning-quarterbacking after having seen the result, and people browbeat the decision-maker in part because of the bad result. But in this spot, the eventual result presents the exact reason why going for it is a good decision! If we knew the Bengals were going to be able to go down the field and score a touchdown with no time left, obviously the Jaguars should be pushing for a touchdown in that spot. 

But let’s take a look at “the analytics” and see how we might have evaluated that question.

SIS has a win probability model that is trained on the last few years of basic game state information, and it can also incorporate a measure of recent team strength. For fourth down decision-making purposes, we consider the three possible choices (punt, kick, and go for it) and pit them against each other based on historical performance.

I’ll start with the top-level result: we also would have this as a “go”. I’ll explain in a bit why there could be some slight caveats there.

Field goal: 77% win probability

Based on recent league history, we estimate that a field goal from this spot has a 98% chance of bringing your lead to 6. The result is likely to be the Bengals starting their upcoming drive down by 6 at around their own 30, which puts the Jaguars at about a 77% chance to win according to our model, all else equal.

The Jaguars might think their kicker is better or worse than that average, but there’s really only downside in that wiggle room, given how short the kick is. So that’s not something that makes the decision any more difficult to make.

Would that number be meaningfully different if we incorporated the strength of the opponent? Not really, because by this point in the game there isn’t that much time for the strength or weakness of a team to exert itself. Anything can happen in a one-drive sample.

Go for it: 84% win probability

With the go-for-it option, there are some more considerations. 

If the Jaguars moved the chains, recent data suggests a better-than-even chance of making it into the end zone on that play, but even if they didn’t, we’re looking at a 94% chance of winning. You’re still very likely to score a touchdown, you could kick a field goal and burn a bunch of clock or timeouts, or you could burn clock and bury the Bengals in the shadow of their own goal post.

If they don’t make it, they’ve still set the Bengals up for a long drive to make. They could force overtime with a field goal (producing a ~50/50 situation), or drive the full distance for a winning score in regulation. In that situation, the Jaguars have a 74% chance of winning.

We estimate the success rate for a 4th-and-5 near the goal line to be 45%. Right around a coin flip, so right around halfway between the success and failure situations.

The result is that we estimate a pretty meaningful advantage to going for it over kicking the field goal. Of course, 7 percent is a much less impactful difference late in the fourth quarter than it is in the first, so we wouldn’t have this as a 100% no-brainer, but it’s a decent chunk of value.

Is the answer that simple?

When we talk about sports analytics, it’s important to acknowledge two things: “analytics” isn’t some single methodology that produces a monolithic conclusion, and analytical research is not the final answer to any question.

Our model is going to produce a different win probability for all of these situations than plenty of other models. We’re all using similar inputs, but we’re employing them differently, making different underlying assumptions, and deploying different modeling approaches. If our model is +/- 5 percent different from another model in most situations (not that much in the scheme of things), then it’s possible that another model thinks that there’s a slight lean in favor of a field goal in this spot.

And what about factors the model doesn’t consider? Were the Jaguars playing below-average football at that time, and therefore could be assumed to have a lower expected success rate on the fourth down try? With our model, that expected success rate would need to be as low as 15% to flip the decision. That’s a tough sell on its own, but maybe a couple specific features of this situation conspire to tip the scales. 

That’s why you want a give and take between the team’s model and the decision makers on the ground. The model is going to consider a whole bunch of factors and operationalize years of relevant history, but it might not be reflecting that the Bengals’ quarterback got hurt earlier in the game, or that the Jaguars’ passing game was generating under 5 yards per attempt in the second half. NFL rules are fairly constrained on what tech can and cannot be employed during the game, so there isn’t much leeway to do in-game model adjustments.

All of these recommendations should have a fudge factor, which is something we employ in our own fourth down reports. We’ll have some very strong recommendations, but given the combination of the model’s wiggle room and the specific context of the game, there are a ton of situations in the squishy middle. It’s in that squishy middle where teams can use some of this context to their advantage; they just have to be conscious of treating that new information with the appropriate number of grains of salt.