As I start writing this, it’s been about two hours since I found out about the death of legendary behavioral psychologist and Nobel laureate Daniel Kahneman. And the fact that I’m already writing an article about it should tell you how much of a loss I felt it to be. 

A psychology major in college, I was well aware of the impact of his research (much of which was done alongside Amos Tversky). He wrote a bestseller in Thinking, Fast and Slow, and his work was chronicled by Michael Lewis in The Undoing Project, and there’s plenty else to look up if you fancy yourself more than a pop psychology enthusiast.

I don’t have a robust thesis here, I kind of just want to talk about some of his areas of research and how immediately relevant they are in the context of sports research.

Substituting simpler questions for more complex ones

I find that psychology research is often an exercise in formalizing (through rigorous study) our understanding of things we intuitively know about ourselves. 

One idea that Kahneman presented to help us answer difficult questions was to try to reframe them into simpler ones that can be answered much more immediately, then map the answer back to the original question. 

For example, you could reframe a question like,
“How much would you contribute to save an endangered species?”

to

“How much emotion do you feel when you think of dying dolphins?”

The idea is that it’s easier to evaluate questions that address things that are right in front of your face, immediate both in physical space and time. You can address the relationship between the two questions later, but first you want to get an answer to the question you have a better chance of evaluating in any fashion.

This applies to an incredibly broad range of research, because we’re constantly looking for ways to evaluate complex questions without the ability to assess them directly. 

How good will this quarterback be when he transitions to the pros? Is this promising rookie going to keep it up? How does the increase in the use of Cover 2 defenses impact the success of NFL offenses?

We can’t go at these questions head on, but we should start by thinking about what answerable questions we can define that would map to those questions, and then make the translation afterward.

The Law of Small Numbers

We know that large samples are more useful than small samples, within reason.

If I’m building an expected points model for football, I don’t want to use one season of data. There are not nearly enough plays in a year to cover all of the situations I’d want to evaluate. At the same time, I wouldn’t want to use 10 years, because the game has changed enough that teams would approach the same situation differently now than they did 10 years ago.

The trouble that Kahneman and Tversky found was that most people don’t apply sufficient suspicion to small samples, assuming that they operate roughly like large ones in many ways. In particular, we don’t account for the increased likelihood that a small sample is biased in some way.

We’re inundated with situational breakdowns like platoon splits and performance against certain defensive coverages, sure. But we don’t acknowledge often enough that even full seasons of performance are subject to a lot of random variation. We need to do a better job of starting from the point of “how much sample is enough to trust the result?” and using that as a guidepost.

Heuristics, for better or worse

I imagine most everyone has encountered some of these heuristics explored by Kahneman and Tversky. 

• Anchoring: our tendency to stick close to what we’ve already observed or thought.
• Representativeness: our tendency to respond more to examples of a group that match what we think of as the prototype for that group
• Availability: our tendency to judge how common something is by how easily we can think of examples of it

We’re in NFL Draft month, so this section feels extremely topical. Scouting as a general practice is rife with examples of these heuristics at work. They are useful tools to save us time—we probably don’t need to worry much about scouting a 170-pound defensive tackle—but they can get us in trouble.

We knew before the year that Caleb Williams and Marvin Harrison Jr. were likely to be top five picks, so maybe we were a little rosier in our evaluation of them during the 2023 season. (Not saying that’s what happened, but what very likely could have.)

We don’t see a lot of small interior defensive linemen making an impact at the NFL level, so Aaron Donald falls through the cracks (relatively speaking).

We give a lot of credit to offensive linemen for pancake blocks, despite them being far from common.

Perhaps the best way to integrate analytics and scouting is to use data to put guardrails around our perception of a player on film, to correct for the downsides of these heuristics.

Prospect Theory

Kahneman espoused an integrated theory that merges ideas like diminishing returns and loss aversion into a single framework. Again, everyone knows these ideas to be true, but Prospect Theory adds graphs!

Internally at Sports Info Solutions, we’ve spent a decent amount of time pondering how we might change the way people talk about football win probability using this framework. 

One of the biggest gripes about the nerds with the models is that they don’t properly account for the downside of missing on a bold fourth down. And while from a purely in-game situational perspective that’s not a fair critique, it’s worth entertaining the possibility that we could incorporate ideas like loss aversion and diminishing returns into our measures of the value of making certain choices.

For example, say you’re already 90% likely to win. If an overall superior choice presents the possibility of dropping to 75% against a possible improvement to 95%, we might want to encourage a nonlinear win probability measure that considers the risk of bringing the opponent back in the game to be less desirable. 

It’s absolutely worth noting here that I haven’t put a ton of research into any of the words that preceded these, but I feel his work was important enough to drop other things on my plate to talk about. He’s an extremely important person in the sports research world, even if his work wasn’t directly impactful on the game.