How Is PFF Changing the Game of Football?

Behavior change is hard. Actively changing another person's behavior is even harder. But there are plenty of companies out there that have been able to crack that code. They understand that people don't always or even rarely act rationally. I'm Kristen Berman, co founder of Irrational Labs. Over the years, we've helped countless companies understand and leverage behavioral science to increase the health, wealth, and happiness of their users. The idea behind the Science of Change was to create a podcast that demystifies the psychology of decision making. So here's how it works. Each week, I get to chat with visionary product and organizational leaders to uncover their insights about user behavior and how they've managed to change it. I mash this up with some of the latest research in the field to get to the bottom of our core psychologies. So let's dive in and unlock the secrets behind the science of change. The Science of Podcast is presented by Setsale, the world's number one revenue execution platform. This episode comes with a disclaimer. We're going to be talking about football. Like, a lot. In fact, this whole episode is about football. So why the disclaimer? Well, it also happens that I know nothing about football, and I suspect that a lot of our listeners don't either. But here's the thing. There is a statistical revolution going on in football, in all sports, really, that is all about behavior change. It's classic behavioral science playing out on the field every week. So why are we talking about football? My guest today is Eric Eager of the data collection and analysis firm Pro Football Focus, or pff. Basically, PFF is a resource for teams and coaches who are looking to get a leg up on the competition by digging into the numbers and developing a strategy around that data. Eric has had to approach his users the same way all of my guests have. How can I get people to change their behavior, to challenge conventional wisdom? And how can I leverage data to do it? So it doesn't matter that I don't know the difference between a first down and a field goal. In a lot of ways, we speak the same language. So maybe we start with the basics. What is pff? What does it do? Who are you?

Yeah, so we're a data collection and analysis firm, and we spend the majority of our resources watching over National Football League NCAA football games, collecting data about 250 records per play for every single player on every single game. And what we then do is use that data to analyze how well players are playing, how well teams are using their players, and information for how teams can sort of acquire or make decisions about the players they currently have. And then we also use it to help broadcasters and folks in the media better understand the games that they're covering for the fans and make that experience enhanced.

Nice. You've basically brought more data to the game of football in a probably quite revolutionary way. Who exactly purchases the products?

Yeah, so right now we have all 32 National Football League teams that purchase our product. There's about 130what are called FBS college football teams. So the very highest division of college football, we have over 100 of those teams as clients. We also have about 15 clients, whether it's like the Canadian Football League or lower levels of college football that also buy our data. We have that as well. So an expansive client list that has taken the better part of a decade to build.

So basically, most teams are using your data to make decisions on and off the field. Is that a fair assessment?

Yeah, I think so. And it certainly varies. Some teams are just using the raw data and they have, you know, I think, extensive research and development capabilities themselves. And then some teams are using some of the research and development tools we've built ourselves. So it's certainly a varied group. And then we also have a consumer product that's last month went over a million dollars in sales where, like, we're talking about fantasy football players. Fantasy football is a game where you pick your own team and you play against people in your family or friends group. And whoever scores the most points wins games, and then whoever wins the title, you win a pool of money or something like that. So we have products that help people play those derivative games. We have products that help people bet on games. We have products that just help people consume the game more fun things like podcasts and videos and all that. So in addition to being sort of a B2B group, we're also a B2C group and helping people consume a game that is increasingly popular.

Maybe if we step back and say, how has data changed how people play? And I'll give you the analogy with basketball is just with basketball stats, the game has changed. When data comes on the scene, what do you see happening in football? Is it comparable to basketball or not so much?

Yeah, I think it's moved a little bit more slowly. I mean, you look at baseball, you look at basketball the way that we folks who grew up watching the games, it's almost unrecognizable. Now baseball, you have the proliferation of what are the three true outcomes, which are strikeouts, walks, and home runs. You don't have as many balls in Play as you used to in basketball, as you said, there are far fewer two point plays than there were because people have realized that at the percentages that folks shoot three pointers, you multiply three by that percentage and it's higher. The expected value is higher than if you multiply the percentage of makes on two pointers times two. Right. So it's an easy math problem. Football is a little bit more complicated. It's also a, I would say, in a lot of ways, stuff. Stuffier game as far as, like, it's very insular, outsiders have a harder time getting in. And so it's been resistant, I think, to football analytics for a long time. That being said, where analytics is getting a foothold is that analytics, unlike in baseball and basketball, in some people's mind, I think everybody believes that analytics makes football more fun to watch. So, you know, analytics would say pass the ball more. We're seeing historic passing rates. Analytics would say don't kick as many field goals and don't punt as much. Go for more fourth downs. And we're seeing that. And in fact, fourth and one, fourth and two. So some of those borderline plays that we've seen, teams historically do the boring thing and kick the ball, the other team, we've seen them keep the ball and try for what many people believe is a risky play, but what the math says is actually the right play. We've seen that a lot more in the last few years, and especially this 2021 season. So you've seen it happen. You've seen the conversations change a little bit. You've seen fans of teams be less tolerant of conservative game plans or game plans that don't necessarily give their favorite team the best chance to win. You've seen the conversation change. I think that's a lot of fun because especially in football, like I said, it makes the game more entertaining. And I think with sports, we have so many options now with sports, we have so many options with entertainment. These leagues need to embrace things that make their product more entertaining to people.

What he's talking about is something called omission bias. Essentially, omission bias is when we judge action that causes damage as worse than inaction that causes the same damage. One classic example of omission bias comes from referees. They have immense pressure of making the right decision, especially in the final moments of a tense game. The fear of calling out an incorrect foul scares the referee into not calling out fouls at all. This is omission bias. They prefer not to do something than to do something. NBA statistics have shown that referees call 50% less fouls during the tense moments of the game. Historically in football, coaches have been guilty of omission bias in two main deciding between punting or going for the fourth down and when deciding between kicking the field goal after a touchdown instead of taking the more difficult two point option. The two point option is an action and people judge action that causes damage. Losing the game if you miss it worse than inaction that causes damage. Why? Negative outcomes losing are perceived as worse when people can easily imagine doing something different. If you flubbed the two pointer, you constantly replay what would have happened if you hadn't gone for it and instead stuck with the extra point. Conservative options like punting or kicking the extra point have become conventional wisdom. Rules of thumb over time. But the kind of data that PFF provides to coaches has kind of turned those rules upside down. How do you actually get them to change their mind and go away from just their expert intuition and follow the trail of data? How has that actually happened?

Well, I think you just have to change the baseline of what everybody believes is true. Because I think the issue is, and we see this in all walks of life. When I was a professor, you saw this in education. There's a lot of incentive to fail with the pack than to take a risk away from the pack. And so, like, for example, in education, folks would teach the Prussian method of lecturing, even though we all knew it was bad. We all did it together and education failed, but we all went with the crowd and no one stuck out as the reason everything was bad. You know, and like, there were brave people that would build these flipped classrooms or things like that. And a lot of students didn't like it, right? And so they would get bad teaching evaluations and some people would lose tenure over it. And even though they were doing the right thing. And I think in football it's the same thing. If you do the same thing, if you play by the book, which is not really the book, it's just what everybody is historically done. And you fail, you're failing the same game that everybody else failed. In a way, you're sort of in a more honorable space than somebody who does something differently and takes that chance. Now, we, we've seen teams. So for example, in 2017, the Philadelphia Eagles went for more fourth downs than basically everybody else did. And luckily for them, they made a lot of them and they won a Super Bowl. And so they got the right end of variance and so enlightened a lot of us as to what the possibilities are when you play against the grain. But there are other teams. The 2019 Baltimore Ravens, just two years later, probably lost a playoff game that they were favored to win because they went for fourth downs and they didn't make two of them and they ended up losing. And they probably would have went with the grain. Who knows what would have happened? They might have lost more slowly than they did going out there and being more risky. By everybody else's view, there's inherent risks in going away from those things. So what I think we have to do to make people act more rationally is just make those actions more a part of mainstream conversation. So that when you do that, everybody's habit of mind is not to say, well, what the heck is going on? And then by the time everybody's questioning what's happening, the mathematicians can't write fast enough on the board, we've moved on. And so that, you know, the truth can't get on the phone before the lie is sort of gotten around the globe. Right. And so I think that's part of it. You just have to like, build trust, build people's intuition, and eventually that Overton window sort of shifts to a point where we're starting to do a lot of things rationally. It just, it won't happen overnight because of the risks and because of the incentives. If you're one of 32 head coaches in the NFL, this is your dream job. There's an incentive to stay in that position longer and maybe not necessarily do as well than there is to stick your neck out and get fired right away.

This is called risk aversion, where we don't necessarily want to take a risk if the cost is high. You mentioned the Eagles. What are other franchises that are maybe leading the pack in this thinking or exemplifying what you would say is good data practices and following your lead here?

Well, I think. I think the Baltimore Ravens have been a team that's led the way for a number of years in a row. Every time they do something, whether it's trade a player or their decisions in game, they very much have a habit of mind that I think follows a rational sort of approach to things. I think they have a price to whatever everything they're doing. They don't take bad risks. They don't put themselves in position to have to be desperate. And I think that that's what doing things analytically puts you in a position for. So they're one. I think the Cleveland Browns are an up and comer there. I think the Los Angeles Chargers in game are making a lot. They're nine for nine, I believe on fourth down so far this year. So not only are they going for a lot of them, they're getting lucky and, and so they're getting a lot of the attention. But those are some teams that I've, I've really admired over the past few years in the case of the Ravens and Browns, and then recently in the case of the Chargers.

Remember, omission bias. The interesting thing is that having access to more data might reverse this. Omission bias is when we favor not taking an action because we can easily see the downsides of what happens if we take that action. We don't appreciate the downsides of what happens if we take no action, but maybe it reverses. Let me explain. In soccer penalty kicks, goalkeepers choose their action before they can clearly observe the kick direction. An analysis of 286 penalty kicks in top leagues and championships worldwide shows that given the probability distribution of kick direction, the optimal strategy for goalkeepers is to stay in the goal's center. Did you catch that? The best option according to the stats, is to take no action. And yet goalies rarely stay in the center. Why, they would be ridiculed, laughed at. The norm of jumping left or right is just too high. It would seem lazy for the goalkeeper to keep his feet planted. Action and risky action is the norm. It's possible that now with all this data and norm changes, that in the future when coaches don't go for it on fourth down or don't take the two point option, they will face the same potential ridicule that goalkeepers face if they choose to stay in the center. But back to American football at some level comes down to some trust in the data that's happening. That coaches are really moving forward and saying this is a strategy because we have such and such data to say that we should go forward with it. Is that fair that people have to start a cult around the data versus trusting their intuition? Is that the leverage point here?

I think it is. And it's also just what you can squeeze out of the data. So for example, there are a number of pieces of data that we collect that are descriptive. This happened, but it's not necessarily going to predict what's going to happen in the future. So there are situations. So for example, how well a quarterback does when there are other players sort of in his area, we call that pressure. Those statistics, they'll tell you what happened in a game. But. But as far as predictive, they're a lot more noisy than anything else. So you can use that data, but you're maybe not necessarily going to gain an edge over a team that is searching for other parts of the data. So having solid research and development, having solid statistical practices gets you out of wasting time using data that won't help you versus data that will help you. So for example, with quarterbacks, how well they do when no one's around them is way more stable than how well they do in somebody's bearing down on them, which I think makes sense. But if you sort of weigh them equally, maybe you're not gaining as big of an edge as a team that weighs them properly. So that's one of them. I think that that's a big deal. And that again is part of like we do help with some of the R and D for teams, but the teams that do their own R and D, they're focusing on the the value system that their team has. And the value system that their team has is aligned with good. What I would consider football key performance indicators. They're probably going to do great if they're optimizing for something else. For example, trying to win seven out of the 17 games and keep their job for another year, they might not necessarily be having this success that a team that's optimizing for winning the super bowl would have. So you can give information to folks and they can use it however they want. There are different what I consider like what are called local optima, like local pieces in the data where you're like, I'm optimizing for this and that's great. And then there's global. There's a global optimum where it's just like I'm optimizing for everything and teams are on different parts of their journey there. And some teams take the data and use different things and aren't optimizing for anything. So it's really an interesting landscape.

Behavior under pressure is a different kind of animal. We tend to focus on distracting stimuli when we have a stressful task we need to complete. In the case of football, maybe the distraction is the 250 pound linebacker rushing at you while the task at hand is completing a pass to pick up a first down. So when the stakes go up, so does the stress. One study found that when skilled golfers competed for a cash prize in front of an audience, they were 20% less accurate than they were without the reward. And the audience. As Eric says, those kinds of distractions tend to vary from situation to situation. So in football, maybe the stress is the person running at you. Or maybe another time the stress is that there's only 30 seconds left on the clock. Or maybe it's that the quarterback has an injured shoulder. These variables are hard to plan and practice for. If you can't practice for it, it's hard to predict it. So what about the people who seem to thrive under pressure? There always seems to be people who perform their best when the stakes are highest. A study by Harvard Business School researcher and assistant professor Alison Wood Brooks found that those who are able to see a nerve wracking situation as an opportunity and not a threat perform better. Specifically, she says, those who reappraise or recategorize their anxiety as excitement end up showing more enthusiasm and perform better in subsequent tasks. Telling yourself things such as I am excited or get excited positively correlates with better results in a stressful situation. Maybe we'll go back to that insight that you mentioned, which is if a quarterback is by himself and he has a play to make, that his behavior in that moment may be more predictive of future behavior versus his behavior under stress. Can you unpack that a little bit more? You'd imagine that you should look at a situation if they're under stress, and that should be used to predict future situations where people are under stress and not the opposite. So why do you think that is?

Yeah, it's a good question. I think you can boil it down to how life is, right? It's like some people are their own worst enemies and they're the reason for the stress that comes onto their life. And then there are a lot of people for whom stress is not their fault. And so for a quarterback, for example, in the football, sometimes quarterbacks just struggle seeing the pressure, but it's their assignment to do so. And so that could be the reason why they struggle under pressure. There could be quarterbacks for whom the offensive line is the reason that they're under pressure and the pressure comes. So for example, the 11 men that play defense, sometimes the opposing team will rush six at the guy. And so if you beat that, there's only five players for your players to beat. Whereas if you can get pressure on a quarterback rushing just three men, then there's eight to beat. It's almost as well as like pressure comes in different ways. And the sort of like baseline expectation for you is so muddy in those circumstances. Whereas when you're clean, generally speaking, you're beating the same defenses as a lot of other players are. So the other variables are more fixed. And so you're really unlocking Sort of true talent level in the latter versus the former, where a lot of times you're. What you're measuring is a person's circumstances. That's important in any walk of life. If you're looking, and I, like I said I was in education. If you're looking at a student's GPA and you're also not looking at the socioeconomic situation that they're growing up in or the family dynamics that they have, or even just the day that they took a test, for example, like, you're measuring their circumstances and not their talent. And I think in the case of pressured plays for quarterbacks, you're really measuring their circumstances, whereas with clean pocket stuff, you're measuring their talent. And talent is obviously what we want, because when you assign a player from a different team or you draft a player from college to the NFL, you. You're drafting their talent and you can craft their circumstances. And that's one of the basic principles that we put into a lot of the models that we give teams is, hey, take this player, create circumstances where he can succeed and see how good he can be, and measure him there. Don't measure him by what his college coach did, good or bad.

Are there other examples of mistakes that coaches are making as they try to win the game?

I think the biggest one is just communication. When I talk to former execs and former coaches that I think the biggest issue when they have gripes with why things went wrong is that expectations weren't that clear. Big question that I have is, well, did you tell this person that this is what you expected out of them, or was there a language in which you could hold somebody accountable? And I think data is very much that thing. If you want your office of coordinator to run more trick plays, let's say, well, you have to be able to document that, and you have to be able to hold that person accountable. And data is a great way to do that. But the mistake is not even thinking that it's possible in the first place. The hard part is what I call fake trust. It's sort of like we've all, you know, a lot of times, you know, football is sort of the good boy club, and we've all been on the same team a bunch of times before, and we all trust each other. And oftentimes that can be the biggest disaster because I call it fake trust. I implicitly trust this person. They're actually not getting the job done, but I have no way of actually measuring that or holding that person accountable. I think that's the big one. And I think ultimately I like where football is going and that's becoming less of a deal because we have more ways of holding people accountable. And that means football is going to be more accessible to more people because your means of getting in and having jobs and having resources in football are going to be more on the merits and less on who you know and what you've done before. And I think that that's good. And I think in any walk of life, that's what analytics should. And I know we failed as a. You know, I talk about math and statistics. You know, Weapons of Math Destruction is sort of a book I think everybody should read. We failed a lot of times at the promise of that, but the promise is still exists. And that data can help equalize a lot of the opportunity space that we give people because it can be the way in which we reveal a lot of the biases we have and then get rid of them.

I often say in this show, we are what we measure. This is true in sports, too. The analytics revolution in sports actually began about 20 years ago in professional baseball. The author Michael Lewis, wrote a book called Moneyball, which told the story of Billy Bean, the general manager of the Oakland Athletics. Go A's. Bean shocked the establishment by doing away with the status quo when it came to evaluating players. For decades, coaches and scouts depended on the same kinds of rules of thumbs we talked about earlier. The conventional wisdom pointed to batting average, stolen bases, and runs batted in. They needed to be strong or fast or both. They needed to be able to perform well under pressure by getting hits when their teammates were in a position to score. But Beans started noticing that none of those factors could predict successful outcomes. The only things that mattered in baseball were how often a player got on base and how hard they hit the ball when they did make contact. This was very controversial, and for a while he was the laughingstock of the league and struggled to see eye to eye with his own players and coaches. Today, he's credited as a visionary who completely changed the game of baseball. Next, basketball experts started applying statistical analysis to their game, and it completely changed their sport. Football was late to the party, but Eric says it's taken off even faster than it did with other sports. But he says there's still more room to change. What is one stat you think coaches should be holding players accountable to? More? So in baseball, there's the Billy Beane story. What is it for football that you think people are not measuring as much as they should be?

It's so hard because every position is different. Like in baseball, most players jobs is either you pitch well or you hit well. And like, so it's kind of the same. Like you want catchers to get on base a lot and you want first basemen to get on base a lot. For football it's a little different because obviously everybody has different traits and their jobs are different. I would say, for example, I talk about putting pressure on the quarterback. In football there are two things. Defensive linemen are often judged by what are called sacks, which is basically if you tackle a quarterback behind the line of scrimmage before he can throw the ball, that's a humongous play. But like the league leaders and sacks in a season are generally generating just like 20 a year. And so like they play about a thousand snaps. You can think that's a 2% of plays, then we're doling out. And you can actually look at contracts and say like, oh my goodness, they pay for these, they pay for sacks. If you get a lot of sacks, you're going to make a lot of money. We found that year to year, if you just look at things like R squared, the pressure you generate on a quarterback can predict sacks much better than the sacks you actually get. So as you talk about process versus results, pressure is the process that generates sacks. Sacks are the outcome. And yet teams pay for pressures. Teams pay for sacks more than they do pressure. So the thing is like we should look at, you know, and we should pay and we should reward the process, which is pressures, and we should enjoy the sacks. And when they don't happen, we should wait for them to happen because they're part of the results base. But we should always monitor pressures versus that. So that's one very easy one and one that I think people are starting to understand a little bit more is that's one place where again, results are what front offices pay for. They're sexy, they win games. But the process is really what we need to look at and what we need to reward and where we can see sort of the emerging players, the players that haven't gotten as much attention. We should look at that statistic for possible breakouts and possible places for buy low candidates.

Are there other ways that this data has changed the way that coaches or front offices draft and trade?

I mean, there's a couple. I think the biggest one is that some teams know now that there is extreme noise inherent in picking players and there's extreme noise inherent in picking players at certain positions. So a lot of the smart teams, what they're doing, for example, in the NFL draft, where basically in the NFL draft they'll take the teams and reverse order them by record. So the worst team picks first, the best team picks last, and they pick once every round. And oftentimes teams will trade these picks. So for example, if you're picking later in the round, but you really like a player early, you'll package a bunch of picks together and you'll move up and get the player you want, but it'll cost you three or four players to get one. And generally speaking, statistically we're not good enough yet. And I think that a lot of this variance is irreducible. We're not good enough yet at deciding who the best players are. Like, we can get them within range. Like a first round pick is generally better statistically than a third round pick. But your bet, if you take two second round picks and package them up to get a first rounder, your gamble is actually better on the two second round picks. So the really smart teams, what they're doing is doing the trading back. They're finding the teams that are willing to take a risk and take two picks and make one and they're accumulating these assets. That's one that a lot of the smart teams are doing and a lot of the teams that are a little bit behind are allowing them to do by trading them that capital. That's one example. I think we're, we're really seeing some real progress.

This is something called opportunity cost neglect. It basically means that we tend to ignore what we give up when we make a choice. So Eric is saying that when it comes to drafting players, some teams look for the best player available or try to address a pressing need by drafting the best player at a certain position. Other teams, when they have the opportunity to go and get the best player, instead try to trade that pick to another team willing to give them two lower picks. Which is the better strategy? When opportunity costs are made more salient versus less salient, our decisions change. Imagine for a moment that I just handed you $10. It's all yours. And then I gave you the option to buy one of these. Option A, an insulated metallic portable coffee mug for $10. Option B, a simple ceramic mug at $3.99. Think for a second. Which would you go for? Option A or option B? Now imagine if instead you had received a slightly different version of the same options. Option A remains the same. An insulated metallic portable coffee mug for $10. Option B has a few words added to the end. A simple ceramic mug at $3.99, leaving you with an extra $6.01 cash to spend on something else. How do you think that would impact the results? One study actually found out and the group that did not receive the disclaimer about the extra cash 40% chose the cheaper mug. And the group that did receive the extra context 60% chose the cheaper one. What a difference those few extra words made. The dollar amounts and options are identical across the two groups. But making the opportunity cost salient made a big jump on choosing the second option. Coaches who opt to pay for the best player are not paying attention to the the opportunity cost that comes with giving up those two good players. And it seems Eric might agree. If franchises pay or overpay for a player, this could cost them in other areas of the game. They're making a trade off inherently by overpaying for a player. Can you talk more about what this means and the trade offs that they're making?

Well, I think that's what makes football sort of fun in that they have what's called a salary cap, meaning that like every single year, basically every team can only spend up to a certain amount. And so inherently any money you spend on one player can't be used on another player. Same thing's true about I talked about draft picks where you only get seven per year. Now you can trade around and get as many as you want as long as you know you have trade partners. So the issue, and one of the reasons why analytics is important is that you can't just do what the New York Yankees do and like if you make a mistake, just spend more money on a player and actually and override it. So there are consequences to the decisions that they make monetarily. And what we found is that other than quarterback, which is the most important position in team sports, every other position in football is what is called weak link. Meaning for example, if I look at my coverage players or the players designed to cover the pass, if I look at who's the most influential player in a game, it's generally speaking the team's third or fourth best player there. Meaning if you spend a ton of money on getting the best player in that group or the most money is to get the first or second best, but you slough off on players 3, 4, 5 and 6 or you have injuries and player 3, 4, 5 and 6 becomes 2, 3, 4, 5, then you have an issue as far as performance is concerned. And so you don't want to be too top heavy, you don't want to concentrate risk too much on any position other than quarterback. And that's, I guess, like, we think about every other thing. You, like when you're spending for retirement, you don't want to concentrate any of your bets on any one stock or whatever. So it makes some sense. But a lot of times what we end up seeing is, like, the biggest players make the most money, and sometimes the smartest teams are the ones that allow the best players to go to different teams and make that money and sort of distribute their capital or their risks out more evenly. But that's a hard thing to do again, because, you know, we fall in love with these players. We want to see them be successful. So there's also that human dynamic to it that makes it hard. But you're right. I mean, you've hit on something which is it's really hard when you have perceptions and you overpay for those perceptions, you have to slough off other places, and sometimes those creep up and make it so that you can perform at your peak.

So I confessed early on that I am not in any way an expert on football. I don't watch it. I've never played it. I don't understand it. But there are a lot of things that I've discovered while researching this episode that I found really interesting. And one of them is this idea of teams drawing up trick plays meant to literally deceive the other team and make them think they're doing X while they're really doing Y. And it turns out some of these plays will catch on like wildfire. And all of a sudden, every team is trying to use the same tricks. I wanted Eric to tell me if this phenomenon is driven by data or do these things just peak and crash like all fads do? The most recent one was something called the Wildcat offense, which is when the quarterback. Okay, I'll just let Eric explain it.

Yeah. In 2008, the Miami Dolphins, who did not have a great quarterback, were going up against, you know, New England Patriots. And at the time, actually, the Patriots quarterback was injured, but their backup was still quite a bit better than the Dolphins quarterback. And so what Miami did was they took their quarterback out of the game and called what's called the Wildcat play. And the team that they beat was previously undefeated the year before. And they ran a completely different offense and won by like three or four touchdowns in a game. And ultimately that became the fad. And teams still run the Wildcats. I think the data would suggest it's not effective, but catching a team off guard in A one game sample. It certainly was. That was a cool thing, right? Like this happens all the time in markets too. Like you have this ingenious idea by a content or a subject matter expert. It gets ingested in there and it creates havoc for a while. And the ones that sort of like have the fundamentals to stay around, stay around and they become incorporated in what is true, you know, what happens all the time. And the ones that don't, they eventually get weeded out. But I feel like the interesting thing is like in games and markets and things like that, like you, we give it a chance to happen. And the Wildcat was one of those that had its day, but ultimately was not, you know, it was a fat offense because it just didn't have the fundamentals to work long term because the other side eats their Wheaties too. And they know they can study the play and sort of see how to defeat it.

So what he's saying football teams do is not uncommon. In our real life decision making, we tend to extrapolate insights from one or two data points and make judgments based on just those data points. For instance, think of availability bias when it comes to a natural disaster like a hurricane. We'll buy hurricane insurance right after a hurricane because we think that if it just happened, it will likely happen again. Or there's the halo effect where you meet someone and they tell you about their taste in music and they're almost identical to yours. So you are more likely to assume you have similar taste in movies. You see one thing about someone and assume you agree on everything else. We do these things all the time and apparently so do football teams. I like a play that is determined by how much you can deceive the other team, which also probably is hard to measure, is who's the best deceptive quarterback out there.

Yeah, deception works in football and it's extremely important because of what I described you. When every team is spending the same amount of money, they have to be deceptive in order to achieve their goals because they're all working with the same raw material.

Maybe we think about other ways that outside of pure talent that teams can win. Do you want to talk about the home field advantage and how much this actually helps teams and then the other things that may not be purely related to talent on the field.

Yeah, one of the interesting things about COVID 19 was it created a natural experiment where home field advantage was completely taken. You know, the people historically used to believe that home field advantage, which was worth about 3 points per game and had been waning so it had been decreasing. In 2019, there was basically zero points of home field advantage per game, meaning that like the road team scored about as many points as home teams, road teams won as many games as home teams. And folks attributed that to noise because there were other seasons in the NFL where that happened just because of random variability. The 2006, I believe was one of them. Etc. So folks were still kind of like, hey, maybe this is noise. But there was also the book Scorecasting. We talked about, you know, some of the reasons which have gone away, which are things like travel, you know, it's a lot, you know, they travel a lot better now, the professionalization of the sport. So like they're in shape all year. So some things like travel don't necessarily affect their physiology as much. Teams are being smarter. So for example, when San Francisco would play Cleveland and Cincinnati and back to back weeks, they'd stay in Youngstown in a week in between. And so they would get rid of some of the home field issues. But there was also referee bias. And referee bias is very much a thing where if you look at pass interference penalties called on the home side of the field versus the road side of the field, they were different. And you had significant differences before replay or when you look at baseball, like balls and strikes are just called differently for home teams and road teams. And it's the crowd having this sort of effect that is not trivial. And when you looked at Covid games, where games were predominantly without fans, the home field advantage almost went away completely at the fundamental level, like balls and strikes and things like that. And what's interesting is when we've had fans return in 2021, we've seen a lot of the same 2019 dynamics, which is that home field is basically gone away. And so that's a really interesting thing. And when you're modeling sports for the sake of betting fantasy football, deciding who, who's going to win or not, you know, that's a non stationary effect that has a very non trivial effect on who's going to win a game.

And final question, this is one we ask most guests. When feature films were just invented and the first feature film was done, the camera was actually stationary. So you'd see the film, but you'd only see it from one angle. And in hindsight you kind of look at this and be like, but you could just move the camera. But because people were used to photography where you had one angle for one picture, the idea just didn't hit until much later. So if you think about football and data now, and you fast forward yourself to the future and you look back at now, what are we missing?

It's starting to go away now. But I do think having defined positions for players will be something that in 15 years we will have thought was a very quaint and antiquated thing. And you see it in baseball now, where the guy's the second baseman, but half the time he's playing on the shorts outside of second, and we're all like, oh, that was cute. He always used to stand in the same spot. And I think in football, like we're starting to see it. But like this idea that, like the five offensive linemen have to stand in the same spot every play, or the quarterback has to stand here every play, or it has to be the same player every play, I think is going to be an antiquated thing and it'll be fun when it happens, but we're going to look back and be like, man, we really restricted ourselves by holding ourselves to this standard that isn't even a part of the rules.

That was Eric Eager, VP of Research and development of Pro Football Focus. So the idea of data beating human predictions is not new. Since the 1950s, researchers have documented the many types of predictions in which algorithms outperform humans. Algorithms beat doctors and pathologists in predicting the survival of cancer patients, occurrence of heart attacks, and severity of diseases. Algorithms predict recidivism of parolees better than parole boards, and they predict whether a business will go bankrupt better than loan officers. What is new is that we're finally starting to trust them. In the 50s, we were generally reluctant to believe anything that the algorithms told us. That started to change in the early 2000s, but it helped. If you made the algorithm feel more human, maybe it apologizes to you or has a cute name. In 2015, the term algorithm aversion became popular, and we started seeing papers coming out saying that when given the opportunity, people still chose human forecasters. But times have changed, and quickly. Today, the data is now pointing to the data. Our comfort level has gone up. We trust data more. There's a series of studies conducted by researcher Jennifer Long and others that asked more than 1200 online participants in the US to make a variety of predictions. For example, they asked some people to forecast the occurrence of business and geopolitical events. Others were asked to predict the rank of songs on the Billboard Hot 100. Each participant was asked to make a numerical prediction based on their best guess on a particular topic. After their initial guess, they were given some advice and additional context and were then allowed to revise their prediction if they wanted to. But the advice came in the form of another prediction, which was labeled as either coming from a human or an algorithm. The participants were told that if they revised their predictions, the closer they were to the actual answer, the greater their chances of receiving a monetary prize. The researchers were shocked to find that people relied more on the same advice and when they thought it came from an algorithm rather than from other people. So when doesn't this hold? It may mean the hardest person to get to trust the algorithm is the coach. They are fighting their own point of view and intuition and in order to believe the algorithm, they need to downgrade their judgment, which is a very hard thing to do. And like most of the companies we profiled here on the Science of Change, PFF is trying to make this easier. And whether you understand football or not, it certainly looks like they are succeeding. The Science of Change podcast is presented by Setsale. Act on the moments to lead to a sale. Visit them at Setsale co or on LinkedIn to learn more about the world's number one revenue execution platform. This podcast is produced by me, Kristen Berman and StudioPod Media. Our executive producer is Rachel Roberts and all episodes are written by Jack Buer.