400. Unveiling Survivorship Bias: How It Shapes Business Decisions (Refreshed Episode)

Welcome to episode 400 of the Brainy Business. Understanding the Psychology of why people Buy. Today's episode is all about survivorship bias. Ready? Let's get started.

You are listening to the Brainy Business podcast where we dig into the psychology of why people buy and help you incorporate behavioral economics into your business, making it more brain friendly. Now, here's your host, Melina Palmer.

Hello. Hello, everyone. My name is Melina Palmer and I want to welcome you to the Brainy Business Podcast. Today's refreshed episode is about survivorship bias, and if I may, can we take a quick moment to celebrate the milestone that is today? 400 episodes of the Brainy Business. Wow. Wow, wow, wow, wow. Thank you for being here. Whether you've been around since the beginning almost six years ago, or this is your first episode with us, thank you. You're in good company. Did you know this show has been downloaded well over a million times in over 170 countries? Amazing. It has been my absolute honor to create content for you and I plan to do more of it in the weeks and years to come. So if you have thoughts, questions, requests, ideas for great guests or topics or anything else, please do come share it with me on social media. Share, friends, all that good stuff. You'll find me as the Brainy Biz pretty much everywhere and as Melina Palmer on LinkedIn. There are links in the show notes to make it easy to connect, rate, review, share, as well as find my top related past episodes, books and articles. Those notes are found within the app you're listening to and at the brainy business.com 400 all right, let's talk about survivorship bias. It may sound like it's only a life or death thing, and while that is part of how it was discovered, it's more than just about surviving. And like all the biases you hear me talk about on the show, your brain is using this one all the time and it can absolutely impact the decisions you make in your business. Survivorship bias impacts entrepreneurs for sure, but it's also leading people astray in all sorts of roles in every every business and industry. Understanding this concept and being on the lookout for it can help you make better decisions in what to invest in both money and time. It can make your calculations and predictions of your work more accurate and generally increase the likelihood that your endeavors are more successful. Let's start with the story of how this bias was discovered, which will require us to journey back to the days of World War II. In a war, the slightest edge can be the difference between success and failure. When it came to pilots During World War II, the odds were not great and making it through the war was basically a coin flip. Helping more planes and their pilots to survive their trips out could be the difference between winning and losing. So doing what any generals might do, the Allies decided to have the planes analyzed to see where they should reinforce with stronger metals and to make them more likely to return. While it would be nice to be able to reinforce an entire plane, there's a reason that tanks can't fly. Too much of the super strong metal would mean the planes couldn't get off the ground. So it was important to be strategic and pick the right spots to reinforce to achieve that delicate balance. The man leading the analysis was a mathematician named Abraham Wald. And as part of the research process, he had the Air Force look at all the planes and to see where they had damage, bullet holes and the like. And mark the spots on a card for each one. When you stack them up on top of each other and look at all the bullet holes, you can see where the damage is in clusters. It's kind of like a heat map. And a couple of the articles and videos I've linked to in the show notes include the images of the playing card data if you want to check that out. When reviewing those images, you can pretty clearly see that the most damage is on the wings, tail and some of the central body. And there are some notable blank spots with no damage. The generals looking at the results did what all our brains tend to do when we see information like this. They said something like capital analysis, Wald. We now know where all the gunfire is hitting and we can reinforce the spots with the most holes. Thanks to which he said no. Do you see the problem with that conclusion? Think about the title of the episode for a minute to help you. The problem with reinforcing the spots on the planes that have received the most bullets is that it doesn't account for a very large and important part of the data set. As the title of the TED talk I've linked to in the show notes explains, this conclusion is missing. What's missing? In the case of the World War II planes, the analysis can only include the planes that made it back. Planes are hit everywhere, so it's not like the gunfire couldn't reach those points and so they didn't need to be reinforced. There's nothing special about those points on the planes that came back. And in fact, the reasonable assumption is that those are the parts that matter the most. The engines, the cockpit, and when the planes were hit there, the damage was fatal and they didn't return to be part of the study. So in fact, those blank spots are where you want to reinforce the planes to make them stronger in those places so they can take some more fire there and not go down. The allies, of course, took Wald's advice and reinforced the planes in the proper spots, and for this and many other reasons, they won the war. When you hear and see it pointed out in this way, it's incredibly obvious the flaw in the thinking. And that's probably why the concept is so clear to understand and get the problem now is understanding how that same bias applies to business. Most of us aren't in the business of planes taking heavy fire, so this concept may seem inconsequential to us. An interesting anecdote, but this bias shows up in so many other ways of thinking. One common example is when you seek advice on how to be successful. And this brings us to the little cartoon from 100 Behaviors that inspired the timing of this episode. The title of it is what We See and it has a little cartoon alarm clock underneath which it says wakes up at 5am every day and points over to a little dancing person and says become CEO. And then as you swipe to the next image which is titled what We Ignore, it says also wakes up at 5am every day and doesn't become CEO. And there's a group of people around this alarm clock or looking a few decades back when people would say things like Bill Gates and Steve Jobs both dropped out of college and became billionaires with amazing companies, if I drop out of college, I'll be a billionaire too. Well, not really. Right? You're ignoring all the other people who dropped out of college and don't go on to found successful billion dollar companies, which I've linked to a Forbes article in the show Notes on dropout rates and it's so much more than I realized. This article from 2018 says that nearly half of first time full time students who started a four year degree had not achieved it six years later, almost half. And only 28% of students complete a bachelor's degree in the time it's expected to take. Putting it another way, 2 million of the students who start college each year will drop out before graduating. That's shocking, but it puts things into perspective with our Gates Jobs survivorship bias rule. If 2 million people drop out of college each year, and we consider the total number of billion dollar companies and the people who founded them, there are obviously a lot more People who drop out of college and don't found successful billion dollar companies than those who do. Another article I found pointed out that in reality, about 95% of the most successful people in business do have college degrees. But we don't like those stories as much. Our brains want to find that easy cause and effect connection to help us find the path to our own. But if you only look at the successful people and ignore those who failed, you aren't getting the full picture. There are a lot of people who get up at 5am and don't become the CEO of a huge company. Way more people who start YouTube channels and don't become influencers, great singers who try to make it on Broadway and never get a part, people who started restaurants that went under or who lose money at the casino but no one is writing articles and books about them or asking the people who failed to write a book about why they failed. This creates availability bias. So the only stories you see are the successful ones. And our brain's optimism bias makes us believe we're going to beat the odds and be lucky just like them. We don't hear the millions of stories of all the people who lost the lottery. We just see the few who win and it makes it seem like they're more common than they really are. The other problem is asking the survivors, winners and successful, about how they got there. As we know from the focusing illusion, as people look back, they cherry pick things that they think made them successful. But that hindsight isn't necessarily accurate. They don't often account for the luck that made it all possible. The serendipitous moments and little quirks they don't even realize were a factor. As we look back on our own lives, we see things we did that we think got us to where we are. But those on their own are not the answer. Waking up at 5am or running every day or meditating or dropping out of school are all one piece of a very complex web that got you to where you are. I'm reminded of when I was in business school and we had this computer simulation that we used to create fake companies selling cameras and we were on teams that would compete against other groups in the class. You could choose if you wanted to invest in a lot of R and D and marketing and be really high quality, or if you wanted to be of a lower quality and a smaller price point and you would compete against everyone else for a set point of maybe five years. At each point that we made our decisions and turned in the numbers of what our choices were. It represented a quarter. You could say, you know, this quarter we're going to acquire a new building and invest $50,000 in research and make our price $1 more expensive. And then you would just wait and see how everything shakes out as all the decisions come in together in the software and decide how you're going to react quarter by quarter. I also used to volunteer every year with a camp called Washington Business Week where high school students got to live on a college campus for a week and they would run pretend businesses using this same basic software. And here's the thing. Students would use one strategy and win the game and then come back the next year and say, hey, this is what worked for my company last year and we won. And then everyone in their new group would say, great, you're in charge of the inputs. And then they would do exactly the same investments and choices as the year before. That was the winning strategy. And lose. There are lots of factors that determine success. The decisions by your own business, of course, but also the environment. Maybe you came in at a low price point the first time around and nobody else was, so you were able to get ahead and win. But this year there are five other companies that are doing the same thing, or someone really invested in research that didn't the time before. If you do exactly the same thing year after year, you're never going to have the exact same end result because all these other factors matter and can change what happens. Survivorship bias was making them only look at what they did and assume that it's the winning formula no matter what. But it just isn't the whole picture. As I've said, there are several articles on survivorship bias in the show Notes for you, and you also have a free worksheet there you can use to take notes on the concept. So let's talk about some more business applications where survivorship bias can haunt us. So you can start looking at ways where you might be a victim to this bias so you can turn it around and have some benefits in life and business. I've mentioned already looking for that quick win tip that you can use to be successful, like Mark Cuban or Oprah or Steve Jobs, believing that one thing that all successful CEOs do is making some big assumptions about correlation and causation. In case you aren't familiar with those concepts, here's a little crash course for you. There are a lot of data points that are correlated, meaning they can be seen together. They follow the same trend. Our brains like to think that means something that we can learn from it and apply one of those items to our own situation and replicate the results. But just because two things are seen at the same time going in the same direction doesn't mean that one caused the other to occur. And this is the difference between correlation and causation. To make a conclusion for something you can apply and attempt to replicate or make predictions from causation is needed. If you do a Google search for ridiculous correlation examples, you can see some for yourself. And to quote some funny ones from Fast Company, who knew that the number of people who drowned by falling into a swimming pool correlates with the number of films Nicolas Cage appeared in? And who could anticipate that the per capita consumption of cheese correlates with the number of people who died by becoming tangled in their bedsheets? And as buzzfeed showed us, global warming was actually caused by a shortage of pirates. These statements are of course ridiculous, and they show that just because two data sets appear to go together are correlated doesn't mean that one caused the other to happen. And it's the same thing with survivorship bias. Just because we see something in the data or that we notice the holes in the planes doesn't mean one caused the other to happen. It's the same with survivorship bias. Just because there are successful CEOs that get up at 5am doesn't mean that getting up at 5am will cause you to become a successful CEO. The correlation exists for at least some of them. And again, there are lots of people that are getting up at 5am that are not successful CEOs. And there are lots of successful CEOs that don't get up at 5am so there are at least two parts of the data pool that you're missing in your conclusions. And even if you do see a correlation, it doesn't mean that those two things are tied together and that one will always cause the other to happen. Now I'm going to share some more examples of places where this shows up in life and business, and I want to start with a shocking stat that I found that really drives this home. Maybe it's just for me right now as I'm working on my own book, but this case is about successful authors. We see all the New York Times and Wall Street Journal bestsellers and people that sell millions of book copies and assume that most people have some increased level of success when they have their books published, and that if we write books, we should achieve that too. But as this video explained, for every popular book there are 100 authors who wrote and published a book that doesn't sell. The world is full of books that didn't make back their advances. And for every one of those 100 unsuccessful authors, there are a hundred more who finished a manuscript and couldn't get a publisher. And for every one of those people, there are another 100 unfinished manuscripts in a drawer that will will never be finished. That means that for every single popular book out there, 1 million unsuccessful books and their authors are the other side of this Survivorship Bias phenomenon crazy. And I want you to know that it's not my intention to bum anybody out with this episode. I feel like I say this a lot, and I talked about this on my episode a couple weeks ago with my tips on how to start and grow a successful podcast. There isn't a magic pill or a silver bullet to win. It takes trial and error and a lot of hard work. And while looking at other successful people for inspiration is great and you should continue to do that, don't forget about everyone else and don't miss what's missing. Who has failed that you can ask about their experience and learn from that. Take a step back and determine what works for you. Your business, your audience, your goals. Right now. Just because someone else got a million followers on Twitter or Instagram by doing X, Y or Z doesn't mean it's an automatic win. There are lots of factors and you can't forget about all the other people out there who did those same things and only have 100 followers. Being ready to invest and do the hard work, which includes looking at your goals and problems from all angles. Taking the tendency of survivorship bias into account will put you miles ahead of your competitors and make it more likely that you will succeed as well. Some other areas where survivorship bias can cause us problems is when we say things like I will win because I have a better product or service than they do. The smartest, best, most innovative products and services are often not the market leaders. There are lots of solid products that didn't make it as the lead for a whole host of reasons. TiVo, Microsoft, Zune, the Slanket, which came out four years before the Snuggie and everyone agrees is a better product. They all lost out to a competitor and in some cases those competitors did not have a better product. Being the best doesn't mean you will automatically win. Being first to market doesn't mean you'll win. The lesson here is to look at all the possible data points and not focus on just one aspect. What is missing that you would otherwise miss? What have you not yet thought about or taken into account that could make a big difference for your business venture? Another place where survivorship bias comes into play is on customer satisfaction surveys and other questions you're asking your current customer base. To say, we asked all our customers and no one has any complaints. They love us. There's nothing we have to work on Is, you know, good to know that your customers like you, but it doesn't necessarily mean that there isn't any other work to be done. You forgot about this giant base of important information. What about all the people who did not choose you? What were their concerns or complaints? What could you have done differently to win their business? Why did they go in another direction? And is it that there are, you know, 10% of the population that don't like what they're seeing when they go to your website, or is it 80%? If you're not looking at this large group, you really don't know what is missing and what you don't know. And similarly, you can't only focus on the customers who complain to reduce churn. They might be too far gone. And you're forgetting about all the borderline unhappy people in the middle. Using Yelp reviews as your only customer feedback tool is a bad idea as well, because you only get the most extreme examples. What about all the people who thought the experience was fine, but nothing exciting? Those might be the bulk of your customers. And how many people with a nothing special experience take the time to post a review? Probably not many. And that means if they are the majority, you're completely ignoring their experiences and only looking at the feedback from those giving 5 stars or 1 star reviews across the board. If you aren't considering the full data set of people, your results, and consequently the actions you take based on those results, will be biased. How can you structure your quest for feedback so it's a more complete data set? And why do you want and need the answers that you're seeking? If you think about what you're trying to accomplish, you can kind of work backwards and find those who would otherwise get lost in the survivorship trap. A good rule of thumb is to stop, take a breath and ask, what about everyone else? Or who have we not thought of? Who else is there? You don't have to research every possible group in every single scenario, but if you don't think about them, you could be oblivious to a huge gap in your work. You're missing what's missing, and it's more than just People, there are problems in the products and services that you're missing. Remember those planes in the bullet holes? It has to do with all these other areas where what you're seeing, what you're looking for, isn't necessarily the only problem or solution that exists. For example, when researchers do their work, the only stuff that gets published is what was statistically significant and related to the hypothesis they were looking for. Proving something is what you thought it would be doesn't mean it's more right than something else you didn't test or look into. And there are lots of results that would otherwise be interesting but aren't publishable. Those sitting in the figurative top drawer, collecting dust. A finding that didn't fit the goal of the research project may be a victim of survivorship bias if you aren't careful, and so are all those things that you didn't look into when you were doing the research. It's important to know the question you are trying to answer, and you need to start from a hypothesis that you want to prove or disprove. That's just the nature of this field. But how can you be more open to the things you aren't looking for? What if you planned for every study to have a follow up where you dug in on an anomaly in the data? Or where you tested something that you don't think is going to work and that just seems crazy? Just because you haven't noticed or looked for it doesn't mean it isn't there and that it isn't true. As Rory Sutherland says in his book Alchemy, the opposite of a good idea can still be a good idea. And that which you haven't found yet doesn't mean it doesn't exist and that it isn't telling a story that's very important for what it is that you're trying to accomplish. There's always more to the story. And bringing it back to our planes in World War II, even though it seems really obvious now, know that it wasn't at the time. It was novel and shocking and not everyone believed it. I'm sure. Taking the time to slow down and consider what might be missing, what the data could look like from another angle. Reframing your question to see how the results and insights change, it can all make a huge difference. Look at the survivors, winners and success stories, but don't forget about everyone else. Their lessons can keep you grounded and help you to become your own winning success story. So what got your brain buzzing as you learned about survivorship bias today? For me, in addition to being in awe of this being the 400th episode of the show, I still can't believe it. I love the idea of survivorship bias and how much people have told me this episode and concept have resonated with them over the years. It's so easy to miss what's missing. We aren't wired to look for that most of the time, and we can have huge gaps in our planning and knowledge and reflection when we don't account for it. Sometimes those mistakes could be massive. So it's so helpful to understand about survivorship bias and how it impacts our lives, decisions, and work. In case you're wondering why I chose to refresh this one today, in addition to just reflecting and thinking back and as I've been Preparing for episode 400, it's because of my guests. Yes, plural, on this upcoming Friday's episode. If you've been around for a while, you know that I like to celebrate each new century of episodes with a guest that's really significant or meaningful to me in some way. Like Dan Ariely Joining me for 101, the author of my favorite book, Warren Berger joining me for episode 200 and Jonah Berger, no relation, on to talk about magic words in episode 301. So for episode 401, I was delighted to discover that Dan Simons and Chris Shabri, the authors of the Invisible Gorilla, and that fascinating research, had a book out recently called called Nobody's Fool. This felt like the perfect fit because the Invisible Gorilla and the idea of selective attention and inattentional blindness are so foundational to me for behavioral economics as well. If you haven't yet heard of this study or seen the video, check the link in the show notes. And of course, tune in on Friday to hear my conversation with Chris and Dan kicking off the next hundred episodes of the Brainy Business. In the show notes, you're also going to find links to top related past episodes, books, a timestamped summary of the episode, and more. It's all waiting for you in the app you're listening to and at the brainy business.com 400 and just like that, episode 400 on survivorship bias is done. Join me Friday for a brand new episode with Dan Simon and Chris Chabri discussing their books the Invisible Gorilla and Nobody's Fool. It's going to be a lot of fun. You don't want to miss it. Until then, thanks again for listening and learning with me, and remember to be thoughtful.

Thank you for listening to the Brainy Business podcast. Melina offers virtual strategy sessions, workshops and other services to help businesses be more brain friendly. For more free resources, visit thebrainybusiness.com.