E28: Randomness Is Not Random

What if our brains see patterns that aren't there? Welcome to Invisible Solutions. I'm your host, Steven Shapiro. Each week we tackle your most complex problems using the lenses from my book, Invisible Solutions. If you need the lenses, go to getthelenses.com and with that, let's get started with today's episode. And in this week's episode, I want to start off with a. A little bit of a confession. I have psychic powers. And if you don't believe me, let's try a little experiment. If you've got the opportunity, you can grab a sheet of paper and a pen. Give this a try. Also grab a coin, a penny, nickel, dime, doesn't matter, Euro, as long as it's a coin that you can flip. That's all that matters. So here's what I want you to do. On one sheet of paper, take the sheet of paper, rip it in half. On one half of the paper, I want you to list a series of random x's and o's, just totally random. So it'll be xoxo, xxo, xo, xxo, xoxo. You get the point, 25 of them in a row. Remember, you want to make sure that the list is random. You want to have a random list of X's and O's, 25 of them. Okay? Now take out the other half of the sheet of paper and this time take your coin and. And flip it 25 times. Each time it lands on heads, you're going to write an X. Every time it is A, tails you're going to write an O. So this is going to give you a different string of 25 X's and O's. Now, remember, the first sheet you wrote on is the random list. And the second sheet you wrote on is the coin flip list. What I want you to do is take the two pieces of paper and I want you to mix them up so that I can't tell which list is which. Now, you need to remember, but I can't tell. You're going to take those two pieces of paper and you're going to study them. You're going to mentally, telepathically send me the lists, and I'm going to try to predict which one is the random one and which one is the coin flip one. Okay? So I'm going to turn on my psychic powers. Give me a few seconds here. All right? I'm going to think about this. Your thoughts have a long way to travel. Okay. It's getting clear, though. I see it. Okay, I can now tell you which list is which. Now, hopefully you actually tried this out, because there is a pretty exciting point to this exercise here. One of the points is that I don't actually have psychic abilities. And that's probably not a surprise to you, especially since this is a podcast that's prerecorded and I have no idea predicting what you are going to write on your sheet of paper. But I can tell you which list is which without even looking at them, or at least I can with a high probability of accuracy. So what I want you to do is for each list, I want you to count the length of the longest string of X's and O's. So maybe you had two O's or three X's in a row on the random list, and that would be your longest string. So count the number of X's and O's on the random list. And now I want you to count the X's and O's on the coin flip list. Now, my prediction is that for your random list, you probably only had maybe two, three, if you're lucky, four in a row, four X's in a row, or four O's in a row. But the coin flip list is almost always going to have a longer string. In most cases, I've seen it to have 5, 6, or even 7 x's or o's in a row. Now, here's what's fascinating is the patterns on the coin flip paper are random. The ones you wrote in the first list are actually not random. Randomness looks different than we actually think it is. Randomness is not alternating back and forth. XOXO xo that's not randomness. Randomness has inherent patterns, but these patterns mean nothing. They're just random. Like if you put it on your phone and you put on some music and you put it on random play, and it's supposed to play song completely randomly across all your artists, all your songs, and then it plays three songs in a row by the same artist. You might think it's a software flaw, but it's not. That's actually randomness. Now, sometimes they might build into the software something to prevent that level of randomness, but that's actually very natural. And unfortunately, as human beings, we like to find meaning in everything. We like to find meaning in everything. Even things that are just random events or statistically probable eventualities, we find patterns in them. And this isn't inherently bad, but it sometimes might lead us to make bad conclusions and bad decisions. I remember a long time ago, a friend was sitting on a plane next to a stranger, and she decided to strike up a conversation. And eventually she discovered that this stranger was in fact the uncle of a woman who was married to a guy she knew 20 years ago. And she asked me, like, wow, you know, what are the odds of that? Well, the odds are actually pretty good. Eventually, like, how many strangers have you sat next to that you ended up being strangers by the end of the call? Of course, it's a lot of them. So with this whole six degrees of separation, or maybe three degrees, whatever the number actually is, if you try hard enough, you can always find a connection with pretty much every person you meet. And I talked about this in a previous podcast. I talked about how synchronistic events can be explained with probability. It was in episode 24 where I talked about the birthday game. And if you missed that, definitely take a listen, because I think it's a really powerful example of how we make certain assumptions around randomness and the likelihood of certain events taking place. And here's the bottom line is as human beings, we strive for meaning, and we want to find patterns even when they aren't there. And there are ramifications for innovation when it comes to this. The brain sees what it wants to see. And I think even more importantly, the brain sees what it expects to see. We take shortcuts because if we didn't take shortcuts, it would take us way too long to process information. So, based on past experiences, we make certain shortcuts, certain decisions, based on what we've seen in the past, assuming that they're accurate today. So we might see a pattern, and we assume that it's a pattern and not randomness, when in fact it might be randomness, or in some cases, we might actually see the wrong patterns. As a result, we end up often solving the wrong problems because we make a ton of assumptions. We often assume our assumptions are correct. And I know there's a little bit of a pun in there somewhere, but I think that's really the case, is we tend to assume that our beliefs are accurate and what we are seeing is an accurate reflection of the world, when in fact what we are seeing is filtered by our own beliefs. We might be thinking, hey, you know, this is a customer service problem, so let's solve it the way we've always solved our customer service problems. Or maybe, maybe we'll borrow a best practice from someone else and solve the problem the way they solved it. But we often don't look beneath the surface to see if there's something else going on Maybe the patterns and the shortcuts that we are taking in our brain are leading us to faulty decisions. I remember a client who had a large call center and customers would call in to add a new service, cancel a service, report a problem, pay bills, and much more. And it was a really great world class call center. And then all of a sudden, without warning, wait times started to increase uncontrollably and they had no idea what was going on. And it became a matter of urgency because wait times were going up, call volumes were going up. It was just a. It was a mess. And so with urgency, they decided the problem they needed to solve was how can we improve our call center performance? And they bring all the call center people together and they say, what's going on? And everybody's like, we don't know, we're getting more calls. The pattern they saw was a deterioration in call center response times. And this would be a logical pattern. It was actually a correct pattern, it was an accurate pattern. But they didn't initially look for the underlying pattern because beneath what they were looking at was something more sinister. As it turns out, just around the time that things started to go south for this company, they had made some changes to their billing system and cryptic messages started appearing on customer bills that confused people. There were new charges that they hadn't previously seen. And in some cases they were the same charges, but they were displayed differently and broken up in different categories. And it just confused the heck out of people. So as it turns out, it wasn't a customer service problem, it was a billing problem. It was a different department. And if they spent all their time training call center people, teaching them how to work faster, how to answer questions faster, using newer technology to route people faster, to have shorter wait times, they still would have never solved the problem. When they went back and they diagnosed the problem as being a billing issue, and billing went back and started making some simple changes, relatively simple changes. Pretty much overnight, the call center problem went away. Because now people understood their bills again. They weren't calling about questions on their bill. The data we collect and how we interpret the data is going to impact our ability to see what's actually going on. And I feel this is a good example of lens number 21, which is the real problem lens. And the real problem lens basically says that make sure you're solving the right problem, not the symptom. Look, we make a lot of assumptions and we often solve the wrong problem because the patterns we see are not what is actually there. In this case, call center issues were the symptom, not the problem. When we look at the data, and this is really important, when we look at the data, there's always going to be some kind of inherent bias. There's going to be an inherent bias caused by the observer. Now, we could also look at this from the other side of things if we want to get into physics and quantum physics. And we could argue that the process of observing changes the behaviors of the one being observed, but we're going to save that one for a different podcast. The point is, the way we collect data and the way we analyze and interpret data is going to have biases, and these biases are going to create patterns because these patterns are based on shortcuts and that we use to make decisions quickly. In a previous episode, I talked about confirmation bias very briefly, but I talked about confirmation bias and how it can negatively impact the quality of the decisions you make. Basically what confirmation bias is, it's the brain's way of filtering out anything that doesn't agree with our deep seated beliefs. And I did talk about this back in, I think it was episode seven when I talked briefly about the dangers of loving your ideas. But this is a really important point, is if you have a belief, even if it is a subconscious belief, when you get evidence that supports your belief, you're going to magnify it. And when you get evidence that's not in line with your belief, you will filter it out or find arguments against it. So let's talk about some science. Actually, this is a really fascinating piece of research and I'll put a link to it in the show notes. This is a great example of confirmation bias in action. And I wrote about this initially in Best Practices Are Stupid. And then I referred to it briefly in Invisible Solutions. And I just think it's really worth bringing up right now because it gets at the power of patterns and the danger of patterns. So there were two researchers, Martin Tolcott and Freeman Marvin, and they gave trained U.S. army intelligence analysts a battlefield scenario. And they asked the analyst to determine the enemy's most likely avenue of approach and the level of confidence they had with this analysis. Later, they were given three updated intelligence reports, each which contained some information that supported their hypotheses, yet there were other pieces of information that would disprove them. They were then asked to rate each information item in terms of the degree to which it supported or contradicted their hypothesis. Now, the way this was structured, you would think they would have a lower level of confidence. But what was interesting Is they found that the analysts actually held more tightly to their initial hypotheses. They had even higher levels of confidence. Even though there was significant evidence that disproved their beliefs, the new information somehow increased their confidence levels. This is confirmation bias. They wanted to believe that their initial scenario was correct. And so when they were given new information, and we could do this, where 90% of the information actually disproves their hypothesis, yet they're going to amplify the 10% that supports their belief. This is really important when it comes to making decisions in life and in business. So when the researchers ran the test again with the same participants, but this time they showed them how confirmation bias impacted their decision making the first time around. During the second trial, they were given visual reminders designed to help foster their awareness of alternative hypotheses. So, in other words, when they were going through the new data, they were reminded that their natural tendency was to focus on what was going to support their belief. And these tools that they gave them, these visual reminders, were designed to help them think that maybe their original hypothesis was not correct. So when they did this, the result was fascinating. In general, there was actually a lower level of confidence, greater consideration of alternative enemy courses of action, and more willingness to reverse early decisions based on new evidence. In fact. In fact, during the second trial, 50% of the participating teams changed their hypotheses at least once during the exercise. The point is, even people who are highly trained intelligence analysts, people who are doing this for a living, to look at data, analyze data, and make decisions that are literally having life and death consequences, are making bad decisions because of the patterns they're seeing. It is human nature to believe what we want to believe. When we get an idea in our head, we are subconsciously going to focus our energies on proving those beliefs to be true. But being right can be the enemy of good innovation. So, as an innovator, don't get too attached to your ideas, because the patterns you see might be your brain just playing tricks on you. So one of the things which I always suggest is that you somehow proactively have a devil's advocate team or someone who's looking for disproving data, something that's going to provide alternative hypotheses to give equal or greater weight to the contradictory evidence. And this is going to help you make better decisions on which innovations to pursue and which ones to kill. I'm going to say something I know I've said in previous episodes. Failure is an inevitable outcome of innovation, but it is not the goal, it is not desirable. I don't believe that you want to glorify failure the way we have so much. What we want to do is get people to be better at looking at the data in the correct way and seeing what's really there versus what they want to believe is there. And there's a whole set of things we could talk about around experimentation. How do you do good experiments and how can you reduce the risk of failure? But the point is, failure happens often because we make faulty assumptions based on patterns that we see that aren't there. And therefore we make decisions that are inaccurate based on the information that's actually available to us. So we see patterns everywhere. We know this. But speaking of patterns, I just want to close with this because I think this is something fun, which I've. For the past year, not one person has commented on it. So I know it's been certainly under the radar. If you have a cover of my book, you have a hard copy cover of my book, take a look at it. And if you don't have the hardcover version of the book, maybe have the ebook, or if not, just even go onto my website. I'm gonna post a picture of the COVID zoomed in because there's something I want you to see. Because even though I've been talking about seeing patterns that aren't there, in this case there's actually a pattern, an invisible pattern that's been right in front of your eyes and yet you probably never saw it. So when you look at the COVID of my book, it's going to look like the invisible man wearing a jacket, tie, hat, and there's also glasses. And when you look at the clothing, it looks solid, but if you look closely, you're going to see there's a pattern. In fact, the pattern is comprised of the 25 lenses. The words associated with the 25 lenses. Leverage, deconstruct, reduce, eliminate. Those words are all around the clothing, on the clothing, on the jacket, on the tie, on the hat. And what I'll do is I'll post a zoomed in version, zoomed in image of this on the blog entry for this episode, just so you can see what I'm talking about. So there's patterns everywhere that we don't see. There are patterns that don't exist that we do see. And sometimes they're just patterns right in front of our nose and we just completely miss them. And I feel like this is a great way to wrap up this week's episode where we talked about the fact that we tend to see patterns when they're not there because it's a great way to create shortcuts to decisions to help speed up the process. But if we don't take the time to question the patterns, if we don't take time to question our assumptions, we might be making faulty decisions. Check your confirmation bias. Look to find ways of disproving what you believe to be true and you might have a greater chance of succeeding rather than failing. As always, if you want to submit problems to be solved in this podcast, go to invisiblesolutionspodcast.com there you'll also find every episode easy access to them. And if you want to download the 25 lenses, go to getthelenses.com and don't forget if you want to see a zoomed in pattern of the jacket, the hat and the tie with the 25 lenses, go to my blog and the most recent episode which will be related to this one, you'll see there a zoomed in image of it. I hope you enjoyed this episode and I look forward to being with you next week. And until then, happy problem solving.