Hindsight Bias: Why You Make Terrible Life Choices - -Nir And Far

Looking back into the past, we often think we can understand how things really were. The idea we can look back on history and understand it should be viewed with skepticism. This hurts our ability to learn from history. It is much easier after the event to sort the relevant from the irrelevant signals. After the event, of course, a signal is always crystal clear.

Welcome to the near and Far Podcast. This is the voice of Nir Eyal, but it's not really me. This episode is an AI rendering of one of my recent articles. I hope you enjoy this episode and for the latest posts, Please subscribe@nearandfar.com.

Why youy Make Terrible Life Choices In 2000, a 69 year old man began experiencing a persistent cough, chest discomfort and weight loss. His physician recommended a radiograph of his chest to identify the root of the issue, which revealed a large tumor. A biopsy confirmed the malignant thymoma, a cancer hiding between the lungs, consuming the patient's body from the inside out. Though the tumor had been growing for years, if left untreated, the man would die of the disease within 16 months. Shocked, the man struggled to understand how this could have happened. Three and a half years earlier, he'd had the same chest radiograph done as part of a routine examination. The radiologist who did the original exam found nothing out of the ordinary. Now, because of the bad test results, there was little time for treatment. The patient decided to sue the doctor who missed the tumor. In the trial, the patient's attorney showed the initial X rays to other radiologists who had no trouble identifying the tumor. There it was. The doctors corroborated. They could all see it. The doctors agreed the patient had suffered a clear case of medical malpractice. Or had he? The radiologist's attorney defended his client by claiming that the other radiologists only spotted the cancer because they already knew what to look for. After all, why else would a lawyer ask doctors to give an X ray a second look? The radiologist was innocent, his lawyer claimed, because without knowing there was something to find in the scans, he had made a reasonable diagnosis. It was only in hindsight that the cancer could be easily spotted. The other radiologists opinions couldn't be trusted because they had succumbed to hindsight bias. The jury had to decide whether to convict the radiologist based on how much they attributed to the benefit of hindsight. Was the cancer as obvious and the radiologist as incompetent as they now seemed? Or did knowing the outcome make it impossible to blame the radiologist for making the wrong call? What is hindsight bias? Hindsight bias occurs when people feel that they knew it all along, when they believe that an event is more predictable after it becomes known than it was before it became known. In other words, when we're looking back at an event after it already happened, knowing that outcome influences our perception of the events leading up to it. We experience hindsight bias because our brains are constantly trying to make sense of the world. To do that, we're constantly connecting causes and effects, connecting chains of events. Understanding cause and effect is an essential skill for survival. But when we succumb to hindsight bias, we oversimplify those explanations. We see unpredictable events as obviously I knew it all along. We end up thinking just as the radiologists were more likely to spot the malignancy when they knew something was wrong with the patient's X rays. We allow our knowledge of the present to affect our interpretation of the past every day of our lives. Examples of Hindsight Bias Hindsight bias influences our perception of all sorts of life events. An investor might think of course the stock market crashed this year. I knew it would, despite not having sold out when there was a chance a disappointed birthday girl might tell her party guests, I just knew it was going to rain on my birthday. Even though there's no way she could have predicted the weather a sports fan might claim with confidence, I always knew my team would win the championship this year, while she or he had little clue when the season began. What these three examples have in common is the tendency to see the past as certain, but only after the fact. Hindsight bias isn't just about the past, though. It hurts us the most by manipulating how we make decisions for the future. How hindsight bias tricks you into making terrible life choices. When we misinterpret our thinking that preceded an outcome, it can color our judgments for future predictions. The important thing to know about hindsight bias is that it not only changes how you see the world, but also how you see yourself in it, neil Rose, a professor of marketing at the Kellogg School of management at Northwestern University, 20 told the New York Times. You begin to think, I'm really good at figuring out what's going to happen. You begin to see outcomes as inevitable that were not, for instance, once burned. By not getting out of the stock market before a crash, investors might avoid getting back in the markets even when they logically know it's the right thing to do. In contorting the past to fit a narrative that he had known the market would crash all along, even when he didn't, the investor might listen to his incorrect intuition and and get burned yet again. If you feel like you knew it all along, it means you won't stop to examine why something really happened, wrote Ross. It's often hard to convince seasoned decision makers that they might fall prey to hindsight bias. One way this misinterpretation happens is in assuming the causes of past events were much simpler than they were. Kathleen Voes, a social scientist at the University of Minnesota's Carlson School of Management, found the bias's consequences included myopic attention to a single causal understanding of the past to the neglect of other reasonable explanations. This can also lead to incorrectly assigning blame or credit for an outcome. Overly simplistic explanations of the past make it easier to pin the consequences on individuals rightly or wrongly. Incorrectly assigning credit can also cause distorted thinking. According to Voese, the simplistic explanations of the past provided by hindsight bias breeding breed overconfidence in the certainty of one's judgments. It makes people think they can look back at past events and interpret something. It makes them think they have new ability to predict, drew Boyd, executive director of the University of Cincinnati's MSc Marketing program, explained to the BBC. When we concoct such false explanations of the past and of our own predictions, we can't learn and adapt for future decisions. When we don't take what really happened to heart, we fail to learn from our mistakes and tend to repeat them. How does hindsight bias affect certain fields? The effects of hindsight bias are especially significant in certain areas business, law, medicine, politics and history. Business Nobel Prize winning American economist Richard Thaler believes business executives may be more prone to hindsight bias than people in other fields. In one study, researchers surveyed 705 entrepreneurs from failed startups. Before the failure, 77.3% of entrepreneurs believed their startup would grow into a successful business with positive cash flow. But after the startup failed, only 58% said they had originally believed their startup would be a success. Such a disconnect clouds business leaders judgment and ability to learn from the past. It makes people think they can look back at past events and interpret something. It makes them think they have new ability to predict, said Boyd to the BBC. It happens in business a lot. When you think that something that has happened before is going to happen again, it seems to make sense. But then it doesn't happen again and you wonder what happened. Business people will decide on a strategy because it worked for them before, but the conditions in the next environment are going to be different. It's a different market situation, different people, and it's a mistake to immediately assume that what worked before is going to work again, he continued. This tendency makes it more likely that business leaders will take on risky and poorly thought out ventures. Law as you saw in the radiologist lawsuit described earlier, legal disputes and deliberations can also be uniquely affected by hindsight bias. Judges and juries knowing the outcome of a situation at the beginning of a trial are uniquely susceptible to the oversimplifications of hindsight bias when assessing blame for crimes. Separating the facts of the case from the blurred vision caused by knowing the outcome is extremely difficult.

Medical professionals are also susceptible to hindsight bias. In one classic experiment, a group of physicians were given the medical case history for a patient with a list of four possible diagnoses and asked to estimate the probability of each being correct. Some of the physicians the hindsight group were told that one of the four diagnoses was correct before making their estimates. The hindsight group assigned far greater probability estimates to the diagnoses they were told were correct than the rest of the subjects. Hindsight bias significantly altered their perception of the symptoms and their cause. Politics Political judgment is also clouded by hindsight. Winston Churchill remarked that politics is the ability to foretell what is going to happen tomorrow, next week, next month, and next year, and to have the ability afterward to explain why it didn't happen. This is clear in political punditry, in which commentators always have neat explanations for election results or legislative decisions regardless of their own advance predictions. In an experiment conducted by psychologists at Loyola University in Chicago, students were asked to predict the likelihood that then President Bill Clinton would be convicted or acquitted in his impeachment tr. They gave estimates at 22 days and three days before the verdict, and then they were asked to recall those estimates 4 days and 11 days afterwards. 11 days after the Senate's acquittal vote, students inflated their original estimates by about 10 percentage points, while their estimates of what a friend would have guessed showed no such inflation. Political campaigns are aware of voters susceptibility to hindsight bias and exploit it to win votes. With the benefit of hindsight, it's easy for voters to develop a narrative explaining the results of a politician's service, and campaigns are happy to help craft it. If a swing voter made a difficult decision in the previous election, for example, a challenger can exploit the bias to create voters remorse for the incumbent as the challenger. You need to win over some of those voters, a former advisor to President George W. Bush, Mark McKinnon, said to the New York Times. You need to give them an out for their voters remorse. Hindsight may make it easy to justify their initial reservations about the incumbent and make them more willing to vote for the challenger. Of course, this can also work in the opposite direction. Voters are just as likely to justify their original decision, which may be why incumbents win more often. History finally, hindsight bias is also difficult to avoid in studying history itself. According to Professor Nick Chater of Warwick Business School in an interview with the History News Network, looking back into the past, we often think we can understand how things really were, like what caused the 2008 financial crisis, the collapse of communism, or the First World War, because we know how things turned out. But now we know about hindsight bias, we should be suspicious of this feeling of understanding. The idea we can look back on history and understand it should be viewed with skepticism. This hurts our ability to learn from history. In 1962, the historian Roberta Wohlstetter wrote in her book in reference to the attack on Pearl harbor, it is much easier after the event to sort the relevant from the irrelevant signals. After the event, of course, a signal is always crystal clear. No one wants to imagine that they might have owned slaves had they been a white man in the American south during antebellum. It's incomprehensible to think of ourselves as Nazis had we lived in Germany in the 1930s. However, unconscionable acts were carried out by normal, everyday citizens just like us. We don't want to think of ourselves as capable of hideous acts, so we use hindsight bias to protect ourselves from the truth. We'd much rather believe bad people do bad things or assign blame to a sinister mastermind like Adolf Hitler. But that rationale misses the real lessons of history. By refusing to understand the circumstances that made good people like us do horrible things, we risk not learning from and therefore potentially repeating the mistakes of the past. The History of Hindsight Bias Though the term is modern, examples of hindsight bias have been documented for centuries. In his 1867 novel War and Peace, Leo Tolstoy described hindsight bias as the law of retrospectiveness, which makes all the past appear a preparation for events that occur subsequently. Hindsight bias was formally named, documented, and studied in the 1970s. In 1975, Baruch Fischoff, a doctoral student of Nobel Prize winning psychologist Daniel Kahneman, conducted the first experimental study on the concept. Fischoff wanted to test what he called the creeping determinism hypothesis, in which people tend to perceive outcomes as having been relatively inevitable. In the study, participants were given brief descriptions of real historical events with four potential outcomes for each and asked to predict the probability of each outcome being the true one. However, some participants were told that one of the choices was the true outcome in advance, though the given answer was not always the correct one. They were then asked to answer as though the answer hadn't been given. The results supported Fischoff's hypothesis. When given the true outcome in advance, participants were significantly more likely to view it as inevitable. Further, when asked to explain the reasoning behind their choices, they emphasized the data points supporting the chosen outcome as most relevant. Being given an answer in advance, they were both more confident in their choice and more cognizant of information confirming it. Fischoff concluded that creeping determinism not only bias their impressions of what they would have known without knowledge of the outcome, but also their impressions of what they themselves and others actually did know in foresight. In the decades since Fischof's initial study, researchers have conducted many more experiments documenting hindsight bias in a variety of ways. The bias has been found in a wide range of task types, such as confidence judgments in the outcome of events, choices between alternatives, and estimations of quantities. It has also been documented in a wide span of knowledge domains, including political events, medical diagnosis, outcomes of scientific experiments, economic decisions, autobiographical memory, and general knowledge. In a 1991 meta analysis of hindsight bias studies conducted by JJJ Christensen Solansky, they found the effects of hindsight bias to be statistically strong across the findings, concluding the bias to be robust and difficult to eliminate. Why do we have hindsight bias? The consequences of hindsight bias are clear. What's less clear is why we're so susceptible to it in the first place. Why would we so naturally rewrite history when it can have such negative effects on our learning and decision making? Like most cognitive biases, it's the result of mental processes that work well in many situations but backfire in others. To understand why, we first need to become familiar with those processes that combine to create hindsight bias. For one, we naturally rely on information that is easiest to recall, not that is most relevant or accurate. For example, our fear of flying will often increase after hearing of a crash on the news. Even though the statistical risk is unchanged, the vividness and recency of the memory weighs more heavily than than objective data. Once we know an outcome, we retrieve memories that align with the result more easily than those that contradict it. We are also often overconfident in our abilities 93% of Americans rate themselves as better than average drivers, for example. Overconfidence leads us to create a rosier picture of ourselves than we deserve. Thus, it's natural to retroactively look back at our interpretations of events as being obvious predictors of an outcome with the benefit of hindsight. The anchoring effect is another component of hindsight bias. Anchoring refers to our mental tendency to latch onto any available information as a reference point for interpretation and action, regardless of its relevance. Once we know a result, we naturally anchor to it when considering the preceding events. In most cases, we can't help it. In one study, researchers asked participants to write down the last two digits of their Social Security number, then state whether they'd pay the equivalent dollar figure for a variety of consumer items. Participants with higher numbers were willing to pay higher prices for the items. For the highest numbers, the prices were higher by a factor of three. They were anchored to the arbitrary numbers, despite them having no real relevance to the product's value. These biases work together for hindsight bias. The anchor of the outcome likely leads to the ease of recall for coinciding data points via the availability heuristic. Then overconfidence crafts the story of how we knew it all from the beginning. Beyond these biases, the way we observe the world contributes to our tendency for one, our brains have evolved to find patterns even when none exist. The ability to fill in the blanks between disparate pieces of information made it easier for humans to spot and avoid predators, for instance, creating a major evolutionary advantage. Superior pattern processing is the essence of the evolved human brain, johns Hopkins neurologist Mark Matson explained to psychology today. It is the fundamental basis of most, if not all, unique features of the human brain, including intelligence, language, imagination, invention, and the belief in imaginary entities such as ghosts and gods. The success of pattern recognition also can lead to overcompensation, though the same mental processes that aided in primitive survival helps breed conspiracy theories. When we observe an event happening, we are more likely to assume it was the result of an intentional series of events rather than randomness. This is called the fundamental attribution error, or our tendency to prefer dispositional explanations to situational ones. Similarly, aligning past events with a known outcome creates a neat and easy to understand narrative. Aside from recognizing patterns, the mind is also designed with an adaptive thinking process that modifies perception as new information is received. As we take in new information, our mind naturally recalibrates its understanding of the world around us. Some researchers argue that hindsight bias is simply a feature of that adaptive process. They contend that humans are naturally iterative thinkers who update our priors as we receive new information so it's obvious that we would update our perceptions of the past with the new information of the results. The knew it all along effect is simply a byproduct of that framework. Since we are cognitive misers with limited mental capacity, it's more efficient to update our interpretation of past events like we would for anything else. How to Overcome Hindsight Bias Perhaps the most effective tactic to overcome hindsight bias is to document the inputs to decisions and predictions as they happen to, leaving a clear record that isn't at the mercy of biased memory. The surest protection against hindsight bias is disciplining ourselves to make explicit predictions. Showing what we did in fact know that record can also provide us with some protection against those individuals who are wont to second guess us, producing exaggerated claims of what we should have known and perhaps should have told them, states John Scott Armstrong, professor of marketing at the Wharton School and editor of Principles of Forecasting. Richard Thaler supports this method as well, according to an interview with McKinsey, one suggestion I make to my students is write stuff down. I have a colleague who says if you don't write it down, it never happened. Any company that can learn to distinguish between bad decisions and bad outcomes has a leg up. When documenting the inputs that went into making decisions, be sure to weigh them as well. We try to teach people to use what we call Bayesian thinking, said Drew Boyd to the BBC 18th century English statistician Thomas Bayes premise was to consider all sources of information but weight them. Some information is more valuable, but all information has some value. Weight that information appropriately and you tend to make the best decision. Make decisions based on what the data says is likely to happen, not what you think is going to happen. For example, imagine deciding whether or not to invest in a company. There are many inputs that factor into the choice profit margins, growth rates, industry trends, cash flow, share price, and so on. When making the decision, the investor should document each data point and assign a weight to each one, quantifying its perceived importance to the outcome. Documenting such weights ahead of time avoids the revisionist history that accompanies hindsight bias. When evaluating outcomes that were not properly documented like this, our mind will retroactively assign probabilities. The paper trail counteracts that tendency. If we haven't documented the information leading up to the event, then we can consider the opposite. When you construct an interpretation of the past, you should flip it and think, how could this not have gone the way you anticipated? Once you have your little narrative in your mind, think, how could the outcome go in a different direction or not occur at all, kathleen Voes told the BBC. If you flip the script like that in a number of ways, you can reduce hindsight bias. This also helps disrupt our natural overconfidence. Anything that reduces people's confidence in predicting something will happen or the pathway in which something will happen is a good way to go about it, voes added. Conclusion and Summary in the case of the radiologist accused of malpractice, hindsight bias was a core part of the defense attorney's strategy. He argued that reviews of the original scans by other experts could never be accurate. One of the experts on the defense team supported the view that hindsight bias was impossible to avoid in such a case, even when the graphs were shown without any context of the case. I've never had an attorney bring me a normal radiograph, he said. Whenever an attorney shows or sends me radiographs, the first and only question that comes to my mind is what was missed on these films? The plaintiff's experts, of course, argued otherwise. They claimed they had reviewed the initial radiographs before they had looked at subsequent studies and before they had been told that the case centered on an alleged missed lung tumor. But didn't you surmise that if an X ray was brought to you for review by an attorney, that X ray would necessarily be abnormal? Asked the defense attorney. Not at all, answered one of the radiology experts retained by the plaintiff. The lawyer could well bring normal radiographs. The jury bought the latter argument, voting the defendant radiologist liable for malpractice by a 102 vote and awarded $872,000 to the family of the deceased patient. Hindsight bias has real consequences the question of how much the future influences our interpretation of the past has important implications. Due to many of our cognitive processes, we cannot help but let the knowledge of an outcome influence our perception of the events preceding it. We can, however, mitigate these retroactive judgments with clear records. By documenting predictions and conditions surrounding events, we can have a more accurate understanding of the past. With that, we can better prepare for the future.

Sam.