Daniel Kahneman on Cutting Through the Noise

Hi everyone, and thanks for your listenership. Conversations with Tyler has grown bigger than ever this year, and I hope in some ways at least, better than ever. Now, there's a new way you can support the show. Today through the end of the year, you can visit conversationswithtyler.com/donate to make a financial contribution. All donations will go to the actual production of the show including new conversations every other week. Note that unlike many other podcasts, we often travel to guests, or maybe they travel to us. We've never done a remote interview. Plus, I need a lot of books for my research. We also put on live shows in Northern Virginia, San Francisco, New York, with more to come. And there are full transcripts of every episode enhanced with helpful links. This year, the podcast has featured some of the best-known thinkers and doers in the whole world, like today's guest, Danny Kahneman, or recently, Eric Schmidt. No less importantly, the success of the podcast has given a broader platform to underrated thinkers such as Agnes Callard, Juan Pablo Villarino, and Michelle Dawson. I think of the unifying theme behind Conversations with Tyler is learning how to learn. It's about how I learn from all of these other people. That to me is what ties all of the episodes together, and I hope that makes the world a wiser place. If you believe in this theme, please consider supporting the show at conversationswithtyler.com/donate. Thank you. Conversations with Tyler is produced by the Mercatus Center at George Mason University, bridging the gap between between academic ideas and real-world problems. Learn more at mercatus.org. And for more conversations, including videos, transcripts, and upcoming dates, visit conversationswithtyler.com. Thank you for coming, Danny. You've worked on so many topics. Let me start with the issue of happiness. If you have an experience, it seems that how happy you are at the end of the experience, it depends on the end of the experience and how good was the peak or how bad was the bottom. Given that result, should we aim to deliberately structure our experiences so they give us more happiness?

Well, I mean, if you want good memories, Good endings are really important. The question is how important good memories are relative to the experience itself, but no question, ends are very important. They're particularly important in the context of goal striving. That is, whether you achieve a goal or don't achieve a goal colors the whole experience of trying to to get it, to get to it. So ends are very important for memories.

So do people structure their vacations to meet the standard, or there's a kind of market failure? If they listen to you, they would have better vacations.

I'm not at all sure that people— my guess is that people are conscious of, you know, that they don't want the peak to be too far from the end. That's my guess.

And why does duration of pain seem to matter so little for how we evaluate painful experiences?

Well, you know, if you were asking what is the evolutionary value, then the duration of pain is really not very important. What's important is the intensity because the intensity is a measure of the severity of threat. The duration is really something else. It's very striking that it You know, it's completely insignificant when people— in many situations, it's completely insignificant. Quite a striking result.

You also have a paper on happiness with Alan Krueger using what you call the day reconstruction method, how much people enjoy different experiences. And one result from that paper is how much people enjoy spending time with their friends. If that's so much more enjoyable at the margin, why don't people do more of it?

Well, altogether, I don't think that people maximize happiness in that sense. And that's one of the reasons that I actually left the field of happiness. I'm not involved in that. I was very interested in maximizing experience, but this doesn't seem to be what people want to do. They actually want to maximize their satisfaction with themselves and with their lives, and that leads in completely different directions than the maximization of happiness.

And do you think that telling people you'll be happier a particular way changes their behavior much, or they still stick to maximizing a sense of satisfaction with their lives?

No idea. I haven't tried. You know, there is a lot of work these days on trying to make people happier and on trying to coach people. In the UK in particular, it's a whole You know, there is a— I wouldn't call it an industry, but it's sponsored by government. My friend Lord Layard has started a movement that promotes happiness. There's a great deal of that happening, and I don't know how successful it is because the criterion for evaluation, it's very difficult to conduct evaluations on those things because people who know they've been subjected to interventions cannot really answer those questions honestly, or even if they try. So the way to test whether things are successful would be to ask a person's friends, has he become or she become happier? And that hasn't been done.

And that people want to maximize their overall sense of how their life has gone. Do you think that has ultimately Darwinian roots, or why is that the equilibrium? Happiness feels good, right?

Yeah, happiness feels good in the moment, but you know, it's in the moment. What you're left with are your memories. That's the very striking thing, that memories stay with you and the reality of life is gone in an instant. And so memory has a disproportionate weight because it's with us. It stays with us. It's the only thing we get to keep.

If you think of your own life, have you maximized happiness or the overall sense of how your life has gone? Neither. Neither. Citations? No. If you miss a flight due to a traffic jam outside your control, would you rather be 2 hours late or just 1 minute late?

Oh, I mean, I'm like everybody else. I'd rather be 2 hours late.

And you think even knowing about this doesn't change that? You can't talk yourself out of the bias.

You can talk yourself out of some biases, I think. I mean, I wouldn't generalize on that, but it would take, you know, I could possibly talk myself out of that one by, you know, by really sort of repeating to myself how stupid it is, but it would take a lot of work. It's not that you can decide once and for all, I will not be subject to that bias. Doesn't work that way.

Do you think we overinvest or underinvest in memories overall?

We certainly invest heavily in memories. I mean, you know, vacations for many people are investment in the formation and maintenance of memories. So there is a lot of investment. Whether it's too much or too little, it probably depends a lot on people's— on the amount of consumption of memory that people engage in. So I, for one, I'm certainly biased, but I do not consume my memories a lot. I almost never go back to photographs, not deliberately. If I stumble on something, it will move me, but the idea of going back to relive a vacation, that's not what I do. So I have little empathy for this.

If we think about, say, sports, they're a form of bias, right? Most people root for a home team. Or they root for their country in the Olympics. Music arguably is a form of bias. There's soundtrack music. It affects how you view the movie, even though it's not changing any facts. To what extent should we think of bias as the main thing that gives our lives an overall structure, just as a musical soundtrack is what gives structure to a movie?

Well, I mean, you know, that's a tendentious way of labeling things, to call them biases. I wouldn't call the effect on music a biasing effect. It completes the experience. And what were your other examples?

Well, sports. You're consuming bias, right? You don't actually think your team is better.

No, but you identify. I mean, you know, it's not— there are emotions over which you have very little control, and it's a fact that you feel pride when your team wins. In fact, you feel pride if a stranger who lives on your street gets a prize. So that tendency to identify with what's around us and with things that we are connected to is very powerful. We derive a lot of emotion from it, and I wouldn't call that a bias because you can call any emotion a bias.

There's a well-known article by John List where he argues if you study how experts trade assets, that a lot of what are called biases go away and become quite small. What's your reaction to his research?

It's beautiful research. I'm convinced it's right. And indeed, you don't have to go as far as he does to find cases in which people act fairly rationally. People act fairly rationally in routine transactions. So if there is a thing that's loss aversion, that it plays a large role, and that's List's research in novices. They get attached to things and then they don't want to sell them and they get over it over time. And in routine transactions, when I go and I spend some money to get shoes, I feel no loss aversion for the money and certainly the person who sells me the shoes feels no loss aversion for the shoes. It's a routine transaction and it's a whole domain.— in which loss aversion doesn't apply.

Now, much of your last book is about bias, of course, and much of your next book will be about noise. If you think of actual mistakes in human decision-making, how do you now see the relative weight of bias versus noise?

Well, I would say this. So first of all, let me explain what I mean by noise. I mean just randomness. And it's true within individuals, but it's especially true among individuals who are supposed to be interchangeable in, say, organizations. Can I spend 3 minutes to explain that?

Of course.

Sure. So I'll tell you where, you know, the experiment from which my current fascination with noise arose. I was working with an insurance company And we did a very standard experiment. They constructed cases, very routine standard cases, expensive cases. We are not talking of insuring cars. We're talking of insuring financial firms for risk of fraud. So you have people who are specialists in this. This is what they do. So cases were constructed completely realistic, the kind of thing that people encounter every day. And you have 50 people reading a case and putting a dollar value on it. And now I could ask you, and I asked the executives in the firm, and it's a number on which just about everybody agrees. So suppose you take 2 people at random, 2 underwriters at random, you average the premium they set You take the difference between them and you divide the difference by the average. So by what percentage do people differ? Well, would you expect people to differ? And there is a common answer that you find, you know, when I just talk to people and ask them, all the executives have the same answer. It's somewhere around 10%. That's what people expect to see in a well-run firm. Now what we found was 50%, 5 zero, which by the way means that those underwriters were absolutely wasting their time. I mean, in the sense of assessing risk. So that's noise. And you find variability across individuals, which is not supposed to exist. And you find variability within individual depending morning, afternoon, hot, cold. I mean, a lot of things influence the way that people make judgments. Whether they are full or whether they've had lunch or haven't had lunch affects judges and things like that. Now, it's hard to say what there is more of, noise or bias, but one thing is very certain, that bias has been overestimated at the expense of noise. I mean, virtually all the literature and a lot of public conversation is about biases, but in fact, noise is, I think, extremely important, very prevalent, And there is an interesting fact that noise and bias are independent sources of error so that reducing either of them improves overall accuracy. And so there is room for— and the procedures by which you would reduce bias and reduce noise are not the same. So that's what I'm fascinated by these days.

Do you think of low intelligence as yet a third independent source of error? Or is that somehow subsumed in bias and noise?

You mean plain stupidity.

In some cases.

Yeah. It wouldn't really be necessarily the same as either bias or noise. I mean, you know, getting inadequate information or not getting adequate information when it's available is a stupid thing to do and a very common thing, and it's not exactly bias and it's not necessarily— it would contribute more to noise than to bias, by the way, by and large. When people collect too little information or are swayed by the first thing that comes to mind, you get noise rather than bias.

And do you see the wisdom of crowds as a way of addressing noise in business firms? So you take all the auditors and you somehow construct a weighted average?

Well, the wisdom of the crowds will work and pooling opinions will work when errors are independent. That is, when everybody is inclined to make the same mistake, which is then a bias. Right. Then having multiple individuals engaged in it, if they share their biases, you'll get the bias, it's going to be worsened and everybody will have much higher confidence in their biased views because other people share them. Wisdom of the crowd works under quite specified conditions. With respect to the underwriters, I would expect certainly that if you took 12 underwriters assessing the same risk, you would eliminate the noise. You would be left with bias, but you would eliminate one source of error, and the question is just price. Google, for example, when it selects— when it hires people, They have a minimum of 4 individuals making independent assessments of each candidate, and that reduces the standard deviation of error at least by a factor of 2.

So is the business world in general adjusting for noise right now or only some highly successful firms?

I don't know enough about that. All I do know is that when we pointed out the results the bewildering results of the experiment on underwriters, and there was another unit, people who assessed the size of claims, again about— actually it's more than 50%, like 58%. The thing that was the most striking was that nobody in the organization had any idea that this was going on. So it took people completely by surprise. So my guess now is that wherever people exercise judgment, There is noise. And as a first rule, there is more noise than people expect. And there's more noise than they can imagine because it's very difficult to imagine that people have a very different opinion from yours when your opinion is right, which it is. So that's, that's the way it works.

So if you're called in by a CEO to give advice, and I think sometimes you are, how can I reduce the noise in my decisions, the decisions of the CEO? When there's not a simple way to average, the firm doesn't have a dozen CEOs, what's your advice?

My advice is divide and conquer. That is, there is one thing that we know that improves the quality of judgment, I think, and this is to delay intuition. I think there is in the audience a friend of mine, Gary Klein, who is violently opposed to what I'm saying, as are many others. But I'm here, so— I think delaying intuition is a very good idea and delaying intuition until the facts are at hand and looking at dimensions of the problem separately and independently is a better use of information. And the problem with intuition is that it forms very quickly. So that you need to have special procedures in place to control it, except in those rare cases, and Gary Klein and others have demonstrated that, where you have intuitive expertise. That's true for athletes. They respond intuitively. It's true for chess masters. It's true for firefighters, captains, as Gary Klein has shown. So that's intuitive expertise. I don't think CEOs encounter many problems where they have intuitive expertise. They haven't had the opportunity to acquire it. So they'd better slow down.

And just take more time on each decision.

Break the decision up. It's not so much a matter of time because you don't want people to get paralyzed by analysis, but it's a matter of planning how you're going to make the decision and making it in stages and not acting without an intuitive certainty that you're doing the right thing, but just delay it until all the information is available.

And does noise play any useful roles either in businesses or in broader society, or is it just a cost we would like to minimize?

Well, I mean, you know, there is one condition under which noise is very useful, and that if there is a selection process. Evolution works on noise, so you have random variation and then selection. But when there is no selection, noise is just a cost.

But say it were always transparent who would be the winners and who would be the losers from a given decision. Wouldn't we be too emotional, too polarized, engaging in too much rent-seeking, and having an ambiguity as to cause and effect is in part what allows us to get along with each other?

I mean, you are sort of making a lot of assumptions I'm not used to in this question. You seem to assume that, you know, there is something very competitive that could be alleviated.

But there's the old saying, say from the Soviet Union, that meritocracy is very hard to live under. That if you really know how many people are better than you are, which say a chess player might, there's something psychologically oppressive to being downgraded. Whereas noise, you can be overconfident more easily. And we all know overconfidence—

You don't need noise for that. I mean, you know, Bias will do it for you, and there is a lot of bias in that direction. I mean, people clearly overestimate what they can do and how good they are, and that's a blessing undoubtedly.

Are there groups of people you feel are less subject to biases? So there's some papers, for instance, showing that autistics, they have weaker framing effects, smaller endowment effects, maybe because top-down processing works in a different way. Do you have an opinion on that literature?

No, I don't know it well enough.

If you think of the literature on what are called cognitive disabilities, so ADHD, do you think of that as bias or somehow in a different logical category or?

I mean, you know, I don't think it's a bias. No, I think it's an attention deficit. So it means that people have difficulty controlling their attention, focusing on what they want to focus on and staying focused. That's neither bias nor noise. You know, bias and noise do not cover the universe. There are other categories.

And if you think about the issue of when people think about the world, they find some kind of transactions repugnant. So sometimes they just don't like to sell what they have. Other times they seem to object to markets, say in kidneys or kidney transplants. Do you view that as bias or where does that come from?

Well, I mean, in the sense that this is a norm and there are things that we are trained or socialized to find disgusting, to find repugnant. And so there are repugnant transactions and you have to treat them as you treat every other moral feeling. We have lots of moral feelings, things that we find unacceptable without without any ability to really explain why they're unacceptable. There is such a thing as moral emotion. There is such a thing as indignation, as moral disgust, and that's what we're talking about here.

So you're pessimistic about the ability of psychologists to develop structural explanations of where feelings of repugnance come from.

Well, in some cases, we know, and you can do that associatively. I mean, it really depends on the associative structure that is imposed by a given culture. So to give you a sense of the way that works, there is a psychologist, Paul Rosen, who's done some brilliant experiments on that. And in one of the experiments, so he has people and they have a glass of orange juice and They have a sticker and they're asked to write on that sticker cyanide and to stick it on the juice and then to drink the juice, and they don't want to. Now, this is something, it's an emotion over which people have no control, and our socialization has created those emotions in us. And we're conditioned to have them under some conditions. Other cultures are disgusted by other things.

Philip Tetlock has argued that if we set up long-run tournaments with forecasting and we measure results and we test teams against each other, that we can in the longer run reduce, I think, both noise and bias. Do you agree? And do you think there are factors he's overlooking in how his tournaments are set up?

I mean, Phil Tetlock is another friend, but he's also a hero. I mean, I think this is beautiful research. I think it's proved beyond a shadow of a doubt that when when you have people making forecasts for the medium term, up to 6 months in many situations, you can have people thinking carefully without any training who do better than CIA analysts. That's fundamentally what he has shown. And he really knows why, or he knows how they do it. And the tricks are very simple. I mean, if you made a list of intelligent ways to go at problems, That's what people do. So, you know, they view the problem as an instance of a category and then they switch to looking at the problem from the inside and essentially they adopt different points of view. It's not the same thing as what I was saying earlier about breaking up a problem into dimensions and averaging. There's no averaging, but there is looking at a problem from multiple dimensions and collecting a lot of information. And that's basically what creates superforecasters.

So if you're picking the Daniel Kahneman superforecasting team, what qualities are you looking for in individuals?

Well, Phil Tetlock really has a comprehensive list, which I'm not going to remember.

But at the margin, how would you modify it?

They will be intelligent. They will be numerate. They will be open-minded. They'll be curious, interested in learning, eager to train their mind.

But is there a bias left in how Tetlockians pick their teams?

He picks the teams by results. So what he has, he has people competing in making probabilistic forecasts of strategic or economic events in the medium and short term. And some people are more accurate than others. And after a year of that, you select the top 2% and you call them superforecasters. And that gives them a very good feeling to be labeled superforecasters., and they do not regress to the means. That is, the second year, they're just about as good as the first year. That's the basic finding.

If you're picking doctors where maybe results are hard to measure in some cases, what do you look for when selecting doctors in broad terms?

Well, I will do the conventional thing. I will ask about their reputation because that's the best measure we have. If it's a surgeon that I'm looking for, then there are real indices. The main one being the number of times he's performed the operation in question. So that, you know, that you know this is what you've got to examine because people really do get better over time. And so measuring how much practice they've had, and the practice is fairly specific on different operations. So I think I would know how to pick a surgeon.

There's a good deal of evidence that people in businesses are overconfident, but do you think they're more overconfident than they should be?

Well, overconfidence has many virtues. I mean, you know, in the first place, it's nice. I mean, it's pleasant to be overconfident, especially if you're an optimist. I mean, optimism is valuable much more than overconfidence. Overconfidence is sort of a side effect. To exaggerate the odds of success is a very useful thing for people. It will make them more appealing to others, they will get more resources, and they will take risks. And it's not necessarily good for them. The expected utility, you know, of taking risks in the economy is probably marginally negative. But for society as a whole to have a lot of optimists taking risks, that's what makes for economic progress. So I call that the engine of capitalism, really, that sort of optimism.

There's a collaboration between a human being and a machine, and occasionally the human being overrides the machine. Do you feel the human beings in those situations are on average either too overconfident or too optimistic?

Well, I mean, you know, there are certain criteria that you would want to apply before you put a machine to work. You want to validate that. But once you have a machine making decisions, the conditions under which it's a good idea for humans to override them are really well known and well understood. And it's not that when you get a feeling that the machine is wrong, that's not enough. I'll give you an example where it would be okay to override a machine. So suppose you have a computer that approves loans and then you're the banker and you see that the person who was approved for a loan has just been arrested for fraud. Then you will override the machine. That's about the conditions under which it's worth it. Otherwise, there have been many experiments And when people override formulas, by and large, they do worse than if they hadn't intervened.

So do you side with the analysts such as Martin Ford, who see really a very large number of jobs being potentially automatable with artificial intelligence, machine learning, or will we always need the human beings to work with the machines?

That— we will need human beings. Is, I think, an illusion. It's really very— I mean, take chess, for example. Kasparov was beaten 20 years ago, and he went on for a while, and it was true for a while, saying that teams of chess players with grandmasters, programs with grandmasters, would be stronger than either. And it was true for a while. It is true no longer. The programs do not need the grandmasters. And you know how it happened, and it's likely to happen in many other fields. I mean, it's happening in dermatology. Dermatology, the diagnosis is now better done by programs than by people, and they are not going to need the person very often. That is, to have a person intervene with the right to intervene, they will sometimes correct mistakes, but they will more often, I think, introduce mistakes. So when you have a well-running program, leave it alone.

So we as professors won't need to grade exams anymore, and I don't just mean multiple choice. You run machine learning on papers, you find what correlates with a good paper, you put the paper through the program.

Look, I mean, The point is there is so much noise in essay grading that it's quite easy to imagine a program that would look at various indices and that would do better than, you know, hurried and tired professors.

If you consider people working in psychology or maybe economics or just social sciences, do you think people persist with their professional and research projects too long or not long enough? Where's the bias?

My guess is too long, but it's a personal bias.

Because of sunk costs.

Because of sunk costs. And I think sunk cost is really the enemy when you're doing research, innovative research. You ought to recognize that something isn't working and just move on. And there are different views on that, but my sense is that this is the direction of the bias here, sunk cost.

Michael Nielsen, who's a scientist and he works at Y Combinator, he tweeted today, "If it weren't for sunk costs and my respect for them, I wouldn't ever get anything done." What do you think?

I mean, you know, sunk costs—

It keeps you at things, right?

Yeah, it keeps you at things.

You stay loyal to your friends, you become more trustworthy.

Well, that's not, you know, when we talk about sunk costs, we talk about something else. So it is not true that attachment Growing attachment to things that you're familiar with and that you like and love and increasingly trust, that's not sunk cost, that's something else. Sunk cost is a fairly specific thing, that is that you are putting a different value on a move or an investment that you make because of investment that you have already made than you would if you were looking at that de novo. Sunk costs by and large, I think, are a negative. We know that when you get a new CEO in place in organizations, the new CEO has one big advantage. He's got no sunk costs with respect to poor ideas that the exiting CEO had and couldn't let go of.

If you had a perfectly rational pure Bayesian, Would anyone else trust that person?

Well, you know, I mean, would he be nice? I don't think so. You know, that's what would matter. I don't think, you know, if you could get me a nice Bayesian, that would be fine.

Some questions about psychologists outside of what you've worked on, but may be related. Freud. What do you think of Freud's body of work, and has it influenced you at all? If I think of Freud's two principles of mental functioning, right, the notion of pleasure principle, reality principle, it's a little bit like thinking fast and slow in some ways with big differences.

Well, I mean, you know, all dichotomies are alike, you know, in some ways. And yeah, you know, there are similarities. Oddly enough, there is one aspect of Freudian work that I think did influence me. And he has, for some reason, when I was a graduate student, it's too long a story, but I was exposed to Chapter 7 in The Interpretation of Dreams, and I spent a summer studying Chapter 7 of The Interpretation of Dreams. And in Chapter 7, there is basically a theory of attention. And like 25 years later, I published a theory of attention, and when it was done, I realized that It resembled Chapter 7, you know, quite a bit. So, yes.

Personality psychology and Five-Factor Personality Theory, is that for you a useful way of thinking about human beings?

Well, you know, it's a proven way of thinking. It's sort of boring, but— and I mean that seriously. This Five-Factor thing, you know, that's about 20 years old and it dominates personality psychology because it works. And it's sort of, you know, it used to be more exciting to have more complicated mechanisms, but you have something that seems to work.

What did you draw from Herbert Simon?

Directly, nothing. Indirectly, a lot. And retrospectively, a lot. What I mean by indirectly is that, you know, the air I breathe, was influenced by Herbert Simon. He had the notion of heuristics and it was in the language and it affected me. Of course, it affected the whole zeitgeist, it affected the whole culture. Retrospectively, when I learned Simon, but that was after I was in the field and after I had made some contributions to the field, I discovered that I was following in his footsteps. You know, that's not what I had been doing originally. I hadn't viewed myself, and in fact, I wasn't following in his footsteps. Retrospectively, you find, oh yeah, this is what I did, you know, in the historical perspective.

And also from classical psychology, either Jung or Piaget, did you draw anything from them, or is that just a foreign stream?

Yeah, just completely foreign.

Completely foreign. If you think about your early work, On vision and on Israeli bus drivers. How did your later work on biases and Thinking Fast, Thinking Slow come out of your very earliest papers?

It didn't. It was a completely separate thing. I worked originally on a concept for quite a few years on the notion of effort, mental effort. And when I started work on heuristics and biases with Amos Tversky, That wasn't on our mind and it had very little effect. When I wrote Thinking, Fast and Slow, like 10 years ago when I was doing that, then it turned out that I put together all my life's work and the early work did get into Thinking, Fast and Slow, but it had no effect on my work with Amos Tversky.

But the idea of attention-switching costs, so Israeli bus drivers, it takes time for them to switch attention from one event to another. Is that not an underlying micro-foundation of your, say, 1980s papers on bias? That people aren't switching their attention to the new problem?

No.

No.

You know, it's not. We didn't think of it. That really happens a great deal, and quite often it happens in a different way. It happens when somebody is insulted because you didn't cite him. And so, you know, he looks at your work and he says, "That's just the same as what I'd said before." But, and in some way it may be true, there may be some resemblance, but it may be true and yet you were completely uninfluenced by that. And it's the same thing. I was uninfluenced by my earlier work, I think.

Now your basic distinction between System 1 and System 2, thinking fast and thinking slow, to the extent that particular results do not replicate, do you view that as undercutting the System 1 versus System 2 distinction, or is that immune?— to the degree of replicability?

Well, I think, you know, that there were whole sets of results that I published in Thinking, Fast and Slow that I wish I hadn't published because they're not reliable. Whether it undercuts the— the idea of two systems is really anchored in a basic sort of fact of experience that the process by which You know, you get 2 2 is fundamentally different from the way that you get 17 × 24. And so one of them happens automatically, associatively, quickly, you have no control. The other demands effort and is slow and so on. That's immune to replication.

But if there's a bias in individuals and noise, why should we trust our experience about this apparent sense of having two methods.

Is it three? Is it four? Well, it is not only— well, in the first place, those are extremes. It doesn't mean that there aren't others. It doesn't mean that there is not a continuum, but there is at least a continuum to be explored of those two extremes. Of that, I'm quite confident.

Do you think that working outside of your native language in any ways influenced your ideas on psychology? It makes you more aware of thinking fast versus thinking slow or not?

It's something I used to think about in the context, you know, I'm from Israel and it was thinking whether there was something in common to Israeli intellectuals operating in a second language. And I thought that in a way it can be an advantage to operate in a second language, that there are certain things that you can think about the thing itself not through the words.

It's like lower sunk costs in a way.

I don't know exactly how to explain it, but I haven't been— I thought that this was not a loss for me to do psychology in a second language.

Do you have thoughts on the potential cognitive advantages of bilingualism or trilingualism?

You know, it's an empirical matter. It's not a matter of thinking, and I don't know enough. It appears to be advantageous, but I don't know the literature.

If we think of therapists, psychiatrists, internists who are trying somehow to fix, improve, or cure people, are they underinvesting in a knowledge of what might be called behavioral economics or your work on psychology? Should they be using more of it? Is that their bias?

I have an opinion on that, and I think it is supported by evidence, but there is one line of therapy that clearly works and it's evidence-based and it's supported time and again, and that is one style and it's cognitive behavioral therapy. That works and we know it does. Other things work and we don't— some of them do, some of them don't, and it primarily seems to depend on the personality of the therapist and on the interaction between the personality of the therapist and the personality of the patient. Whereas cognitive behavioral therapy is a technique, and it's a technique that works. So that's a fact, and the rest is a lot of bias.

In a society such as Argentina that relies so heavily on psychoanalysis, as a psychologist, do you see that as bias?

Is it a placebo?

Is there a placebo effect in psychoanalysis?

You know, you seem to attribute— you seem to think that they think of bias all the time.

I can't imagine why that's my bias.

They don't think of sex all the time. No, I mean, I really don't think of bias that much, but if you want to apply it, then clearly there is a lot of psychoanalysis in Argentina and there's no indication that it makes them more sane.

If you were to express, what is the question about gender and your own work that interests you the most? Maybe you've never done it. But what would that be?

Because I have really never been interested in anything to do with gender. So I have never studied, looked at differences between gender in the kind of research we did. I've never been very interested in individual differences and not in gender either. So I don't know.

So it's the means really that interest you the most.

It's, yeah, it's the means and it's some extremes, but it's not cutting and dicing into categories.

Being an Israeli, and surely you've traveled to many, many countries, at the very least Sweden, right, among others, there are papers on cross-cultural differences in bargaining or in decision biases. How much stock do you put in those results?

Oh, I mean, I think there's no question that there are cultural differences. For one thing, for example, There are major cultural differences in the attitude to optimists, to optimism. I mean, in quite a few European countries, optimism is considered rather foolish. You know, it's for children. In the United States, optimism is clearly a desirable trait. And similarly, there are differences on whether risk-taking is considered a good thing or a bad thing. And so there are certainly cultural differences.

And do you think of those in functionalist terms? So some people might argue, well, Israelis, they have a tendency to speak directly because they've had a lot of crisis situations where you can't beat around the bush, you need to say what you think, or we don't know.

I don't like those kinds of explanations. They look, you know, facile to me.

Right now in psychology, in your own work, what are the open questions you're most interested in?

Well, I mean, like everybody else, I think, like many others, there are two exciting developments now that one would want to know about. I mean, one would want to know how the brain works or to know more about how the brain works than we do. And one would want to know about artificial intelligence and when it, and if, when and how it will become more human-like. In what it can do.

And you're optimistic on that front, or?

I'm optimistic on virtually nothing. But that AI is developing faster than anybody could have anticipated, no question. And so if it continues to develop at that rate, meaning a lot faster than we expect, then things are going to happen relatively quickly.

And what do you think are the main obstacles? So some people in Silicon Valley will argue AI is stuck at a kind of local optimum. Driverless cars, although they're ahead of the pace we thought 10 years ago, they may be behind the pace we thought 2 years ago. There's always a problem with emergency situations, a policeman waving you on. The last 1% maybe is very, very difficult.

Yeah. I mean, I can't evaluate that. That's a technical problem, you know, how long it will take to get to clean up the last 1%. The question that are of my interest as a psychologist is when can you simulate common sense? You know, when, you know, there is the really serious question that people raise about computers is whether they know what they're talking about, you know, whether they understand what they're talking about. And without sense organs and without the perceptual apparatus that we have and the ability to cause things by acting on the world, They can't be exactly like us, so, but that sense of understanding, nobody actually today would, I think, claim that even the most sophisticated programs have it.

And do you think we've learned anything general about common sense by having some artificial intelligence?

What we have learned is that our basic ideas about what's difficult and what's easy, what's going to be simple and what's going to— have undergone a series of sort of revolutionary So we used to think that perception would be easy and thinking would be difficult. Turns out that thinking was relatively easy and perception was difficult. Now there are ways of handling perceptual problems, and so thinking is difficult again. And it's a very interesting, developing thing.

Moving the chess piece is often harder than figuring out the best move for the program. Looking back on your collaboration with Amos Tversky, which has been written about widely, of course, there's the famous Michael Lewis book, but what is there about that collaboration or about Amos that you feel one could read everything that's out there, but still has been underappreciated or undervalued?

I mean, you know, so much has been written that I couldn't point out to anything that people have completely ignored. You know, it was just actually the thing that when I think about him, it was the mental energy, just the joy of thinking and the mental energy. And that made him very charismatic. And he was also very funny. And being funny is a major asset in social life. And it turned out to be a major asset in our work because our work, our joint work, had a touch of irony to it. And the fact that we were laughing continuously as we were doing the work was very important to the nature of what we did.

And that stimulates discovery? It breaks down sunk cost bias? Or what does it do in formal terms?

I mean, what it does is it makes you look for funny things about the— for us, what it did for us, I can't generalize. For us, we were examining our own thinking. And finding stupid things in their own thinking and finding that delightful and very funny. So we were very lucky in our choice of topic in many ways. Our choice of topic lent itself to a lot of things that are virtually impossible in other fields.

And your current collaborators on the Noise book, how would you describe that collaboration and tell us who they are?

Well, one of them is Cass Sunstein, who is a very famous jurist and also known for writing like 3 or 4 books a year. So he writes very easily and I write with difficulty, and so it's not an accident that we teamed up. And the other collaborator is a brilliant Frenchman that you haven't heard of. He was for 25 years at McKinsey and he became a director of McKinsey, and then he got bored with that and he got a PhD and he teaches. Which is just extraordinary. So I'm very lucky.

And what will the main theme of that book be?

Well, it will be that noise is an underestimated problem, and it will be that there is something deep about two ways of thinking that I was working on in Thinking, Fast and Slow, which I called statistical versus causal.. And noise is clearly a statistical way of looking at things, and bias is inherently causal. And so the interplay of those forms of thinking, and then the idea that if you want to reduce noise, we have a pretty good idea of what you should do in order to induce greater uniformity to overcome sort of the vulnerability of people to all sorts of irrelevant influences.

And when will that book be out?

Who knows? It was supposed to be out in the fall of 2020, and I think our publishers just remembered that there is going to be a presidential election at that time and that probably a lot of other more interesting books are going to be appearing. And so that postponed it to spring 2021.

We now have some time for questions, but Daniel Kahneman, thank you very much.

I have my hearing aid on, but if I can't hear the question, you'll repeat.

I will repeat, but people will go up to the mics. There are mics on each side. I will call on you. And please, questions only. This is our chance to hear from Danny Kahneman. If you start making a long speech or statement, I will cut you off. And I also have questions from the iPad, so please get in line if you would like. To start with the questions here, first question: Could prediction markets reduce both bias and noise?

Well, noise certainly, but then averaging does it. And whether prediction markets consistently beat averaging is, I think, not yet fully established. Bias, no. If there is a general bias, unless the people who are unbiased also know that they're unbiased, I mean, unless they have a way of being sure so that they can invest more than others and move the price toward the correct answer. But without that, without this asymmetry of knowledge, If there is a bias, it will not be reduced. Noise will be reduced.

First question over here. Good evening.

I have two questions, but they're short. My first question is, you briefly talked about moral emotions. Do you see any benefit to shame? Because I've read conflicting theories there. So the moral emotion of shame in your thoughts. And two, what is the impact of counterfactual thinking on happiness in your study? Oh, about shame, I really have no idea. You know, it's there. So I don't— should one wish that it weren't? It's probably a force that induces better behavior in lots of people who cannot be, would not be controlled in other ways. So I don't know how important or how useful it is. It's painful to the people. Who feel it, and it might be useful to others who might be affected by bad behavior. As for counterfactuals and happiness, I think that what you refer to— there are counterfactual emotions. Regret is a counterfactual emotion. Guilt is a counterfactual emotion. And you can ask, in the sense that they're driven by something that didn't happen. That could have happened but didn't. And some of these emotions seem to be completely superfluous, like regret. I mean, you know, and I think people by and large would be better off without regret. But it may also be, I mean, notice what regret is. Regret is what happens the next morning. And, uh, and you know, if we didn't have it, then who knows what we might do.

So, uh, next question over here. Yes, sir. On this topic of, uh, delaying intuition, and I'm delighted that Mr. Klein is in the audience because I spent over a decade myself as an intuitive expert and found myself mostly using recognition-primed decision-making. And I'm curious how much you think, uh, availability bias, confirmation bias, et cetera, was still affecting my recognition-primed decision-making? And is recognition-primed decision-making still useful? Is it just the best option in a temporally constrained environment?

Oh, you know, I think obviously recognition-primed decision-making is going to be wonderful if people really can recognize things accurately. So if they can diagnose the situation accurately, and do it quickly and act intuitively on that basis, then of course it's beneficial. And there are conditions under which this applies. Gary Klein and I became friends over a period of 6 years when we were trying to find out what are the boundaries. I mean, I'm sort of a critic of intuition and he is very much in favor of intuition. Of expert intuition, and we were trying to find out what are the boundaries, because it's clear that sometimes intuition is wonderful and sometimes it's awful. And we ended up with a fairly obvious set of conclusions about what it is. You're going to have Gary Klein-type intuition, expert intuition, if you have a regular world,— that's condition number 1— if there are regularities that you can pick up, and if you have a lot of experience, and if the feedback is rapid and unequivocal, and if you have those 3 conditions, which are true for chess players and they're true for spouses recognizing the emotion of their spouse on the telephone, to give you a completely different example, then intuition will develop and it will be perfect. If those conditions do not develop, I don't think that we can trust people who say that they're experts.

Another iPad question. Tech entrepreneur Daniel Gross suggested that growing up in Israel was a forcing function for the tech sector. How much was Israel a forcing function for your thinking?

I don't really completely understand the term forcing function in this context. I know that Israel afforded many opportunities when I was growing up, and it probably still does. I grew up very early in the history of Israel when the state was small and everyone could make a difference and you really could make a difference. I mean, I was as a lieutenant in the army, age 21 or 22, I made a difference. I created an interviewing system for the whole army. So those kinds of experiences that you can do things and that seemed impossible or unlikely, that is certainly very liberating and encouraging and induces creativity. And I think some of that is actually present now that the state is bigger and more established. I see it, I think I was telling you earlier, how my grandson in the Israeli army, the kinds experiences that he has as a sergeant. He feels very free, you know, in an intelligence unit. He feels that he can use his mind and he can speak his mind, and it's going to be wonderful for his future.

Next question over here. You mentioned earlier that you view many things in the world through a basic lens of pessimism, but if you were going to challenge yourself to identify something going on in the world now now or in the near term about which to be optimistic? If not for yourself, for your grandchildren, say, or very young people, what should they be optimistic about?

I'm going to pass.

Over here.

I'm curious about what beliefs you currently hold that you think in the next 5 to 10 years might be proven incorrect, and alternatively, the same question of social science broadly. Well, you know, if I knew how I would change my mind, I would have changed my mind. So my guess is that there will be completely different frameworks. There will be different ways of thinking. It's not going to be this or that detail. It's going to be, and this is what happens to ideas or to frameworks, they at some point become irrelevant. This is, you know, and I know that this is going to happen to everything that I believe, that, you know, give it a few decades, it's going to be irrelevant. And I wish I, you know, I wish I could peer into the future and know what comes next, but, you know, I can't.

From the iPad: Why did the replication crisis take so long to arrive in social psychology?

I would question that. It didn't take very long. I mean, the replication crisis started, was studied first in medicine and where there were provocative claims by Stanford. I don't know, he's not a statistician, is he? There were provocative claims that most published research in medicine are false and it started there. And then psychology came very soon after. I mean, and in fact, psychology is considered to have been quite rapid in adopting it. And there was a crisis and many results were questioned, I think correctly. There were aggressors and there were defenders and both sides, I think, behaved quite often quite badly. But the result in psychology, and it's amazing, within a decade Psychology has changed. Many areas of psychology have changed, and it clearly is a better science than it was 10 years ago because of the replication crisis.

Next question.

I have two questions. The first one is the one that you just answered, not exactly the same. It's what do you think about the replication crisis in psychology that is happening recently? And the second one is the psychologist Martin Seligman, who is also working on happiness for years, and he believes there are several dimensions that consist happiness. Do you share the same opinion or not?

The second question was Martin Seligman, his work on happiness, that there are several dimensions of happiness. Do you share his opinion or not? And the first was just more on the replication crisis. No. Next question over here.

I have a question about bounded rationality over time.

With the rise of the internet, the rise of more readily available information, you have so many prices, things that you can see on Amazon, all this price discrimination and differentiation across products that's grown with that. Do you think that people's biases are improving or getting worse over time as more information, for example, over the past 20, 30 years has become more readily available?

Well, I mean, to the extent that you think of biases as representing human nature in a broad cultural context, it hasn't changed over the last 30 years. Human nature hasn't changed. In certain domains, it's much easier to be rational when you can look things up. So when you can search on the computer instead of going out and searching as you had to when I was a young person, then of course you can achieve more rational results than you could. But whether it has changed anything significant, I doubt it. And what is very striking over the last few years is that it's not only information that is readily available, it's misinformation is also readily available. So the net effect, you know, it used to be very clear that this is all to the good, but what we're seeing in the last few years is that there is a very heavy cost to the availability and the ease of expression that transmits itself over the internet.

Next question. I'm aware that I suffer from biases and I try to hold myself to account and think better. And I'm resistant to that, of course, because I always want to think that whatever thought I'm having at the moment is the exception and I'm thinking it for good reasons. But I fight against that. When I'm trying to persuade somebody else to listen to one of my opinions with an open mind, is there some particular technique that you would recommend for persuading other people to do battle with their own biases? Because of course they're even more resistant to that than I am when I challenge myself.

Uh, you know, it's a game one primarily plays with one's spouse, and, uh, I —and it doesn't work, I think, by and large.

Related question from the iPad: How can we use behavioral economics to reduce political polarization?

It's not that I have an answer and I'm suppressing it. Here is a topic where I'm optimistic, but I have no idea. You know, I don't have an answer. But I think the kind of thinking that is going on where you're trying to look at practical practical manipulations that can be— the word manipulation is a bad word, but I intend it as a good thing. When you look at practical moves that can make a difference in the way that people think, that way of thinking should be effective in improving the quality of life and improving the quality of How polarization can be reduced is too big a problem for me and I think currently for behavioral economics. Next question.

On a practical note, my high school psychology students ask how they can best use your research to make choices about college and career.

I mean, that's not— my research adds absolutely nothing to this. There are sensible ways of choosing colleges and I think they're well known and you have to collect a lot of information and you have ask yourself what the student really wants and where, you know, he or she will really fit. And so there are obvious ways of doing this.

I have nothing to add. But say you have a student who has a gut feeling that he or she ought to go to some college for a reason he or she cannot articulate. Are you telling us they should dismiss that feeling and defer to the algorithm?

As a general rule, I would try to probe and understand, you know, why, where does that feeling come from? I mean, I don't think— are you asking me as a parent, say, or as an imagined parent? I would really probe. I would feel free. If that's a very expensive college and strong— I think I would feel free to probe. Where does that strong wish come from? And can we discuss it? Next question. So many behavioral economists use the notion of rationality in neoclassical economics as a normative benchmark., and you have said that you don't think that's necessarily a good normative benchmark, and instead something like reasonableness is a better way to think about those things. Could you say more about how we might identify or define and identify this reasonableness? The rational agent models are built on the notion of consistency being the one guiding principle. So your beliefs and your preferences have to be internally consistent. Nobody can tell you what to believe. Nobody can tell you what you want, but your beliefs and— the only thing we know is that you ought to be consistent, otherwise you're not rational. You know, that's, as a normative principle, that consistency is the only normative principle, that strikes me as pretty odd. There are other things that seem to matter. There is human nature, and human nature is not consistent. I mean, we are context-dependent, so our emotions are context-dependent. We ought to have normative theories that are adapted to who we are as people, as humans, and the idea of consistency is an infeasible— it's completely unfeasible for finite mind, and we have finite minds. So on that ground alone, it would be questionable as the principle for a normative model. But one would want a normative theory that takes into account human nature, which the principle of consistency doesn't. Next question over here.

So you talked a little bit about cultural differences. I was just wondering, do you think that there are like some cultural aspects that are costly and some that are really good? And do you think that something like increased migration or open borders would kind of push— would dissolve these cultural differences and push toward like a more optimal equilibrium?

Way beyond what I can talk about, you know, responsibly. You know, that migration— automatically causes, you know, cultural amalgamation, you know, that's questionable. I have no idea how to answer your question.

Next question. Over on this side.

We've run out. Daniel, thank you very much.

It's been a great honor to have you. Thanks for listening to Conversations with Tyler. Please don't forget to visit conversationswithtyler.com/donate to support the show before the year's end.