Daniel Kahneman: Thinking Fast and Slow, Deep Learning, and AI | Lex Fridman Podcast #65

The following is a conversation with Daniel Kahneman, winner of the Nobel Prize in Economics for his integration of economic science with the psychology of human behavior, judgment, and decision making. He's the author of the popular book Thinking Fast and Slow that summarizes in an accessible way his research of several decades, often in collaboration with Amos Tversky, on cognitive biases, prospect theory, and happiness. The central thesis of this work is the dichotomy between two modes of thought. What he calls System one is fast, instinctive, and emotional. System two is slower, more deliberative, and more logical. The book delineates cognitive biases associated with each of these two types of thinking. His study of the human mind and its peculiar and fascinating limitations are both instructive and and inspiring for those of us seeking to engineer intelligent systems. This is the Artificial Intelligence Podcast.

If you enjoy it, subscribe on YouTube.

Give it five stars on Apple Podcast, follow on Spotify, support it on Patreon, or simply connect with me on Twitter lexfriedman Spelled F R I D M A N I recently started doing ads at the end of the introduction. I'll do one or two minutes after introducing the episode and and never any ads in the middle that can break the flow of the conversation. I hope that works for you and doesn't hurt the listening experience. This show is presented by Cash App, the number one finance app in the App Store. I personally use Cash App to send money to friends, but you can also use it to buy, sell and deposit Bitcoin in just seconds. Cash App also has a new investing feature. You can buy fractions of a stock, say $1 worth no matter what the stock price is. Brokers services are provided by Cash App Investing, a subsidiary of Square and member sipc. I'm excited to be working with Cash App to support one of my favorite organizations called First, Best known for their.

First Robotics and Lego competitions.

They educate and inspire hundreds of thousands of students in over 110 countries and have a perfect rating at Charity Navigator, which means that donated money is used to maximum efficiency effectiveness. When you get Cash App from the App Store, Google Play and use code lexpodcast, you'll get $10 and cash app will also donate $10 to first, which again is an organization that I've personally seen inspire girls and boys to dream of engineering a better world. And now here's my conversation with Daniel Kahneman.

You tell a story of an SS soldier early in the war, World War II in Nazi occupied France and Paris where you grew up. He picked you up and hugged you and Showed you a picture of a boy, maybe not realizing that you were Jewish?

Not maybe. Certainly not.

So I told you I'm from the Soviet Union. That was significantly impacted by the war as well, and I'm Jewish as well. What do you think World War II taught us about human psychology broadly?

Well, I think the. The only big surprise is the extermination policy. Genocide by the German people. That's when you look back on it and, you know, I think that's a major surprise.

It's a surprise because it's a surprise.

That they could do it. It's a surprise that they. That enough people willingly participated in that. This is a surprise now. It's no longer a surprise, but it's changed many people's views, I think, about human beings. Certainly for me, the Eichmann trial teaches you something, because it's very clear that if it could happen in Germany, it could happen anywhere. It's not that the Germans were special. This could happen anywhere.

So what do you think that is? Do you think we're all capable of evil? We're all capable of cruelty?

I don't think in those terms. I think that what is certainly possible is you can dehumanize people so that you treat them not as people anymore, but as animals. And the same way that you can slaughter animals without feeling much of anything, it can the same. And when you feel that, I think the combination of dehumanizing the other side and having uncontrolled power over other people, I think that doesn't bring out the most generous aspect of human nature. So that Nazi soldier, you know, he was a good man. I mean, you know, and he was perfectly capable of killing a lot of people, and I'm sure he did.

But what did the Jewish people mean to Nazis? So what the dismissal of Jewish as worthy of.

Again, this is surprising that it was so extreme. But it's not one thing in human nature, I don't want to call it evil, but the distinction between the in group and the out group, that is very basic. So that's built in the loyalty and affection towards in group and the willingness to dehumanize the out group. That is in human nature. And that's what I think probably didn't need the Holocaust to teach us that. But the Holocaust is a very sharp lesson of what can happen to people, what people can do.

So the effect of the in group and the out group, you know, it's.

Clear that those were people. You know, you could. You could shoot them. You could. You know, they were not Human. There was no empathy or very, very little empathy left. So occasionally, you know, there might have been. And very quickly, by the way, the empathy disappeared. If there was initially, and the fact that everybody around you was doing it, that completely, the group doing it and everybody shooting Jews, I think that makes it permissible.

Now.

How much? Whether it could happen in every culture or whether the Germans were just particularly efficient and disciplined so they could get away with it.

That is a question.

It's an interesting question.

Are these artifacts of history or is it human nature?

I think that's really human nature. You know, you put some people in a position of power relative to other people and. And then they become less human, they become different.

But in general, in war, outside of concentration camps In World War II, it seems that war brings out darker sides of human nature. But also the beautiful things about human nature.

Well, you know, I mean, what it brings out is the loyalty among soldiers. I mean, it brings out the bonding. Male bonding, I think, is a very real thing that happens. And there is a certain thrill to friendship and there is certainly a certain thrill to friendship under risk and to shared risk. And so people have very profound emotions up to the point where it gets so traumatic that little is left. But.

So let's talk about psychology a little bit. In your book Thinking Fast and Slow, you describe two modes of thought. System one, the fast, instinctive and emotional one. And System two, the slower, deliberate, logical one. At the risk of asking Darwin to discuss the theory of evolution, can you describe distinguishing characteristics for people who have not read your book of the two systems?

Well, I mean, the word system is a bit misleading, but it's at the same time it's misleading. It's also very useful. But what I call System one, it's easier to think of it as a family of activities. And primarily the way I describe it is there are different ways for ideas to come to mind. And some ideas come to mind automatically. And the example, standard example is two plus two. And then something happens to you. And in other cases you've got to do something, you've got to work in order to produce the idea. My example, I always give the same pair of numbers as 27 times 14.

I think you have to perform some algorithm in your head, some steps.

Yes, and it takes time. It's a very different. Nothing comes to mind except something comes to mind, which is the algorithm. I mean, that you've got to perform. And then it's work. And it engages short term memory and it engages executive function and it makes you incapable of doing other things at the same time. So the main characteristic of System two, that there is mental effort involved and there is a limited capacity for mental effort, whereas System one is effortless, essentially. That's the major distinction.

So you talk about there. You know, it's really convenient to talk about two systems, but you also mentioned just now and in general, that there is no distinct two systems in the brain from a neurobiological, even from psychology perspective. But why does it seem to. From the experiments you've conducted, there does seem to be kind of emergent two modes of thinking. So at some point these kinds of systems came into a brain architecture. Maybe mammals share it, but. Or do you not think of it at all in those terms, that it's all a mush and these two things just emerge?

You know, evolutionary theorizing about this is cheap and easy. So it's. The way I think about it is that it's very clear that animals have a perceptual system, and that includes an ability to understand the world, at least to the extent that they can predict. They can't explain anything, but they can anticipate what's going to happen. And that's a key form of understanding the world. And my crude idea is that what I call System two, well, System two grew out of this. And there is language and there is the capacity of manipulating ideas and the capacity of imagining futures and of imagining counterfactual things that haven't happened. And. And to do conditional thinking. And there are really a lot of abilities that without language and without the very large brain that we have compared to others would be impossible. Now, System one is more like what the animals are. But System One also can talk. I mean, it has language, it understands language. Indeed, it speaks for us. I mean, you know, I'm not choosing every word as a deliberate process. The words, I have some idea, and then the words come out, and that's automatic and effortless.

And many of the experiments you've done is to show that, listen, System One exists and it does speak for us. And we should be careful about the.

Voice it provides because, well, I mean.

You know, we have to trust it because it's the speed at which it acts at System two. If we dependent on System two for survival, we wouldn't survive very long. Because it's very slow.

Yeah, crossing the street.

Crossing the street. I mean, many things depend on their being automatic. One very important aspect of System One is that it's not instinctive. You use the word instinctive. It contains skills that clearly have been Learned so that skilled behavior, like driving a car or speaking in fact, skilled behavior has to be learned. And so it doesn't, you know, you don't come equipped with driving. You have to learn how to drive and you have to go through a period where driving is not automatic before it becomes automatic.

So yeah, you construct, I mean this is where you talk about heuristic and biases is you to make it automatic, you create a pattern and then system one essentially matches a new experience against a previously seen pattern. And when that match is not a good one, that's when the cognitive, all the, all the mess happens. But it's most of the time it works. And so it's pretty.

Most of the time the anticipation of what's going to happen next is correct. And most of the time the plan about what you have to do is correct. And so most of the time everything works just fine. What's interesting actually is that in some sense system one is much better at what it does and system two is at what it does. That is there is this quality of effortlessly solving enormously complicated problems which clearly exists so that the chess player, a very good chess player, all the moves that come to their mind are strong moves. So all the selection of strong moves happens unconsciously and automatically and very, very fast. And all that is in system one. So system two verifies.

So along this line of thinking, really what we are are machines that construct pretty effective system one. You could think of it that way. So we're now talking about humans, but if we think about building artificial intelligence systems, robots, do you think all the features and bugs that you have highlighted in human beings are useful for constructing AI systems? So both systems are useful for perhaps.

Instilling in robots what is happening these days is that actually what is happening in deep learning is more like a system one product than like a System two product. I mean deep learning matches patterns and anticipate what's going to happen. So it's highly predictive what deep learning. And you know, many people think that this is a critical. It doesn't have the ability to reason. So there is no system to there. But I think very importantly, it doesn't have any causality or any way to represent meaning and to represent real interaction. So until that is solved, you know, what can be accomplished is marvelous and very exciting, but limited.

That's actually really nice to think of. Current advances in machine learning is essentially system one advances. So how far can we get with just system one? If we think of deep learning and artificial intelligence systems, I Mean, you know.

It'S very clear that DeepMind has already gone way, way beyond what people thought was possible. I think, I think the thing that has impressed me most about the developments in AI is the speed. It's that things, at least in the context of deep learning, and maybe this is about to slow down, but things moved a lot faster than anticipated. The transition from solving chess to solving Go was. I mean, that's bewildering how quickly it went. The move from AlphaGo to AlphaZero is sort of bewildering, the speed at which they accomplished that. Now, clearly there, there are. So there are many problems that you can solve that way, but there are some problems for which you need something else.

Something like reasoning.

Well, reasoning and also, you know, the. One of the real mysteries. Psychologist Gary Marcus, who is also a critic of AI I mean, what he points out, and I think he has a point, is that humans learn quickly. Children don't need a million examples, they need two or three examples. So clearly there is a fundamental difference. And what enables a machine to learn quickly what you have to build into the machine, because it's clear that you have to build some expectations or something in the machine to make it ready to learn quickly. That at the moment seems to be unsolved. I'm pretty sure that DeepMind is working on it, but if they have solved it, I haven't heard yet.

They're trying to actually, them and OpenAI are trying to, to start to get. To use neural networks to reason. So assemble knowledge. Of course, causality is temporal. Causality is out of reach to most everybody. You mentioned the benefits of system one is essentially that it's fast, allows us to function in the world.

Fast and skilled. Yeah, it's skill and it has a model of the world. You know, in a sense, I mean, there was the early phase of AI attempted to model reasoning and they were moderately successful. But reasoning by itself doesn't get you much. Deep learning has been much more successful in terms of what they can do. But now it's an interesting question whether it's approaching its limits. What do you think?

I think absolutely. So I just talked to Kean LeCun. He mentioned, you know, I know him. So he thinks that the limits, we're not going to hit the limits with neural networks that ultimately this kind of System one pattern matching will start to start to look like System 2 without significant transformation of the architecture. So I'm more with the. But the majority of the people who think that, yes, neural networks will hit a limit in Their capability.

On the one hand, I have heard him tell Demi Sasabis essentially that, you know, what they have accomplished is not a big deal, that they have just touched, that basically they can't do unsupervised learning in an effective way. But you are telling me that he thinks that the current, within the current architecture, you can do causality and reasoning.

So he's very much a pragmatist in a sense. That's saying that we're very far away, that there's still. Yeah, I think there's this idea that he says is we can only see one or two mountain peaks ahead and there might be either a few more after or thousands more after. Yeah, so that kind of idea.

I heard that metaphor.

Yeah, right. But nevertheless it doesn't see the final answer, not fundamentally looking like one that we currently have. So neural networks being a huge part of that.

Yeah, I mean, that's very likely because. Because pattern matching is so much of what's going on.

And you can think of neural networks as processing information sequentially.

Yeah, I mean, you know, there is an important aspect to. For example, you get systems that translate and they do a very good job, but they really don't know what they're talking about. And for that I'm really quite surprised. For that you would need an AI that has sensation, an AI that is in touch with the world.

Yes. Self awareness and maybe even something resembles consciousness, kind of ideas.

The awareness of, you know, awareness of what's going on so that the, the words have meaning or can get. Are in touch with some perception or some action.

Yeah, so that's a big thing for Jan and is what he refers to as grounding to the physical space. So.

So that's what. Talking about the same thing.

Yeah. So, but so how you ground, I.

Mean, the grounding without grounding, then you get a machine that doesn't know what it's talking about because it is talking about the world.

Ultimately, the question, the open question, is what it means to ground. I mean, we're very human centric in our thinking, but what does it mean for a machine to understand what it means to be in this world? Does it need to have a body? Does it need to have a finiteness? Like we humans have all of these elements? It's, it's a very, it's an open question.

You know, I'm not sure about having a body, but having a perceptual system. Having a body would be very helpful too. I mean, if, if you think about human mimicking humano. But having a perception that seems to be essential so that you can build, you can accumulate knowledge about the world. So if you can imagine a human completely paralyzed, and there's a lot that the human brain could learn, you know, with a paralyzed body. So if we got a machine that could do that, that would be a big deal.

And then the flip side of that, something you see in children and something in machine learning world is called active learning. Maybe it is also is being able to play with the world. How important for developing system one or system two, do you think it is to play with the world, to be able to interact with a lot of.

What you learn as you learn to anticipate the outcomes of your actions? I mean, you can see that how babies learn it, you know, with their hands, how they learn, you know, to connect, you know, the movements of their hands with something that clearly is something that happens in the brain and the ability of the brain to learn new patterns. So, you know, it's the kind of thing that you get with artificial limbs that you could.