Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI | Lex Fridman Podcast #43

The following is a conversation with Gary Marcus. He's a professor emeritus at nyu, founder of Robust AI and Geometric Intelligence. The latter is a machine learning company that was acquired by Uber in 2016. He's the author of several books on natural and artificial intelligence, including his new book Rebooting AI Building Machines We Can Trust. Gary has been a critical voice highlighting the limits of deep learning and AI in general, and discussing the the challenges before our AI community that must be solved in order to achieve artificial general intelligence. As I'm having these conversations, I try to find paths toward insight towards new ideas. I try to have no ego in the process. It gets in the way. I'll often continuously try on several hats several roles. One, for example, is the role of a three year old who understands very little about anything and asks big what and why questions. The other might be a role of a devil's advocate who presents counter ideas with a goal of arriving a greater understanding through debate. Hopefully both are useful, interesting, and even entertaining at times. I ask for your patience as I learn to have better conversations. This is the Artificial Intelligence podcast. If you enjoy it, subscribe on YouTube, give it five stars on iTunes, subscribe support it on Patreon, or simply connect with me on Twitter lexfridman, spelled F R I D M A N. And now here's my conversation with Gary Marcus.

Do you think human civilization will one day have to face an AI driven technological singularity that will in a societal way modify our place in the food chain of intelligent living beings on this planet?

I think our place in the food chain has already changed. So there are lots of things people used to do by hand that they do with machine. If you think of a singularity as like one single moment, which is I guess what it suggests, I don't know if it'll be like that, but I think that there's a lot of gradual change and AI is getting better and better. I mean, I'm here to tell you why I think it's not nearly as good as people think, but the overall trend is clear. Maybe, maybe Ray Kurzweil thinks it's an exponential and I think it's linear. In some cases it's close to zero right now. But it's all going to happen. I mean we are going to get to human level intelligence or whatever you want what you will artificial general intelligence at some point. And that's certainly going to change our place in the food chain because a lot of the tedious things that we do now we're going to have machines do And a lot of the dangerous things that we do now we're going to have machines do. I think our whole lives are going to change from people finding their meaning through their work, through people finding their meaning through creative expression.

So the singularity will be a very gradual, in fact, removing the meaning of the word singularity. It'll be a very gradual transformation. In your view?

I think that it'll be somewhere in between. And I guess it depends what you mean by gradual incitement. I don't think it's going to be one day. I think it's important to realize that intelligence is a multidimensional variable. So people sort of write this stuff as if like IQ was one number and the day that you hit 262 or whatever, you displace the human beings. And really there's lots of facets to intelligence. So there's verbal intelligence and there's motor intelligence and there's mathematical intelligence and so forth. Machines in their mathematical intelligence far exceed most people already in their ability to play games. They far exceed most people already in their ability to understand language. They lag behind my five year old, far behind my five year old. So there are some facets of intelligence the machines have grasped and some that they haven't. And we have a lot of work left to do to get them to, say, understand natural language or to understand how to flexibly approach some kind of novel, MacGyver problem solving kind of situation. And I don't know that all of these things will come at once. I think there are certain vital prerequisites that we're missing now. So for example, machines don't really have common sense now. So they don't understand that bottles contain water and that people drink water to quench their thirst and that they don't want dehydrate. They don't know these basic facts about human beings. And I think that that's a rate limiting step for many things. It's a rate limiting step for reading, for example, because stories depend on things like, oh my God, that person's running out of water, that's why they did this thing. Or if only they had water, they could put out the fire. So you watch a movie and your knowledge about how things work matter. And so a computer can't understand that movie if it doesn't have that background knowledge. Same thing if you read a book. And so there are lots of places where if we had a good machine interpretable set of common sense, many things would accelerate relatively quickly. But I don't think Even that is like a single point. There's many different aspects of knowledge. And we might, for example, find that we make a lot of progress on physical reasoning, getting machines to understand, for example, how keys fit into locks or that kind of stuff, or how this gadget here works and so forth and so on. Machines might do that long before they do really good psychological reasoning because it's easier to get kind of labeled data or to do direct experimentation on a microphone stand than it is to do direct experimentation on human beings to understand the levers that guide them.

That's a really interesting point, actually, whether it's easier to gain common sense knowledge or psychological knowledge.

I would say that common sense knowledge includes both physical knowledge and psychological knowledge.

And the argument I was physical versus psychological.

Yeah, physical versus psychological I was making is physical knowledge might be more accessible because you could have a robot, for example, lift a bottle, try putting a bottle cap on it, see that it falls off if it does this. And see that it could turn it upside down. And so the robot could do some experimentation. We do some of our psychological reasoning by looking at our own minds. So I can sort of guess how you might react to something based on how I think I would react to it. And robots don't have that intuition. And they also can't do experiments on people in the same way, or we'll probably shut them down. So if we wanted to have RO figure out how I respond to pain by pinching me in different ways, like, that's probably, you know, it's not going to make it past the human subjects board, and, you know, companies are going to get sued or whatever. So, like, there's certain kinds of practical experience that are limited or off limits to robots.

That's a really interesting point. What is more difficult to gain a grounding in? Because to play devil's advocate, I would say that human behavior is easier expressed in data in digital form. And so when you look at Facebook algorithms, they get to observe human behavior, so you get to study and manipulate even a human behavior in a way that you perhaps cannot study or manipulate the physical world. So it's true why you said pain is like physical pain, but that's again, the physical world. Emotional pain might be much easier to experiment with, perhaps unethical, but nevertheless, some would argue it's already going on.

I think that you're right, for example, that Facebook does a lot of experimentation in psychological reasoning. In fact, Zuckerberg talked about AI at a talk that he gave nips. I wasn't there, but the conference has been renamed Norips, but he used to be called Nips when he gave the talk. And he talked about Facebook basically having a gigantic theory of mind. So I think it is certainly possible Facebook does some of that. I think they have a really good idea of how to addict people to things. They understand what draws people back to things, and I think they exploit it in ways that I'm not very comfortable with. But even so, I think that there are only some slices of human experience that they can access through the kind of interface they have. And of course, they're doing all kinds of VR stuff, and maybe that'll change and they'll expand their data, and I'm sure that that's part of their goal. So it is an interesting question.

I think love, fear, insecurity, all of the things that I would say, some of the deepest things about human nature and the human mind could be explored to digital form. You're actually the first person just now that brought up. I wonder what is more difficult, because I think folks who are the slow and we'll talk a lot about deep learning, but the people who are thinking beyond deep learning are thinking about the physical world. You're starting to think about robotics in the home, robotics. How do we make robots manipulate objects, which requires an understanding of the physical world, and it requires common sense reasoning. And that has felt to be like the next step for common sense reasoning. But you've now brought up the idea that there's also the emotional part. And it's interesting whether that's hard or easy.

I think some parts of it are and some aren't. So my company that I recently founded with Rod Brooks from MIT for many years and Roomba and so forth, we're interested in both. We're interested in physical reasoning and psychological reasoning, among many other things. And there are pieces of each of these that are accessible. So if you want a robot to figure out whether it can fit under a table, that's a relatively accessible piece of physical reasoning. If you know the height of the table and you know the height of the robot, it's not that hard. If you wanted to do physical reasoning about Jenga, it gets a little bit more complicated and you have to have, you know, higher resolution data in order to do it. With psychological reasoning, it's not that hard to know, for example, that people have goals and they like to act on those goals, but it's really hard to know exactly what those goals are.

But ideas of frustration, I mean, you could argue it's extremely difficult to understand the Sources of human frustration as they're playing Jenga with you or not. You could argue that it's very accessible.

There's some things that are going to be obvious and some not. I don't think anybody really can do this well yet, but I think it's not inconceivable to imagine machines in the not so distant future being able to understand that if people lose in a game that they don't like, that that's not such a hard thing to program. And it's pretty consistent across people. Most people don't enjoy losing, and so that makes it, you know, relatively easy to code. On the other hand, if you wanted to capture everything about frustration. Well, people get frustrated for a lot of different reasons. They might get sexually frustrated, they might get frustrated, they can get their promotion at work, all kinds of different things. And the more you expand the scope, the harder it is for anything like the existing techniques to really do that.

So I'm talking to Garry Kasparov next week, and he seemed pretty frustrated with his game against T Blue.

Yeah, well, I'm frustrated with my game against him last year because I played him. I had two excuses. I'll give you my excuses up front, but it won't mitigate the outcome. I was jet lagged and I hadn't played in 25 or 30 years, but the outcome is he completely destroyed me and it wasn't even close.

Have you ever been beaten in any board game by a machine?

I have. I actually beat or played the predecessor to Deep Blue, Deep Thought I believe it was called, and that too crushed me.

And after that, you realize it's over for us.

There's no point in my playing Deep Blue. I mean, it's a waste of Deep Blue's computation. I mean, I played Kasparov because we both gave lectures at this same event and he was playing 30 people. I forgot to mention that not only did he crush me, but he crushed 29 other people at the same time.

I mean, but the actual philosophical and emotional experience of being beaten by a machine, I imagine, is. I mean, to you, who thinks about these things may be a profound experience or. No, it was a simple mathematical experience.

Yeah. I think a game like chess, particularly where you have perfect information, it's two player closed end and there's more computation for the computer. It's no surprise the machine wins. I mean, I'm not sad when a computer. I'm not sad when a computer calculates a cube root faster than me. I know I can't win that game. I'M not going to try.

Well, with a system like AlphaGo or.

AlphaZero, do you see a little bit.

More magic in a system like that, even though it's simply playing a board game, but because there's a strong learning component?

I find you should mention that in the context of this conversation, because Kasparov and I are working on an article that's going to be called AI is Not Magic. And neither one of us thinks that it's magic. And part of the point of this article is that AI is actually a grab bag of different techniques, and some of them have, or they each have their own unique strengths and weaknesses. So you read media accounts and it's like, ooh, AI. It must be magical or it can solve any problem. Well, no, some problems are really accessible, like Jess and Go, and other problems like reading are completely outside the current technology. And it's not like you can take the technology that drives AlphaGo and apply it to reading and get anywhere. DeepMind has tried that a bit. They have all kinds of resources. They built AlphaGo and I wrote a piece recently. They lost. And you can argue about the word lost, but they spent $530 million more than they made last year. So they're making huge investments. They have a large budget and they have applied the same kinds of techniques to reading or to language. It's just much less productive there because it's a fundamentally different kind of problem. Chess and Go and so forth are closed end problems. The rules haven't changed in 2,500 years. There's only so many moves you can make. You can talk about the exponential as you look at the combinations of moves, but fundamentally, the go board has 361 squares. That's it. Those intersections are the only places that you can place your stone, whereas when you're reading the next sentence could be anything. It's completely up to the writer what they're going to do next.

That's fascinating that you think this way. You're clearly a brilliant mind who points out the emperor has no clothes. So I'll play the role of a person who says, you're going to put.

Clothes on the emperor. Good luck with it.

It romanticizes the notion of the emperor period, suggesting that clothes don't even matter. Okay, so that's really interesting that you're talking about language. So there's the physical world of being able to move about the world, making an omelet and coffee and so on. There's language where you first understand what's being written and then Maybe even more complicated than that, having a natural dialogue. And then there's the game of go and chess. I would argue that language is much closer to go than it is to the physical world. It is still very constrained. When you say the possibility of the number of sentences that could come, it is huge, but it nevertheless is much more constrained. It feels maybe I'm wrong than the possibilities that the physical world brings us.

There's something to what you say in some ways in which I disagree. So one interesting thing about language is that it abstracts away. This bottle, I don't know if it'll be in the field of view is on this table. And I use the word on here. And I can use the word on here, maybe not here, but there's all that one word encompasses in analog space, a sort of infinite number of possibilities. So there is a way in which language filters down the variation of the world, and there's other ways. So we have a grammar, and more or less, you have to follow the rules of that grammar. You can break them a little bit, but by and large, we follow the rules of grammar. And so that's a constraint on language. So there are ways in which language is a constrained system. On the other hand, there are many arguments that say there's an infinite number of possible sentences. And you can establish that by just stacking them up. So I think there's water on the table. You think that. I think there's water on the table. Your mother thinks that. You think that. I think that water is on the table. Your brother thinks that maybe your mom is wrong to think that. You think that. I think right. We can make it sentences of infinite length, or we can stack up adjectives. This is a very silly example. A very, very silly example. A very, very, very, very, very, very silly example, and so forth. They're good arguments that there's an infinite range of sentences in any case, it's vast by any reasonable measure. And for example, almost anything in the physical world we can talk about in the language world. And interestingly, many of the sentences that we understand we can only understand if we have a very rich model of the physical world. So I don't ultimately want to adjudicate the debate that I think you just set up, but I find it interesting. Maybe the physical world is even more complicated than language. I think that's fair.

But you think that language is really, really complicated.

It's hard. It's really, really hard. Well, it's really, really hard for machines, for linguists, people trying to understand it. It's not that hard for children. And that's part of what's driven my whole career. I was a student of Stephen Pinker's and we were trying to figure out why kids could learn language when machines couldn't.

I think we're going to get into language, we're going to get into communication, intelligence and neural networks and so on. But let me return to the high level, the futuristic, for a brief moment. So you've written in your book, in your new book, it would be arrogant to suppose that we could forecast where AI will be, where the impact will have in a thousand years or even 500 years. So let me ask you to be arrogant. What do AI systems, with or without physical bodies look like 100 years from now? You can't predict. But if you were to philosophize and imagine.

Can I first justify the arrogance before you try to push me beyond it? Sure. I mean, there are examples like people figured out how electricity worked. They had no idea that that was going to lead to cell phones. Right. I mean, things can move awfully fast once new technologies are perfected. Even when they made transistors, they weren't really thinking that cell phones would lead to social networking.

There are nevertheless predictions of the future which are statistically unlikely to come to be. But nevertheless is the best.

Asking me to be wrong?

Asking you to be successful?

In which way would I like to be wrong?

Pick the least unlikely to be wrong thing, even though it's most very likely to be wrong.

I mean, here's some things that we can safely predict. I suppose we can predict that AI will be faster than it is now. It will be cheaper than it is now. It will be better in the sense of being more general and applicable in more places. It will be pervasive. I mean, these are easy predictions. I'm sort of modeling them in my head. On Jeff Bezos famous predictions, he says, I can't predict the future. Not in every way I'm paraphrasing, but I can predict that people will never want to pay more money for their stuff. They're never going to want it to take longer to get there. You can't predict everything, but you can predict some things. Sure, of course it's going to be faster and better. We can't really predict is the full scope of where AI will be in a certain period. I think it's safe to say that although I'm very skeptical about current AI, that it's possible to do much better. There's no, in principle, an argument that says AI is an insolvable problem, that there's magic inside our brains that will never be captured. I mean, I've heard people make those kind of arguments. I don't think they're very good. So AI is going to come and probably 500 years of planning to get there. And then once it's here, it really will change everything.

So when you say AI is going to come, are you talking about human level intelligence?

I like the term general intelligence. I don't think that the ultimate AI, if there is such a thing, is going to look just like humans. I think it's going to do some things that humans do better than current machines, like reason flexibly and understand language and so forth. But that doesn't mean they have to be identical to humans. So for example, humans have terrible memory and they suffer from what some people call motivated reasoning. So they like arguments that seem to support them and they dismiss arguments that they don't like. There's no reason that a machine should ever do that.

So you see the limitations of memory as a bug, not a feature.

Absolutely. I'll say two things about that. One is I was on a panel with Danny Kahneman, the Nobel Prize winner last night, and we were talking about this stuff and I think what we converged on is that humans are a low bar to exceed. They may be outside of our skill right now as AI programmers, but eventually AI will exceed it. So we're not talking about human level AI, we're talking about general intelligence that can do all kinds of different things. Things and do it without some of the flaws that human beings have. The other thing I'll say is I wrote a whole book actually about flaws of humans. It's actually a nice bookend to the counterpoint to the current book. So I wrote a book called Kludge, which was about the limits of the human mind. Current book is kind of about those few things that humans do a lot better than machines.

Do you think it's possible that the flaws of the human mind, the limits of memory, our mortality, our bias is a strength, not a weakness. That is the thing that enables from which motivation springs and meaning springs.

I've heard a lot of arguments like this. I've never found them that convincing. I think that there's a lot of making lemonade out of lemons. So we, for example, do a lot of free association where one idea just leads to the next and they're not really that well connected and we enjoy that and we make poetry out of it and we make kind of movies with free Associations and it's fun and whatever. I don't think that's really a virtue of the system. I think that the limitations in human reasoning actually get us in a lot of trouble. For example, politically, we can't see eye to eye because we have the motivational reasoning I was talking about and something related called confirmation bias. So we have all of these problems that actually make for a rougher society because we can't get along, because we can't interpret the data in shared ways. And then we do some nice stuff with that. So my free associations are different from yours, and you're kind of amused by them, and that's great, and hence poetry. So there are lots of ways in which we take a lousy situation and make it good. Another example would be our memories are terrible. So we play games like Concentration, where you flip over two cards, try to find a pair. Can you imagine a computer playing that? Computer's like, this is the dullest game in the world. I know where all the cards are. I see it once. I know where it is. What are you even talking about?

Stuff.

So we make a fun game out of having this terrible memory.

So we are imperfect in discovering and optimizing some kind of utility function. But you think in general, there is a utility function, there's an objective function that's better than others.

I didn't say that.

But see the presumption when you say.

I think you could design a better memory system. You could argue about utility functions and how you want to think about that, but objectively, it would be really nice to do some of the following things to get rid of memories that are no longer useful. Objectively, that would just be good. And we're not that good at it. So when you park in the same lot every day, you confuse where you parked today with where you parked yesterday with where you parked the day before and so forth. So you blur together a series of memories. There's just no way that that's optimal. I mean, I've heard all kinds of wacky arguments, people trying to defend that, but in the end of the day, I don't think any of them hold water.

Trauma memories of traumatic events would be possibly a very nice feature to have to get rid of those.

It'd be great if you could just be like, I'm going to wipe this sector. You know, I'm done with that. I didn't have fun last night. I don't want to think about it anymore.

Whoop, bye, bye, gone.

But we can't.

Do you think it's possible to build a system.

So you said human level intelligence is.

A weird concept, but.

Well, I'm saying I prefer general intelligence. I mean, human level intelligence is a real thing. And you could try to make a machine that matches people or something like that. I'm saying that per se shouldn't be the objective, but rather that we should learn from humans the things they do well and incorporate that into our AI, just as we incorporate the things that machines do well that people do terribly. So, I mean, it's great that AI systems can do all this brute force computation that people can't. And one of the reasons I work on this stuff is because I would like to see machines solve problems that people can't that combine the strength or that in order to be solved would combine the strengths of machines to do all this computation with the ability, let's say, of people to read. So I'd like machines that can read the entire medical literature in a day. Seven thousand new papers or whatever the numbers comes out every day. There's no way for any doctor or whatever to read them all. A machine that could read would be a brilliant thing. And that would be strengths of brute force computation combined with kind of subtlety in understanding medicine that a good doctor or scientist has.

So if we can linger a little bit on the idea of general intelligence. So Yann Lecun believes that human intelligence isn't general at all. It's very narrow. How do you think?

I don't think that makes sense. We have lots of narrow intelligences for specific problems. But the fact is anybody can walk into, let's say, a Hollywood movie and reason about the content of almost anything that goes on there. So you can reason about what happens in a bank robbery or what happens when someone is infertile and wants to go to IVF to try to have a child, or you can. The list is essentially endless. And not everybody understands every scene in the movie, but there's a huge range of things that pretty much any ordinary adult can understand.

His argument is that actually the set of things seems large to us humans because we're very limited in considering the kind of possibilities of experiences that are possible. But in fact, the amount of experience that are possible is infinitely larger.

I mean, if you want to make an argument that humans are constrained in what they can understand, I have no issue with that. I think that's right. But it's still not the same thing at all as saying here's a system that can play go. It's been trained on 5 million games and then I say can it play on a rectangular board rather than a square board? And you say, well, if I retrain it from scratch in another 5 million games, it can. That's really, really narrow. And that's where we are. We don't have even a system that could play Go and then without further retraining, play on a rectangular board, which any good human could do with very little problem. So that's what I mean by narrow. And so it's just wordplay to say then it's semantics.

Yeah. Then it's just words then. Yeah. You mean general in a sense that you can do all kinds of go board shapes flexibly.

Well, I mean that would be like a first step in the right direction, but obviously that's not what it really meaning. You're kidding. What I mean by general is that you could transfer the knowledge you learn in one domain to another. So if you learn about bank robberies in movies and there's chase scenes, then you can understand that amazing scene in Breaking Bad when Walter White has a car chase scene with only one person, he's the only one in it. And you can reflect on how that car chase scene is like all the other car chase scenes you've ever seen and totally different and why that's cool.

And the fact that the number of domains you can do that with is finite doesn't make it less general. So the idea of general is you could just do it on a lot of. Transfer it across a lot of domains.

Yeah, I mean, I'm not saying humans are infinitely general or that humans are perfect. I just said a minute ago it's a low bar, but it's just a low bar, but the bar is here and we're there and eventually we'll get way past it.

So speaking of low bars, you've highlighted in your new book as well, but a couple years ago wrote a paper titled Deep Learning A critical appraisal that lists 10 challenges faced by current deep learning systems. So let me summarize them as data efficiency, transfer learning, hierarchical knowledge, open ended inference, explainability, integrating prior knowledge, causal reasoning, modeling an unstable world, robustness, adversarial examples, and so on. And then my favorite probably is reliability and engineering of real world systems. So whatever people can read the paper, they should definitely read the paper, should definitely read your book. But which of these challenges, if solved in your view, has the biggest impact on the AI community?

That's a very good question and I'm going to be evasive because I think that they go together a lot. So Some of them might be solved independently of others. But I think a good solution to AI starts by having real, what I would call cognitive models of what's going on. So right now we have an approach that's dominant, where you take statistical approximations of things, but you don't really understand them. So you know that bottles are correlated in your data with bottle caps, but you don't understand that there's a thread on the bottle cap that fits with the thread on the bottle and that's what tightens. And if I tighten enough that there's a seal and the water will can come out. There's no machine that understands that. And having a good cognitive model of that kind of everyday phenomena is what we call common sense. And if you had that, then a lot of these other things start to fall into at least a little bit better place. Right now you're learning correlations between pixels when you play a video game or something like that, and it doesn't work very well. It works when the video game is just the way that you studied it. And then you alter the video game in small ways, like you move the paddle and break out a few pixels and the system falls apart because it doesn't understand, it doesn't have a representation of a paddle, a ball, a wall, a set of bricks and so forth. And so it's reasoning at the wrong level.

So the idea of common sense, it's full of mystery, you've worked on it, but it's nevertheless full of mystery, full of promise. What does common sense mean? What does knowledge mean? So the way you've been discussing it now is very intuitive. It makes a lot of sense that that is something we should have and that's something deep learning systems don't have. But the argument could be that we're oversimplifying it because we're oversimplifying the notion of common sense because that's how we, it feels like we as humans at the cognitive level approach problems.

So a lot of people aren't actually going to read my book, but if they did read the book, one of the things that might come as a surprise to them is that we actually say common sense is really hard and really complicated. So they would probably my critics know that I like common sense. But that chapter actually starts by us beating up not on deep learning, but kind of on our own home team, as it will. So Ernie and I are first and foremost people that believe in at least some of what good old fashioned AI tried to do. So we believe in symbols and logic and programming things like that are important. And we go through why even those tools that we hold fairly dear aren't really enough. So we talk about why common sense is actually many things and some of them fit really well with those classical sets of tools. So things like taxonomy. So I know that a bottle is an object or it's a vessel, let's say, and I know a vessel is an object and objects are material things in the physical world. So I can make some inferences. If I know that vessels need to not have holes in them, then I can infer that in order to carry their contents. Then I can infer that a bottle shouldn't have a hole in order to carry its contents. So you can do hierarch inference and so forth. And we say that's great, but it's only a tiny piece of what you need for common sense. We give lots of examples that don't fit into that. So another one that we talk about is a cheese grater. You've got holes in a cheese grater, you've got a handle on top. You can build a model in the game engine sense of a model so that you could have a little cartoon character flying around through the holes of the grater. But we don't have a system yet. Taxonomy doesn't help us that much. That really understands why the handle is on top and what you do with the handle, or why all of this circles are sharp or how you'd hold the cheese with respect to the grater in order to make it actually work.

Do you think these ideas are just abstractions that could emerge on a system like a very large deep neural network?

I'm a skeptic that that kind of emergence per se can work. So I think that deep learning might play a role in the systems that do what I want systems to do, but it won't do it by itself. I've never seen a deep learning system really extract an abstract concept. What they do, principled reasons for that stemming from how backpropagation works, how the architectures are set up. One example is deep learning. People actually all build in something called convolution, which Yann Lecun is famous for, which is an abstraction. They don't have their systems learn this. So the abstraction is an object, looks the same if it appears in different places. And what Lecun figured out, and essentially why he was a co winner of the Turing Award, was that if you program this in innately, then your system would be a whole lot more efficient. In principle. This should be learnable but people don't have systems that kind of reify things and make them more abstract. And so what you'd really wind up with if you don't program that in advance is a system that kind of realizes that this is the same thing as this. But then I take your little clock there and I move it over and it doesn't realize that the same thing applies to the clock.

So the really nice thing, you're right, that convolution is just one of the things that's like, it's an innate feature that's programmed by the human expert.

But we need more of those, not less.

Yes, but the nice feature is it feels like that requires coming up with that brilliant idea, can get you a touring award. But it requires less effort than encoding and something we'll talk about the expert system. So encoding a lot of knowledge by hand. So it feels like there's a huge amount of limitations which you clearly outline with deep learning. But the nice feature of deep learning, whatever it is able to accomplish.

It.

Does a lot of stuff automatically without human intervention.

Well, and that's part of why people love it. But I always think of this quote from Bertrand Russell, which is it has all the advantages of theft over honest toil. It's really hard to program into a machine a notion of causality or even how a bot model works or what containers are. Ernie Davis and I wrote a, I don't know, 45 page academic paper trying just to understand what a container is, which I don't think anybody ever read the paper, but it's a very detailed analysis of all the things, well, not even all some of the things you need to do in order to understand a container. It would be a whole lot nice. And you know, I'm a co author on the paper. I made it a little bit better. But Ernie did the hard work for that particular paper and it took him like three months to get the logical statements correct. And maybe that's not the right way to do it. It's a way to do it. But on that way of doing it, it's really hard work to do something as simple as understanding containers. And nobody wants to do that hard work. Even Ernie didn't want to do that hard work. Everybody would rather just like feed their system in with a bunch of videos with a bunch of containers and have the systems infer how contain containers work. It would be like so much less effort. Let the machine do the work. And so I understand the impulse, I understand why people want to do that. I just don't think that it works. I've never seen anybody build a system that in a robust way can actually watch videos and predict exactly which containers would leak and which ones wouldn't or something. I know someone's going to go out and do that since I said it, and I look forward to seeing it. But getting these things to work robustly is really, really hard. So Yann Lecun, who was my colleague at NYU for many years, thinks that the hard work should go into finding an unsupervised learning algorithm that will watch videos, use the next frame, basically in order to tell it what's going on. And he thinks that's the royal road and he's willing to put in the work in devising that algorithm. Then he wants the machine to do the rest. And again, I understand the impulse. My intuition, based on years of watching this stuff and making predictions 20 years ago that still hold even though there's a lot more computation and so forth, is that we actually have to do a different kind of hard work, which is more like building a design specification for what we want the system to do, doing hard engineering work to figure out how we do things, like what Jan did for convolution, in order to figure out how to encode complex knowledge into the system. The current systems don't have that much knowledge other than convolution, which is again the this, you know, objects being in different places and having the same perception, I guess I'll say same appearance. People don't want to do that work. They don't see how to naturally fit one with the other.

I think that's. Yes, absolutely. But also on the expert system side, there's a temptation to go too far the other way. So it's just having an expert sort of sit down and encode the description, the framework for what a container is, and then having the system reason the rest. From my view, one really exciting possibility is of active learning, where it's continuous interaction between a human and machine. As the machine, there's kind of deep learning, type extraction, information from data patterns and so on, but humans also guiding the learning procedures, guiding both the process and the framework of how the machine learns whatever the task is.

I was with you with almost everything you said except. Except the phrase deep learning. What I think you really want there is a new form of machine learning. So let's remember deep learning is a particular way of doing machine learning. Most often it's done with supervised data for perceptual categories. There are other things you can do with deep Learning, some of them quite technical. But the standard use of deep learning is I have a lot of examples and I have labels for them. So here are pictures. This one's the Eiffel Tower, this one's the Sears Tower, this one's the Empire State Building, this one's a cat, this one's a pig, and so forth. You just get millions of examples, millions of labels. And deep learning is extremely good at that. It's better than any other solution that anybody has devised. But it is not good at representing abstract knowledge. It's not good at representing things like bottles contain liquid and have tops to them and so forth. It's not very good at learning or representing that kind of knowledge. It is an example of having a machine learn something, but it's a machine that learns a particular kind of thing, which is object classification. It's not a particularly good algorithm for learning about the abstractions that govern our world. There may be such a thing. Part of what we counsel in the book is maybe people should be working on devising such things.

So one possibility, just I wonder what you think about it, is deep neural networks do form abstractions, but they're not accessible to us humans in terms of we can't.

There's some truth in that.

So is it possible that either current or future neural networks form very high level abstractions which are as powerful as our human abstractions of common sense? We just can't get a hold of them. And so the problem is essentially we need to make them explainable.

This is an astute question, but I think the answer is at least partly no. One of the kinds of classical neural network architectures is what we call an auto associator. It just tries to take an input, goes through a set of hidden layers and comes out with an output. And it's supposed to learn essentially the identity function that your input is the same as your output. So you think of it as binary numbers. You've got like the 1, the 2, the 4, the 8, the 16 and so forth. And so if you want to input 24, you turn on the 16, you turn on the 8, it's like binary 1, 1 and a bunch of zeros. So I did some experiments in 1998 with the Precursors of contemporary deep learning. And what I showed was you could train these networks on all the even numbers and they would never generalize to the odd number. A lot of people thought that I was, I don't know, an idiot or faking the experiment or wasn't true or whatever, but it Is true that with this class of networks that we had in that day, that they would never, ever make this generalization. And it's not that the networks were stupid, it's that they. They see the world in a different way than we do. They were basically concerned what is the probability that the rightmost output node is going to be a one. And as far as they were concerned, in everything they'd ever been trained on, it was a zero. That node had never been turned on. And so they figured, why turn it on now? Whereas a person would look at the same problem and say, well, it's obvious. We're just doing the thing that corresponds. The Latin for it is mutatis mutandis. We'll change what needs to be changed. Changed. And we do this. This is what algebra is. So I can do F of x equals y plus 2, and I can do it for a couple of values. I can tell you If y is 3, then x is 5, and if y is 4, x is 6. And now I can do it with some totally different number like a million. Then you can say, well, obviously it's a million and two, because you have an algebraic operation that you're applying to a variable. And deep learning systems kind of emulate that, but they don't actually do it. The particular example, you could fudge a solution to that particular problem, but the general form of that problem remains that what they learn is really correlations between different input and output nodes. They're complex correlations with multiple nodes involved and so forth, but ultimately they're correlative. They're not structured over these operations over variables. Now, someday people may do a new form of deep learning that incorporates that stuff, and I think it will help a lot. And there's some tentative work on things like differentiable programming right now that fall into that category. But the sort of classic stuff like people use for imagenet doesn't have it. And you have people like Hinton going around saying symbol manipulation. Like what? Marcus, what I advocate is like the gasoline engine. It's obsolete. We should just use this cool electric power that we've got with a deep learning. And that's really destructive because we really do need to have the gasoline engine stuff that represents. I mean, I don't think it's a good analogy, but we really do need to have the stuff that represents symbols. Yeah.

And Hinton as well would say that we do need to throw out everything and start over. So, I mean, there's a. Yeah, you.

Know, Hinton said that to Axios and I had a friend who interviewed him and tried to pin him down on what exactly we need to throw out. And he was very evasive.

Well, of course, because we can't. If he knew, then he'd throw it out himself.

But I mean, you can't have it both ways. You can't be like, I don't know what to throw out, but I am going to throw out the symbols. And not just the symbols, but the variables and the operations over variables. And don't forget the operations over variables. The stuff that I'm endorsing and which John McCarthy did when he founded AI, that stuff is the stuff that we build most computers out of. There are people now who say we don't need computer programmers anymore, not quite looking at the statistics of how much computer programmers actually get paid. Right now we need lots of computer programmers and most of them, they do a little bit of machine learning, but they still do a lot of code. Code where it's like if the value of X is greater than the value of Y, then do this kind of thing like conditionals and comparing operations over variables. There's this fantasy you can machine learn anything. There's some things you would never want to machine learn. I would not use a phone operating system that was machine learned. You made a bunch of phone calls and you recorded which packets were transmitted and you just machine learned it be insane. Or to build a web browser by taking logs of keystrokes and images, screenshots, and then trying to learn the relation between them. Nobody would ever, no rational person would ever try to build a browser. They would use symbol manipulation, the stuff that I think AI needs to avail itself of in addition to deep learning.

Can you describe what your view of symbol manipulation in its early days. Can you describe expert systems and where do you think they hit a wall or a set of challenges?

Sure. So I mean, first I just want to clarify, I'm not endorsing expert systems per se. You've been kind of contrasting them. There is a contrast, but that's not the thing that I'm endorsing.

Yes.

So expert systems try to capture things like medical knowledge with a large set of rules. So if the patient has this symptom and this other symptom.