François Chollet: Measures of Intelligence | Lex Fridman Podcast #120

The following is a conversation with Francois Chollet, his second time on the podcast. He is both a world class engineer and a philosopher in the realm of deep learning and artificial intelligence. This time we talk a lot about his paper titled on the Measure of Intelligence that discusses how we might define and measure general intelligence in our computing machinery. Quick summary of the sponsors Babel Masterclass and Cash App. Click the sponsor links in the description to get a discount and to support this podcast. As a side note, let me say that the serious, rigorous scientific study of artificial general intelligence is a rare thing. The mainstream machine learning community works on very narrow AI with very narrow benchmarks. This is very good for incremental and sometimes big incremental progress. On the other hand, the outside the mainstream renegade, you could say AGI community works on approaches that verge on the philosophical and even the literary without big public benchmarks. Walking the line between the two worlds is a rare breed, but it doesn't have to be. I ran the AGI series at MIT as an attempt to inspire more people to walk this line. DeepMind and OpenAI for a time and still on occasion walk this line. Francois Chollet does as well. I hope to also. It's a beautiful dream to work towards and to make real one day. If you enjoy this thing, subscribe on YouTube, review it with five stars on Apple Podcast, follow on Spotify, support on Patreon, or connect with me on Twitter lexfriedman. As usual, I'll do a few minutes of ads now and no ads in the middle. I try to make these interesting but I give you time stamps so you can skip. But still, please do check out the sponsors by clicking the links in the description. It's the best way to support this podcast. This show is sponsored by Babbel, an app and website that gets you speaking in a new language within weeks. Go to babbel.com and use code LEX to get three months free. They offer 14 languages including Spanish, French, Italian, German and yes, Russian. Daily lessons are 10 to 15 minutes. Super easy, effective, designed by over 100 language experts. Let me read a few lines from the Russian poem Noch ulize fanarptjeka by Alexander Blok that you'll start to understand if you sign up to Babel. Now I say that you'll start to understand this poem because Russian starts with a language and ends with vodka. Now the latter part is definitely not endorsed or provided by Babbel and will probably lose me this sponsorship, although it hasn't yet. But once you graduate with Babel you can enroll in my advanced course of late night Russian conversation over vodka. No app for that yet, so get started by visiting babbel.com and use code LEX to get three months free. This show is also sponsored by Masterclass. Sign up@masterclass.com lex to get a discount and to support this podcast. When I first heard about Masterclass, I thought it was too good to be true. I still think it's too good to be true. For $180 a year you get an all access pass to watch courses from to list some of my favorites. Chris Hadfield on space exploration. Hope to have him in this podcast one day. Neil DeGrasse Tyson on scientific thinking communication Neil 2 Will Wright, creator of SimCity and Sims on game design, Carlos Santana on guitar, Garry Kasparov on chess, Daniel Negrano on poker and many more. Chris Hadfield explaining how rockets work and the experience of being launched into space alone is worth the money. By the way, you can watch it on basically any device. Once again, sign up@masterclass.com lex to get a discount and to support this podcast. This show finally is presented by Cash App, the number one finance app in the App Store. When you get it, use Code lexpodcast. Cash App lets you send money to friends, buy Bitcoin, and invest in the stock market with as little as $1. Since Cash App allows you to send and receive money digitally, let me mention a surprising fact related to physical money. Of all the currency in the world, roughly 8% of it is actually physical money. The other 92% of the money only exists digitally and that's only going to increase. So again, if you get Cash App from the App Store, Google Play and use Code Lex podcast, you get 10 bucks and Cash App will also donate $10 to first, an organization that is helping to advance robotics and STEM education for young people around the world. And now here's my conversation with Francois Chollet. What philosophers, thinkers or ideas had a big impact on you growing up and today?

So one author that had a big impact on me when I read his books as a teenager was Jean Piaget, who is a Swiss psychologist, is considered to be the father of developmental psychology. And he has a large body of work about basically how intelligence develops in children. And so it's fairly old work. Most of it is from the 1930s, 1940s, so it's not quite up to date. It's actually superseded by many newer developments in developmental psychology. But to me it was very interesting, very striking, and actually shaped the early ways in which I started thinking about the Mind and the development of intelligence.

As a teenager, his actual ideas or the way he thought about it, or just the fact that you could think about the developing mind at all.

I guess both. Jean Piaget is the author that really introduced me to the notion that intelligence and the mind is something that you construct throughout your life and that children construct it in stages. And I thought that was a very interesting idea, which is of course very relevant to AI, to building artificial minds. Another book that I read around the same time that had a big impact on me and there was actually a little bit of overlap with Jean Piaget as well, and I read it around the same time is Jeff Hawkins on Intelligence, which is a classic. And he has this vision of the mind as a multiscale hierarchy of temporal prediction modules. And these ideas really resonated with me. Like the notion of modular hierarchy of potentially of compression functions or prediction functions. I thought it was really, really interesting and it reshaped the way I started thinking about how to build minds, the.

Hierarchical nature, which aspect. Also he's a neuroscientist, so he was thinking actual. He was basically talking about how our mind works.

Yeah, the notion that cognition is prediction was an idea that was kind of new to me at the time and that I really loved at the time. And. Yeah, and the notion that there are multiple scales of processing in the brain.

The hierarchy. Yes, this is before deep learning.

These ideas of hierarchies in AI have been around for a long time, even before art intelligence. They've been around since the 1980s and yeah, that was before deep learning. But of course I think these ideas really found their practical implementation in deep learning.

What about the memory side of things? I think he was talking about knowledge representation. Do you think about memory a lot? One way you can think of neural networks as a kind of memory, you're memorizing things. But it doesn't seem to be the kind of memory that's in our brains or doesn't have the same rich complexity, long term nature that's in our brains.

Yes, the brain is more for sparse access memory so that you can actually retrieve very precisely, like bits of your experience.

The retrieval aspect. You can like introspect. You can ask yourself questions.

Yes, you can program your own memory and language is actually the tool you use to do that. I think language is a kind of operating system for the mind. And you use language. Well, one of the uses of language is as a query that you run over your own memory. Use words as keys to retrieve specific experiences or specific concepts. Specific Thoughts, like language is the way you store thoughts, not just in writing, in the physical world, but also in your own mind, and it's also how you retrieve them. Imagine if you didn't have language, then you would not really have a self internally triggered way of retrieving past thoughts. You would have to rely on external experiences. For instance, you see a specific sight, you smell a specific smell, and that brings up memories. But you would not really have a way to deliberately access these memories without language.

Well, the interesting thing you mentioned is you can also program the memory, you can change it and probably with language.

Yeah. Using language. Yes.

Well, let me ask you a Chomsky question, which is like, first of all, do you think language is like, fundamental, like, there's turtles. What's at the bottom of the turtles? They don't go, it can't be turtles all the way down. Is language at the bottom of cognition of everything? Is, like language the fundamental aspect of what it means to be a thinking thing?

No, I don't think so. I think language.

You disagree with Noam Chomsky.

Yes. Think language is a layer on top of cognition. So it is fundamental to cognition in the sense that, to use a computing metaphor, I see language as the operating system of the brain, of the human mind, and the operating system is a layer on top of the computer. The computer exists before the operating system, but the operating system is how you make it truly useful.

And the operating system is most likely Windows, not Linux, because language is messy.

Yeah, it's messy and it's pretty difficult to inspect it. Introspective.

How do you think about language? Like, we use actually sort of human interpretable language. But is there something like, deeper? It's closer to like, like logical type of statements, Like.

Yeah.

What is the nature of language? Do you think, like, is there something deeper than like, the syntactic rules we construct? Is there something that doesn't require utterances or writing or so on?

Are you asking about the possibility that there could exist languages for thinking that are not made of words? Yeah, yeah, I think so. I think so. The mind is layers, right? And language is almost like the outermost, the uppermost layer. But before we think in words, I think we think in terms of emotion, in space, and we think in terms of physical actions. And I think babies in particular probably expresses thoughts in terms of the actions that they've seen or that they can perform and in terms of motions, of objects in their environment before they start thinking in terms of worlds.

It's amazing to think about that as the Building blocks of language. So like the kind of actions and ways the babies see the world as like more fundamental than the beautiful Shakespearean language you construct on top of it. And we, we probably don't have any idea what that looks like, right? Like what? Because it's important for them trying to engineer it into AI systems.

I think visual analogies and motion is a fundamental building block of the mind, and you actually see it reflected in language. Like, language is full of spatial metaphors. And when you think about things, I consider myself very much as a visual thinker. You often express your thoughts by using things like visualizing concepts in 2D space. Or you solve problems by imagining yourself navigating a concept space. I don't know if you have this sort of experience.

You said visualizing concept space. So like. So I certainly think about. I certainly meth. I certainly visualize mathematical concepts, but you mean like in concept space, visually you're embedding ideas into some. Into a three dimensional space you can explore with your mind, essentially.

Should be more like 2D, but yeah. 2D.

You're a flatlander. You're okay. No, I do not. I always have to. Before I jump from concept to concept, I have to put it back down. It has to be on paper. I can only travel on 2D paper, not inside my mind. You're able to move inside your mind.

And even if you're writing like a paper, for instance, don't you have a spatial representation of your paper paper? You visualize where ideas lie topologically in relationship to other ideas. Kind of like a subway map of the ideas in your paper.

Yeah, that's true. I mean, in papers. I don't know about you, but there feels like there's a destination, there's a key idea that you want to arrive at. And a lot of it is in. In the fog. And you're trying to kind of. It's almost like. What's that called? When you do a path planning search from both directions from the start and from the end, and then you find you do like shortest path. But like, you know, in game playing, you do this with like a star.

From both sides and you see where they join.

Yeah, so you kind of do. At least for me, I think, like, first of all, just exploring from the start from like first principles. What do I know? What can I start proving from that?

Right.

And then from the destination, if they're backtracking, like, if, if I want to show some kind of sets of ideas, what would it take to show them? And you kind of backtrack but like, yeah, I don't think I'm doing all that in my mind, though. Like, I'm putting it down on paper.

Do you use mind maps to organize your ideas? Yeah, I like mind maps.

Let's get into this. Because I've been so jealous of people. I haven't really tried it. I've been jealous of people that seem to, like. They get, like, this fire of passion in their eyes because everything starts making sense. It's like Tom Cruise in the movie was, like, moving stuff around. Some of the most brilliant people I know use mind maps. I haven't tried, really. Can you explain what the hell a mind map is?

I guess a mind map is a way to make kind of like the mess inside your mind, to just put it on paper so that you gain more control over it. It's a way to organize things on paper. And as kind of like a consequence of organizing things on paper, it starts being more organized inside your own mind.

What does that look like? Do you have an example? Like, what's the first thing you write on paper? Or what's the second thing you write?

I mean, typically, you draw a mind map to organize the way you think about a topic. So you would start by writing down the key concept about that topic. Like you would write intelligence or something. And then you would start adding associative connections. Like, what do you think about when you think about intelligence? What do you think are the key elements of intelligence? So maybe you would have language, for instance, instead of motion. And so you would start drawing notes with these things. And then you would see what. What do you think about when you think about motion and so on. And you would go like that. Like a tree.

It's a tree or a tree mostly. Or is it a graph too? Like a tree.

Oh, it's more of a graph than a tree. And it's not limited to just writing down words. You can also draw things. And it's not supposed to be purely hierarchical. Right. The point is that you can start. Once you start writing it down, you can start reorganizing it so that it makes more sense, so that it's connected in a more effective way.

See, but I'm so OCD that you just mentioned intelligence and language, emotion. I would start becoming paranoid that the categorization is imperfect like that. I would become paralyzed with the mind map that, like, this may not be so. Like the. Even though you're just doing associative kind of connections, there's an implied hierarchy that's emerging. And I would start becoming paranoid that it's not the proper hierarchy. So you're not just one way to see mind maps is you're putting thoughts on paper. It's like stream of consciousness. But then you can also start getting paranoid. Well, is this the right hierarchy?

Sure. But it's mind maps. Your mind map. You're free to draw anything you want. You're free to draw any connection you want. And you can just make a different mind map if you think the central node is not the right node.

I suppose there's a fear of being.

Wrong if you want to organize your ideas by writing down what you think, which I think is very effective. How do you know what you think about something if you don't write it down? If you do that, the thing is that it imposes much more syntactic structure over your ideas, which is not required with a mind map. So a mind map is kind of like a lower level, more freehand way of organizing your thoughts. And once you've drawn it, then you can start actually voicing your thoughts in terms of paragraphs.

It's a two dimensional aspect of layout too.

Right.

It's a kind of flower, I guess. You start, there's usually you want to start with a central concept.

Yes. Typically it ends up more like a subway map. So it ends up more like a.

Graph, a topological graph without a root node.

So like in a subway map, there are some nodes that are more connected than others and there are some nodes that are more important than others. Right. So there are destinations. But it's not going to be purely like a tree, for instance.

Yeah. It's fascinating to think if there's something to that about the way our mind thinks. By the way, I just kind of remembered obvious thing that I have probably thousands of documents in Google Doc at this point that are bullet point lists, which is you can probably map a mine map to a bullet point list. It's the same. It's a. No, it's not. It's a tree. It's a tree.

Yeah.

So I create trees, but also they don't have the visual element. Like, I guess I'm comfortable with the structure. It feels like the narrowness, the constraints feel more comforting.

If you have thousands of documents with your own thoughts in Google Docs, why don't you write some kind of search engine, like maybe a mind map, a piece of software, a mind mapping software where you write down a concept and then it gives you sentences, all paragraphs from your thousand Google Docs document that match this concept.

The problem is it's so deeply unlike mind maps. It's so deeply rooted in natural language. So it's not semantically searchable, I would say, because the categories are very. You kind of mentioned intelligence, language and motion. They're very strong semantic. Like it feels like the mind map forces you to be semantically clear and specific. The bullet points list I have are sparse, disparate thoughts that poetically represent a category like motion as opposed to saying motion. So unfortunately that's the same problem with the Internet. That's why the idea of semantic web is difficult to get. It's most language on the Internet is a giant mess of natural language that's hard to interpret, which. So do you think, do you think there's something to mind maps as you actually originally brought it up? As we're talking about kind of cognition and language, do you think there's something to mind maps about how our brain actually deals like think reasons about things?

It's possible. I think it's reasonable to assume that there is some level of topological processing in the brain, that the brain is very associative in nature. And I also believe that a topological space is a better medium to encode thoughts than a geometric space.

What's the difference between a topological and a geometric space?

Well, if you're talking about topologies, then points are either connected or not. So the topology is more like a subway map. And geometry is when you're interested in the distance between things. And in subway map, you don't really have the concept of distance. You only have the concept of whether there is a train going from station A tower to station B. And what we do in deep learning is that we're actually dealing with geometric spaces. We are dealing with concept vectors, word vectors that have a distance between expressed in terms of dot product. We are not really building topological models usually.

I think you're absolutely right. Distance is a fundamental importance in deep learning. I mean, it's the continuous aspect of it.

Yes, because everything is a vector and everything has to be a vector because everything has to be differentiable. If your space is discrete, it's no longer differentiable. You cannot do deep learning in it anymore. Well, you could, but you could only do it by embedding it in a bigger continuous space. So if you do topology in the context of deep learning, you have to do it by embedding your topology in a geometry.

Yeah. Well, let me, let me zoom out for a second. Let's get into your paper on the measure of intelligence that did you put on 2019?

Yes.

Okay. Yeah, November, November, yeah. Remember 2019? That was a different time.

Yeah, I remember. I still remember.

It feels like a different, different, different world.

You could travel, you could, you know, actually go.

And I think there's some non zero probability there'll be a textbook one day, like 200 years from now on artificial intelligence. Or it'll be called like just intelligence because humans will already be gone. It'll be your picture with a quote. This is, you know, one of the early biological systems would consider the nature of intelligence and, and there'll be like a definition of how they thought about intelligence. Which is one of the things you do in your paper on measure intelligence is to ask like, well, what is intelligence? And how to test for intelligence and so on. So is there a spiffy quote about what is intelligence? What is the definition of intelligence according to Francois Chollet?

Yeah. So do you think the superintelligent AIs of the future will want to remember us the way we remember humans from the past? And do you think they will be, you know, they won't be ashamed of having a biological origin?

No, I think it'll be a niche topic. It won't be that interesting. But it'll be like the people that study in certain contexts, like historical civilization that no longer exists, the Aztecs and so on, that that's how it'll be seen. And it'll be study in also the context on social media. There'll be hashtags about the atrocity committed to human beings when, when the, when the robots finally got rid of them, like it was a mistake. It'll be seen as a, as a giant mistake, but ultimately in the name of progress. And it created a better world because humans were over consuming the resources and we're not very rational and we're destructive in the end in terms of productivity and putting more love in the world. And so within that context there'll be a chapter about these biological systems.

Seems to have a very detailed vision of that feature. You should write a sci fi novel about it.

I'm working on a sci fi novel currently. Yes. Self published. Yeah.

The definition of intelligence. So intelligence is the efficiency with which you acquire new skills at tasks that you did not previously know about, that you did not prepare for. So it is not. Intelligence is not skill itself. It's not what you know, it's not what you can do, it's how well and how efficiently you can learn new things.

New things, yes. The idea of newness there seems to be fundamentally important.

Yes. So you would see intelligence on Display, for instance, whenever you see a human being or an AI creature, adapt to a new environment that it has not seen before, that its creators did not anticipate. When you see adaptation, when you see improvisation, when you see generalization, that's intelligence in reverse. If you have a system that when you put it in a slightly new environment, it cannot adapt, it cannot improvise, it cannot deviate from what it's hard coded to do or what it has been trained to do, that is a system that is not intelligent. There's actually a quote from Einstein that captures this idea, which is the measure of intelligence is the ability to change. I like that quote. I think it captures at least part of this idea.

There might be something interesting about the difference between your definition and Einstein's. I mean, he's just being Einstein and clever. But acquisition of new, ability to deal with new things versus ability to just change, what's the difference between those two things? So just change in itself. Do you think there's something to that? Just being able to change?

Yes. Being able to adapt, so not, not change, but certainly change its direction. Being able to adapt yourself to your environment, whatever the environment is, that's a big part of intelligence. Yes. And intelligence is most precisely how efficiently you're able to adapt, how efficiently you're able to basically master your environment, how efficiently you can acquire new skills. And I think there's a big distinction to be drawn between intelligence, which is a process, and the output of that process, which is skill. So for instance, if you have a very smart human programmer that considers the game of chess and that writes down a static program that can play chess, then the intelligence is the process of developing that program, but the program itself is just encoding the output artifact of that process. The program itself is not intelligent. And the way you tell it's not intelligent is that if you put it in a different context, you ask it to play Go or something, it's not going to be able to perform well without human involvement. Because the source of intelligence, the entity that is capable of that process, is the human programmer. So we should be able to tell the difference between the process and its output. We should not confuse the output and the process. It's the same as do not confuse a road building company and one specific road, because one specific road takes you from point A to point B. But a road building company can take you from, can take you make a path from anywhere to anywhere else.

Yeah, that's beautifully put, but it's also, to play devil's advocate, a little bit you know, it's possible that there's something more fundamental than us humans. So you kind of said the programmer creates the difference between the. The choir of the skill and the skill itself. There could be something like, you could argue the universe is more intelligent, like the. The deep. The base intelligence of. That we should be trying to measure is something that created humans. We should be measuring God or what the source of the universe, as opposed to, like there's. There could be a deeper intelligence. Sure, there's always deeper intelligence.

I guess you can argue that, but that does not take anything away from the fact that humans are intelligence. And you can tell that because they are capable of adaptation and generality. And you see that in particular in the fact that humans are capable of handling situations and tasks that are quite different from anything that any of our evolutionary ancestors has ever encountered. So we are capable of generalizing very much out of distribution. If you consider our evolutionary history as being, in a way, our training data.

Of course, evolutionary biologists would argue that we're not going too far out of the distribution. We're, like, mapping the skills we've learned previously, desperately trying to jam them into these new situations.

I mean, there's definitely a little bit of that, but it's pretty clear to me that we're able to. Most of the things we do any given day in our modern civilization are things that are very, very different from what our ancestors a million years ago would have been doing in a given day. And your environment is very different. So I agree that everything we do, we do it with cognitive building blocks that we acquired over the course of evolution, and that anchors our cognition to a certain context, which is the human condition very much. But still, our mind is capable of a pretty remarkable degree of generality, far beyond anything we can create in artificial systems today. The degree in which the mind can generalize from its evolutionary history, can generalize away from its evolutionary history, is much greater than the degree to which a deep learning system today can generalize away from its trained data.

And the key point you're making, which I think is quite beautiful, is we shouldn't measure. If you're talking about measurement, we shouldn't measure the skill. We should measure, like the creation of the new skill, the ability to create that new skill. Yes, but there it's tempting. Like, it's weird because the skill is a little bit of a small window into the. Into the system. So whenever you have a lot of skills, it's tempting to measure the skills.

Yes. I mean, the skill is the only thing you can objectively measure. But yeah. So the thing to keep in mind is that when you see skill in a human, it gives you a strong signal that that human is intelligent because, you know, they weren't born with that skill. Typically, like you see a very strong chess player, maybe you're a very strong chess player yourself.

I think you're, you're saying that because I'm Russian and now you're prejudiced, you assume all Russians are prejudiced. I'm biased, cultural bias.

So if you see a very strong chess player, you know, they weren't born knowing how to play chess. So they had to acquire that skill with their limited resources, with their limited lifetime. And you know, they did that because they are generally intelligent. And so they may as well have acquired any other skill. You know, they have this potential. And on the other hand, if you see a computer playing chess, you cannot make the same assumptions because you cannot just assume the computer is generally intelligent. The computer may be born knowing how to play chess in the sense that it may have been programmed by a human that has understood chess for the computer and that has just encoded the output of that understanding in a static program. And that program is not intelligent.

So let's zoom out just for a second and say, what is the goal of the on the measure of intelligence paper? What do you hope to achieve with it?

So the goal of the paper is to clear up some long standing misunderstandings about the way we've been conceptualizing intelligence in the AI community and in the way we've been evaluating progress in AI. There's been a lot of progress recently in machine learning and people are extrapolating from that progress that we are about to solve general intelligence. And if you want to be able to evaluate these statements, you need to precisely define what you're talking about when you're talking about general intelligence. And you need a formal way, a reliable way to measure how much general intelligence a system possesses. And ideally, this measure of intelligence should be actionable. So it should not just describe what intelligence is. It should not just be a binary indicator that tells you this system is intelligent or it isn't. It should be actionable. It should have explanatory power, so you could use it as a feedback signal. It would show you the way towards building more intelligent systems.

So at the first level, you draw a distinction between two divergent views of intelligence. As we just talked about, intelligence is a collection of task specific skills and a general learning ability. So what's the difference between kind of this memorization of skills And a general learning ability. We talked about it a little bit, but can you try to linger on this topic for a bit?

Yes. So the first part of the paper is an assessment of the different ways we've been thinking about intelligence and the different ways we've been evaluating progress in AI. And the history of cognitive sciences has been shaped by two views of the human mind. And one view is the evolutionary psychology view, in which the mind is a collection of fairly static, special purpose, ad hoc mechanisms that have been hard coded by evolution over our history as a species, over a very long time. And early AI researchers, people like Marvin Minsky, for instance, they clearly subscribed to this view and they saw the mind as a kind of collection of static programs, similar to the programs they would run on mainframe computers. And in fact, I think they very much understood the mind through the metaphor of the mainframe computer, because that was the tool they were working with. And so you had these static programs, this collection of very different static programs operating over a database like memory. And in this picture, learning was not very important. Learning was considered to be just memorization. And in fact, learning is basically not featured in AI textbooks until the 1980s with the rise of machine learning.

It's kind of fun to think about that learning was the outcast, like the weird people working on learning, like the mainstream AI world was. I mean, I don't know what the best term is, but it's non learning. It was seen as like reasoning would not be learning based.

Yes, it was seen. It was considered that the mind was a collection of programs that were primarily logical in nature, and that all you needed to do to create a mind was to write down these programs and they would operate over knowledge which would be stored in some kind of database. And as long as your database would encompass everything about the world and your logical rules were comprehensive, then you would have a mind. So the other view of the mind is the brain as sort of blank slate, right? This is a very old idea. You find it in John Locke's writings. This is the tabulaza. And this is this idea that the mind is some kind of information sponge that starts empty, that starts blank, and that absorbs knowledge and skills from experience. So it's, it's a sponge that reflects the complexity of the world, the complexity of your life experience, essentially, that everything you know and everything you can do is a reflection of something you found in the outside world, essentially. So this is an idea that's very old that was not very popular, for instance, in the 1970s, but that had gained a lot of vitality recently with the rise of connectionism, in particular deep learning. And so today deep learning is the dominant paradigm in AI. And I feel like lots of AI researchers are conceptualizing the mind via a deep learning metaphor. They see the mind as a kind of randomly initialized neural network that starts blank when you're born and then that gets trained via exposure to training data, that acquires knowledge and skills via exposure to training data.

By the way, it's a small tangent. I feel like people who are thinking about intelligence are not conceptualizing it that way. I actually haven't met too many people who believe that a neural network will be able to reason, who seriously think that rigorously, because I think it's an actually interesting worldview and we'll talk about it more. But it's been impressive what neural networks have been able to accomplish. And I, to me, I don't know. You might disagree, but it's an open question whether scaling size eventually might lead to incredible results. To us, mere humans will appear as if it's general.

I mean, if you ask people who are seriously thinking about intelligence, they will definitely not say that all you need to do is like the mind is just in your network. However, it's actually a view that's very popular, I think in the deep learning community that many people are kind of conceptually, you know, intellectually lazy about it. Right?

It's it. But I guess what I'm saying exactly right, It's, I, I mean, I haven't met many people and I think it would be interesting to meet a person who is not intellectual lazy about this particular topic and still believes that neural networks will go all the way. I think Yahoo. Is probably closest to that with people.

Who argue that current deep learning techniques are already the way to general artificial intelligence and that all you need to do is to scale it up to all the available training data. And if you look at the waves that OpenAI's GPT3 model has made, you see echoes of this idea.

So on that topic, GPT3, similar to GPT2, two actually have captivated some part of the imagination of the public. There's just a bunch of hype of different kind that's. I would say it's emergent, it's not artificially manufactured. It's just like people just get excited for some strange reason in, in the case of GPT3, which is funny that there's I believe, a couple months delay from a release to hype. Maybe I'm not historically correct on that. But it feels like there was a little bit of a lack of hype and then there's a phase shift into. Into hype. But nevertheless, there's a bunch of cool applications that seem to captivate the imagination of the public about what this language model that's trained in unsupervised way without any fine tuning is able to achieve. So what do you make of that? What are your thoughts about GPT3?

Yeah, so I think what's interesting about GPT3 is the idea that it may be able to learn new tasks after just being shown a few examples. So I think if it's actually capable of doing that, that's novel and that's very interesting and that's something we should investigate. That said, I must say I'm not entirely convinced that we have shown it's capable of doing that. It's very likely, given the amount of data that the model is trained on, that what it's actually doing is pattern matching a new task you give it with a task that it's been exposed to in its training data. It's just recognizing the task instead of just developing a model of the task.

Right, but sorry to interrupt. There's parallels to what you said before, which is it's possible to see GPT3 as like the prompts it's given as a kind of SQL query into this thing that it's learned, similar to what you said before, which is language is used to query the memory.

Yes.

So is it possible that neural network is a giant memorization thing, but then if it gets sufficiently giant, it'll memorize sufficiently large amounts of things in the world or it becomes more intelligence, becomes a querying machine.

I think it's possible that a significant chunk of intelligence is this giant associative memory. I definitely don't believe that intelligence is just a giant associative memory, but it may well be a big component.

So do you think GPT 3, 4, 5, GPT 10 will eventually. What do you think? Where's the ceiling? Do you think you'll be able to reason, no, that's a bad question. Like what is the ceiling? Is the better question.

How well is it going to scale? How good is GPT N going to be? Yeah, so I believe GPT N is going to. GPT N is going to improve on the strength of GPT 2 and 3, which is it will be able to generate, you know, ever more plausible text in context, just monotonically performance. Yes. If you train a bigger model on more data, then your text will be Increasingly more context aware and increasingly more plausible, in the same way that GPT3 is much better at generating plausible text compared to GPT2. But that said, I don't think just scaling up the model to more transformer layers and more training data is going to address the flaws of GPT3, which is that it can generate plausible text, but that text is not constrained by anything else other than plausibility. So in particular it's not constrained by factualness or even consistency, which is why it's very Easy to get GPT3 to generate statements that are factually untrue, or to generate statements that are even self contradictory. Right. Because its only goal is plausibility and it has no other constraints. It's not constrained to be self consistent, for instance. And so for this reason, one thing that I thought was very interesting with GPT3 is that you can predetermine the answer it will give you by asking the question in a specific way, because it's very responsive to the way you ask the question, since it adds no understanding of the content of the question. And if you ask the same question in two different ways that are basically adversarially engineered to produce certain answer, you will get two different answers, two contrary answers.

It's very susceptible to adversarial attacks, essentially.

Potentially, yes. So in general, the problem with these models, these generative models, is that they are very good at generating plausible text, but that's just not enough. I think one avenue that would be very interesting to make progress is to make it possible to write programs over the latent space that these models operate on, that you would rely on these self supervised models to generate a sort of flag pool of knowledge and concepts and common sense, and then you would be able to write explicit reasoning programs over it. Because the current problem with GPT3 is that it can be quite difficult to get it to do what you want to do. If you want to turn GPT3 into products, you need to put constraints on it, you need to force it to obey certain rules. So you need a way to program it explicitly.

Yeah. So if you look at its ability to do program synthesis, it generates, like you said, something that's plausible.

Yeah. So if you try to make it generate programs, it will perform well for any program that it has seen it in its training data. But because program space is not interpolative. Right. It's not going to be able to generalize to problems it hasn't seen before.

Now that's currently, do you think, sort of an absurd. But I think Useful, I guess intuition builder is, you know, the GPT3 has 175 billion parameters. Human brain has 100, has about a thousand times that or or more. In terms of number of synapses, do you think? Obviously very different kinds of things, but there is some degree of similarity. Do you think what, what do you think GPT will look like when it has 100 trillion parameters? You think our conversation might be in nature different because you've criticized GPT3 very effectively now, do you think?

No, I don't think so. So to begin with, the bottleneck with scaling up GPT3, GPT models, generative pre trained transformer models is not going to be the size of the model or how long it takes to train it. The bottleneck is going to be the trained data. Because OpenAI is already training GPT3 on a crawl of basically the entire web. Right. And that's a lot of data. So you could imagine training on more data than that, like Google could train on more data than that, but it would still be only incrementally more data. And I don't recall exactly how much more data GPT3 was trained on compared to GPT2, but it's probably at less like 100 or maybe even 1000x don't have the exact number. You're not going to be able to train a model on 100 more data than what, what you're already doing.

So that's brilliant. So it's easier to think of compute as a bottleneck and then arguing that we can remove that bottleneck, but we.

Can remove the compute bottleneck. I don't think it's a big problem. If you look at the pace at which we've improved the efficiency of deep learning models in the past few years, I'm not worried about training time bottlenecks or model size bottlenecks. The bottleneck in the case of these generative transformer models is absolutely the trained data.

What about the quality of the data?

Yeah, so the quality of the data is an interesting point. The thing is, if you're going to want to use these models in real products, then you want to feed them data that's as high quality, as factual, I would say as unbiased as possible. But there's not really such a thing as unbiased data in the first place. But you probably don't want to train it on Reddit, for instance. Sounds like a bad plan. So from my personal experience working with large scale deep learning models, so at some point I was working on a model at Google that's trained on 350 million labeled images, it's an image classification model that's a lot of images that's like probably most publicly available images on the web at the time. And it was a very noisy data set because the labels were not originally annotated by hand by humans. They were automatically derived from tags on social media or just keywords in the same page as the image was found and so on. So it was very noisy. And it turned out that you could easily get a better model, not just by training. Like if you train on more of the noisy data, you get an incrementally better model, but you very quickly hit diminishing returns. On the other hand, if you train on smaller data set with higher quality annotations, annotations that are actually made by humans, you get a better model and it also takes less time to train it.

Yeah, that's fascinating. It's the self supervised learning. There's a way to get better doing the automated labeling.

Yeah. So you can enrich or refine your labels in an automated way. That's correct.

Do you have a hope for. I don't know if you're familiar with the idea of a Semantic Web, Semantic Web just for people who are not familiar. And is the idea of being able to convert the Internet or be able to attach like semantic meaning to the words on the Internet, the sentences, the paragraphs, to be able to convert information on the Internet or some fraction of the Internet into something that's interpretable by machines, that was kind of a dream for, I think the Semantic Web paper was in the 90s. It's kind of the dream that, you know, the Internet is full of rich, exciting information. Even just looking at Wikipedia, we should be able to use that as data for machines.

And so far it's not really in a format that's available to machines. So no, I don't think the Semantic Web will ever work simply because it would be a lot of work to provide that information in structured form. And there is not really any incentive for anyone to provide that work. So I think the way forward to make the knowledge on the Web available to machines is actually something closer to unsupervised deep learning. Yeah. So GPT3 is actually a bigger step in the direction of making the knowledge of the Web available to machines than the Semantic Web was. Yeah.

Perhaps in a human centric sense, it feels like GPT3 hasn't learned anything that could be used to reason, but that might be just the early days.

Yeah, I think that's correct. I think the forms of reasoning that you see it perform are basically just reproducing patterns that it has seen in string data. So of course, if you're trained on the entire web, then you can produce an illusion of reasoning in many different situations, but it will break down if it's presented with a novel situation.

That's the open question between the illusion of reasoning and actual reasoning.

Yes, the power to adapt to something that is genuinely new. Because the thing is, imagine you could train on every bit of data ever generated in the history of humanity. That model would be capable of anticipating many different possible situations. But it remains that the future is going to be something different. For instance, if you train a GPT3 model on data from the year 2002, for instance, and then use it today, it's going to be missing many things. It's going to be missing many common sense facts about the world. It's even going to be missing vocabulary and so on.

Yeah, it's interesting that GPT3 even doesn't have, I think any information about the coronavirus.

Yes. Which is why a system that's. You tell that the system is intelligent when it's capable to adapt. So intelligence is going to require some amount of continuous learning, but it's also going to require some amount of improvisation. It's not enough to assume that what you're going to be asked to do is something that you've seen before or something that is a simple interpolation of things you've seen before. Yeah. In fact that model breaks down for even very tasks that look relatively simple from a distance. Like L5 self driving, for instance. Google had a paper a couple of years back showing that something like 30 million different road situations were actually completely insufficient to train a driving model. It wasn't even L2. And that's a lot of data. That's a lot more data than the 20 or 30 hours of driving that a human needs to learn to drive, given the the knowledge they've already accumulated.

Well, let me ask you on that topic. Elon Musk, Tesla Autopilot, one of the only companies I believe is really pushing for a learning based approach. Are you, you're skeptical that that kind of network can achieve level four?

L4 is probably achievable. L5 probably not.

Was a distinction there as L5 is completely. You can just fall asleep.

Yeah. L5 is basically human level.

Well, it will drive. You have to be careful.