Conversation (YouTube clip) where François Chollet questions the notion of an intelligence explosion. He argues that intelligence is not an isolated property of a brain but emerges from interactions among brain, body (embodiment), and environment; therefore merely improving an AI's internal algorithm is unlikely to produce recursive exponential gains. Chollet uses historical/counterfactual examples (Einstein as a patent clerk) to illustrate that opportunity and environment are necessary for expressed intelligence, distinguishing potential intelligence from expressed intelligence.
Chapters
So, yeah, so intelligence explosion, I'm sure you're familiar with the idea, but it's the idea that if you were to build general AI problem solving algorithms, well, the problem of building such an AI, that itself is a problem that could be solved by your AI, and maybe it could be solved better than what humans can do. Human, your AI could start tweaking its own algorithm, could start making a better version of itself and so on iteratively in a recursive fashion. And so you would end up with an AI with exponentially increasing intelligence.
That's right.
And I was basically questioning this idea, first of all, because the notion of intelligence explosion uses an implicit definition of intelligence that doesn't sound quite right to me. It considers intelligence as a property of a brain that you can consider in isolation, like the height of a building, for instance. But that's not really what intelligence is. Intelligence emerges from the interaction between a brain, a body, like embodied intelligence, and an environment. And if you're missing one of these pieces, then you can actually define intelligence anymore. So just tweaking a brain to make it smarter and smarter doesn't actually make any sense to me.
So first of all, you're crushing the dreams of many people. Right? So there's. Let's look at like Sam Harris, actually a lot of physicists, Max Tegmark, people who think, you know, the universe is an information processing system. Our brain is kind of an information processing system. So what's the theoretical limit like? It doesn't make sense that there should be some. It seems naive to think that our own brain is somehow the limit of the capabilities of this information. Just, I'm playing devil's advocate here. This information processing system, and then if you just scale it, if you're able to build something that's on par with the brain, you just. The process that builds it just continues and it will improve exponentially. So that, that's the logic that's used actually by almost everybody that is worried about superhuman intelligence.
Yeah.
So you're. You're trying to make. So most people who are skeptical of that are kind of like, this doesn't. Their thought process. This doesn't feel right. Like, that's for me as well. So I'm more like, it doesn't. The whole thing is shrouded in mystery where you, you can't really say anything concrete. But you could say, this doesn't feel right. This doesn't feel like that's how the brain works. And you're trying to, with your blog post and now making it A little more explicit. So one idea is that the brain isn't exists alone, it exists within the environment. So you can't exponentially, you would have to somehow exponentially improve the environment and the brain together Almost. Yeah. In order to create something that's much smarter in some kind of. Of course, we don't have a definition of intelligence.
That's correct. That's correct. I don't think if you look at very smart people today, even humans, not Even talking about AIs, I don't think their brain and the performance of their brain is the bottleneck to their expressed intelligence, to their achievements. You cannot just tweak one part of this system, like of this brain, body, environment system, and expect capabilities like what emerges out of this system to just explode exponentially. Because anytime you improve one part of a system with many interdependencies like this, there's a new bottleneck that arises. And I don't think even today for very smart people, their brain is not the bottleneck to the sort of problems they can solve. In fact, many very smart people today, they're not actually solving any big scientific problems. They're not Einstein, they're like Einstein, but the patent clerk days. Einstein became Einstein because this was a meeting of a genius with a big problem at the right time. But maybe this meeting could have never happened and then Einstein would have just been a pattern clerk. In fact, many people today are probably genius level smart, but you wouldn't know because they're not really expressing any of that.
Wow, that's brilliant. So we can think of the world Earth, but also the universe as just as a space of problems. So all these problems and tasks are roaming it of various difficulty. And there's agents, creatures like ourselves and animals and so on that are also roaming it. And then you get coupled with a problem and then you solve it, but without that coupling, you can't demonstrate your quote unquote intelligence.
Exactly. Intelligence is the meeting of great problem solving capabilities with a great problem. And, and if you don't have the problem, you don't really express any intelligence. All you're left with is potential intelligence, like the performance of your brain or how high your IQ is, which in itself is just a number.
