Learning with a Thousand Brains - A Conversation with Dr Viviane Clay

Welcome to the Future Learning Design Podcast.

If you think about it, there isn't really a single animal or intelligent thing in the universe that doesn't move. Everything that has a brain moves. So instead of learning like statistic regularities in a large dataset, the brain learns structured models of features at locations.

Hi everyone, and welcome back to the podcast. My name's Tim Logan, and the podcast is produced by Good Impact Labs. After learning about the amazing work of my guest this week, I had an idea that I wanted to test out. Perhaps our current obsession with large language models and generative AI is revealing of our historic obsession with a narrow cognitivist view of how human learning happens, and therefore how schooling is structured. Memorize the largest possible bank of static data and then output plausible propositions in response to prompts from a teacher or a standardized exam. Obviously, that's a bit of an unfair caricature of the industrial schooling system, but it's also not a million miles from the truth. As you'll have seen from my explorations over the last few years in episodes on ecological psychology, play, movement, active inference, and 4E cognitive science, I believe that there is an emerging understanding of how radically embodied and embedded in place and in relationships human learning actually is. So that's why I was so excited to learn about a vastly different approach to artificial intelligence that is based on a sensory motor learning framework and Jeff Hawkins' Thousand Brains Theory of the neocortex. So it's fantastic to have Dr. Vivienne Clay with me this week. She's the executive director of the Thousand Brains Project and a scientist with years of experience in neuroscience, AI, and machine learning, and a PhD in cognitive computing. The Thousand Brains Project grew out of decades of research at Numenta, a leading company in neuroscience-based AI technology, and is developing Monty, a new type of artificial intelligence based on the sensory motor principles observed in the brain. You can find links to additional information about Monty and about Viviane's work at the Thousand Brains Project in the show notes.

Right. Thank you so much, Viviane, for joining me on the podcast. It's a real pleasure to meet you. I was totally inspired after hearing you and Jeff, talking about the 1,000 Brains Project, and Monty. So yeah, really happy you could join.

Thank you so much.

Well, yeah, thanks for inviting me.

Brilliant. I think as a non-specialist, let's get that out in front. I'm certainly not a specialist in any of these topics, but I'm very interested in this collective obsession with generative AI that we seem to have had over the last few years. It's the late— sometimes it seems like the latest silver bullet of technology that's gonna, you know, change the world. Maybe that's true. Maybe it has already, maybe not. But one of the things I was just, as I say, totally interested by hearing you and Geoff Hawkins talking about your approach is that it seemed to be quite radically different to the large language models that, as I say, everyone seems to be very excited about. So maybe we could start there, just if you wouldn't mind. One of the things Geoff said in a conversation with Tim Logan that you had was the variety we see in nature, we're going to see in AI as well. Which again, for non-specialists, we, we take for granted what the so-called specialists out there are saying, and large language models equals AI and deep learning equals AI. But as you are showing, that maybe isn't true. So maybe you could, yeah, we could start there and just set out a little bit of what is different about the Thousand Brains approach and Monty.

Yeah, sure. Yeah. So first of all, whatever I'm gonna say in this podcast, I'm, I'm not saying that large language models aren't impressive or don't do great things. They are, really, really impressive technology and can do a lot of cool stuff, even things that humans can't do so well.

Yeah.

Particularly because languages seems like such a difficult task for us to see a, a machine produce such good language is impressive. But having studied the brain and knowing how the brain works, we can also see that they work fundamentally different from how our brain works. So it's not a human-like intelligence. So if you look at how these large language models learn, they're basically shown a huge amount of data, basically whatever text was ever produced and put on the internet, and then they're asked to predict the next word, next token. So they're basically just large prediction machines that learn on a huge static dataset. If you compare that to how children learn, It's totally different. Children start out by interacting with the world. They move their eyes 2 to 3 times a second looking around. They grasp things, they play with stuff, they interact with the world. And then language is actually something that comes as one of the last things that we learn. So after we've learned how objects work, how you interact with objects, then someone can come along and tell us, This is called a cat, and we can make an instant connection between the word and the object and how the cat looks like, how the cat acts, how you interact with a cat. And we have this really rich representation of what a cat is and how you play with a cat and what sound a cat might make versus large language models. They just have read all the text that was ever written about a cat and have learned some statistical correlations between words, but they have never seen a cat or touched a cat played with a cat. So they have a totally different representation. Their knowledge of cats is not grounded in physical reality. They didn't learn about the world by interacting with it, how children would. So yeah, I think that's hopefully, uh, the most accessible comparison.

Absolutely. You know, amazing. And it would be great if we can, stretching the limits of my own knowledge, go a bit deeper as well as we go through the conversation. But I mean, perhaps listeners can already hear maybe why I was interested in this from a learning perspective and from an education, you know, and from all the themes that come up in this podcast, what this podcast really is about. Because in a way, this propositional tyranny, as John Verveke calls it, this obsession with naming something as though that is the experience of the thing, you know, memorizing a token about cats, all the facts about cats, as though that is experiencing what it means to interact and relate to cats. Everything you've just been saying. Is in a way the kind of the caricature, you could say, of the very traditional didactic education system where we encourage young people to just use the textbook, memorize the facts about the world, rather than the often counter to that, which is going out into the world to experience, to move through the world and experience what it means to relate to the world through our senses and learn in that way, in a more constructivist Deweyan kind of way. Clearly, there are, you know, that's an unhelpful binary often. But I hear resonances of the things that you've just set out about the difference between the way we have approached large language models as intelligence in a way, and the way that you're approaching it. Would you say that is a fair connection to, you know, broadly from an educational perspective, bearing in mind you're not an educator yourself, and I know you're keen not to make pronouncements for what teachers and schools should do, but does it feel like there's a broad resonance with that comparison?

Yeah. Yeah. I think a, a general theme or, or a thing to keep in mind is I guess the purpose of the brain is to help us interact with the world. In fact, the only output of the brain are actions, motor commands. So like even if we produce like language outputs, this is still muscle— the output of the brain are muscle movements. To your mouth to, to produce these words or to your hand to write down an answer. And that's like the, the highest level kind of tasks that humans are able to perform. But basically the reason why we have brains in the first place and why we have intelligence in the first place is so we can interact with the world in the best way possible. So just kind of thinking of, of that purpose that we have intelligence to act in the world. In an intelligent way, I think is, is really helpful. And then in that sense, if you turn that around, there really wouldn't be any intelligence if there wasn't a need to interact. And these large language models, they basically model huge amounts of static datasets. They don't interact with the world in any way. So it's really a, a fundamentally different approach. I'm not, not 100% sure how that now relates to like children memorizing things and I guess that's one way of learning, but again, like you mentioned, I'm not an educator, so I'm not gonna say one's better than the other, but kind of thinking about how we learn by interacting with the world and by moving, how, how are we moving all the time? Even when we read, our eyes are moving all the time. It seems only natural to think about learning kind of requiring a lot of movement and interaction to, to be efficient.

Yeah, absolutely.

Like useful representations.

Yeah. And certainly, as I say, it's an unhelpful binary in a way, because of course we use, you know, we use language and as you say, it's, that's almost an action that we perform in the world or writing or, you know, we use propositional language to navigate the world as well. So it's incredibly valuable. But I think I definitely hear a lot, again, a lot of resonance with what you're saying in the conversations around learning and around education and the way that we do it. The other thing that I was interested in was you mentioned this idea of the LLMs using a predictive model. Clearly it's just a statistical prediction to, you know, what's the next most plausible word in the sequence. But there's also an interesting predictive model that is again a different way of thinking about a predictive model of the way we navigate the world. And the, the reason I bring this in is 'cause I've had previous conversations on the podcast, for example, with Laura De Zire de Paulo about active inference. And I think that is also a really fundamentally different way of thinking about learning is that, you know, we've got all these essentially predictive models of what we think we are going to experience, and then we are constantly evaluating our sensory data against those predictive models. And that seems to be at a very, you know, crude level what active inference is saying. And what a lot of the work around, like Stanislav Dehaene and his approach to learning, is saying about how the brain actually works. It's not just a reality out there that we're trying to memorize or process and then create representations of. It's a constant interaction with a predictive model that we have and the world out there and how do these things match. Again, would you, does that chime with your understanding, neuroscientific understanding, and also your kind of the way that you're applying that into Monty and the Thousand Brains work.

Yeah, yeah, definitely. So yeah, the brain makes predictions all the time. Like you said, whenever I, I move my eyes somewhere, my brain makes a prediction about what I'm gonna see. And actually, if that, if my brain makes a wrong prediction, that immediately draws my attention there. And I, I try to figure out why I predicted something in the wrong way. Like if I suddenly hear some noise that I didn't expect, I'll turn my head and I'll look there and I figure out what happened there. So like predictions and prediction errors are a huge part of how our perception works and also a big part of the Thousand Brains Theory, which Monty, the actual AI implementation that we wrote, is based on.

Mm-hmm.

So in that implementation, we also make predictions all the time. And I guess if you compare to the kind of next token prediction that large language models do, or even to a similar extent active inference, the kind of predictions that our system makes or that the theory describes how, how the brain would make these predictions is by using structured models of the world. So instead of learning like statistic regularities in a large dataset, the brain learns structured models of features at locations. So for example, in the brain, there's things called grid cells and place cells, and they encode physical locations in space. So as you move through a room in different locations, different cells will fire. And when you learn the layout, the map of a room or some environment, you can learn that different features, different landmarks are at different locations in that room. And that way you can make predictions about if I go to this location, I'm going to see this object or this landmark. In the environment, and you can make, use those models, those structured models to make predictions. And that's much more efficient than trying to just learn statistic regularities without, um, some kind of structured model. It is kind of an inductive bias in the sense that it assumes that we are sensing a physical structured world and that we know how, like if I go from A to B to C, I can go from C back to A. Without going the same path. That's, that's what's called path integration, that we can move in like a kind of metric space. And that's an assumption that's made. But if you make this assumption, you can model our environment very efficiently and you can then use these structured models to extrapolate to new tasks that you've never performed before and to reason about how to interact with the world, how to combine new things together, smashing a, a pot with a spoon, what will happen, or or using a tool, you can use your structured models for that in a very efficient way.

Yeah, interesting. So maybe I could just ask some clarifying questions again for the audience just to connect it back.

How—

I suppose one of the questions would be, where is the input data coming in as we form those structured models? So as I've understood it, the cortical columns in the neocortex are processing or sensing tiny amounts of input or sensory data and then essentially kind of integrating, synthesizing all of those. As Geoff— I heard Jeff say, it's like looking through a straw. So, you know, you're in a way, you're looking through a straw at a particular thing at any one moment. And as you say, you're moving your eyes constantly. And the reason it feels like we're looking at a whole scene is because we've got structured models, predictive models of what that scene looks like based on prior experience or whatever. And it almost feels like we're looking at the whole thing, but actually we're at any one moment, we're looking at particular parts of it. Again, is that not a complete crude representation of the work?

No, that's a, that's a really great summary. So as you look around, you, you think you're perceiving the entire room around you, but actually you only see high resolution in the very center of your vision. And then the further you go out, you see very blurry and, and then you don't even see color anymore, but you don't notice that because once you move your eye somewhere, You know what you will see there and you will see that there because you have these structured models and exactly that metaphor that you just used, that's one that we often use to describe it, like looking at the world through a straw. Each kind of individual modeling system gets a tiny patch of input at a time, but then it moves. So if you think about looking through a straw at a coffee cup, and you want to learn a model of that cup. At one time you only see a little part of the coffee cup, but then you, you move the straw and you know how you're moving it. That's important. You know the movement and you know what you're seeing. So you, you see a color or a certain curvature or a texture or something. Or you can also think about your finger, a touch sensor. You also just have a little patch on your skin that senses something, but yet then you're moving the finger along the rim of the cup and then to the handle. And as you're moving your finger, you know how you're moving and you're sensing certain features at locations. And that way you can learn a structured model of like at what locations do I sense which feature. And then when you see that cup again or you touch that cup again, you can use that model to make predictions. If I'm gonna move my finger over here now, I should be sensing a handle. And if those predictions are correct, you, you pretty quickly infer that you're touching the same cup or seeing the same cup again.

Yeah, it, it's, I love it because it, again, this is my own confirmation bias, so I want to, I want it to be true, but, but at the same time, I, I love it because it, the whole thing is predicated on movement, right? Learning then is predicated on movement. So you only, if you stayed still, forever, you would only get a very poor sensory representation of the world, you know, in all ways. So as you say, moving your finger across the coffee cup or moving your eyes around it is where the learning comes from.

Yeah, exactly. And if you think about it, there isn't really a single animal or intelligent thing in the universe, at least that we all agree on as intelligent, that doesn't move. Everything that has a brain moves.

Yeah.

So how we think of it, movement is actually a really important feature to intelligence, a really important feature to learning. If you go to people who do deep learning and who try to use that for interactive tasks like deep reinforcement learning and robotics and things like that, often kind of this movement and this idea that the data changes constantly, and if your policy changes, the data that you sample changes again. That's kind of a hindrance and they try to add a bunch of little crutches and fixes to avoid that problem, to get back to that assumption that you can just sample data randomly from a dataset. And they really try to kind of get rid of that. But in our theory, in, in our approach, that's a really important feature that you are moving and you can change the things that you're sampling. Mm-hmm. So yeah, that's a big difference, I guess, between approach and deep learning.

Amazing. And it, it's great cuz I've also, I've got a, The next episode, I've got a special on embodiment science and embodied practice in education with a few different embodiment specialists, because I think, you know, again, I just, it feels healthy on one level for young people to be moving rather than sitting still constantly, but it also feels fundamental to learning. And again, it's just another convergence, I think, of different perspectives on what, why we radically need to shift the kinds of spaces that young people are you know, the idea that they're just sitting there in the classroom. There is a significant lack of movement in the way that we've structured educational systems. In a secondary education, the time they really move is between lessons, not dur— not when the actual learning is supposedly happening, right? So, but maybe if I could just ask you, and again, this is maybe a bit of a reach on my part, but one of the interesting stories I heard, Descartes often gets a bad rap in the conversations around education now because of the mind-body separation and Somehow we're recovering from the mind-body separation that he was instrumental in. But at the same time, one of the things I've understood about when he was coming up with some of his, I think, therefore I am stuff, was that he was lying in his room on his bed in a room where there was tiled walls and tiled floors and tiled ceilings. And that's, again, maybe it's apocryphal and not actually true, but somehow he saw a fly moving around the room and then started imagining the fly in space in relation to these, you know, what then became Cartesian planes, right? X, Y, Z. And it was just interesting to me that there's maybe there's some resonance with the idea of these reference frames that you're using, or as you said, grid cells, place cells, that actually maybe intuitively he was kind of onto something, even though he, he then went dualist and, and separated the mind from the body. But, but actually there's something important in as you said, this idea of the way we understand the world is somehow rooted to the place where things happen.

Yeah, I hadn't heard of this story before, but that's a really cool anecdote. And yeah, I, I would definitely say that this is something really fundamental to the way we think, thinking about things in relation to each other and how they like physically relate to each other. Even when we think about abstract spaces and when we talk about more abstract concepts like democracy or math, mathematics, or things like relationships between people, you usually still kind of think of them in, in physical terms. So like, if you talk about relationships between people, you actually use physical terminology. You say these two people are close, or they grew apart, or you use kind of physical terminology to describe it. Or if you think about like a family tree, you often think about it actually drawn on paper in these physical relationships of like the parents and then underneath are the children and underneath are the grandchildren. It's like this kind of hierarchical structure. It's not some abstract space that you can't even imagine. You can very concretely imagine it. And also with mathematics, if you try to teach math, it's easiest if you can somehow visualize it. And if you can somehow ground it in reality, like if you learn about division, if you can tell students about like cutting a cake in half or cutting like a cake in 3 pieces. You kind of try, always try to find these physical metaphors to explain the more abstract concepts. So it seems like this is really a fundamental part of how the brain understands things as like spatial relationships of, of stuff. And it's another thing that is quite different from our implementation and our approach to deep learning that you have this strong reliance on like shape and physical structure of things. So if you look at deep neural networks, they often are very biased towards like low-level texture. That's— I don't want to go too deep in technicalities there, but there's things called adversarial examples where you kind of perturb a couple of pixels and it suddenly classifies a stop sign as a banana, stuff like that. Yeah. That doesn't really pay attention to the shape, but more to like the basic texture of things versus in our approach, it really first looks at the shape and the physical arrangement of features and then at the features themselves, kind of thinking if you wanna recognize a face, you can recognize a face made out of fruit where the eyes are tomatoes and the mouth is a banana. No problem. The features are all wrong. You, you don't have actual eyes and mouth and nose. You, you have fruits, but the physical arrangement matches. So you say this looks like a smiling face.

Yeah.

On the other hand, if you have like eyes and nose and mouth randomly arranged in space, you wouldn't say, oh, this is a smiling face. You would say, yeah, that's true. I don't know. It's just a bunch of face parts.

Right. Is there, it's the relationality proximity that is familiar as an overall arrangement somehow.

Yeah, exactly.

So just, it's really hard for me to imagine exactly what, because we, we get into this abstract space, right? Thinking about democracy in the same terms as thinking about a coffee cup is really, it's really quite difficult to imagine. But I, I mean, That's one of the things again, I really liked about the approach is that it doesn't say there's one, one set of things going on here, which is about the real world and is about skills, often gets called skills in an educational context. And then there's this other thing going on over here, which is conceptual and that is totally separate and is a different way of interacting with the world. And actually this is the one we kind of prioritize cuz it's about knowledge and academic abstract knowledge of the world. Actually in your Thousand Brains approach, it feels like those are just the same kind of exploration of the physical or conceptual world, but, but just, I don't know, somehow in different, using different sensory data maybe, is it?

Yeah, exactly. So actually the reason why it's called the Thousand Brains Theory is because one of the fundamental ideas in it is that there's this repeating computational unit in the brain called the cortical column. Neuron. So it's like about the size of a grain of rice and there are thousands of them in the brain. But the idea is that each of those implements a general purpose algorithm, and that's the same general purpose algorithm anywhere you look in the neocortex. It doesn't matter if it's in the visual cortex or the auditory cortex or the prefrontal cortex where you have more abstract reasoning going on. It's always the same general algorithm. The only difference is what kind of input data it gets. So if it gets input from the eyes, then that's the visual cortex and it will learn visual models of how things look like. If it gets input from a fingertip, it will learn tactile models, but it's the same algorithm, the same mechanism. So even if you go higher up in places like the prefrontal cortex, it's still the same basic structure. It's just that now it gets maybe highly processed input from previous regions that are more like already kind of pooled representations of, oh, a cup or a president or something like that, that it then kind of models the relationship of.

Yeah. Amazing. And then, so thinking about how you then, cuz as we started with AI, how does this then relate to Monty? So you maybe you can introduce Monty a bit and explain Yeah. How are you thinking about training, if that's the right word, Monty in the world to learn in a, again, a rather different way as the LLMs are being trained in inverted commas?

Yeah. So Monty is our nickname that we gave our first implementation of the Thousand Brains Theory. It's named that after the neuroscientist Vernon Mountcastle. He was the first one who proposed this idea of the cortical column as a repeating functional unit. So we wanted to name it after him.

Sorry, just to dwell on that for a moment, just to clarify. So that's, you know, this is not a new idea really. This idea of cortical columns and the way the neocortex actually is structured and functions is, is not a new idea. Is that correct?

Right. Yeah. This has been proposed a long time ago by Vernon Mountcastle, and there's a lot of experimental evidence for it. The Thousand Brains Theory really just makes it very concrete. A lot of people still don't don't think about it this way, that if you have a repeating functional unit and you have a sensory motor system, then each of those individual units needs to be a sensory motor system itself. And since each of these units just gets a small patch of input, it needs to be able to kind of integrate that over time. Right. Each of those columns learns complete models. It doesn't just learn like textural features. It learns structured models through movement. Because you can think about looking through a straw and learning like that, in that extreme case, only one of those columns would get input. So you can't rely on like a large area trying to, to model that object together.

Yeah, perfect. Sorry. No, sorry to interrupt. I just, just wanted to clarify that it's, again, it's just one of those things that's a really powerful idea that's kind of somewhat been overlooked historically.

Yeah, exactly. So Monty basically takes the Thousand Brains Theory as a foundation and the evidence from neuroscience in general, which the Thousand-Brain Theory is based on and implements that. And we, we basically— the idea of like, if you really want to understand something, you need to be able to build it. And we, we had this theory and we wanted to see, does it actually work in a practical system? So about 3 years ago, we started implementing the system where a cortical column is implemented as something we call a learning module. And then we have sensor modules, which basically take a specific input from a specific modality, like a camera or touch sensor or LiDAR, ultrasound, anything, converts it into like a common language and then sends it to a learning module. And a learning module then tries to model the stream of input data and can output suggested next actions and thereby also can kind of make the sensor move in an intelligent way. And you can have many learning modules communicating with each other as well using this kind of communication protocol we define. And that way you can like integrate multiple modalities. You can like stack them hierarchically to learn compositional models. And basically it's, it's the first implementation of what we call a Thousand Brains system. And it's a real alternative type of AI. So it's, it's not using deep learning, it's using the, the principles of the Thousand Brains theory. So sensory motor learning, learning structured models of the world, having a, a repeating general purpose computational unit, making predictions about the world. And it's worked out really well. So at first it was an internal effort at Numenta. We didn't talk about it for, for a while, but we proved to ourselves that it actually works. And we have a paper in the pipeline right now where we demonstrate a lot of really great capabilities of the system that you don't have with deep learning, like being able to learn from several orders of magnitude less computation, being able to do continual learning, like you show a new task and you learn it without forgetting any of the previous knowledge.

Right.

Being able to deal with noise, being able to generalize, being able to detect symmetry just naturally because you have structured models. So it has all these really nice properties and works a lot more how the brain works, given like what we see from neuroscience evidence and psychological experiments.

Amazing. So exciting. And are you, what are your hopes for it in the sense of as you learn more and then as you integrate more and more of these learning modules together? What are your ult— not ultimate hopes, but what does a 3 or 5 year kind of improvement or transformation from where you are now using this approach look like?

So we definitely hope to have a positive impact on a, a wide range of applications where particularly where deep learning doesn't do well, like areas where you have little data, areas where you have to generalize to new tasks, learn very quickly. Deal with adversarial conditions. But more generally, we think that it will change the way that people think about AI. So we're not gonna, we don't think that it will replace deep learning, but we think that ultimately when people think about intelligent machines, it will be based on principles like those outlined in the Thousand Brains Theory. Yeah. And that it will be actually interacting with the world. Or interacting with simulated worlds, depending on, on where you use it. It's a, it's a general purpose system. But yeah, working on very different principles, like deep learning systems will still be useful, but it'll be more like calculators today. I mean, calculators were a big thing when it started replacing manual calculations by humans, but now it's a tool that, that we use. We wouldn't say that they're particularly intelligent. Yeah. But it's still a very important and powerful tool. Same with deep learning. It's, it's a general function approximator, so it can do lots of really cool stuff modeling protein dynamics.

I mean, it's, it's, yeah, I mean, it's incredible, right? But it's just not this, not the same version of intelligence. I mean, it's quite a different understanding of what actually intelligence means, I would say.

Exactly. Yeah.

It's, it's amazing. And again, I hear you kind of saying, you know, it could be in a virtual space, but would you say this is an embodied approach? To AI, or is that going a step too far? Because it feels like as you're talking about all these sensory inputs, you know, you're using these sensory modules to allow the sense data to come in. Is that somewhat similar to, you know, almost building a body for the AI?

Yeah, so it depends a bit on how you define embodied. People have different definitions.

Sure.

It is in the sense that like movement is very important and being able to move your sensors is like core to the whole approach. But it doesn't require a physical body. So you could also think about like sensors navigating webpages on the internet and learning kind of the structure of how links between websites link to each other, or like navigating cyberspace or working in like a simulated environment that may have different physics or whatever. It doesn't have to be like a physical body, like a robot in the real world, but it does require being able to move, or at least being able to— you don't necessarily have to actively control the sensor's movement, but you need to know how the sensor is moving, and the sensor needs to be able to move. So that's, that's a key kind of requirement.

Yeah.

Okay. No, that makes sense. And it, it also links to one of the things I was interested in at the end of— so, I mean, just for context for listeners, I guess Jeff Hawkins is the one who'd Tim Logan, who's kind of been working on this for quite a long time. And he's got a few different books around it, which I'll put links to in the show notes. But yeah, the end of his Thousand Brains book, the last chapter is an interesting one. It was intriguing to me about this idea that it felt like he was going a bit post-human in the sense that he, with this idea, the emotional and maybe the more animal appetites, the instincts, the, you know, the things that really are core to us as organisms feel somehow that they may be redundant ultimately. He seemed to be that he felt like the 1,000 Brains idea would just kind of render the emotional parts of us somehow redundant and almost like you could upload the brain into the cloud and we don't need these meat suits anymore. I don't know whether that's a fair representation of what he's saying, but I also, if it is, I radically disagree. But anyway, that's a—

Well, yeah, it's an important point to make that the Thousand Brains Theory is focused on the neocortex, which isn't the entire brain. So it is kind of the, you might say, newer part of the brain. It's that big sheet at the top of your brain that you usually think about when you would draw a brain or something. But there are also like these deeper structures of the brain that are older. But the neocortex is really like where people would say our intelligence is or where we learn models of the world and where we learn how to perceive the world and how to interact with the world. So we focus on that because it's really like, we would say, the center of intelligence. And then the other parts of the brain are also very important, but they do other things like control your bodily functions and yeah, part also emotions, actually like deciding what you should do, communicating your desires to the neocortex. And then the neocortex is more like a map that can be used to figure out how to achieve these goals, like finding food. So if you then think about like having a robot that's implementing these principles of the neocortex, actually to me it's a very comforting thought that we are not implementing those old parts of the brain that want to find food and mate and whatever, but we are just, we are implementing them, the map that can be used.

Yeah.

To accomplish a task, and then the human can basically provide the task, and the AI uses this very powerful modeling system, those very good models, to then figure out how to accomplish this task. But the AI doesn't itself have these kind of drives and desires. It doesn't suddenly decide, oh, I wanna, I wanna do this even though you told me to do that.

Yeah. Interesting. So is that a, is that somehow a safeguard in the approach, or is it, as I understood what Geoff was saying, that it that actually the emotional parts, the more ancient parts of the brain are just something we can transcend. You know, they're actually unhelpful because they provide, as you say, these kind of more biophysical urges, needs that we have to fulfill if we have a carbon-based body.

Yeah. So on the one hand, it's definitely a safeguard. It's something that makes it not just run wild or whatever, or start a war, or I don't know. Things that humans do, have anger issues and whatnot. And yeah, it's a different question. Like, I, I wouldn't argue that humans shouldn't have emotions anymore. Like, it's a question of like what makes us human and all, but we are really just working on implementing the intelligent part of our brain, implementing the intelligence system and not implementing the emotional parts.

Yeah.

Maybe you might say the more irrational parts. Of the brain. So I mean, it's a philosophical question, like, do you in the end want those kind of all the more irrational parts to continue that start wars and fight with each other and all these kind of things? Is it worth it for the positives? Right. But yeah, I guess it's a bit further away from—

it's a, it's a deep question, right? Because those parts also love and care and, you know, actually in a way want to live, right? That's one of the things that I think is so core to it. It's like Life wants to live, and that somehow is an emotional statement. That's not a computational statement that it was, it makes sense efficiently or computationally to live. It's like I have something in me that cares about sustaining life, my own life maybe. But there's, yeah, I guess it's a pretty fundamental question if you imagine this Thousand Brains approach replacing humans or just being an augmentation to the way that we live and we actually can use this almost like, you know, a very, very sophisticated tool to support us. But as you say, it won't go rogue because it actually doesn't care enough about any of those things because it hasn't got any of those parts built in.

Yeah, that's more how we're thinking of it. It's more like a tool for us, something that can increase our standard of living and help us and do all these things that we currently can't do, help us colonize Mars, whatever, but not, at least in, in the near term, anything that we want to replace humans or so.

Yeah.

Yeah. I, I would totally agree with you that emotions, our ability to enjoy things and to care and all that is like a very fundamental part of what makes us human. So wouldn't want to replace that, but also don't want to build that in a machine.

Yeah. Right, right. No, no. And apologies bringing that on you, but, and also it's just quite interesting to notice in myself that my head goes to this kind of weird dystopian space. I mean, you know, it's just like when we start talking about AI and all these amazing capabilities that it has and can have even more, particularly in this approach. Somehow we, I don't know, my imagination runs wild and I imagine it replacing humans, but actually that's probably quite an unhelpful rabbit hole to go down. Maybe if just to finish off to, again, just as a slightly tangential, but I find very interesting question, just to clarify, you mentioned earlier, I think that it was, it takes less computational power running these. And, and I'm just interested in this energy and water. Question that, you know, we're seeing a lot of stories around how energy and water intensive the large language models are. And then there was the, you know, when DeepSeq came out and there was an interest, you know, people were intrigued by the fact that it was less ecologically damaging, one could say. Where does this 1,000 Brains and Monty approach sit in that frame?

Yeah, definitely is a lot more computationally efficient for learning because it, it learns those structured model. So it, it works with, I don't know how, how deep I should get into the technical things, but basically deep learning uses backpropagation, which is a very global update. So you, you get input and you calculate activations in all of the neurons in the network, and then you calculate the error and you backpropagate it and you update basically all of the weights in the network versus in the brain. First of all, when you get input, only a very, very small percentage of your neurons will actually fire or respond to that input. And then when you learn, only a very, very small percentage of the synapses, or what you might compare to the weights in the network, are being updated or learning. And you have much more local learning, but learning is also faster. So you have this kind of fast local associative learning, something in the brain people often talk about Hebbian learning. So having this fast associative local learning versus these global updates and backpropagation is much, much more efficient. So if people compare about making deep learning systems more efficient, they usually talk about making it maybe 1.5 or 2 times more efficient, or maybe 3 to 10 times perhaps. But when we compare our system to like Vision Transformers, since we compared on a vision task, it was orders of magnitude more efficient. So like, wow, 12,000 times more efficient if you just compare it to fine-tuning. If you consider that vision transformers are usually pre-trained on millions of images, it's more like over 100,000 times more efficient.

Wow.

It's really huge differences just because of the amount of training data that's required and the speed of learning. So it's just a totally different approach. And if you compare it to the brain, the brain is much, much more efficient.

Yeah, exactly. Ah, amazing. I mean, It just strikes me like, why do, why do more people not know about this? This is like radically different and super, super exciting. So yeah, I'm, hopefully I can contribute to people learning about this. Yeah.

Yeah. That, that I hope that too. I'm, I'm super excited by it. It has only been public for the past half year. So that's when we started putting all of the code in the open source and we are uploading all of our research meetings on YouTube and really sharing everything now and wanna build an open community around it. We started the Thousand Brains project now as a separate company from Numenta as a, as a nonprofit. But it is always highlighting that it is still very much in a, in a research stage. So it's not like, um, fully developed product where, where someone can come now and take it out of the box and apply it to their problem. So it definitely hasn't had all the hundred thousands of engineering hours and, um, of dollars poured into it so far.

But no, I so appreciate you sharing about it and thank you. And when's the paper? When are you releasing the paper?

Hopefully whenever you can finish it. I'll let you know.

Yeah. Okay. Perfect. You know, cause I can share that with it, but no, thank you Vivian. This is absolutely amazing.

Yeah. You can follow us on like social media at LinkedIn, Blue Sky X, all those channels and we'll share the paper there.

Okay. Fantastic. Well, yeah. I'll put all of that in the show notes. And no, thank you. And thank you, Geoff, vicariously through you. And no, just thanks. Thanks so much for all of the work. It's absolutely fascinating.

Yeah, thanks for inviting me and thanks for the fun conversation.

We hope you've enjoyed this episode. Please feel free to continue the dialogues with our guests, with us at Good Impact Labs, on our blog or on social media, or within your own networks.