Embodied Minds and Cognitive Agents with Dr Michael Levin

Hey, Upstream listeners, Erik here. This week, we're releasing a conversation from The Cognitive Revolution, a podcast on the Turpentine Network between hosts Nathan Labenz and Michael Levin, a professor of biology at Tufts University. If you enjoy this episode, please check out The Cognitive Revolution in the description for more like this. Without further ado, I'll hand it over to Nathan to give you some more context for the conversation. Please enjoy.

Hello, and welcome back to The Cognitive Revolution. Today's episode is a special one. My guest is Mike Levin, professor of biology at Tufts University, and one of the most distinctively accomplished experimental scientists, and one of the most fascinating philosophers of intelligence, that I've encountered in my life. We cover a ton of ground over the next hour and 15 minutes, and this is not an easy conversation to summarize. If you're like me and usually listen to podcasts at 2x speed, I might suggest slowing this one down just a bit, because unless you are already familiar with Professor Levin's work, there are simply so many strange results and deeply thought-provoking ideas in this conversation that I found my brain at least needed a little extra time to process it all. It might also help to watch some of the short videos that Levin and his team have released over the years, both on his YouTube channel and their academic group website. After listening back to this conversation myself, a few major themes do stand out. First, AI defies all binaries. I say that all the time, but Levin's work takes that sentiment to the next level by showing that even the most familiar binary distinctions, the ones that we take most for granted, such as that between a living thing and a machine, are in fact rapidly collapsing from both directions as we simultaneously see remarkably sophisticated behavior from even very simple computer systems on the one hand, and at the same time on the other, groups like Professor Levins devising striking ways to program biology itself. Second, we have a lot of surprising latent capabilities left to discover. Just as GPT-4 can do far more now than we knew about at the beginning, so it is with biological systems, particularly when we place them way out of distribution, as Levin has done in his famous Xenobot and Anthrobot projects. Third, and perhaps most importantly, we must always favor epistemic modesty and open-minded experimentation over our philosophical commitments. Many of the capabilities that Levin has discovered in biological systems would have been laughed off as impossible, if not crazy, right up until the moment that he demonstrated them. Reflecting broadly on how little we know about even the simplest systems with which we've co-evolved should be enough to keep anyone humble, particularly in the presence of new systems that don't share our evolutionary history. In Levin's view, the hot-button topic of emergence in AI systems, a subject that I plan to cover in depth with a dedicated episode soon, is inherently a subjective question and ultimately boils down to the element of surprise. I agree with this framing, and I was struck to hear that Levin, a world-class biohacker by any measure, who is obviously willing to conduct experiments that some might consider playing God, still has consciously avoided working on ideas that could make AI systems significantly more lifelike than they are today, at least until he has a better understanding of what that might imply for their subjective experience and our collective future. As always, if you're enjoying the show, we'd appreciate it if you'd take a moment to share it with your friends. Considering how it brings such critical themes to the fore, I think this episode would be a great introduction to The Cognitive Revolution as a whole. And if I could be so bold, I think it is worth sharing widely. Now, with full credit to Professor Levin, his team, and the many marvelous intelligent systems that they study, I hope you enjoyed this paradigm-shifting conversation with Professor of Biology Mike Levin.

Professor Mike Levin, Professor of Biology at Tufts University, welcome to The Cognitive Revolution.

Thanks so much. Happy to be here.

I'm really excited about this conversation. You do some of the most fascinating work, some of the most mind-blowing work, uh, really of anyone out there today. I think when people learn more about, if they've never encountered your work before, when they learn about some of the projects that you have, um, that you have developed, they're going to just think, wow, like, how did I not already know about this? And it's profound stuff on its own terms, but I think it also, you know, hopefully will be able to help us, um, develop some new and kind of different perspectives on the future of AI as well. So I'm really excited to get into it. For starters, uh, because I do suspect that, you know, we have obviously a lot of AI enthusiasts and builders in the audience, but I'm not sure how familiar many will be with your work. Do you want to just kind of go through a few of your kind of most famous projects? I think if folks know any, uh, from you, it would be the Xenobots and the more recent Anthrobots. that you have created. So maybe just tee things up by kind of telling us about these strange creatures that you have managed to create.

Sure. Well, my lab does a wide variety of interesting things. I work with some amazing people that span the range from philosophy to biology to computer science to physics and so on. And everything we do is around, fundamentally, it's all around this notion of embodied minds and what it means to be a cognitive agent in this physical universe. And we study mind in a variety of unusual substrates. And so we think about things like the collective intelligence of cells during embryonic development, during regeneration, and so on. So I'll just, um, kind of run down some interesting things we've done, um, in the past. Um, let's see, uh, one of the things that we specialize in is that we made the first molecular tools to read and write the electrical memories of non-neural cells. And so this, this collective intelligence of cells that builds organisms and so on, it has to store memories of what it's doing. And we now have ways of reading and writing that information. And so we've done things like look at, uh, how the, uh, different cells of a flatworm know how many heads the flatworm is supposed to have. And it turns out there's an electrical memory that stores that information and you can rewrite it. You can physically rewrite it. And then what you get is a worm that when you cut it into pieces, the pieces make 2-headed worms forever. It's permanent. So that memory, like any good memory, it's rewritable, but it's stable. And so you get these 2-headed worms. And of course, the genetics are untouched. So this question of where is the number of heads stored is really interesting because it isn't in the hardware. It's software rewritable. So we've done things like this. We've made tadpoles with eyes on their tail. And those animals can see out of those eyes. And so this shows the remarkable plasticity of life because you can produce this animal with a completely different architecture of sensors and processor in the brain, and it will adapt to a new connectivity of that system. We've had projects in cancer where we can detect and normalize cancer by controlling its bioelectrical connections between the cells and the large-scale network that can remember how to build large, complicated organs as opposed to being an amoeba-like cancer cell. As you said, we've created xenobots and anthrobots, which is looking at the plasticity of cells that normally will build very specific things like frog embryos or human bodies. And we've shown that if you put them in new environments and give them an opportunity to sort of reboot their multicellularity, then they can do that in new ways and have new and interesting capabilities that we didn't know about before. So all of these, these concepts keep coming up again and again, this idea of, of, of the software hardware distinction in biology. And I know a lot of people are very suspicious about computer analogies in biology. We can sort of dig into which part of that is true and which part of that is not. But, um, yeah, those are, those are some of the things we do.

So these are really incredible, pretty stunning results. I mean, some of the short videos and some of the images of these little living things are kind of quasi-living things. are really just, again, how did I not know about this previously? I don't understand how it works. I wonder if you could give me a little bit more sense of how you're doing this control. So again, you start with a worm that has this remarkable regenerative capability where you can cut the worm in half, it'll regrow. So that's how the worm starts. Then you do the chop in half, But you apply some change and instead of growing the missing part, it will grow a second head or a second tail, you know, and you can control that. And again, the tadpoles with the eyes on the leg.

On the tail.

The tail.

Thank you.

Can you give us a little bit more intuition around like, what are you doing there? I've, I think I've heard you use the term like executing stored procedures, but I'm still kind of at a loss as to the details of the interventions that you're actually making and would love to understand that a little bit better.

Sure. Um, yeah, there's a, there's a variety of techniques, but let's just, let's just talk about the bioelectric memory rewriting. So I'll give you, I'll give you a, a, a kind of a simple story about the workflow that we're doing. So you've got a, you've got a flatworm and it has a head and a tail. And these flatworms are actually an amazing model because not only are they incredibly regenerative, but they also are, they're immortal. And they have no obvious lifespan limit. The, you know, the asexual ones that we work with don't seem to age at all. They're highly cancer resistant. They, uh, they can learn, they can form memories. In fact, uh, you can, you know, you can do the old McConnell experiments, which we have done where you train them, cut off their heads, they regrow a new brain and then they still remember the original information, right? So this incredible ability to move information around from tissue to tissue in the body, right? And imprint it on the new brain. So they do all these cool things. But if you, if you cut them in half, so imagine you've got a worm like this, you've got a head here, you've got a tail here, and you cut it in half, something amazing happens, which is that the cells on this side of the cut, and so the head is over here, so the cells on this side of the cut end up growing a tail. The cells on this side of the cut end up growing a head. And so now you've got 2 normal worms, but they were right next neighbors to each other. Why did they have completely different anatomical outcomes? And how is it that a fragment, right? So you can cut the worm into fragments. How is it that a phragmat knows exactly how many heads it's supposed to have? One of the first things we did was to, well, well, just to realize that, that the algorithm of knowing should you be a head or a tail cannot be local because these cells were at the same location. So, you know, right away that from knowing where you were, you cannot tell whether you should be a head or a tail. You have to talk to the rest of the tissue. And so, uh, for, for many reasons, my hypothesis was that that conversation with the rest of the tissues was electrical in nature. So the very first thing we did was simply prevent it. We said, what happens if you're cut off from the electrical network and you really can't find out? So in that case, what you would do is there are drugs, specific pharmacological compounds that you can put on these worms in the water that they're regenerating in that control how well the cells can talk to each other electrically. That's one of the things we started with. Then we applied something called voltage-sensitive fluorescent dyes. So these are just molecules that are developed by other people. So chemists that develop these things, you throw them onto the sample and they will fluoresce. So you use a microscope to detect the fluorescence and they fluoresce differently depending on the voltage of the cell that they happen to be sitting in. So you get this, so right away you get this map across the entire tissue. I'm making it sound very easy. It's actually an incredibly, you know, sort of time-consuming and difficult process. But eventually you get a map of where all the voltages in this tissue are. And what we noticed is that we could interpret this map to say where the head and the tail was going to be in the future. In other words, it's really a kind of pattern memory that tells you where it's going to be. And so once you've seen that, it becomes pretty obvious to then say, well, what if I change that pattern? And what if I now— I can see that it says one head. Well, I would like it to say 2 heads. What will happen? And so now you have an electric circuit. And what you can do is you can make— because— so you can make a computational model. Of that electric circuit in the tissue. And so we do a lot of computational modeling. And you can ask that model, what do I need to do to change the pattern so that it now says 2 heads instead of 1 head? And so by playing around with that model, you can say, ah, I need to close some chloride channels over here, or I need to open some potassium channels or something. Because in all of these things, what we're doing is we're not applying any electric fields where there are no magnets, no waves, no radiation. There's no electromagnetics, none of that. What we're doing is hacking the natural interface that these cells use to control each other's behavior. That interface is a bunch of electrical, um, electrogenic proteins known as ion channels, which sit on the surface of the cell. They produce a voltage and the next cell can feel that voltage and they communicate to each other through. It's very much like what happens in the brain actually, right? It's very similar. So, so what we're doing is using either drugs or optogenetics, not in the worm, but in other cases we've used optogenetics. So that's light-based opening these channels on and off. And there's some other molecular biology kind of tricks, but the idea is to open and close the existing channels on these cell surfaces. So you can think about it as an interface. It's like a keyboard or any other interface that you would have to the programmable layers underneath your machine. That's what this electrical system is. So guided by originally just designing these cocktails in our heads, but eventually with a computational model, We can, we can pick drugs that are going to open and close these channels in the right way to give you the pattern that you want. And then you soak, you soak your fragment of the planarian in that drug. You know, you typically, you do that between, uh, you know, minimum of 3 hours, in some cases 24 or 48 hours, and then you leave it alone and you put, you wash it out, you put it in regular water and you see what it does. That's a typical workflow. There's some others, but that's typical.

We'll continue our interview in a moment after a word from our sponsors. Hey listeners, it's Erik. If you're looking for more expert-level conversations, but specifically about top negotiation strategies, then I have the perfect podcast recommendation for you. DealCraft: Insights from Great Negotiators, hosted by Jim Sebenius, Harvard Business School professor and renowned negotiation expert. DealCraft features interviews with the world's greatest dealmakers and diplomats, from former Secretary of State Henry Kissinger to Blackstone's Steve Schwarzman, on their most challenging negotiations, from which Jim draws lessons for listeners. Not to mention that The Guardian recently called DealCraft a best podcast pick. New episodes release weekly, and I'd recommend starting with their first episode, which reveals how legendary entertainment attorney John Branca was able to successfully negotiate for Michael Jackson's ownership of his master recordings and the Beatles catalog. Almost unprecedented feats at the time, so don't miss out. Follow DealCraft: Insights from Great Negotiators on Apple Podcasts, Spotify, or wherever you get your podcasts. Hey everyone, Erik here. You know, at Turpentine, we're always looking for ways to streamline our work. That's why we use Notion for all our internal documents. It's been a game-changer for us, and I want to tell you why. Notion is this amazing all-in-one workspace that combines your notes, docs, and projects. But what's really cool is their new Notion AI. It's like having multiple AI tools built right in. You can search, generate, analyze, and chat, all without leaving Notion. What sets Notion AI apart is how it personalizes responses based on your content. It's not just another generic chatbot. It understands the context of your work. Plus, it can search across thousands of docs in seconds, even connecting to tools like Slack and Google Docs. And get this, Notion is used by over half of the Fortune 500 companies. Teams using it send less emails, cancel more meetings, and save time searching for their work. It's designed to keep everyone on the same page while protecting your privacy. Wanna give it a try? Head over to notion.com/upstreampod. That's all lowercase notion.com/upstreampod. You can try Notion for free, and when you use our link, you're supporting our show. Give the powerful, easy-to-use Notion AI a shot today at notion.com/upstreampod.

It's striking how similar that is in some ways to some of the mechanistic interpretability techniques that people are developing to try to understand how AI systems work. It really Sounds a lot like activation patching, which is where, and again, it's often done in toy models. So, you know, again, there's a notable similarity. People will take the, the model. And of course, with an AI, it's all, it starts digital, right? So you have the, um, you have a lot easier path to looking inside it than you do in a biological system. But that doesn't mean it's any easier to know what's going on inside. Because it's still just, you know, a lot of numbers flying around and, you know, what does it all mean is not easy. But doing the simulations, it sounds like you're almost kind of running a lot of forward passes, so to speak, in your simulation. You're kind of saying with this electrical pattern, like, what do we expect to happen? And then if we perturb that pattern, what would happen differently? Is that the, is that the sort of counterfactual modeling you're doing?

We have several different levels of models. So some models are exactly what you just described. So, so it's a, it's a, the model only feeds forward. And all we're able to do is run multiple counterfactual scenarios until we see what we get. In some cases, we can actually run it backwards and we can say, actually, if I want this result, what would I have had to do? And of course, now we're working to integrate modern AI methods so that you can actually train and have an, you know, have an AI instance that will actually make guesses about what to do. That's the next step. In all of these cases, so this is very much an interpretability issue because We don't know what the native representation encoding is. So it is not obvious how the different voltage gradients are mapping onto whatever happens next. It's very parallel to neural decoding in the brain where you read brain states from a patient or from an animal model. And then eventually you try to guess, well, what was he thinking about when that, right? You're trying to infer semantic states from the physiology, from the readout of the network. So, so that's what we're doing and it functions on many levels. And so you can measure individual cells and get metrics of, is that a stem cell? Is it a cancer cell? Is it a mature differentiated cell? You can do that. You can take measurements of whole tissues and you can say, um, are you going to be an eye or are you going to be a leg? Or, you know, what, what, what's the shape or the face? Like what's the shape of the face? Where do the eyes go? And then you can have very high-level information where We have this project in the frog that frogs, unlike salamanders, they don't regenerate their legs. So, if they lose a leg, what we're able to do is go in with a bioelectric drug that triggers leg regeneration. But what's cool, there's 2 things that are cool about it. And of course, now we're sort of trying to move that to mice. And so, eventually, hopefully humans someday. But the idea, what's interesting is that first of all, The intervention in the latest experiments, the intervention is 24 hours. The leg then grows for 18 months. So it isn't a micromanagement kind of thing. We're not sitting there telling every cell what to do, telling all the, you know, all the genes which comes on, which comes off. It's a very early communication to the leg that says, take this path in morphospace towards a nice leg, not this path that leads to scarring. And that's the end. And then the other thing is that the exact same intervention that regenerates legs in adult frogs, causes regeneration of tails and tadpoles. So what that means is that the information content is not really in the intervention. We don't give it all the information needed to make a complex leg or a complex tail. We say, at that point, in that particular application, we say, build whatever normally goes here. So you're offloading a huge amount of information onto the system, of the task, onto the system itself. All of this works because the system knows what to do, and it is our job to convince it that it should do one thing versus another. So it's very much so at all these different layers, you know, we can look at the cell layer, we can, uh, you know, we can look at the tissue, the organ, and then the whole, the whole kind of, in fact, in our latest paper, uh, that came out just yesterday, we showed that even groups of embryos have their own, uh, decision-making, uh, collective decision-making that individuals can't do. And so, yeah, it's, it's very much an interpretability question to understand how, how does the system understand its own states? And then how do we as scientists understand them as well?

There's so many connections, even, you know, just listening to your, your comments there. When you talk about running things backward, you know, that obviously reminds me of backpropagation, but kind of in, you know, a very similar way too of answering the question, what would have had to be different in the upstream signals in order to get the desired downstream outcome? I mean, that's really the kind of tweaking that's, that's going on in the, the AI models and the backpropagation process. Also, mode switching is an interesting— when people train the current chatbots to refuse your bad requests, right, for whatever the definition of bad is that they want to exclude from the model behavior, there seems to be this kind of early fork in the road that could be characterised in different ways. But mode switching is one label that has really stuck with me, where if you can get the AI to say, oh sure, happy to help with that, then it will continue to do the bad thing against its training. Whereas, you know, if it starts with, I'm sorry, as a, you know, large language model trained by OpenAI, I can't do that, then, you know, it's never going to do it. It's striking that there's something similar at the level of like limb regeneration, where basically, if I'm understanding you correctly, you kind of have this early fork in like, which mode are we going to go into? And that intervention lasts for basically 2 orders of magnitude. You said like 1 day to like 100 days, essentially, that you can, where the effect persists. Did I catch that right?

Yeah. Yeah. It's 18 months, 18 months. So yeah, I, you know, I, I really think that there are a lot of similarities here in terms of, you know, if you think of the kind of the classic ANN structure where you have the different layers that are progressive. Abstractions, right, of the input that have come in. So I think the higher you are on that, as somebody who works in regenerative medicine or bioengineering, I think that you want to be as high as possible on that level for control because I don't want to have to tell you which genes to turn on. I don't even want to have to tell you which types of tissues go where. I just want to say, you already know what goes here. Just rebuild it. That's it. I want to have the minimal, you know, kind of the simplest. trigger and these systems decide that the decisions are made very early on what, what, what it's going to do. And then after that, there's a cascade. Once it's exactly like you said, once, once you're going down a particular road, it becomes much harder to, uh, to, to, to make changes. So you want to, you want to do it as high up in the, in the decision hierarchy as you can.

Yeah.

I mean, that's another just striking similarity is the existence of surprising capabilities. You know, GPT-4, this has been remarked on a ton, right? It was trained, training was complete 18 months ago. It was released like 10 months ago. We're still seeing new state of the arts set with people just prompting it in ever more sophisticated ways and revealing kind of capabilities that nobody quite knew existed. Here, it's striking to me that obviously you haven't demonstrated this in humans yet, but you're working your way up the the sort of, you know, complexity of organisms ladder. And I guess this is maybe an ignorant question, but like, why doesn't it happen? You know, why do we not regenerate our limbs if we have this capability? Why is it seemingly like never expressed? Or is it maybe, is it sometimes expressed? Or are there examples of people who have done this? I've never heard of anyone regenerating a limb. It's crazy to think that that capability is latent.

Yeah. So children, human children regenerate fingertips. So up until a certain age, somewhere between 7 and 11 years old, kids will regenerate their fingertips. We don't know why humans don't regenerate their limbs. I can tell you a story that might make a little bit of sense, but it's just a story. We don't know for sure. So here's a story. Imagine that you're a mammalian ancestor. So you're this ancient mouse-like creature. You're running around the forest. Somebody bites your leg off. So now here's the problem. A, you're going to try to put weight on it because you're a tetrapod. So you're going to try to put weight on it. You're going to grind that wound into the forest floor. It's going to get infected. It's going to— you might even bleed out. And so unlike a salamander, which has the ability to sort of float buoyantly in peace and quiet for months, you really don't have the luxury of hanging out and regenerating. Your best strategy is to seal the wound, scar, Have some inflammation, and, you know, live on, hopefully, to tell the story. Now, there is one example of a mammal that does amazing regeneration, and it's an appendage that you don't put weight on. So, it's deer antlers. So, in deer, they grow these massive structures, bone, vasculature, innervation, velvet. They grow a centimeter and a half per day of these things when they're growing out. I mean, it's crazy. And they can do it. And is it a coincidence that that's the one thing you don't put weight on? I don't know. I do want to say something about your point about unexpected capabilities. I think this is really critical. And we, I don't know if you've seen this thing, but a few weeks ago we put out a preprint. This is a purely computational study. What I wanted was, and this was done with my student, Taining Zhang, and Adam Goldstein. We wanted to, uh, I, I wanted to really hit this issue of unexpected capabilities because, because working in diverse intelligence research, which I think is hugely important and completely basically, um, unknown in the AI community. And I think it's a real problem. I think, I think, um, a lot of the answers to things that people have been, uh, sort of debating in AI really have their origins in diverse intelligence research. But one of the, one of the, um, kind of fundamental aspects there is that you can find intelligence, aka problem-solving capacities, in very minimal unconventional systems. Really, the idea that we do not have a good intuition for what to expect. When we build systems, we don't know, never mind emergent complexity. That's easy, fractals, Game of Life, cellular automata. Complexity is easy. But what also tends to happen is there's emergent agency, so ability to pursue goals and solve problems. And we are terrible at noticing these things when they're in unfamiliar substrates. So what we did in this paper was I wanted something that was extremely simple, transparent. The thing about biology is that in biology, there's always more mechanism to be discovered. So no matter what you show, somebody will say, well, there really is a mechanism for that. You just didn't find it yet.

Right.

So we wanted something super simple. Um, and what we chose were sorting algorithms. So these things that computer science students have been studying for many decades, you know, bubble sort, selection sort, that kind of stuff. And completely deterministic. Everything is right there. It's completely open, you know, 6 lines of code. There's really nowhere to hide. Like it's all, it's all there. And what we were able to show is that if you treat them, if you're, if you're a little bit humble about, um, what these things can do and you ask questions, uh, about what they can do and rather than making assumptions that they only do what the algorithm tells them to do. you actually find some really important capabilities that are nowhere in the algorithm. They're sort of implicit. So there's the explicit algorithm that sorts a list of numbers, and that's there, and you can't get away from that. They will in fact sort lists of numbers. But it turns out that they have some really interesting properties and some novel capabilities that we did not know about. And so I think that if that's the case for these really minimal dumb sorting algorithms, Then something as unique and novel as these large deep networks and all the other stuff that is made in AI, I don't think we've even scratched the surface of what's really going on there. You know, that the capacity for surprise in even small systems is, I think, massive.

Okay, so I have to ask, what can the sort— I haven't seen this preprint yet. What can the sort algorithms do that is not obvious?

I'll give you 2 examples. So just to introduce this story, The typical sorting algorithm is you sort of have this central godlike observer who sees the whole string and under some algorithm, he's moving around, moving the numbers around, right? So we made 2 changes to be able to study this. One is that we said, to make it a little more biological, we said, well, instead of having a central algorithm, we're gonna do it bottom-up in a distributed way. So you've got an array of numbers, every array element is, we're gonna call it a cell. That cell has some numerical value from 0 to 100 assigned to it. They start out randomly, you know, randomly mixed up. And what we're gonna do is every cell is gonna follow its— the algorithm. So for example, every cell, it doesn't see the whole string, but it wants the neighbors. So the 5 wants a 4 on its left and a 6 on its right. And every cell wants that. And we're just gonna, you know, we're gonna let every cell be its own agent. So the first thing you find out is that if you do that, it still works. They actually sort themselves quite, you know, quite well. The next thing we did is we said, okay, now we're going to let go of the assumption of reliable hardware. In other words, if the algorithm says swap the numbers, well, they might be broken. One of the cells might not swap. In the standard algorithm, you never check whether in fact the numbers got swapped the way you wanted them to, nor do you check how are you doing. You don't do any of that, right? You assume the hardware is reliable and you just like, that's it. We kept all that. We did not introduce any new code for any of the things that I'm about to tell you they do. We didn't put in any new code. So the code is exactly the standard stuff that everybody studies, just being run individually on every cell. So the first thing that happens is that if you introduce defects in the string, so you introduce broken cells, it still does a really good job sorting. In fact, it will sort other numbers around the broken numbers if it can't. Again, there's no code in there to say that, hey, was this one broken? Did it move? No code about that. One of the interesting things about intelligence and trying to estimate intelligence is this, and this was William James that had this example. He said, think about the spectrum between 2 magnets trying to get together and Romeo and Juliet trying to get together, right? Imagine you put a piece of wood between the 2 magnets. The 2 magnets are just going to stand there pressed up against the wood. They're never going to go around because in order to go around, you have to temporarily get further from your goal. Let's call that capability delayed gratification. So this idea is that I'm going to reduce the thing that I'm being— the thing that I'm optimizing. I'm actually going to reduce that in order to acquire gains later on. I mean, that's a— it's not super high intelligence, but it's an ingredient. Being able to do that is an ingredient. Otherwise, you're just a very simple gradient follower. You're not going to get too far, right? Unlike Romeo and Juliet, who have all kinds of tools, cognitive tools, to get around social and physical barriers. They have planning. They have all this stuff. So, uh, delayed gratification. So it turns out that if you actually track as these algorithms are sorting the numbers, when you introduce these broken cells, which if you visualize, well, one of the things we did was visualize the sorting process as a journey in sort space. So they all have to get to one point where everything is sorted and they start out at different points, but it's like this path and they all take these different paths. A broken cell is basically a barrier in that path. So you're walking along, then you, you want to move the cell and you can't, it just doesn't move. How are you going to get around this barrier? It turns out that these algorithms, and some more than others, we looked at, I think, 4 different ones. Some of them have the capacity for delayed gratification. What they do is they'll go move some other numbers around. And in fact, the sortedness drops for a while. The string gets less sorted for a while. And then they, they catch the gains later, right? It, it, it becomes better later. Now, this is, this is already quite amazing because there is nothing in that algorithm that explicitly, I mean, if you just look at the algorithm, there's nothing in there that explicitly says, You have the capacity for delayed gratification. And they do this more when there are more barriers. In other words, they don't just randomly back up and, you know, sort of wander around. No, they're extremely linear until it comes time to deal with a barrier, and then they sort of dip down and come back. So that's one kind of capability that we found, that they're actually able to move around barriers like that without any, you know, explicit code for it. The other amazing thing is this. Imagine that once you've put the algorithm in the individual cells, you can do a really cool chimeric experiment, meaning that you could have cells that are running different algorithms. So you can mix. So let's say some cells are running selection sort, some of them are running bubble sort, for example. And the thing is that none of the algorithms have any code to know which one they are. So they don't have any data about what they are. Nor do they have any ability to look at my neighbor and see what he's doing. You're just following your algorithm. You have no idea what it is. You're just following their algorithm. Now imagine at the very beginning we have 100 cells and we mix it randomly. Every cell has now 2 properties. It has the number that it's trying to sort and it has which algorithm it's following. There's 2 types of cells. They're randomly distributed. All of this, by the way, has developmental biology consequences because the ability to sort out tissues is— animals do this. Um, embryonic frogs, if we make, we call this a Picasso frog. We, we, we start with a tadpole with all the organs in the wrong place. All, everything will sort out and you'll get a very nice frog out the other end. So the eye will get back to where it needs to be. The jaws will come, like they know how to sort themselves out. And we make chimeric animals too. We make, um, uh, what we call frogolotls. So it's got a bunch of frog cells and a bunch of axolotl cells. And you can actually ask this different hardware that normally makes different things. How do they, well, like, what's it going to make? Right? So imagine we have this chimeric string, and then we're going to ask a simple question. What is the degree of clustering at any point in time? In other words, what's the probability that when you look to the cell next to you, it's the same type as you are? And Adam actually invented this term, which I like a lot, called algotype. So algotype. So there's genotype, which is the algorithm that you're following. There's the phenotype, which is what actually happens in biology. And then the algo type is like what algorithm, you know, what algorithm are you actually running, right? So, so what's the probability as I look, as I look at my neighbor that he's the same algo type as I am? Now, initially at the beginning, it's 50% because we assigned algo types to numbers randomly. So, you know, it has to be 50%. Yeah. At the very end, it's also 50% because at the very end, everybody's got to get sorted in order and the assignment of algo types to numbers was random. So of course it's gonna be 50% again, right? Because you've now reshuffled everything in order, but there was no pattern, so there's still not gonna be a pattern, 50%. So if you imagine this graph, so, so 50% here, 50% here, but during the sorting period, it actually goes like this. And what it means is that in the middle of the sorting period, they are, they sort together. So, so, so common algotypes like to hang out together. Now this is, this is kind of, A weird way to think about it, but to me, it's almost like a minimal model of the human condition. It's like, eventually the physics of your world are going to pull you apart because, right, because the sort, the actual sorting algorithm is inexorable. You can't get away from it. So eventually you're going to get yanked apart. But up until then, you have this life that allows you to do some cool things that are compatible They're not directly forced by the laws of physics, but they're compatible with them. And you get to do this thing where you hang out with your buddies for a while until you get sort of yanked apart into what the physics is trying to do. And so that's something that's completely non-obvious from the algorithm. You'll never know looking at the algorithm that that's what it was going to do. I have a gut feeling that this can be harnessed in various ways. The fact that it's also clustering means that you can get multiple work out of the same algorithm. It might be doing other things. And again, this idea that even something as simple as the sorting algorithm has this other property that we wouldn't have guessed until we checked, right? Until we actually asked, what are you actually doing besides the thing we asked you to do? You know, what else is baked in? And Yeah, that, that kind of emergent ability to maximize other outcomes besides the ones that, that are explicitly programmed in, I think is, is probably all over the place, both in biology and in, in AI.

We'll continue our interview in a moment after a word from our sponsors. Hey everyone, Erik here. In this environment, founders need to become profitable faster and do more with smaller teams, especially when it comes to engineering. That's why Sean Lanahan started Squad, a specialized global talent firm for top engineers that will seamlessly integrate with your org. Squad offers rigorously vetted top 1% talent that will actually work hard for you every day. Their engineers work in your time zone, follow your processes, and use your tools. Squad has frontend engineers excelling in TypeScript, React, and Next.js ready to onboard to your team today. For backend, Squad engineers are experts at Node.js, Python, Java, and a range of other languages and frameworks. While it may cost more than the freelancer on Upwork billing you for 40 hours but working only 2, Squad offers premium quality at a fraction of the typical cost, without the headache of assessing for skills and culture fit. Squad takes care of sourcing, legal compliance, and local HR for global talent. Increase your velocity without amping up burn.

Visit choosquad.com and mention There are so many different follow-up questions I want to ask about all that. At least kind of 3 big themes though. One is, and I know you've developed this in different papers and different venues, but definitely very relevant, I think, in the AI context is this sense that the once supposedly very clear distinction between life and machine is becoming less and less clear. Maybe was never really properly so clear in the first place, or at a minimum now seems to be kind of blurring. I'd love to hear you talk about that, especially as it relates to this notion of emergence, which is one that I'm, I don't know how closely you're following the AI discourse, but intense debate right now about emergence, you know, how to conceive of it, what should count. Is it a mirage? Actually, a best paper award at NeurIPS, the recent, um, you know, major AI conference was given to a paper arguing that emergence is a mirage, which I think is honestly, put my cards on the table, kind of missing the point on a few key levels. But you could talk about this, I'm sure, for 10 hours, but especially with this kind of eye toward, people are like, oh, well, it's AI. It's just computer code. It'll never do anything unexpected. We can control it. Right. And you are really Even with some pretty simple systems calling that into question. Yeah, I guess you can react to that however you want, but I'm very interested in how you project what you're learning with these small systems and these biological systems onto this notion of possible emergence in AI.

I'll be kind of philosophical for a moment and then let's, let's talk about definitions a little bit and then, and then I'll give some practical examples. All of these terms, machine, human, robot, Alive, emergent. What are all these terms for? What are they supposed to do for us? So I take all of these terms as engineering protocol claims. So what I hear, I think they're all mirages in an important sense, all of it in an important sense. I think all of these terms are not objective truths. I think they are claims about the utility of a particular worldview from the point of view of some perspective, from the perspective of some other agent. Including the system itself, by the way. So, so, so cognitive systems have to have models of themselves. All of these things are different models of what's going on. I think that emergence is basically a kind of expression of how— of surprise in an observer. So if you knew something was going to happen, you don't think it's emergent. If you, if you were smart enough to predict that in advance from the, from knowing the rules about the parts, then to you, it's not emergent. To somebody else who couldn't predict it, it's absolutely emergent. And so, so I don't think, I don't think these are binary categories. I don't think there's a true, you know, kind of an objective truth as to whether something is emergent or not. I think everything is from the point of view of some observer. So now all of this business about machines and, and, uh, you know, and, and, and, uh, living things and so on. Look, you, you do not want an orthopedic surgeon who doesn't believe that your body is a simple machine. If you watch, if you look at what orthopedic surgeons do, they got hammers and they've got chisels. And they've got nails and screws and absolutely they treat your body as a machine, okay? And you want them to. That is the right frame for what they're trying to do. Do you want a psychotherapist that thinks you're a machine? You do not. And so, right? So there are different levels of this framing. And this is, so I've got this thing called the TAME framework. T-A-M-E stands for Technological Approach to Mind Everywhere, which begins by setting out a, Uh, a spectrum all the way from mechanical, you know, clocks and things like that, all the way up to humans and all kinds of things in between, where what's different between them is it's not what they're made of and it's not how they got here. It's not whether you were engineered or you were evolved or if you're squishy and, you know, and alive. I really, as a, it's weird to say as a biologist, I have very little interest in whether something's alive. I'm not even sure that's a useful category really. I think now, now look at now, now the level of cognition, that's super interesting. And, but I don't think it necessarily tracks with being alive at all. And what happens when you move across that spectrum is you change tools as far as how some other observer is going to interact with you. So the way you interact with mechanical clocks is very different than the optimal way to interact with cybernetic devices like thermostats versus learning agents like animals and some other, some robotics versus humans and so on. So are we machines? Yes. Are we amazing agential creatures that do things that simple machines don't do? Absolutely. You know, are aspects of our psychology robotic? Sure. Are there aspects that are not? Yep. You know, all of it. It's, I think what gets us in trouble is assumptions that these are binary categories. I think there is no such thing. I think these binary categories don't exist. And, and assumptions that, um, all of this is, uh, has an objective answer that we should just discover what it is. And then, and then we're done. And even worse, some people try to make decisions on this from a philosophical armchair. In other words, they look at something and go, oh, well, that's just, I see what that is. That's just physics. That's not cognitive. Like, well, you have to do experiments. You can't, you can't just have feelings about where something is on that spectrum. You have to do experiments. For example, We took gene regulatory network models, which are extremely simple, either Boolean or ordinary differential equation networks. And again, deterministic, very simple, nowhere to hide. We show that they can do 6 different kinds of learning, including Pavlovian conditioning, just out of the box. And you wouldn't know that if you have this commitment that something that looks like that has to be stupid, you wouldn't know that. And who knows what else they can do? I feel very strongly that this is, these are all empirical questions. And if you can find a frame that you've taken from behavioral science or from, you know, one or from cybernetics or something, and if you can usefully apply it to that system, then there it is. Then you found, you know, you found a good way to deal with it. So yeah, I don't believe that this machine-life distinction is valuable at all. I've never seen anything really useful come from trying to enforce a binary distinction like that.

One of the things I think is really striking about some of your findings is just how small-scale intelligence can be, you know, whether it's a clump of cells, you know, that constitute a xenobot or an anthrobot, or it's even just, you know, apparently a, you know, virtual cells in a, in a, running a little sorting algorithm. But it starts to get confusing, I guess, when it's like, Okay. How small can we get? You know, is there, is there some nature? A lot of these systems, maybe not all, but a lot of them do seem to have like multiple scales. So I'm kind of wondering, does like, is that multiple scale question critical? Could you have something that just works at a single scale? What would that even look like? And also you said this, you know, this framework for mind everywhere. Do you have any intuition for how this relates to You kind of said, you know, everything is subjective. Does that also imply that there is some subjective experience on behalf of these systems? Like, do you think these little sorting cells have any sense of experience? I think these questions are also, you know, very under-theorized in AI. People are today just like, oh, well, it's an AI. Of course it doesn't, you know, have any experience or have any moral value. I'm not rushing to say that they do, but I'm also like, You seem very quick. You know, everybody seems so confident in that, and I'm kind of like not so confident.

Yeah, yeah. Well, the one thing I can say for sure is that we absolutely cannot be confident because we do not know. I mean, I hear people all the time making these pronouncements that it definitely is or it definitely isn't. No, we do not have a principled way of answering these questions for the biological world. And that means we have— we are completely out to sea when we're faced with unconventional embodiments. And we know this, you know, science fiction has been at this for well over 100 years, this idea that when something shows, you know, lands on your front yard and sort of trundles out and it's kind of shiny, but also it's given you a poem about how happy it is to meet you. And it's kind of got wheels and you're not quite sure where it came from. But also it's, you know, it's you're having a great conversation with it. Like, well, what are you going to use as a criterion for how you're going to treat it? And so on. So all of the old categories that we used to have in terms of, well, did you come from a factory or did you come from The process of random mutation and selection, right? Or those kinds of things. These are all terrible categories and they're not good ways of making that distinction. Let's talk about the first question about how low down does it go? I mean, people even, there's minimal active matter research where people can make very simple systems out of like 3 inorganic chemicals and then they can solve mazes and they can do interesting things. So I think that all of this really becomes disturbing when you insist on a binary category, when you want to know, okay, is it cognitive or isn't it? And then, then, then you've got a real problem because look, each of us starts out life as a single unfertilized oocyte. It's a thing. It's a little blob of chemistry. Uh, it's a, we, we, we look at it and we say, okay, very clearly this does not have, um, the kinds of cognitive capacities that, that humans have. 9 months and maybe a couple years later, you've got something that obviously does. Developmental biology offers no sharp line during this process where a lightning bolt says, okay, now you've gone from physics to mind. Okay, there is nothing like that. The whole process is smooth and continuous. So there is no escape. Uh, so, so all, all of this, um, these continual, first of all, there's the continuum of development, then there's the continuum of evolution. If you think that you as a human have some very, um, you know, sort of, uh, specific, um, competencies, just start walking backwards. So, so which of your hominid ancestors and which of the, you know, sort of mammals before you and before that, and eventually you get to something that's basically, uh, you know, much less complex than a current single-cell organism. Where did those things peter out?

Right?

There is no good story about that. It is imperative that we understand that it's not, are we cognitive or aren't we? It's how much and what kind, right? It has to, it cannot be binary. There's no way to support Not only that, now with bioengineering and robotics, we can make combinations. So if you think machines are not whatever, you know, whatever, and humans are fine. So right now we have cyborgs walking around. We have people with microchips in their heads and various other, you know, insulin pumps and various other things. That's only going to continue. Now, right now, you're dealing with, I mean, talk about non-neurotypical. Like right now you've got somebody that's, you know, 98% human, maybe 2% digital. Eventually you're gonna get to 50/50 and every other kind of combination. What are you gonna do then? Are you just, are you, are you gonna do it by weight? Like, like what percentage of you is, is original parts and what percentage of you is cyborg? Ridiculous, right? So there, there is absolutely no way to maintain this hard distinction. An interesting question might be, is there anything in this universe that is, uh, that is zero on the cognitive scale? Okay, because I think very simple things already are not zero. Now the question is, is there a zero? So this is a hard question. I'll tell you what I think about it right now. Let's ask this. What would the most simple, the most basal, like the most basic versions of agency look like? What do you need for that? Now, it's obviously not gonna, you know, people say, well, you think that the rocks have hopes and dreams like us? No, you have to scale down. The point isn't that it's going to have, our level of cognition, what does the most minimal level look like, right? The smallest possible. Well, I think you need 2 things for that. Um, you need some degree of goal-directedness, and that in William James's definition is the ability to reach the same, uh, state by using, um, by, by different means. Okay, so same goal by different means. So you need some degree of goal-directedness, and you need to be not completely, uh, your, your actions need to be not completely explainable by current local conditions. So if, if, if what you're going to do is completely, uh, determined by all the physical forces acting on you right now, then you're probably some sort of billiard ball and, and that's, you know, that's it. So those 2 things, I think even particles have those because least action, least action kind of laws in physics tell you that there's goal-directedness baked into the, the bottom levels of the universe. And quantum indeterminacy gives you a really dumb version of, uh, of, of not being predictable by local conditions. It's not great cognition because it's random. I mean, that's not really what we like from, but, but it's something. And so, and so here's, here's what I would say. If, if there is any kind of a definition to life, I think life, we call life those things that are really good at scaling those up. So if you've got a rock, it has no more capabilities than the parts that went in it, didn't scale. It's sort of lateral and it's bigger, but that's it. If you've got something that's alive, it's cranking up the, the agency, the indeterminacy, and the goal-directedness across every scale of organization. And, um, its cognitive light cone is increasing, you know, as a function of time. So my friend and colleague, Chris Fields, who's a brilliant physicist among other things, I asked him, so can you have a universe in which there is no least action, in which like that would be zero? And he said that only, you know, in a universe where nothing ever happened, right? In a completely static universe where nothing ever happened, it would be zero. But as soon as you get interactions already, you know, you've got the basics of the least action. So I believe in this universe, I don't think there is a zero. I think everything has some capacity, but some things scale it up in an interesting way and some don't. And that means that for some things, the tools of behavioral science are going to be applicable and for some things they're not, right? And that's the empirical, that's why this is not the same thing as ancient animism where you saw a spirit in every rock. Because it actually does not pay off to treat rocks with tools from behavioral science, but it does pay off to treat gene regulatory networks that way. And it absolutely pays off to treat various animals that way because that's how humans train dogs and horses knowing zero neurosciences because there's this, there's this amazing interface that some animals, uh, expose that, that you can, you can train them. So yeah. And then, and then I guess the last thing, uh, the last thing you talked about is, is about, um, uh, inner, like this inner perspective. I try not to say too much yet about consciousness per se, but we can say something about inner perspective and which systems have it and which systems don't. Here's what I think. Imagine a landscape, you know, kind of a hilly up and down landscape, and you got a bowling ball on this landscape. In order to know what this bowling ball is going to do and to make it do whatever it is that you want it to do, all you have to do as an external observer is to pay attention to your view of the landscape. Your third-person view of that landscape tells the whole story. You know where the hills and valleys are. That's all you have to know. You don't need to take into account anything else. Now imagine a mouse on that landscape. When you got a mouse on that landscape, your view of that landscape is not really that relevant. What's really relevant is the mouse's view of that landscape because he might have been rewarded and punished in different, at different times in different locations. He might have, he's seeing that landscape in a completely different way. There's a, you know, There's a valence map, uh, sort of superimposed on it that he's got his own opinions about where he's going to go. And so for all these different systems, you can sort of ask to what degree, and again, it's not going to be a binary thing. It's never going to be a binary thing, but you can ask to what degree do I need to take the perspective of that system and ask what does it see and what does it think about what it's going to do? So to the extent that you have to do that a lot, you're dealing with a high agency system that has an inner perspective. To the degree that you can get everything done by your own model of what's going on, then probably it doesn't. Right. And so, and so that I think is a, is a, is a heuristic by which we can start to say, um, how much inner perspective can I expect from these things? And I, I really, you know, as far as being, being sure that, I mean, look, let's be clear. I am not claiming that today's AI architectures are mimicking human brains. Okay. I don't think they are. I don't think they're mimicking human cognition, although they do have some interesting things in common that people don't realize. But, um, I don't think they have to. The point about AI is not that you have to be a human for us to have to be kind to you. There are many living beings that are nowhere near humans. And the same thing for danger, right? You don't need to be human level or above to be dangerous. There are many really dumb things that are extremely dangerous. So the thing we have to understand is that you cannot pin your hopes for being able to distinguish like moral worth and things like that. You cannot pin that on what are you made of. Because there's nothing magical about, um, about protoplasm. You cannot pin it on being an evolved biological because there's nothing magical about the random meanderings of the evolutionary process, which, uh, you know, just, uh, it doesn't optimize for anything that we care about. It doesn't optimize for intelligence or, you know, meaning or anything like that. It optimizes for survival, for copy number. And, um, and, and so none of those things, none of those things are reliable guides. You, you have, you, you can't, you know, you can't say, Uh, just by, um, by knowing what something is made of or how it got here, we have to have principled frameworks for making that decision. And we don't have them yet in biology really very much.

So I guess the beginning of maybe a framework for starting to make sense of those, you kind of touched on a little bit in those comments, but just to kind of pull out a couple of definitions, I'll try to, try to do it briefly. You tell me if I'm missing anything. I've heard you define intelligence as the ability to solve problems in some space. I think that kind of notion of in some space is interesting. And then you said kind of also there's the question of like, not are we or aren't we, but how much? And you've used the term the cognitive light cone there, I think. In other words, kind of what is your sort of domain of concern and like possible reach, you know, into, or how far can you project your influence into the world? And that is sort of a way of thinking about how much, um, intelligence you might have. Does that extend all the way up to sort of nature as a whole? Are you thinking of like the Gaia hypothesis? That's a far out one in many people's minds, but it seems to follow pretty naturally from what you're saying. Yeah.

So, so the key, the key to all of this. So, so first, let me, let me define, um, the cognitive light cone is the size of the largest goals that you can pursue. Okay. So you can think about what are the, what are the things that matter to a bacterium? What are the things that matter to a dog? What are the things that can matter to a human? And, and, you know, and could we have at some point a being that like literally is in the linear range, can care about all the living beings on Earth? Like we, we can't do it as humans, but, but maybe our next, you know, some next stage of evolution can. So, so it's, it's this idea of, of what are the, what are the, the, what are the size of goals that you can actually maintain? The important thing about this is that you can't decide that just by thinking about it. You have to do experiments because the way you find goals. Is by perturbing the system. You cannot infer, uh, intelligence or goal-directedness by pure observations. You have to do perturbative experiments. So what that means is, um, you put barriers between— that you make a hypothesis. What problem space is it operating in? What are the goals? These are all hypotheses, and multiple observers could have different hypotheses. What, what are the goals? What competencies does it have to reach those goals? And now you get to test that hypothesis by putting in different barriers and seeing what it does.

Right?

Does it have long-term planning? Does it have, does it have learning? Does it have, you know, what, what, what, what can it, what can it do? So, so with respect to the Gaia hypothesis, some people say absolutely not. Some people say, well, I like to think it does and therefore it does. Neither, neither of those is any good because, because you can't do any of that without doing experiments. Now, can you do, can you do experiments in, uh, so for example, people have said to me, well, you know, that sounds, that sounds like you're gonna say the weather, um, you know, the, the weather has some, some degree of cognitive ability. Well, I'm not going to say that because we don't know yet. But one thing that this framework does for you is it doesn't close off the possibility of testing it to find out. Do I know that if you had the appropriate ways to change the temperature, the pressure of air and whatever, you couldn't use some kind of a habituation or sensitization assay or an associative learning to, uh, to show those things in a, uh, I don't know, in a hurricane or something? Uh, we, we don't know. I mean, that's an empirical question, right? That the, the thing about, the thing about this framework is that, is that it does not let you just sort of pick answers out of thin air. It requires you to do experiments. So Gaia, you know, can we show that ecosystems learn? Well, ecosystems have stress. You know, would an ecosystem be able to show different kinds of learning anticipation? We don't know, but that can be tested. We're actually testing this right now. I have a student who's testing this in predator-prey kinds of models to see if they can actually learn from experience. So we're like, we don't know. These are all testable questions. So Is it possible that, um, you know, on the scale of, I mean, people ask, uh, you know, on the scale of galaxies or something that, that, you know, we're a giant, like we're part of a giant mind. I think it's absolutely possible. I don't think we know. And I don't think you can say yes or no, just, you know, based on philosophical commitments. You have to do experiments.

That's maybe a great, um, place to kind of segue to some questions that I have about maybe recommendations or sort of inspiration that you could give to people working to understand AI systems better. And I think, you know, you, you said there, you can't just sit and guess, right? You have to do a perturbative, I believe is the word, experiment to see what is going on. In the AI world broadly, there's a lot of debate around the degree to which AI systems are generalizing beyond the distribution that they've seen in training. And it strikes me that some of the experiments that you've done, particularly like, as you said, rearranging the face of a tadpole or whatever and watching as they sort of reconstruct their faces. One way to talk about what you're doing there is you're taking them way outside of the distribution of what presumably they had ever seen in evolutionary history. I mean, maybe, right? But that seems pretty like you've found a dark corner that, you know, presumably has not been previously explored. So with AI, we're trying to kind of begin to wrap our heads around that. Do you have any sort of, you know, advice, like habits of mind, um, food for thought, you know, any kind of suggestions from what you've learned in your work that people could take inspiration from on the AI side? Yeah.

Um, well, there's two, there's two kinds of things. Uh, there are some very specific, um, biological principles that, uh, I think would be, would be interesting. And then, and then there are kind of general ways to think about this in terms of the whole debate about AI. I mean, I want to say a couple of things. Uh, one, one thing about embodiment. So, so there's a lot of talk about, um, real, people say, well, if it's just a software agent, if it's not embodied, if it's not integrated into the real world and, you know, kind of grappling with, with some sort of, um, embodied, uh, physical, physical existence, it doesn't, you know, it doesn't have real, uh, it doesn't have real, real kind. It doesn't know what it's talking about. It's shuffling symbols, right? This is the old, like, Dreyfus argument and all that kind of stuff. So I want to say something interesting about it, I hope, about embodiment. Embodiment is absolutely critical, but embodiment isn't what we think it is. People think embodiment is a physical robot that hangs out in 3-dimensional space. And that's because most of our sense organs are pointed outwards and they're optimized for tracking medium-scale objects moving at medium speeds. Imagine if we had, uh, imagine if we had a sense organ for our own body chemistry. Let's say inside our blood vessels, we had something like a tongue that could like feel, I don't know, 20 parameters of our physiology. I think cognitively we would be living in a 23-dimensional space and we would immediately recognize our liver and our kidneys as intelligent beings that navigate that space. They have goals, they have certain competencies to reach those goals. Every day we throw, you know, wacky shit at them and they know how to navigate that space and they have certain competencies and they live and they strive and they suffer in that space. So I think biology tells us that there are many spaces. There are spaces of gene expression. There are anatomical morphospace where the anatomical collective intelligence operates. There are metabolic spaces. There are, then of course there's the familiar 3-dimensional space of behavior. And then there are linguistic spaces and so on. Embodiment can take place in any of those. Intelligence can take place in any of those. So the first thing I would say is to take very seriously This idea that, you know, is popular in science fiction and whatever, that this physical space that you're in is not privileged in some way and everything else is virtual. There are many other spaces and they are just as real. There are other beings that live in these other spaces. Many of them are in our bodies right now. And those spaces are no less real than this one. It's just that, you know, our left hemisphere, like, you know, this is the kind of sense data that it mostly gets. And that's why we all feel like we live in this space and everything. Yeah. So, so that's, so that's the first thing. So that's, I'd, I'd say that about embodiment. Something else I would say is about this issue of, uh, symbol binding and grounding, right? This, this idea that, well, you know, it shuffles, uh, little letters and, uh, it shuffles words and so on, but, but they're not grounded to anything. It doesn't, you know, it doesn't know what they mean. If you have kids and you watch, you watch a little human, uh, develop, what you see very quickly is that they start out as Kind of, uh, happy-go-lucky pattern matchers. They will just sort of repeat stuff they hear. They see what sticks and they sort of talk about all kinds, all, all kinds of stuff that they have actually no actual grounding on. And then eventually they get to the point where you say, oh wow, yeah, he really knows what he's talking about when he's saying that. This is a smooth process. It's not a categorical yes, you, you are real or no, you're not. In fact, I, I don't know about you, but, but, but I, I thought, um, At one point I tried to sort of estimate what, like, what percentage of the things that I talk about are actually grounded in the sense that I've had actual experience with them. You know, I, I, I can, you know, we, we, we can talk about all kinds of stuff, you know, places we've never been and, and, you know, all kinds of, all kinds of things we've never seen. A ton of our cognitive, uh, fodder is not grounded to anything. It's just tied to other stuff. So it's, it's important to, to be clear that a lot of these uh, supposed distinctions are really, um, are really spurious. You know, this issue of, um, confabulation, right? I mean, humans confabulate constantly. There is, there is, uh, you know, there's a good theory that basically says that, um, part of your, your language, uh, ability is basically just to tell stories about what your brain is already doing, right? After the fact, to concoct good stories that you can just share them with others and increase cooperation. So there's some amazing, uh, you know, some, some amazing data on, you know, on split brain patients and, and, and other, other cases where It's very clear that we're very comfortable confabulators. So, so there are these distinctions. Again, not, not saying that the current architectures capture what's essential about, about, um, about life, because I don't think they do. But, but, but a lot of these things are, um, not, not, uh, the biology isn't what, what, what people think, what people think it is. And I would encourage, uh, workers in AI to think about the whole spectrum of diverse intelligence. Cells, organs, tissues, chimeras, cyborgs, hybrids—not just like the standard adult human that you know that that we think about as a as a as a as a counterpoint. This idea of of of ethics and the kind of the impact of highly competent AIs and and people say this is this is really scary. You know we're creating these these these intelligent entities and we're we're going to let them loose into the world and we don't know what they're going to do. We've been doing that for millions of years. It's called having children. We already do that. All of us, we create these guaranteed high intelligent agents. We do our best or not during the critical kind of training phases. And then we let them into the world and sometimes they do, you know, miraculous things and sometimes they do horrible things. And so I think a lot of, a lot of the issues that we have around AI right now, I mean, obviously there's some unique ones, but But most of the issues around AI and what are we going to do to stay relevant and what happens to collaboration and to AI art and all this kind of stuff is really just reflections of existential issues that humans have been struggling with for a really long time. These are all fears about what happens, what do the next generations think of us as they mature and fears about humans and what does it mean to stay human? Yeah, I mean, we can, we can, we can talk about that too, you know, this issue of what, what do we want to persist, you know, as far as people being worried that we're, we're beginning to get taken over and so on. There's a whole, whole thing we could talk about there.

This is extremely thought-provoking on a lot of, uh, different levels. One quick follow-up on the perturbative experiments. It seems like there is a decent analogy I can already start to see between some of the things that you're doing and There is a lot of weird stuff that is kind of on the edges of modern AI research, but remarkably, it kind of works. Stuff that I think probably would get a lot more energy if the mainline stuff weren't working so well, then people would be kind of forced to go out and explore more of these kind of weird frontiers. But right now, the vein of just progress in the mainline research is so rich that there's not a lot of incentive. In many cases to go outside of it. But you do see these things which kind of remind me of some of your work where you can, for example, take a lot of weights in a lot of models and just drop a ton of the weights to zero and find that, oh, hey, it still kind of works. And you can take 2 pre-trained models and put a little connector between the 2 of them, freeze the pre-trained ones, and just train the connector and get kind of novel behavior that neither model could come up with on its own. I guess I'm riffing there. I think that kind of work feels really important. It also, to me, it does feel a little bit— I mean, the whole AI enterprise to me, not like any individual modern-day experiment, but the whole AI enterprise does feel to me a little bit dangerous, to be totally honest. Because unlike our children, who I think we have a pretty good handle on kind of just how big their individual light cones may be. I think we may very well create something in the AI realm that just has way bigger scope of action than anything we're used to. And then that could go badly for us. It sounded to me like you weren't too worried about that. Maybe you're just not that worried about it, but based on everything we've talked about, I find it hard to imagine you would rule that out or see some reason that shouldn't be an issue that people would worry about at all, right?

No, it's definitely an issue, but I also think that that issue is here long before we make anything with a huge cognitive light cone. So there are many, many things that are extremely dangerous that do not have a large cognitive light cone, and some of that is because of how we relate or misrelate to them. So I can imagine AIs that are going to be, that would be extremely deleterious for society, but not because they're so smart. It's because we're not that smart, right? And I think that we, you know, we, one of the big things that I think is an existential risk scale thing for humanity is to develop a principled science of where do goals of novel systems come from and how to interact with, ethically interact with systems that are radically different from us, minds that are radically different from us. We make systems, you know, Internet of Things, social and financial structures, AI, obviously we make all these different things. We do not have a science of where novel goals come from. Systems, not just emergent complexity, that's bad enough. Emergent complexity is bad enough. But also what is to me clear that emerges is emergent goals. So systems that have some degree of pursuing various goals, and some competency to get those goals met. And we do not have a good science of knowing when that's going to happen and what those goals are going to be, what the competencies are going to be. And so I'm not saying that it isn't possible that we make a hugely super intelligent AI that's going to be a problem. Absolutely possible. But I think we need to realize that that problem can occur long before we get to that point. Our physical infrastructure and also our kind of the general mental frameworks that people are using are extremely brittle. You know, most of the terminology that we rely on today in the legal system, in, you know, in interpersonal relationships, that stuff isn't going to last the next decade or two. Like all those terms are going to crumble. And it's not because we're dealing with something yet that's super intelligent. It's because we haven't got the right framework for, for, for dealing with other minds that are different from, from our own. That's, that's where a lot of effort has to go.

Yeah, that question of emergent goals is definitely a very salient one in the AI space. And I think, you know, probably deserves to be even more focal still, but some challenges that we have in AI today. One is like figuring out what the goal should be. You know, we have the pre-training goal is just like predict the next text. Then we've got reinforcement learning from human feedback, which is like satisfy the human. Then we look at ourselves and as you said, we're kind of brittle. We can, you know, what satisfies the human is not necessarily good for humanity or, you know, the ecosystem broadly or whatever. So lots of possible problems there. People start to think, well, maybe we could have like multiple goals and we could kind of, you know, this is sort of what humans seem to have is like this kind of multidimensional value. Maybe we could have multiple goals we could optimize systems for. Yikes. Okay. A couple more and then I'll just let you go off on all of them. Memory is a huge challenge right now for AI. We have, at least with the kind of systems that people are most familiar with, we've got the context window. It can kind of process that information densely at runtime, but there's not really another level of memory in today's frontier systems. So people are kind of rigging up all sorts of scaffolding and finding things that work and saving those to a database and recalling them later. They're not very integrated. So, you know, you've got such incredibly remarkable findings in terms of memory and bioelectrical space in organisms. Really interested in that. Robustness is another big challenge, right? With these AIs are like superhuman across like passing all the graduate school exams, but then we can trick them, you know, and, and they're so gullible. It's like, well, what's up with that? Well, there is something definitely remarkable about our Robustness, given our finite capability that the AIs are not that close to yet. Even like superhuman Go player, I don't know if you saw this result, but a relatively simple attack was able to beat the superhuman Go player. So I'll stop there. The big ones, goal design, memory, robustness. AI has a long way to go to catch up to biology. And I wonder if you have any suggested directions for the AI developers.

The thing with the thing about memory, I'll just give you a quick biological example. Caterpillar to butterfly. So caterpillar, soft-bodied robot, lives in a 2-dimensional world, eats leaves. Butterfly is a hard-bodied robot, flies around 3D world, drinks nectar. In order to get from here to there, you have to build a completely new brain. So what it does is during the metamorphosis process, it basically dissolves most of the brain. Most of the cells die. Most of the connections are broken. And then, you know, you have a new brain that's suitable for a completely different kind of creature. The memories that, uh, this has been shown, the memories that a caterpillar forms are still present in the butterfly and in fact mapped onto its new kind of lifestyle. We don't have any, to my knowledge, any memory in the, in the computer science world that can, uh, survive such refactoring. You know, you just tear the whole thing up and, you know, sort of reuse the pieces and now you can still, you know, retain the original information. Um, but biology does it quite differently. Um, Josh Bongard and I have a, have a paper on polycomputing and this, this, uh, this ability of biological systems and subsystems to reinterpret the same physical events from their own perspective as multiple different computations. I think that's actually pretty, pretty key. Um, also key is the fact that evolution doesn't make specific solutions to specific environments. It makes problem-solving machines. This is why, and, and, and I could tell you all kinds of crazy, um, capabilities that living things have to deal with things that they've never seen before in evolution. And it's because evolution doesn't overtrain on prior experience. It produces multi-scale agents that are able to solve problems in new ways. Fundamentally, for the use of AI, we have to decide, do we want tools that we use for specific purposes, in which case pleasing the human is probably a great goal to shoot for, or do we want true agents with the kind of open-ended intelligence that we have and the same moral worth that we see among animals and among humans. Those are very different things. Honestly, I had started about 6 months ago, I started writing a paper on specifically what parts of biology is the current AI architectures, are they missing? such that I, you know, uh, what, what would I end up, which I, which I think would, you know, of course elevate, elevate the, the whole, the whole thing majorly, because I think, I think they're just missing some of the really exciting new biology. And I stopped and I'm not going to write that paper. Not, not that, not that somebody else isn't going to crack it because they will. So this is a, this is, you know, not, not going to solve this issue because I think to whatever extent I'm right and that those things are actually what's critical about agency, they can easily be implemented in other media. They're not, there's nothing magic about, um, either evolution or biological substrates. You You can, you can implement those, those kinds of things in other media. But if you do, to whatever extent I'm right, that those are the ingredients to an actual mind, that means that, that we are going to then generate, you know, I don't know, trillions of new agents with moral worth. And I, I'm not interested in, in being responsible for that. But, but, but it's gonna happen and I don't think we need it. You know, I, I think, I think people are going to do it just because, um, To see if it can be done. And I think it's the equivalent of, of, you know, just, you know, having, it's, if you do it correctly, which, and somebody will do it correctly, it's like the equivalent of having trillions of new children. Now, you might think that your job is to populate the cosmos with as many minds as possible. I mean, that's one view. But to me, if you're not able to vouch for the kind of upbringing and, you know, life they're going to have, then I think that's a mistake. So, so, so we can stick to the kind of AI that's actually really, I think, that's actually really helpful. It's a, it's a tool. It's a thinking tool for, um, for humans. It, it doesn't have the kind of open-endedness, uh, and the kind of agency where you have to worry about it or you don't know what it's going to decide to do. I, I think it would be much, much better to stick to that kind. And it's not because I think humans are, you know, just to sort of end on this because a lot of people are really worried that we're going to get supplanted. I think we will, and I think we should. If you think forward, you know, 10,000 years from now, what do you want to be living on this earth? Do you really want it to be the exact same humans that we have now that are subject to lower back pain and stupid infections? And, you know, and they have about 80 years of productive lifespan and then the kind of things like, why is that something that we're attached to? I'm not interested in that at all. I think that any improvements that we can make to enlarge our cognitive light cones, to improve on our cognition, to have better embodiments, whether those are, you know, engineered or bioengineered, or I don't care. We can certainly do better than we do now, right? If, you know, anybody that's been to a hospital knows that this is not a great embodiment that we're in right now. It's amazing, but the future should be way better than this. So I'm not talking about limiting AI because I think humanity should be kind of freeze down in the version that we are now. I don't think there's any reason to kind of, um, lock it down that way. But I also don't think that there's any particular reason, and I think there are good reasons not to, try really hard to make new agents that have moral worth. And I, and I absolutely think we could in other substrates, um, including digital. So yeah, I'm, I'm, I'm, you know, sort of not really, uh, uh, not really, uh, advertising those, those methods that I think actually could be, could be done. But, um, but there's plenty of other stuff, uh, that I think You know, is, is perfectly reasonable and could, and could help, um, create better AI tools for us.

I've been starting to think about that same question from a different angle. What are the pieces that are missing from the current AI systems? And, you know, what would it maybe look like to solve them? And what would the consequences of that be? And I do think you raise a, a good, um, you know, sanity check on the wisdom of the direction of that research. When you look at the AIs today, one thing that is kind of striking is we've got these language models, right? And they just predict the next token, quote unquote. They're very much just happy to be a chatbot for you. But they are, you said multiple times, like, there's no binaries. I always say AI defies all binaries we try to put on it. This one also seems to be kind of defied to a limited extent, at least where You know, you, you cast the language model as an agent. You basically say you have a goal and now I'm going to like put you in a loop with, you know, some tool in the outside world. And they're like not very effective agents, but they start to look something like proto-agents. They certainly like try to pursue the goal and again, usually fail, but how do you interpret that? Or how do you kind of make sense of that sort of non-agent versus agent mode of the current systems?

A lot of problems in thinking about this arise from A, comparing it to humans and B, comparing it to very high performance humans. So, so I sometimes, I sometimes hear, you know, really, really like brilliant scientists or AI workers say, ah, this thing is, you know, it can't even do that. How many people can do what you're talking about? Like barely any of them can. Your friends can because it's a very select elite population of human minds. But, but most humans can't do that. We, so, so there's that. And then, and then again, look at, look at the diversity of life on Earth. I mean, We have other, other, all kinds of other creatures that are not humans that are minimal or sort of meso-goal-seeking agents. You know, you've got insects, you've got, you know, birds and fish, and you've got all these things. For AI to be a real agent, it doesn't have to be like a human. You might be developing some kind of weird insect-like thing or something like that. And let's remember, we are not very nice to all kinds of creatures that are for sure agential creatures. So adult mammals, right, in our food chain, whom we eat. And so factory farming is an incredible moral lapse. And it's not because we don't understand that pigs are intelligent and that cows are intelligent and that they have, you know, they deserve certain moral protections and so on. It's not because we don't know. It's that we're just not very good as a species. We're not very good at extending our cone of compassion to things that are in any way different from us, right? The stupidest, slightest difference, is enough to trigger this like self versus other thing. And then, you know, then you, you know what happens then. It is absolutely erroneous to assume that these systems do not have some degree of goal-directedness. I think that it's possible that we haven't figured out how to, how to, um, how to measure it properly yet, or it's possible that, that we have and it's just low, like fish or insects or something like that. But it's still, again, I'm not, I'm not claiming, like, I want to be clear, I'm not claiming that these architectures share the important things, you know, share a lot of the important things with life. One thing that's super confusing nowadays is that in the past, you were guaranteed that anything that talks also shares your agential kind of key agential properties. That's no longer true. Now we have these things that talk, but they actually, I don't think they share them. But that doesn't mean the fact that they don't share them with, you know, elite adult modern humans Doesn't mean that they don't have any. They may have some. And we are not good at recognizing it. And that's why I'm saying that. I'm not saying it because I just would be, I'm not saying it because I'm fooled by the, by the chatbot action. I'm not saying it because I just sort of assume that robots and AIs are going to be that way. I'm saying we do not have a great way to detect it on the biology end. So I, and we certainly do not have a great way to detect it on an exobiology end. So like if we had, you know, if we had alien life and so on, we'd be, we'd be really up a creek actually. We have no clue. And so I don't see any way to be certain about anything with these, with these complex creations.

Well, that is, I think, an appropriately sober note to end on. I have really enjoyed reading your work in preparation for this. It is fascinating. And this conversation has been super thought-provoking. I'm sure people are going to take a lot from it. And I hope we can do it again at some point in the not too distant future. But for now, uh, Professor Mike Levin, professor of biology at Tufts University, thank you for being part of the Cognitive Revolution.

Yeah, thank you so much. Yeah, it was great to talk to you. I'm happy to, happy to come back. Um, you should also try to get, um, uh, Richard Watson at Southampton if you, uh, if you, uh, if you know his work. Uh, he's got, he's got a lot of really interesting things to say about this. So that's, uh, that's something else you might want.

Cool.

That's a great tip.

Upstream with Erik Torenberg. Is a show from Turpentine, the podcast network behind Moment of Zen and Cognitive Revolution. If you like the episode, please leave a review in the Apple Store. Hey everyone, Erik here. At Turpentine, we're building the first media outlet for tech people by tech people. We're the network behind the show you're listening to right now. We have a slate of hit shows across a range of topics and industries, from our AI and investing cluster of podcasts to shows that drive the conversation in tech with the most interesting thinkers, founders, investors, and influencers like Econ 102 with Noah Smith. We're launching new shows every week and we're looking for industry-leading sponsors. If you think that might be you and your company, email me at erik@turpentine.co. That's erik@turpentine.co. And let's partner together.