Stephen Wolfram: Cellular Automata, Computation, and Physics | Lex Fridman Podcast #89

The following is a conversation with Stephen Wolfram, a computer scientist, mathematician and theoretical physicist who is the founder and CEO of Wolfram Research, a company behind Mathematica, Wolfram Alpha, Wolfram Language, and the New Wolfram Physics project. He's the author of several books, including A New Kind of Science, which, on a personal note, was one of the most influential books in my journey in computer science and artificial intelligence. It made me fall in love with the mathematical beauty and power of cellular automata. It is true that perhaps one of the criticisms of Stephen is on a human level, that he has a big ego, which prevents some researchers from fully enjoying the content of his ideas. We talk about this point in this conversation. To me, ego can lead you astray, but can also be a superpower, one that fuels bold, innovative thinking that refuses to surrender to the cautious ways of academic institutions. And here especially, I ask you to join me in looking past the peculiarities of human nature and opening your mind to the beauty of ideas in Stephen's work and in this conversation. I believe Stephen Wolfram is one of the most original minds of our time and at the core is a kind, curious and brilliant human being. This conversation was recorded in November 2019, when the Wolfram Physics project was underway but not yet ready for public exploration as it is now. We now agreed to talk again, probably multiple times in the near future. So this is round one and stay tuned for round two soon. This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, review it with five stars on Apple Podcast, support it on Patreon, or simply connect with me on Twitter lexfriedman spelled F R I D M A n. As usual, I'll do a few minutes of ads now and never any ads in the middle that can break the flow of the conversation. I hope that works for you. It doesn't hurt the listening experience. Quick Summary of the ads 2 sponsors ExpressVPN and Cash App Please consider supporting the podcast by getting ExpressVPN at expressvpn.com lexpod and downloading Cash App and using code lexpodcast. This show is presented by Cash App, the number one finance app in the App Store. When you get it, use code Lex Podcast Cash App lets you send money to friends, buy Bitcoin and invest in the stock market with as little as $1. Since Cash App does fractional share trading, let me mention that the order execution algorithm that that works behind the scenes to create the abstraction of fractional orders is an algorithmic marvel. So big props to the Cash App engineers for Solving a hard problem that in the end provides an easy interface that takes a step up to the next layer of abstraction over the stock market. This makes trading more accessible for new investors and diversification much easier. So again, if you get Cash app from the App Store, Google Play and use the code lexpodcast, you get $10 and cash app will also donate $10 to FIRST, an organization that is helping to advance robotics and STEM education for young people around the world. This show is presented by ExpressVPN. Get it at expressvpn.com lexpod to get a discount and to support this podcast, I've been using ExpressVPN for many years. I love it. It's really easy to use. Press the big power on button and your privacy is protected. And if you like, you can make it look like your location is anywhere else in the world. This has a large number of obvious benefits. Certainly it allows you to access international versions of streaming websites like the Japanese Netflix or the UK Hulu. ExpressVPN works on any device you can imagine. I use it on Linux, shout out to Ubuntu, new version coming out soon, actually Windows, Android. But it's available anywhere else, too. Once again, get it at expressvpn.com lexpod to get a discount and to support this podcast. And now, here's my conversation with Stephen Wolfram.

You and your son Christopher helped create the alien language in the movie Arrival. So let me ask maybe a bit of a crazy question, but if aliens were to visit us on Earth, do you think we would be able to find a common language?

Well, by the time we're saying aliens are visiting us, we've already prejudiced the whole story because the, you know, the concept of an alien actually visiting, so to speak, we already know they're kind of things that make sense to talk about visiting. So we already know they exist in the same kind of physical setup that we do. They're not, you know, it's not just radio signals, it's an actual thing that shows up and so on. So I think in terms of, you know, can one find ways to communicate? Well, the best example we have of this right now is AI. I mean, that's our first sort of example of alien intelligence. And the question is, how well do we communicate with AI? You know, if you were to say, if you were in the middle of a neural net and you open it up and it's like, what are you thinking? Can you discuss things with it? It's not easy, but it's not absolutely impossible. So I Think by the time, given the setup of your question, aliens visiting, I think the answer is yes, one will be able to find some form of communication, whatever communication means. Communication requires notions of purpose and things like this. It's a kind of philosophical quagmire.

So if AI is a kind of alien life form, what do you think visiting looks like? So if we look at aliens visiting, and we'll get to discuss computation and the world of computation. But if you were to imagine, you said you already prejudice something by saying you visit. But how would aliens visit by visit?

There's kind of an implication. And here we're using the imprecision of human language, you know, in a world of the future. And if that's represented in computational language, we. We might be able to take the concept visit and go look in the documentation, basically, and find out exactly what does that mean, what properties does it have? And so on. But by visit, in ordinary human language, I'm kind of taking it to be there's something, a physical embodiment that shows up in a spacecraft. Since we kind of know that that's necessary. We're not imagining in. It's just, you know, photons showing up in a radio signal that, you know, photons in some very elaborate pattern we're imagining it's physical things made of atoms and so on that show up.

Can it be photons in a pattern?

Well, that's good question. I mean, whether there is the possibility, you know, what counts as intelligence. Good question. I mean, it's, you know, and I used to think there was sort of a. Oh, it'll be clear what it means to find extraterrestrial intelligence, et cetera, et cetera, et cetera. I've increasingly realized, as a result of science that I've done that there really isn't a bright line between the intelligent and the merely computational, so to speak. So in our kind of everyday sort of discussion, we'll say things like, the weather has a mind of its own. Well, let's unpack that question. We realize that there are computational processes that go on that determine the fluid dynamics of this and that in the atmosphere, et cetera, et cetera, et cetera. How do we distinguish that from the processes that go on in our brains of, you know, the physical processes that go on in our brains? How do we separate those? How do we say the physical processes going on that represent sophisticated computations in the weather? Oh, that's not the same as the physical processes that go on that represent sophisticated computations in our brains. The answer is I don't think there is a fundamental distinction. I think the distinction for us is that there's kind of a thread of history and so on that connects kind of what happens in different brains to each other, so to speak. And it's a. You know, what happens in the weather is something which is not connected by sort of a thread of civilizational history, so to speak, to what we're used.

To in our story, in the stories that the human brains told us. But maybe the weather has its own stories.

Absolutely. Absolutely. And that's where we run into trouble thinking about extraterrestrial intelligence, because it's like that pulsar magnetosphere that's generating these very elaborate radio signals. Is that something that we should think of as being this whole civilization that's developed over the last, however long, millions of years of processes going on in the neutron star or whatever versus what, what, you know, what we're used to in human intelligence? I think in the end, when people talk about extraterrestrial intelligence and where is it, and the whole Fermi paradox of how come there's no other signs of intelligence in the universe, my guess is that we've got sort of two alien forms of intelligence that we're dealing with artificial intelligence and sort of physical or extraterrestrial intelligence. And my guess is people will sort of get comfortable with the fact that both of these have been achieved around the same time. And in other words, people will say, well, yes, we're used to computers, things we've created, digital things we've created, being sort of intelligent like we are. And they'll say, oh, we're kind of also used to the idea that there are things around the universe that are kind of intelligent like we are, except they don't share the sort of civilizational history that we have. And so they're a different branch. I mean, it's similar to when you talk about life, for instance. I mean, you kind of said life form, I think, almost synonymously with intelligence, which I don't think is. The AIs would be upset to hear.

You equate those things, really probably implied biological life.

Right, right, right.

But you're saying, I mean, we'll explore this more, but you're saying it's really a spectrum, and it's all just a kind of computation. And so it's. It's a full spectrum, and we just make ourselves special by weaving a narrative around our particular kinds of computation.

Yes. I mean, the thing that I think I've kind of come to realize is, you know, at some level, it's a little depressing to realize that there's, there's so little that or liberating. Well, yeah, but I mean, it's, you know, it's the story of science, right. And you know, from Copernicus on, it's like, you know, first we were like convinced our planet's at the center of the universe. No, that's not true. Well then we were convinced there's something very special about the chemistry that we have as biological organisms. That's not really true. And then we're still holding out that hope. Oh, this intelligence thing we have, that's really special. Yeah, I don't think it is. However, in a sense, as you say, it's kind of liberating for the following reason that you realize that what's special is the details of us, not some abstract attribute that we could wonder, oh, is something else going to come along and also have that abstract attribute? Well, yes, every abstract attribute we have something else has it, but the full details of our history, of our civilization and so on. Nothing else has that. That's what, you know, that's our story, so to speak. And that's sort of almost by definition special. So I view it as not being such a. I mean, I was initially, I was like, this is bad. This is, this is kind of, you know, how can we have self respect about, about the things that we do? Then I realized the details of the things we do, they are the story. Everything else is kind of a blank canvas.

So maybe on a small tangent, you just made me think of it. But what do you make of the monoliths in 2001 Space Odyssey in terms of aliens communicating with us and sparking the kind of particular intelligent computation that we humans have? Is there anything interesting to get from that sci fi?

Yeah, I mean, I think what's fun about that is the monoliths are these one to four to nine perfect cuboid things. And in the Earth of a million years ago, or whatever they were portraying with a bunch of apes and so on, a thing that has that level of perfection seems out of place. It seems very kind of constructed, very engineered. So that's an interesting question. What is the, you know, what's the techno signature, so to speak?

What's.

What is it that you see it somewhere and you say, my gosh, that had to be engineered. Now the fact is we see crystals which are also very perfect. And the perfect ones are very perfect. They're nice polyhedra or whatever. And so in that sense, if you say well, it's a sign of sort of, it's a techno signature that it's a perfect polygonal shape, polyhedral shape. That's not true. And so then it's an interesting question, what is the right signature? I mean, like Gauss, famous mathematician, he had this idea you should cut down the Siberian forest in the shape of sort of a typical image of the proof of the Pythagorean theorem on the grounds that this is a kind of cool idea, didn't get done, but on the grounds that the Martians would see that and realize, gosh, there are mathematicians out there. It's kind of, you know, it's the, in his theory of the world that was probably the best advertisement for the cultural achievements of our species. But you know, it's a reasonable question, what can you send or create that is a sign of intelligence in its creation or even intention in its creation?

Yeah, you talk about if we were to send a beacon, can you. What should we send? Is math our greatest creation? What is our greatest creation?

I think it's a philosophically doomed issue. In other words, you send something, you think it's fantastic, but it's kind of like we are part of the universe. We make things that are things that happen in the universe. Computation, which is sort of the thing that we are in some abstract sense using to create all these elaborate things we create, is surprisingly ubiquitous. In other words, we might have thought that we've built this whole giant engineering stack that's led us to microprocessors, that's led us to be able to do elaborate computations. But this idea, computations are happening all over the place. The only question is whether there's a thread that connects our human intentions to what those computations are. And so I think this question of what do you send to kind of show off our civilization in the best possible way? I think any kind of almost random slab of stuff we've produced is about equivalent to everything else. I think it's one of these things.

Where such a non romantic way of phrasing it. I just. Sorry to interrupt, but I just talked to Ann Drouyn, who's the wife of Carl Sagan. And so I don't know if you're familiar with the Voyager. I mean, she was part of sending, I think, brainwaves of, you know, wasn't it hers? It was hers, her brainwaves when she was first falling in love with Carl Sagan. Right. It's this beautiful story, right, that, that perhaps you would shut down the power of that by saying we might as well send anything else. And that's interesting. All of it is kind of in. Interesting, peculiar thing that's.

Yeah, yeah, right. Well, I mean, I think it's kind of interesting to see on the, on the Voyager, you know, golden record thing. One of the things that's kind of cute about that is, you know, it was made, when was it, in the late 70s, early 80s. Yeah. And you know, one of the things, it's a phonograph record. Okay. And it has a diagram of how to play a phonograph record. It does. And you know, it's kind of like, it's shocking that in just 30 years, if you show that to a random kid of today and you show them that diagram and I've tried this experiment, they're like, I don't know what the heck this is. And the best anybody can think of is take the whole record, forget the fact that it has some kind of helical track in it, just image the whole thing and see what's there. That's what we would do today. In only 30 years, our technology has kind of advanced to the point where the playing of a helical mechanical track on a phonograph record is, is now something bizarre. So that's a cautionary tale, I would say, in terms of the ability to make something that in detail sort of leads by the nose, the aliens or whatever to do something. It's like, no, best you're going to do. As I say, if we were doing this today, we would not build a helical scan thing with a needle. We would just take some high resolution imaging system and get all the bits off it and then say, oh, it's a big nuisance that they put in a helix in a spiral. Let's just unravel the spiral and start from there.

Do you think? And this will get into trying to figure out interpretability of AI. Interpretability of computation. Being able to communicate with various kinds of computations. Do you think it would be able to, if you're putting, put your alien hat on, figure out this record, how to play this record?

Well, it's a question of what one wants to do.

I mean, understand what the other party was trying to communicate or understand anything about the other party.

What does understanding mean? I mean, that's the issue. The issue is, it's like when people were trying to do natural language understanding for computers, right? So people tried to do that for years. It wasn't clear what it meant. In other words, you take your piece of English or whatever and you say, gosh, my computer has understood this. Okay, that's nice. What can you do with that? Well, so for example, when we did built WolframAlpha, one of the things was it's doing question answering and so on, and it needs to do natural language understanding. The reason that I realize after the fact that the reason we were able to do natural language understanding quite well and people hadn't before, the number one thing was we had an actual objective for the natural language understanding. We were trying to turn the natural language into computation, into this computational language that we could then do things with. Now, similarly, when you imagine you're alien, you say, okay, we're playing them the record. Did they understand it? Well, depends what you mean. If there's a representation that they have, if it converts to some representation where we can say, oh yes, that's a representation that we can recognize, represents understanding, then all well and good. But actually the only ones that I think we can say would represent understanding are ones that will then do things that we humans kind of recognize as being useful to us.

Maybe trying to understand, quantify how technologically advanced this particular civilization is. So are they a threat to us from a military perspective? Yeah, yeah, that's probably the kind of first kind of understanding that would be interested in.

Gosh, that's so hard. I mean, that's like in the Arrival movie. That was sort of one of the key questions is, you know, why are you here? So to speak?

And it's are you going to hurt us?

Right. But even that is, you know, it's a very unclear, you know, it's like the, are you going to hurt us? That comes back to a lot of interesting AI ethics questions because we might make an AI that says, well, take autonomous cars for instance. Are you going to hurt us? Well, let's make sure you only drive at precisely the speed limit because we want to make sure we don't hurt you, so to speak, because that's. And then, well, something. But you say, but actually that means I'm going to be really late for this thing. And that sort of hurts me in some way. So it's hard to know. Even the definition of what it means to hurt someone is unclear. And as we start thinking about things about AI ethics and so on, that's, you know, something one has to address.

There's always trade offs. And that's the annoying thing about ethics.

Yeah, well, right. And I mean, I think ethics, like these other things we're talking about, is a deeply human thing. There's no abstract you know, let's write down the theorem that proves that this is ethically correct. That's a meaningless idea. You have to have a ground truth, so to speak. That's ultimately sort of what humans want. And they don't all want the same thing. So that gives one all kinds of additional complexity.

In thinking about that one convenient thing in terms of turning ethics into computation, you can ask the question of what maximizes the likelihood of the survival of the species.

That's a good existential issue. But then when you say survival of the species, right, you might say, you might. For example. For example, let's say forget about technology, just hang out and be happy, live our lives, go on to the next generation, go for many, many generations, where, in a sense, nothing is happening. Is that okay? Is that not okay? Hard to know in terms of the attempt to do elaborate things. And the attempt to might be counterproductive for the survival of the species. Like, for instance. I mean, I think it's also a little bit hard to know. So, okay, let's take that as a sort of thought experiment. Okay? You can say, well, what are the threats that we might have to survive the super volcano, the asteroid impact, all these kinds of things. Okay, so now we inventory these possible threats and we say, let's make our species as robust as possible relative to all these threats. I think in the end, it's sort of an unknowable thing. What it takes to. So given that you've got this AI and you've told it, maximize the long term. What does long term mean? Does long term mean until the sun burns out? That's not going to work. Does long term mean next thousand years? Okay, they're probably optimizations for the next thousand years. That it's like if you're running a company, you could make a company be very stable for a certain period of time. Like if your company gets bought by some private investment group, then you can run a company just fine for five years by just taking what it does and removing all R and D. And the company will burn out after a while, but it'll run just fine for a little while. So if you tell the AI, keep the humans okay for 1,000 years, there's probably a certain set of things that one would do to optimize that, many of which one might say, well, that would be a pretty big shame for the future of history, so to speak, for that to be what happens. But I think in the end, as you start thinking about that question, what you realize is there's a whole Sort of raft of undecidability, computational irreducibility. In other words, it's, I mean, one of the good things about sort of the, the, the what our civilization has gone through and what sort of we humans go through is that there's a certain computational irreducibility to it in the sense that it isn't the case that you can look from the outside and just say the answer is going to be this. At the end of the day, this is what's going to happen. You actually have to go through the process to find out. And I think that's, that's both. That feels better in the sense it's not a, you know, something is achieved by going through all of this, all of this process. But it also means that telling the AI go figure out what will be the best outcome. Well, unfortunately it's going to come back and say it's kind of undecidable what to do. We'd have to run all of those scenarios to see what happens. And if we want it for the infinite future, we're thrown immediately into sort of standard issues of kind of infinite computation and so on.

So, yeah, even if you get that the answer to the universe and everything is 42, you still have to actually run the universe.

Yes.

To figure out the question, I guess, or the journey is the point.

Right. Well, I think it's saying to summarize, this is the result of the universe, if that is possible. It tells us the whole sort of structure of thinking about computation and so on and thinking about how stuff works. If it's possible to say, and the answer is such and such, you're basically saying there's a way of going outside the universe and you're getting yourself into something of a sort of paradox because you're saying if it's knowable what the answer is, then there's a way to know it that is beyond what the universe provides. But if we can know it, then something that we're dealing with is beyond the universe. So then the universe isn't the universe, so to speak.

So and in general, as we'll talk about, at least for our small human brains, it's hard to show the result of a sufficiently complex computation. I mean, it's probably impossible right on this side ability. So, and the universe appears by at least the poets to be sufficiently complex that we won't be able to predict what the heck it's all going to.

Well, we better not be able to, because if we can, it kind of denies. I mean, it's, you Know we're part of the universe.

Yeah.

So what does it mean for us to predict? It means that we, that our little part of the universe is able to jump ahead of the whole universe. And you know, this, this quickly winds up. I mean, that it is conceivable the only way we'd be able to predict is if we are so special in the universe. We are the one place where there is computation more special, more sophisticated than anything else that exists in the universe. That's the only way we would have the ability to sort of the almost theological ability, so to speak, to predict what happens in the universe is to say somehow we're better than everything else in the universe, which I don't think is the case.

Yeah, Perhaps we can detect a large number of looping patterns that reoccur throughout the universe and fully describe them and therefore. But then it's, it still becomes exceptionally difficult to see how those patterns interact and what kind of complexity looks.

The most remarkable thing about the universe is that it's. Has regularity at all. Might not be the case.

Does it have regularity?

Absolutely. It's full of, I mean, physics is successful. You know, it's full of laws that tell us a lot of detail about how the universe works. I mean, it could be the case that, you know, the 10 to the 90th particles in the universe, they all do their own thing, but they don't. They all follow. We already know they all follow basically physical, the same physical laws. And that's something, that's a very profound fact about the universe. What conclusion you draw from that is unclear. I mean, in the, you know, the early, early theologians, that was, you know, exhibit number one for the existence of God. Now, you know, people have different conclusions about it. But the fact is, you know, right now, I mean, I happen to be interested. Actually I've just restarted a long running kind of interest of mine about fundamental physics. I'm kind of like, I'm on a bit of a quest which I'm about to make more power public to, to see if I can actually find the fundamental theory of physics.

Excellent. We'll come to that. And I just had a lot of conversations with quantum mechanics folks with. So I'm really excited on your take because I think you have a fascinating take on the, the, the fundamental nature of our reality from a physics perspective. So. And what might be underlying the kind of physics as we think of it today? Okay, let's take a step back. What is computation?

It's a good question. Operationally, computation is following rules. That's kind of it. I mean, computation is the result, is the process of systematically following rules. And it is the thing that happens when you do that.

So taking initial conditions or taking inputs and following rules, I mean, what are you following rules on? So there has to be some data, some.

Not necessarily. It can be something where there's a very simple input and then you're following these rules. And you'd say there's not really much data going into this. You could actually pack the initial conditions into the rule if you want to. So I think the question is, is there a robust notion of computation?

That is, what does robust mean?

What I mean by that is something like this. So one of the things in a different, in an area of physics, something like energy, okay, there are different forms of energy, but somehow energy is a robust concept that isn't particular to kinetic energy or nuclear energy or whatever else. There's a robust idea of energy. So one of the things you might ask is there's the robust idea of computation, or does it matter that this computation is running in a Turing machine? This computation is running in a CMOS silicon cpu. This computation is running in a fluid system, in the weather, those kinds of things, or is there a robust idea of computation that transcends the sort of detailed framework that it's running in?

Is there?

Yes. I mean, it wasn't obvious that there was. So it's worth understanding the history and how we got to where we are right now. Because to say that there is is a statement in part about our universe. It's not a statement about what is mathematically conceivable. It's about what actually can exist for us.

Maybe you can also comment because energy as a concept is robust, but there's also, it's intricate, complicated relationship with matter, with mass is very interesting. Of particles that carry force and particles that sort of. Particles that carry force and particles that have mass. These kinds of ideas, they seem to map to each other, at least in the mathematical sense. Is there a connection between energy and mass and computation? Or are these completely disjoint ideas?

We don't know yet. The things that I'm trying to do about fundamental physics may well lead to such a connection, but there is no known connection at this time.

So can you elaborate a little bit more on what, how do you think about computation? What is computation?

Yeah, so I mean, let's, let's tell a little bit of a historical story.

Yes.

Okay. So you know, back, go back 150 years, people were making mechanical Calculators of various kinds. And, you know, the typical thing was you want an adding machine, you go to the adding machine store. Basically, you want a multiplying machine, you go to the multiplying machine store. They're different pieces of hardware. And so that means that at least at the level of that kind of computation and those kinds of pieces of hardware, there isn't a robust notion of computation. There's the adding machine kind of computation. There's the multiplying machine notion of computation. And they're disjoint. So what happened in around 1900, people started imagining, particularly in the context of mathematical logic, could you have something which would represent any reasonable function? And they came up with things. This idea of primitive recursion was one of the early ideas, and it didn't work. There were reasonable functions that people could come up with that were not represented using the primitives of primitive recursion. Okay, so then along comes 1931 and Godel's theorem and so on. And as in looking back, one can see that as part of the process of establishing Godel's theorem, Godel basically showed how you could compile arithmetic, how you could basically compile logical statements, like this statement is unprovable, into arithmetic. So what he essentially did was to show that arithmetic can be a computer in a sense, that's capable of representing all kinds of other things. And then turing came along, 1936 came up with Turing machines. Meanwhile, Alonzo Church had come up with lambda calculus. And the surprising thing that was established very quickly is the Turing machine idea about what computation might be is exactly the same as the lambda calculus idea of what computation might be. And then there started to be other ideas, register machines, other kinds of. Other kinds of representations of computation. And the big surprise was they all turned out to be equivalent. So in other words, it might have been the case, like those old adding machines and multiplying machines, that Turing had his idea of computation, Church had his idea of computation, and they were just different, but it isn't true. They're actually all equivalent. So then by, I would say, the 1970s or so, in sort of the. The computation, computer science, computation theory area, people had sort of said, oh, Turing machines are kind of what computation is. Physicists were still holding out, saying, no, no, no, that's just not how the universe works. We've got all these differential equations, We've got all these real numbers that have infinite numbers of digits.

Yeah, the universe is not a Turing machine.

Right. The Turing machines are a small subset of that are the things that we make in microprocessors and engineering structures and so on. So probably actually, through my work in the 1980s about sort of the relationship between computation and models of physics, it became a little less clear that there was this big sort of dichotomy between what can happen in physics and what happens in things like Turing machines. And I think probably by now people would mostly think. And by the way, brains were another kind of element of this. I mean, Godel didn't think that his notion of computation or what amounted to his notion of computation would cover brains. And Turing wasn't sure either. But although he was a little bit. He got to be a little bit more convinced that it should cover brains. But so, you know, I would say by probably Sometime in the 1980s, there was beginning to be sort of a general belief that, yes, this notion of computation that could be captured by things like Turing machines was reasonably robust. Now the next question is, okay, you can have a universal Turing machine that's capable of being programmed to do anything that any Turing machine can do. And this idea of universal computation, it's an important idea. This idea that you can have one piece of hardware and program it with different pieces of software, that's kind of the idea that launched most modern technology. I mean, that's the idea that launched computer revolution, software, et cetera. So important idea. But the thing that's still kind of holding out from that idea is, okay, there is this universal computation thing, but seems hard to get to. It seems like you want to make a universal computer, you have to kind of have a microprocessor with a million gates in it, and you have to go to a lot of trouble to make something that achieves that level of computational sophistication. Okay, so the surprise for me was that stuff that I discovered in the early 80s, looking at these things called cellular automata, which are really simple computational systems. The thing that was a big surprise to me was that even when their rules were very, very simple, they were doing things that were as sophisticated as they did when their rules were much more complicated. So it didn't look like this idea, oh, to get sophisticated computation, you have to build something with very sophisticated rules. That idea didn't seem to pan out. And instead it seemed to be the case that sophisticated computation was completely ubiquitous, even in systems with incredibly simple rules. And so that led to this thing that I call the principle of computational equivalence, which basically says, when you have a system that follows rules of any kind, then whenever the system isn't doing things, that are in some sense obviously simple, then the computation that the behavior of the system corresponds to is of equivalent sophistication. So that means that when you kind of go from the very, very, very simplest things you can imagine, then quite quickly you hit this kind of threshold above which everything is equivalent in its computational sophistication. Not obvious. That would be the case. I mean, that's a science fact.

Well, no, no, no, hold on a second. So this you've opened with a new kind of science. I mean, I remember it was a huge eye opener that such simple things can create such complexity. And yes, there's an equivalence, but it's not a fact. It just appears to. I mean, as much as a fact as sort of these theories are so elegant that it seems to be the way things are. But let me ask, sort of. You just brought up previously, kind of like the communities of computer scientists with their Turing machines, the physicists with their universe, and whoever the heck, maybe neuroscientists looking at the brain, what's your sense in the equivalence you've shown through your work that simple rules can create equivalently complex Turing machine systems. Right. Is the universe equivalent to the kinds of tutoring machines? Is the human brain a kind of toy machine? Do you see those things basically blending together or is there still a mystery about how disjoint they are?

Well, my guess is that they all blend together, but we don't know that for sure yet. I mean, this. You know, I should say I said rather glibly that the principle of computational equivalence is sort of a science fact. And I was using air quotes, air quotes for the science fact because when you. It is a. I mean, just to talk about that for a second and then we'll. The thing is that it has a complicated epistemological character, similar to things like the second law of thermodynamics, the law of entropy increase. What is the second law of thermodynamics? Is it a law of nature? Is it a thing that is true of the physical world? Is it something which is mathematically provable? Is it something which happens to be true of the systems that we see in the world? Is it in some sense a definition of heat, perhaps? Well, it's a combination of those things and it's the same thing with the principle of computational equivalence. And in some sense, the principle of computational equivalence is at the heart of the definition of computation because it's telling you there is a thing, there is a robust notion that is equivalent across all these systems and doesn't depend on the details of each individual system. And that's why we can meaningfully talk about a thing called computation. And we're not stuck talking about, oh, there's computation in turing machine number 3785 and, et cetera, et cetera, et cetera. That's why there is a robust notion like that. Now, on the other hand, can we prove the principle of computational equivalence? Can we prove it as a mathematical result? Well, the answer is, actually, we've got some nice results along those lines that say, throw me a random system with very simple rules. Well, in a couple of cases, we now know that even the very simplest rules we can imagine of a certain type are universal and do sort of follow what you would expect from the principle of computational equivalence. So that's a nice piece of sort of mathematical evidence for the principle of.

Computational equivalence, just to link on that point. Simple rules creating sort of these complex behaviors. But is there a way to mathematically say that this behavior is complex, that you mentioned that you cross a threshold?

Right? So there are various indicators. So, for example, one thing would be, is it capable of universal computation? That is, given the system, do there exist initial conditions for the system that can be set up to essentially represent programs to do anything you want, to compute primes, to compute PI, to do whatever you want? So that's an indicator. So we know in a couple of examples that, yes, the simplest candidates that could conceivably have that property do have that property. And that's what the principle of computational equivalence might suggest. But this principle of computational equivalence, one question about it is, is it true for the physical world? It might be true for all these things we come up with, the Turing machines, the cellular automata, whatever else. Is it true for our actual physical world? Is it true for the brains, which are an element of the physical world? We don't know for sure. And that's not the type of question that we will have a definitive answer to, because it's, you know, there's a sort of scientific induction issue. You can say, well, it's true for all these brains, but this person over here is really special, and it's not true for them. And you can't, you know, the only way that that cannot be what happens is if we finally nail it and actually get a fundamental theory for physics and it turns out to correspond to, let's say, a simple program. If that is the case, then we will basically have reduced physics to a branch of mathematics in the sense that we will not be right now with physics, we're like, well, this is the theory that this is the rules that apply here. But in the middle of that, right by that black hole, maybe these rules don't apply and something else applies. And there may be another piece of the onion that we have to peel back. But if we can get to the point where we actually have. This is the fundamental theory of physics. Here it is, it's this program. Run this program and you will get our universe. Then we've kind of reduced the problem of figuring out things in physics to a problem of doing some. What turns out to be very difficult, irreducibly difficult mathematical problems. But it no longer is the case that we can say, that somebody can come in and say, whoops, you were right about all these things about Turing Machines, but you're wrong about the physical universe. We know there's sort of ground truth about what's happening in the physical universe. Now, I happen to think, I mean, you asked me at an interesting time because I'm just in the middle of starting to re energize my project to kind of study the fundamental theory of physics. As of today. I'm very optimistic that we're actually going to find something and that it's going to be possible to see that the universe really is computational in that sense. But I don't know, because we're betting against the universe, so to speak. And I didn't. It's not like when I spend a lot of my life building technology and then I know what's in there, right? And it may have unexpected behavior, it may have bugs, things like that. But fundamentally I know what's in there for the universe. I'm not in that position, so to speak.

What kind of computation do you think the fundamental laws of physics might emerge from? Just to clarify. So you've done a lot of fascinating work with kind of discrete kinds of computation that, you know, cellular automata, and we'll talk about it, have this very clean structure. It's such a nice way to demonstrate that simple rules can create immense complexity. But what kind, you know, is that, actually? Are cellular automata sufficiently general to describe the kinds of computation that might create the laws of physics? Just to give. Can you give a sense of what kind of computation do you think would create laws of physics?

So this is a slightly complicated issue because as soon as you have universal computation, you can in principle simulate anything with anything, but it is not a natural thing to do. And if you're asking were you to try to find our physical universe by looking at possible programs in the computational universe? Of all possible programs, would the ones that correspond to our universe be small and simple enough that we might find them by searching that computational universe? We got to have the right basis, so to speak. We've got to have the right language, in effect, for describing computation, for that to be feasible. So the thing that I've been interested in for a long time is what are the most structuralist structures that we can create with computation? So, in other words, if you say a cellular automaton, it has a bunch of cells that are arrayed on a grid, and it's very. And every cell is updated in synchrony at a particular. When there's a click of a clock, so to speak, and it goes a tick of a clock, and every cell gets updated at the same time. That's a very specific, very rigid kind of thing. But my guess is that when we look at physics and we look at things like space and time, that what's underneath space and time is something as structureless as possible. That what we see, what emerges for us as physical space, for example, comes from something that is sort of arbitrarily unstructured underneath. And so I've been for a long time interested in kind of what are the most structuralist structures that we can set up. And actually, what I had thought about for ages is using graphs, networks, where essentially. So let's talk about space, for example. So what is space? Is a kind of a question one might ask. Back in the early days of quantum mechanics, for example, people said, oh, for sure, space is going to be discrete because all these other things we're finding are discrete. But that never worked out in physics. And so space and physics today is always treated as this continuous thing, just like Euclid imagined it. I mean, the very first thing Euclid says in his sort of common notions is a point is something which has no part. In other words, there are points that are arbitrarily small, and there's a continuum of possible positions of points. And the question is, is that true? And so, for example, if we look at, I don't know, fluid, like air or water, we might say, oh, it's a continuous fluid. We can pour it. We can do all kinds of things continuously. But actually, we know, because we know the physics of it, that it consists of a bunch of discrete molecules bouncing around, and only in the aggregate is it behaving like a continuum. And so the possibility exists that that's true of space, too. People haven't managed to make that work with existing frameworks in physics. But I've been interested in whether one can imagine that underneath space and also underneath time is something more structureless. And the question is, is it computational? So there are a couple of possibilities. It could be computational, somehow fundamentally equivalent to a Turing machine, or it could be fundamentally not. So how could it not be? It could not be. So a Turing machine essentially deals with integers, whole numbers at some level. And it can do things like it can add one to a number. It can do things like this.

It can also store whatever the heck it did.

Yes, it has an infinite storage. But when one thinks about doing physics or sort of idealized physics or idealized mathematics, one can deal with real numbers, numbers with an infinite number of digits, numbers which are absolutely precise. Someone can say, we can take this number and we can multiply it by itself.

Are you comfortable with infinity in this context? Are you comfortable in the context of computation? Do you think infinity plays a part?

I think that the role of infinity is complicated. Infinity is useful in conceptualizing things. It's not actualizable. Almost by definition, it's not actualizable.

But do you think infinity is part of the thing that might underlie the laws of physics?

I think that no. I think there are many questions that you ask about. You might ask about physics which inevitably involve infinity. Like when you say, is faster than light travel possible? You could say, given the laws of physics, can you make something even arbitrarily large, even quotes, infinitely large, that will make faster than light travel possible? Then you're thrown into dealing with infinity as a kind of theoretical question. But, I mean, talking about what's underneath space and time and how one can make a computational infrastructure. One possibility is that you can't make a computational infrastructure in a Turing machine sense, that you really have to be dealing with precise real numbers. You're dealing with partial differential equations, which have precise real numbers at arbitrarily closely separated points. You have a continuum for everything. Could be that that's what happens, that there's sort of a continuum for everything and precise real numbers for everything. And then the things I'm thinking about are wrong. And that's the risk you take if you're trying to sort of do things about nature. You might just be wrong. For me, personally, it's kind of a strange thing, because I've spent a lot of my life building technology where you can do something that nobody cares about, but you can't be sort of wrong in that sense. In the sense you build your technology and it does what it does, but I think this question of what, you know, what the sort of underlying computational infrastructure for the universe might be. So it's sort of inevitable. It's going to be fairly abstract, because if you're going to get all these things, like, there are three dimensions of space, there are electrons, there are muons, there are quarks, there are this, you don't get to. If the model for the universe is simple, you don't get to have sort of a line of code for each of those things. You don't get to have sort of the. The muon case, the tau lepton case, and so on.

All of those have to be emergent somehow.

Right.

Something deeper.

Right. So that means it's sort of inevitable that it's a little hard to talk about what the sort of underlying structuralist structure actually is.

Do you think human beings have the cognitive capacity to understand, if we're to discover it, to understand the kinds of simple structure from which these laws can emerge? Like, do you think that's a good pursuit?

Well, here's what I think. I think that. I mean, I'm right in the middle of this right now, so I'm telling you that I think you'll hit a wall. Yeah, I mean, this human has a hard time understanding, you know, a bunch of the things that are going on. But. But what happens in understanding is one builds waypoints. I mean, if you said understand modern 21st century mathematics, starting from, you know, counting back in whenever counting was invented, 50,000 years ago, whatever it was, that will be really difficult. But what happens is we build waypoints that allow us to get to higher levels of understanding. And we see the same thing happening in language. When we invent a word for something, it provides kind of a cognitive anchor, a kind of a waypoint that lets us. Like a podcast or something. You could be explaining. Well, it's a thing which works this way, that way, the other way. But. But as soon as you have the word podcast and people kind of societally understand it, you start to be able to build on top of that. And so I think. And that's kind of the story of science actually, too. I mean, science is about building these kind of waypoints where we find this sort of cognitive mechanism for understanding something, then we can build on top of it. We have the idea of, I don't know, differential equations. We can build on top of that. We have this idea or that idea. So my hope is that if it is the case, that we have to go all the way, sort of from the sand to the computer and there's no waypoints in between, then we're toast. We won't be able to do that.

Well, eventually we might. So if we're, if we're us, clever apes are good enough for building those abstractions, eventually from sand we'll get to the computer. Right. And it just might be a longer journey.

The question is whether it is something that you ask whether our human brains will quotes understand what's going on. And that's a different question because for that it requires steps that are sort of from which we can construct a human understandable narrative. And that's something that I think I am somewhat hopeful that that will be possible. Although as of literally today, if you ask me, I'm confronted with things that I don't understand very well.

So this is a small pattern in the computation, trying to understand the rules under which the computation functions. And it's an interesting possibility under which kinds of computations such a creature can understand itself.

My guess is that within. So we didn't talk much about computational irreducibility, but it's a consequence of this principle of computational equivalence and it's sort of a core idea that the one has to understand, I think, which is the question is you're doing a computation, you can figure out what happens in the computation just by running every step in the computation and seeing what happens. Or you can say, let me jump ahead and figure out, have something smarter that figures out what's going to happen before it actually happens. And a lot of traditional science has been about that act of computational reducibility. It's like we've got these equations and we can just solve them and we can figure out what's on going.