Future of Science and Technology Q&A (January 19, 2024)

Foreign. Welcome to another episode of Q and A about the future of science and technology. I am still not in my natural habitat, so technical things could go wrong here. All right, let's see. All kinds of questions that we have saved up here. One from Pavel here. Can AI be swallowed by more advanced AI? By feeding it virtual input, the motivation could be increased efficiency of larger AIs. Well, let's see, I mean, the question is computation can do all kinds of things. The issue is to align computation with the kinds of things that we humans care about and, and want to do things with. And the main sort of idea in modern times is sort of train the AIs with lots of material made by humans. And you know, there's maybe, I don't know, 5 billion, maybe 10 billion web pages that are sort of reasonably human written. Maybe if you include all of the sort of hidden stuff. There's maybe 100 billion pages worth of, of things humans have written. There's zillions of videos, things like this. There's lots of material that kind of shows what we humans care about and kind of paints the picture of the kind of human relevant world. And that's a place where if you want AIs to do kind of human relevant things, then you've got to kind of feed them human relevant training material that they will then imitate, so to speak. If you want them to do things that are related to, for example, the way the physical world works, then having them observe the physical world and say, oh, I've grown up learning about how chemical reactions work or how proteins fold or something like this. And that's my experience of things. There's a question, it's not completely obvious to what extent the idea of just train with what's already known is enough to tell you what will come next, so to speak, with a lot of the things in the kind of human world. What we're finding is that we humans, possibly because our brains are not that different from neural nets that are sort of put into AIs, that the things that kind of we care about and that we are able to do are things that can readily be reproduced by kind of a trained, you know, a trained neural net kind of approach. Whether that's true of the things that exist in the physical world is much less clear. It's a little bit complicated because there are things that are intrinsic to the physical world where the physical world is doing what it does. And there are aspects of the physical world that we humans have decided we care about. Like for example, if we look at, I don't know, clouds in the sky. The detailed shape of clouds is something that we humans don't usually make much comment on other than to say, you know, that one looks like a chicken or something, but we're. That's something. But on the other hand, the question of, you know, is it raining right now? Is something that we humans notice. And so it could be the case that even though the underlying physics that's making the billowings of the clouds and so on, is not something that is readily aiable, so to speak, that is readily sort of extrapolatable from training data, that it's still the case that something like will it rain or not? Is. And that's the part we humans care about. So this question about whether sort of things in the physical world are what one might call sort of aiable, it could be the case that at some intrinsic level they're not. In fact, that's my belief is that they're not for all sorts of reasons, but. But that it's possible that the ones of human relevance are more likely to be not clear whether they are or aren't. But that's not something to ask about. Sort of the underlying processes we're running into this phenomenon of computational irreducibility that I seem to talk about a lot and that I kind of came up with in the 1980s, this idea that just because, you know, the rules for a system doesn't mean you can immediately kind of jump ahead and say everything about what the system does. You are, in general, have to just go through sort of the irreducible computational process of seeing each step of what happens. And if you are an AI that's sort of really trying to just say, is this something a bit like what I've seen before? Oh, then I know what's going to happen. That's not good enough. If you have computational irreducibility. Computational. Computational irreducibility, you have to follow the steps, and there will always be surprises. And those surprises will be things you can't deduce from just saying, well, what have I seen before? Let me train on what I've seen before as a very practical matter. I was just looking actually at the ability to predict computational processes like cellular automata with things like neural nets, transformers, all the fancy technology and so on. And the answer is it does kind of sort of okay in some cases. And when things get complicated and when sort of unexpected things start to happen in these computational processes, the kind of the AI predictors just sort of fall over and don't work now. You know, there are all sorts of questions about what it's going to take to kind of get get to different levels of kind of sort of human like performance. I mean, that's a thing often of interest to us humans is can we get an AI to do human like things? Again, separate from can we have the AI predict the positions of planets or something at some point in the future? That is driven by physics. Can we get an AI to have human like performance on telling apart pictures of cats and dogs? That's something where sort of the test is can it emulate what we humans do? Not can it follow what the physical world does? But again, more complicated in the case of the physical world is often what we want is not just the raw physical world, but the physical world as perceived by humans in some way or another. So I think we can kind of ask about all these different things about what's possible and what's not. I think one of the things that was obviously the big surprise of a bit more than a year ago now was the whole chatgpt story of, gosh, you can have an AI that produces kind of pretty human like language and sort of why does that work? How does it work? And I think the realization is it probably means that human like language isn't as kind of complicated as we thought it was. It has more regularities that we perhaps should be embarrassed for, for not having noticed in the past. And I think that same thing can be said about a bunch of the places where sort of there's been success in where the AI has been able to sort of jump ahead and do things that seemed magical to us. It's like, well, that's because we didn't notice something that we could have noticed that human brains could have dealt with. We just didn't happen to go in that direction. So there are all sorts of questions about, well, all sorts of questions that that raises. Like one question is, okay, so if the AI notices these sort of regularities in the world that we didn't notice, in a sense, science is about recognizing those regularities. So is it possible that the AI can sort of find regularities that will kind of build, drive forward science? Here's a couple of issues with that. One issue is it's one thing to say, yes, I know what's going to happen. I have this black box model that I sort of feed the input into a neural net and it says it's going to, the answer is going to be three or something. That's sort of case Number one, where it's kind of a black box thing. Case number two is I can tell, kind of a human narrative or I can derive it's going to be the number three, with all kinds of human steps inside. There's, and I think there are plenty of cases, particularly in technology, where getting that camera to focus correctly on the object of interest in the image, it doesn't matter whether there's an explainable algorithm for doing that. It's something where when trying to achieve a technological purpose, that's what we want. It's, I think, less satisfactory in some cases if you say, well, you know, why does this, if you're trying to have science be this kind of, this way of kind of taking what. I mean, what science at some level, natural science at some fundamental level is, is taking kind of the complexity of the world and figuring out how to make a narrative based on that that sort of addresses the main things that are going on in a way that sort of fits in our finite minds. That's kind of the key activity of natural science is to do that and in a sense by saying, well, we just have this black box which can say this is what's going to happen. But there's never going to be a human level narrative about how that works. Isn't our traditional view of how natural science is supposed to be set up? Doesn't mean it's not useful, doesn't mean it's not, but it's different from what's been done in kind of natural science as it's evolved. Now it has to be said that natural science as it's evolved kind of in a sense is evading computational irreducibility. Wherever you say, well, you just have to run the system and see what it does. That's precisely saying there isn't a narrative that can be sort of a reduced narrative that we get to stuff in our brains, but that tells us what's going to happen. So I think this question about sort of when can we expect that sort of you can just learn from experience and see what's going to happen. And whether you can do that for systems in general or for the aspects of systems that we humans care about, those are lots of different kinds of questions. And another issue is this. Maybe it's the case that if you're using something like a neural net and you're trying to take all of this sort of data about what happens in the world and you're effectively making a model for that, in a vastly simpler case, when you make a model for Something you might fit a mathematical formula to some data and you might say, that's our model. In the case of a neural net, you might be fitting, in the end, you are fitting a mathematical formula to the data. But it's a formula with 100 billion parameters in it or something. And it's a formula that has a particular structure that corresponds to this neural net. And there's kind of the question of, well, if you have a model of that kind, sort of how big a model do you need to be able to fit this or that kind of thing? So, for example, let's say we're talking about mathematical formula models. That's often the domain of statistics. It's kind of like, oh, is this thing fit by two Gaussian distributions or something? A formula that involves, let's say, four parameters or something like that. Is that a decent fit to what we're seeing? Can we say. I don't know, I don't know why I'm thinking of this case, but the law of friction, for example, F equals mu times N. It's a linear law. The frictional force is linearly dependent on the normal force. That's a very simple model. It's not precisely correct, but. But it's a good approximation for many purposes. And the fact that that whole collection of sort of physical phenomena is modelable with that simple linear formula is an interesting fact. Now the question is, if you say, well, let's take human language for example, how big a formula do we need effectively to do predictions to model human language as human language is typically used? And the answer we know from ChatGPT is with 200 billion parameters, we can do a reasonable job of sort of essay writing type human language activities. And there's sort of a question of, is that how does that 200 billion parameters or something, how does that kind of relate to what human language is? And for instance, could it be the case that there is sort of a point, a threshold, at which you suddenly start to be able to do human language like things? Oh, if we only had 10 billion parameters, we'll never get there, and so on. Now there's a little bit of kind of circumstantial evidence about these kinds of things. So if you look at the history of machine learning, for example, there have been these particular times when sort of major advances were made that sort of vaguely corresponded to the amount of training time, the size of the neural nets, these kinds of things, the sort of, the early efforts in doing, you know, can we recognize, you know, digits from 0 to 9 that was, you know, 1980s and so on. Even earlier than that, can we recognize, you know, 5,000 different kinds of images? That was 2011, 2012. What? You know, there are these thresholds that seem to be somewhat associated with the sizes of networks and the amounts of training data and so on. And so one of the things that's obviously an interesting speculation is if you want to get to the point of having reasonable human language like stuff, there's a threshold. I tend to think the threshold is considerably lower than the 200 billion parameters because that happens to sort of package in a lot of detailed knowledge about the world that you don't really need to have good sort of common sense kind of linguistic capability. But maybe it's a tenth the size. Maybe it's, you know, 50th the size. It's probably not a million parameters. That's probably not enough. So then the question would be, if we look at kind of the history of biology and we say, okay, you know, the great achievement of our species from a couple hundred thousand years ago or whatever, was we got human language. And notably another great feature of our species is we got a bunch more brain, a bunch more neurons in our brains and more structure in our brains. And you know, is it conceivable that there is a threshold at which you can have sort of reasonable human language? I don't know. I think that you could ask the question, typical human languages have 30,000, 40,000 words in them that get used. I think that's true of all human languages. I'm not sure. I think that's true of 7,000 or so extant human languages. I think that's true of all of them. Now there are constructed languages that have many fewer words, but there are fewer things you can say about the world with those constructed languages. You can even have ones down to a couple hundred words and you can have a kind of general pleasantries, discussion with those languages, but you can't have a. I mean, for example, the conversation we're having right now would not really be accessible to a language with 200 words in it. Now it raises the question of if we scaled up, if we had a language with a million words, could we express a lot more kinds of things than we can express in our language with 40,000 words or whatever? Could we, you know, could it be the case that insofar as we might talk about philosophy with our 40,000 word language, if we had a million word language, that we could have a completely different level of discourse about things? Well, I don't Think we know. But there is this possibility that, for example, getting to kind of reasonable human language requires some threshold, and maybe there's another threshold higher up where we get to something sort of a sort of superhuman language. You could argue both ways on this. I think one argument against it would be the phenomenon, the principle of computational equivalence, phenomenon of universal computation and so on. That's a place where, if you want to sort of get something that can do arbitrary computation, you say, well, I've got to add all these capabilities to the system, and I've got to make a Turing machine. We know from my efforts and so on from the mid 2000s, and this prize that we had for. Is this Turing machine the simplest possible universal Turing machine? Answer yes, we know that the simplest possible sort of Turing machine that can do arbitrary computation is this absolutely tiny thing with two states and three colors. But sort of there's this question of once you've got to the point where you can do universal computation, then there's no further sort of fundamental computational sophistication. You can get beyond that. And in fact, it could be the case that if you're trying to do these things, let's say with a neural net that's incredibly inefficient and that the neural net has to just go sort of, you know, do all sorts of handstands and so on to, you know, and cascades of handstands, so to speak, to be able to get to the point of doing what, if you represented a slightly different way, will be a rather straightforward piece of structure, so to speak. But that, you know, but that. So the argument would be, once the neural net, by all kinds of stuff, has been able to get to this sort of universal computation point, then computationally it's in principle able to go further. Actually, neural nets aren't good universal computers in the way they're usually used, because there's this kind of. One of the features of universal computers is they can kind of run with arbitrarily much memory and for arbitrarily long, and that the typical architecture of neural nets right now, at least, doesn't have that capability. But so the question would be, the phenomenon we see there is that with universal computation, we get to this threshold. And once you're above this threshold, there's no further you can go. And it could be the case that with things like language, human language and so on, that the same thing is true. Once you're above that threshold, you can, yes, you can add more words to the language and you can get more Specialized, you can have more jargon and so on, but fundamentally you're there in terms of being able to have sort of an expression of thoughts that has the sort of this compositionality of thoughts, can nest within other thoughts and so on, and that you're kind of. That there's no further you can go. Or it could be that if we had a million words, we'd be able to sort of get to very different places. I think what we can see is there's a question of quantitative versus qualitative changes. What we can see is that as we kind of formalize things with, for example, computational language, Wolfram language effort, so to speak, as we try and sort of formalize the world in computational terms, which has been our last four decades effort to try and represent the world in this kind of computational way, that it is certainly something as a practical matter where you can build much taller towers with that kind of formalized structure than you can if you're kind of just sort of just using sort of plain language and so on. It allows you to know that your tower is not going to fall over. You don't, you know, the foundations are solid, the bricks are hard, so to speak, as you build the thing up. So, you know, I think another thing to sort of wonder, well, there's all these questions about sort of when we look at the future of, let's say, science, to what extent is AI going to. And this kind of neural net idea and so on, to what extent is that going to change, provide change to what's possible in science? It's worth saying there are things that are the whole using computation to represent and formalize ideas that turn into science. That's clearly a winning thing and that has a long way to run, so to speak. And there are pieces to that. Once you formalize things computationally, there are kinds of things you can do that you couldn't do before. Like you could say something like, let's enumerate all possible theories of this. Let's. Let's enumerate all possible systems that do this or that, that have this or that structure. This is the area that I like to call rul, the study of simple rules, let's say, and what they could possibly do. Or equivalently, let's enumerate all possible chemical structures and see whether any of them do this or that thing. Let's enumerate all possible algorithms of this type. Let's see whether any of them do the thing. We want that idea of kind of ruleology and exploring the computational universe. It's not a very AI ish thing because it's mostly. It's a thing where you're really just computing out there. But perhaps then the question is, can AI help you to figure out, when you're searching this big space of possibilities, can it help you to say that's a more fertile area than that, and so on? That's one thing it could do. Another thing it could do is, is be able to say, once you've got a candidate thing, oh, I can predict more quickly that that's hopeless, or that's interesting to look at. I mean, this type of thing, you know, there are endless efforts to do drug design, for example, using sort of AI ish methods where you're enumerating and sometimes you're enumerating, sometimes you're, you're selecting. I don't think. I mean, you could, you can use the, the AI in a sense. It's very easy to get the AI to be creative because, you know, you just throw a random number and you say, let's put the molecules in this, put the atoms in this configuration and let's see whether that's a candidate. Now the question is, are you, are you, are you arranging it so that whichever way the dice comes down, you'll always get a viable candidate, or at least a plausibly viable candidate. And for that it matters sort of what the underlying model of what these molecules might be like is. And if you set the parameters up correctly, then however you pick them, you're going to get a reasonable molecule. And that's a sort of AI ish thing to do to kind of figure out what's a reasonable parameterization so that you can then drive it with ruliology effectively and just start enumerating things. So I think there are questions like that. I think another type of question is if you're solving math problems, like there was recent announcement about geometry problems, doing geometry proofs. If you say, well, is there a point at which human geometry proofs are kind of. Yeah, we kind of know how to do those because we've seen enough examples that we can kind of piece the pieces together. And for the level of things that are human relevant, we can just say, yes, we've seen enough examples, we can just see what to do. I think the question of can you always do? Well, geometry is a special case because geometry is a decidable mathematical theory. Euclidean geometry is, which means in principle you can just, for example, turn it into an algebra problem. We can do that in Wolf language and Mathematica. You can sort of turn it into algebra problem. You just grind it and you can get to a result. It might take a while, but you know you can do it. There are plenty of other places in mathematics. Even in arithmetic, you say, I've got this equation that involves just integers and it is, you know, x cubed plus Y cubed equals Z cubed plus whatever. And say, find X, Y and Z that satisfy this equation. There is no upper bound in general on how far you have to look to do that. That's a undecidable theory. And so that's a thing where you can't guarantee. If you say, well, is it conceivable that I could just sort of close this off by finding enough sort of rules for how to do it and that then I'd always be able to do it in limited time? And the answer is, no, you can't do that. But I mean, there's this question of if you want to achieve sort of human level performance on some particular task, to what extent, sort of how many parameters do you need to be able to do that, how much training material do you need to be able to do that? And so on. And to what extent are there kind of thresholds of, oh, it doesn't work at all, versus its sort of achieved human level or something? And you might have thought this was a surprise with things like ChatGPT. You might have thought that as you kind of added more neurons or whatever else, added more training data, the thing would progressively get better. But in fact, that's not our experience. In fact, our experience is it was really crummy until it reached above a threshold and then it looked really good. Now, is that a phenomenon of the thing itself or is that more a phenomenon of our perception? I mean, it's kind of like, you could say, I don't know, this is what's a good example. I mean, you know, when something kind of is good enough that we can really use it versus it's, it's kind of a little bit too crummy. What is a good example of that? There must be plenty in sort of mechanical kinds of things. I don't know if you have an automatic screwdriver or something where the thing doesn't quite have enough, can't quite exert enough torque to be able to screw that screw in or something. It's like, well, that was nice, but it doesn't quite make it for the purposes I have, because typical screws in a piece of wood or something require this amount of force and it doesn't really quite make it. And so I might as well just throw that device away. And as soon as it's above the critical torque level, then it's like, okay, great, it's going to work. And it could be the case that, that, you know, we don't really have a good sense for something like language of what that kind of what that thresholding looks like. My own guess is that there are certain, once you've achieved, once you've learned certain regularities, you're above that threshold. Like, for example, I could be below the threshold where I knew basic grammar and I could be continually making up sentences which are just grammatically wrong. You know, cat, dog, chase mouse. I don't know, cat, dog, read big was okay, in some language that word order might be fine, but I can clearly make up sentences which you immediately know are wrong because they just violate standard syntactic grammar in English, for example. And there will be a point at which that no longer happens where it's syntactically correct and how that threshold works is not clear. You know, I think I was yakking on here about a lot of things and not really even addressing the question that was being asked, which was, I think about feeding sort of the next level of AI things that were the experience of the previous level of AI. And you know, I think that is both an opportunity and a danger, so to speak. I mean, in the sense that right now we are getting AIs that do human relevant things because we train them on human produced stuff. By the time we're training them on AI produced stuff, we've got this sort of strange sort of dynamics where the AI thing is, if the AI thing was exactly a perfect model for humans, then training on AI based stuff will be equivalent to training on humans. And it's all good. But assume that the AI based stuff isn't quite equivalent to humans. For example, let's say we've got essays and for whatever reason intrinsic or kind of control, the essays are very bland that the AI produces. And then the next generation of AIs are trained only on very bland essays. Well, obviously then over the course of time, as we train more and more levels of AIs things just get blander and blander and yep, they won't say anything wrong. They might say almost nothing. They might just have platitudes, so to speak, but, but that will be sort of the, the direction. So I think it's probably true that anytime one is using some sort of, one is modeling what the original sort of underlying training Data was one is making one sort of acting as some kind of modeler, observer, whatever of that original training data. The kinds of things that you're doing to be able to model those things are sort of reducing that training data. It's not the whole training data. You've made decisions, you've got what amount to biases, not biases of social or ethical relevance necessarily, they might be, but just ones that the model was of this kind and so it tended to do this or that thing and then you amplify that by going to more and more levels and then it's more about what was. Yeah, I think this is the point. It's more about what was in the model than it was what was in the original data. In other words, if you iterate many levels of, you know, you're going to train from the results of the model that was trained from the results of the model and so on, pretty soon you're reflecting the model more so than reflecting the underlying training data or possibly reality. Now, you know, you can imagine that in sort of the, the, the training of AIs. Let's say you're trying to get them to train on what the physical world is like. Maybe they're robots, maybe they can go and make measurements about the physical world. Maybe they've got grounding in reality, in the real physical world that they can go and, and, and mine more training data effectively from the physical world. It's us humans have only produced a certain amount of stuff. You know, you could imagine some, some terrible scenario where sort of the AIs have decided that they just need more training data from humans. And in some kind of strange inverse matrix type setup, you know, the AIs have just hooked up a bunch of brains and the AI is just sort of poking those brains, just saying we want more training data. We want to see what does this brain do in that scenario. What does it do in that scenario. Let's go explore all these different things. And you have this sort of brain bank that is, that is driving, that is providing the ground truth, so to speak, for the AIs, as the AIs might have ground truth provided by going and exploring the physical world, for example. So that's a, that's a, a rather awful scenario I think, from our point of view. Now, I mean, you might say, gosh, you know, if AI is going to be the next evolutionary level in the development of our sort of, Well, I don't know whether it's a biosphere, but the development of sort of the future of us or something that, gosh, you just want the AIs to be as well trained as possible to carry as much of kind of the human achievement as can be carried into sort of the future of what exists. I think it's, I don't know, as a, as just a simple human myself, I, I tend to be rather human centered, so to speak, and, and rather not, you know, the, the idea of, of. And I suppose it relates to a lot of sort of, in a sense, both personal and societal purpose, so to speak. Is it, you know, does one think it's okay, you know, if, if the, if a civilization wrote down all its great discoveries and they were passed down through the ages, but the civilization was exterminated, you know, is that okay or does one feel bad about that? I kind of think as sort of a, as a human, one feels bad about that as a sort of looked at from the outside, whatever that means, because I don't think it means much. You could say, look, you've got more knowledge, more abstraction, more abstract kinds of things that were produced, even though, sorry, the actual humans that produced it are long gone. You know, it really depends on. And it sort of quickly devolves into questions and this is going to go sort of deep philosophy or something into questions about sort of the purpose of the universe and if it has one, which I don't think it meaningfully can be said to have one. But this question of whether it's sort of good enough to have these kind of engrams of abstract achievement or whether what matters is the details of the actual humans. And I think it quickly devolves into meaninglessness if you say that what is going to be left is just the engram, so to speak, without the actual humans and their experiences. You know, I think it's. Yeah, so that, yeah. So I mean, this question about if you, if you let the AIs train the AIs and so on, my claim would be actually sort of interesting, interesting thing because, okay, in, in biology there are sort of forces of natural selection that are informed by the world as it is. You know, if the climate gets warmer and things like that, there will be changes to the sort of forcing function of what is a successful species. And so the species through natural selection will tend to evolve sort of fixed to the ground truth of the actual physical environment of the Earth for now, at least until we get species on Mars evolving and things like this or on the atmosphere of Venus. That's a more interesting case, I would say. But in any case, we can imagine Some interesting kind of natural artificial selection of, let's make microbes that live in the clouds of Venus and so on, and perhaps do interesting things or things we think are interesting to them. But, okay, so in biological evolution, there's sort of a ground truth from the physical environment. There's also a certain set of constraints associated with kind of the way biological organisms are put together. You know, unless you're going to jump away from proteins and things, it's, we're fixed. We have a certain set of constraints about what can happen unless we. You know, it's pretty hard for organisms to make wheels. I mean, bacteria have little flagella at the back and so on. But fundamentally, you don't see a lot of big organisms running around on or wheeling around on wheels because that isn't something that's particularly easy to make with blood supplies and nerves and all this kind of thing. So there are these constraints about the construction of the organism, and there are constraints that are imposed by the environment. Now, it could be the case that being a wheeled organism would be a fantastic thing for some environment that arises, but you probably won't get there because it's just not something that can be constructed from the components you have. But I think an interesting thing with sort of the AI story here is if you say train from the training, you know, train, train, train in a big. In a big train of training, so to speak, that you will end up with something which is much more dependent on the underlying constraints of the construction of the AI than on the external environment. If you keep going back to the external environment and saying, what can we deduce from sort of the ground truth of the external environment? That's more like what has happened in biological evolution so far. I mean, by the way, we can imagine a similar situation in biological evolution if we're evolving in a Petri dish or maybe even in tumors in humans. I'm not sure where sort of it's in an environment where the environment is sort of fixed. And you don't get to sort of go back and ask more about the environment. And then it's more something which is constrained by the intrinsic nature of the thing that's evolving, so to speak, rather than by what's sort of imposed by the environment. But so my guess would be if you go the sort of the chain of trainings of AIs, that in the end it will be more and more evident that, oh, you're using a transformer with this and that and the other characteristics in the system. Now you can say well, how about if you evolve the architecture of the neural net? Sure, you can do that. I think then you will be recursing again to saying, well, what was the evolver of the architecture of the neural net? And so on. I think unless you get to sort of external ground truth, I think that you are inevitably going to evolve. It'd be an interesting thing to actually study in detail to evolve to something which tends to be kind of, which, which tends to reflect the sort of inner construction mechanism. Okay, that was a super long answer to that. Let's see. Well, okay, let's. There's a whole bunch of questions and comments about AI here. Let me try and address some of these. But X asks, do you think anyone will solve the Riemann hypothesis in your lifetime? Let's talk about that for a minute and we'll talk about whether the someone might be an AI, what that would mean, et cetera, et cetera, et cetera. Reminder about the Riemann Hypothesis formulated around what, 1860 or so by Bernard Riemann. The Riemann Hypothesis is about this thing called the Riemann zeta function. And as it's stated, are all of the zeros, places where zeta of S is equal to zero are all of those at places where on the critical line S equals a half plus I times an arbitrary variable where I is squared minus one. So that's what the thing is. It's stated in terms of this sort of mathematical construct. What it really is about is the randomness of prime numbers. It's really something that relates to the density of prime numbers. And really the Riemann hypothesis asks is there a certain degree of randomness in the distribution of primes? If there is that, then the Riemann hypothesis is true. If there isn't that, then there's certain regularities that we didn't think existed within the primes. And I think when you look at the zeta function, you just sort of plot it out and you see it wiggling. It wiggles a lot. And the RN hypothesis is equivalent to is, when it looks at these wiggles, is there ever a minimum wiggle, A lowest point that's above the axis that's above value equals zero. Okay, so when you see it wiggling, you realize this might be a hard thing to figure out because there's a lot of complicated wiggles there. Now the real question though is what would it mean to figure it out? It really relates to questions about what is mathematics? And the kind of last century or so view has been mathematics is based on. You set down axioms, you say these are the things that I assume are true. Euclid said that these are the things I assume are true. Now let me, as a purely sort of mechanical process, see what I can derive from that. Now that's a tricky thing, because what do you assume is true? Like Euclid, for example, assumed his fifth postulate, assumed that no parallel lines can ever cross. And that postulate turns out to be, well, it's a fine thing to assume. You can do geometry on that basis as a constructive kind of thing. But it turns out it's not true for the physical universe. In the physical universe, space can be curved. That was sort of the discovery of general relativity. And that that means that Euclid's axioms are just sort of this bubble out there that's not related as such to physical reality. And you can invent different axioms for mathematics. And you can say, well, somebody could say, well, I've got the Riemann hypothesis and I'm going to choose these axioms for mathematics that include very obscure things about infinite sets and how they work and so on. And it's like, okay, with these axioms of mathematics and kaboom, we proved the Riemann Hypothesis. You say, well, you know, that was, for example, worst case in areas of theoretical computer science. People sometimes assume in the Riemann hypothesis. So you could say, well, I'm going to make an axiom system and I'm going to get something which is almost assuming the Riemann hypothesis. Then if I prove the Riemann Hypothesis, like, who cares? That was obvious. So the real question about proving the Riemann Hypothesis is you've got to go from some axioms to prove the hypothesis. Now it could be the case that the axioms that you typically use in mathematics for, let's say, arithmetic or something, that those axioms are just not powerful enough to prove or disprove the Riemann hypothesis. It could be the case that the Riemann Hypothesis is something where it's a thing where with those axioms alone, you just don't reach that particular theorem. Roughly, it's a little bit more complicated with the Riemann hypothesis because it has the feature that if you can show that it is undecidable, then it is in some sense true because of just the way that it's stated in terms of for alls and there exists and things like this. But roughly, it's fair to say that sort of, it depends what you have to ask given this particular sort of low level formalization of mathematics. And by the way, most mathematics that's done is not done down to its lowest level formalization. For example, when Fermat's last theorem was proved, the chap who proved it, I asked him, what axioms did you assume? He says, I have no idea, I just did it using sort of common mathematics of the time, which is what everybody thought was the interesting thing to do. To say, kind of what would be the, you know, to say, can you, can you just go mechanically from certain axioms to prove that theorem? That's sort of a different level of question, not one of such interest to most mathematicians. I mean, there are efforts in modern times to make proof assistants, computational systems where you're kind of fitting together puzzle pieces and you're saying, if you can fit this puzzle this way, then you have definitively proved this thing. But it's a really hard road to be able to do that. And then even when you've done it, it's not clear what you've got, because it could be the case that. So what you have to claim is that something like the hypothesis, Fermat's Last Theorem, that there's a gaggle of possible axiom systems for which that a reasonable axiom system makes the thing true, and that you could. That if you tweaked the axiom system so that it wasn't true, lots of other stuff would fall apart. But that's a little bit of a trickier thing. Now, it could be the case that with the kind of standard level mathematics that don't go all the way down to these axiom systems, that you could prove the Riemann Hypothesis. And I don't have a good sense. My own guess would be that you need to go down to this infrastructure level, that in the end it has a dependence on some infrastructure kinds of things. I will say about the Riemann Hypothesis, there is a nice correspondence. There are sort of fairly simple to state mathematical problems like the 3N +1 problem. You take a number, if it's even, you divide it by two. If it's odd, you say 3N +1 and then you ask the question, if you start off with any number, does that thing eventually reach sort of small numbers? It goes to 1 and 4 and things like this. Or does it sort of eventually go off and get bigger and bigger? If it always hits the 3N +1, it's going to get bigger and bigger. If it hits the N over 2, it'll go down to something smaller and it bounces around. You can make these plots, they bounce around all over the place. So it turns out there's a Formulation of the Riemann hypothesis. It's partly due to most of the real work done by a chap called Yuri Matissevich, longtime user of our technology, who's been involved in proving lots of interesting things about kind of what's decidable and what's not. But anyway, he kind of made a version of the Riemann hypothesis which can be stated in terms of these sort of iterations involving integers. I tried to clean it up a bit, got a simpler version, which actually, I think seemed to people at some point so bizarre that, you know, how could this possibly be right? But I think it is. And it's just, you know, it's a one line of wolvesm language code and just says if this iteration N goes to blah, blah, blah, if that iteration doesn't, if that iteration always terminates, then the Riemann hypothesis is true. And so you've reduced it to this problem about sort of iteration of integers. And that kind of makes it a little bit more, kind of, it brings it a little bit closer to these kinds of questions that are more about kind of going down to the low levels of axioms and a little bit less about kind of can we construct this nice geometrical picture that allows us to use sort of traditional mathematics to go and, to go and work through different kinds of things. So I think the answer is, my guess would be that the Riemann hypothesis might be proved to be undecidable from certain axioms. That's a thing that might happen, you know, fairly soon, to know kind of the narrative story, you know, from sort of some common idea of mathematics that isn't as such, the axioms. I think that's a more challenging thing. Let's see. Oh, boy. There's a question here From Jamie Will. AIs. Okay, let me take a quick, quick comment here from rbs. Aren't there thousands and thousands of papers written that assume the Riemann Hypothesis is true? Yes, absolutely there are. And. Let's see, I mean, the status of those. Many of them are algorithmic papers that say this algorithm will be correct, this randomized algorithm will, will be correct if whatever. And my guess is I haven't really thought this through properly, but my guess is that this whole question about what axioms you're assuming to get the result that probably relates to, particularly when it's randomized algorithms that probably tangles itself up with where did the randomness come from that you say, well, I pick these things at random and with overwhelming probability, probability that approaches one, this or that will happen, that somehow the axiomatization of that randomness will be sort of entangled with the axiomatization you're using to prove that. The R hypothesis. That would be my guess about how that will work. Let's see, Jamie's asking, will AIs be the ones to explore space? Well, I mean, it depends. I think that, you know, in space is very, not very human friendly. It's the time scales, you know, the speed of light compared to, and the distances between stars compared to human lifespans. It doesn't look good. And it's, you know, you want to get to the nearby star. It's going to take you many current human lifetimes to do that. And so. Well, what do you do? Well, you can sort of dehumanize the humans at some level. You can say, okay, let's, you know, make the thing that's worked in science fiction forever, but we don't yet know how to make work in practice. You know, cryonic suspension, so to speak. Let's, let's make that work. And then sort of we, we kind of, you know, we, we freeze the humans and then they wake up in 50,000 years and they're at Alpha Centauri or something. And you know, in a sense that's a, you know, in that sense, yes, the humans can explore space. If we manage to make that technology work, which my guess is one will be able to. I think it will be non trivial to have the, you know, the interstellar dust particle into interplanetary interstellar dust particles that, you know, hit the spacecraft, not, you know, over the course of 50,000 years or whatever, not destroy any material we put in place. But let's assume that's also solvable. Let's assume there's little robots that go fix things and things. But in a sense it's like, well, yeah, we got a human to Alpha Centauri in 50,000 years and so what? So to speak. And we could also say, well, we got some kind of AI to Alpha Centauri in 50,000 years. And again, it might be a. So what? Because it's the, you know, this relates to sort of again to these things we were talking about before about, you know, you can create a thing computationally, you can do a thing. But does it relate to anything that we humans actually care about? I mean, I think that it's worth remembering that the one way that we get to kind of explore the universe without going to all that trouble is through things that go at the speed of light. Whether it's light itself, whether it's Gravitational waves, whether it's maybe neutrinos that go at least really close to the speed of light. These are things where they're kind of, you know, they are, they are ways to sort of see different parts of the universe without physically going there. In other words, how important is it to see with your own eyes Alpha Centauri versus how important is it versus can you. If you get photons from there that we can receive here, you know, is that, is that a thing one can do? And then, you know, I think it's. It's worth remembering. It's kind of ironic that for a photon, because it's going at the speed of light and time is infinitely dilated at the speed of light, to a photon, no time passes, you know, from the time the photon is emitted to the time the photon is absorbed. That's, you know, just a moment. There's no. No time has passed. So photons that were emitted right at the beginning of the universe or at the point where the Universe became transparent 100,000 years or something after it began, where there are photons that are just sort of going through, through, coming towards us. And eventually some photon from the cosmic microwave background that lasted 14 billion years of our time and finally got absorbed by some detector, you know, it seems. Feels very sad that it lasted 14 billion years and then splat, we detected it in some. In some detector of microwave radiation. To the photon, no time will have passed to the photon. If the photon was kind of. If we had some kind of assembly of things that was sort of. If we could imagine a mind made of photons, so to speak, then no time would pass for that mind between the creation of the mind and the destruction of the mind, so to speak, at a time which for our minds is vastly later. So, you know, I think there's this question of sort of, we're going to explore the universe, we're going to send out sort of AI probes. They will kind of go out and about in the universe and they'll arrive places 100,000 years from now or something. You know, I don't know. That's a. That's a complicated issue, how one feels about that. I mean, it's kind of like building the pyramids and putting things in there that were sort of intended for the afterlife but end up being relevant for 4,000 years later. And, you know, how do you feel about that? You know, that process of putting things out there into the universe? I have a number of friends who've been involved in sort of putting things out into the universe. So we at least have planted our flag on the universe even long after we might be around or not around and so on. So I think it's. But if one's asking the question, you know, for example, within our solar system, you know, will it be the case that it's, we can have robotic spacecraft going and exploring things, surely the answer is yes. There is a certain sort of coolness factor to having the actual humans show up on Mars or whatever else, rather than just, oh, it's a robotic probe and it's sending data back and we can get, you know, even if we can get very high bandwidth data back from it, it still feels cooler and feels more us related if there's, you know, a friendly human, so to speak, who went there rather than just a robotic probe. But I think, you know, it's clearly vastly easier to have. I mean, you know, it depends a bit on one's view about sort of the value of humans versus the value of robots versus whatever else. I mean, if, you know, in some, to my mind, very undesirable world where, you know, you say, well, humans aren't that valuable, we can send a bunch of them off to Mars, maybe some of them will make it and you know, they're cheap, so to speak, you know, that will be a different world than, than fortunately than the one we seem to live in where it's, you know, where the calculation of what, you know, what's it plausible to do is different. I mean, there are plenty of places in the world today where it's like, well, we could have a factory robot do that, but why bother? Because we can have a human do it sort of for the same price type thing. And I think, you know, you can, you can imagine the same kinds of things with respect to robotic probes versus humans. And insofar as you put the value of, of human prosperity and life for the astronauts or whoever else, very, very high, you know, it makes it much more attractive to, to send the robot out where, yeah, it's, you know, it's kind of just money if it blows up, so to speak. Let's see. Yeah, well, RBS is commenting. You know, if only we could just send, okay, if on Alpha Centauri we had a sort of friendly, cooperative whatever, where we just send a light signal out there and then a 3D printer starts up on Alpha Centauri and makes the thing that we want to make and then it goes and does its study of whatever we want to study and it sends back this wonderful picture of this Totally alien thing that we don't recognize as anything like life or whatever else. Some complicated chemical process that we say, huh, that's kind of interesting. I wonder what that means, so to speak. Yes, but I think that's kind of a. We don't have a way to do that. You know, it's an interesting question whether we could in a sense, whenever we. Well, okay, right now we are taking passive information from, for example, exoplanets. You know, there's light from a star, it maybe goes through the atmosphere of the exoplanet. Maybe we get some wonderful spectroscopic conclusion from that. But we're not, and we're far from being able to say, let's shine a laser in the direction of that exoplanet, wait eight years or something and see what comes back. We're not in a position to do that. The energetics of that don't work out even close right now. I mean, the fundamental problem there is if we could keep a beam of light sort of perfectly collimated, then we might be able to do that, but we can't because of the phenomenon of diffraction. It's inevitable that it will spread out and we can't. And so that idea that then we have to put incredible power into it so that even with the sort of inverse square law of it spreading out, we'll still get something reasonable back. I mean, to give an example of actually probably the sort of the furthest out active things like that are, you send a signal to which one is it? Voyager is still in contact or Pioneer Voyager, I think. Voyager, yeah. So you can send a signal out to the Voyager spacecraft and it will do something and you get data back that's kind of like we put the 3D printer on alpha Centauri, except it isn't even close to Alpha Centauri yet. So the question would be, is there a way without humans having put something at the place where you're having a receiver, could you imagine a situation where there's a thing out in the distance, so to speak, that could receive a light signal and do the useful thing with it and then send the data back? We don't know how to do that right now. But I mean, one could imagine that and that would kind of, that would definitely make it even less sort of attractive to send out that human built piece of material stuff. I mean, in a sense what one's proposing doing is sending out human built photons, but rather than human built physical objects with little pieces of metal and so on in them. Let's see. You know, I mean, I have to say, in our models of physics, the. To us right now, photons look like featureless. You've either got one or you don't. In our models of physics, photons are probably made of this kind of complicated structure made of this network that represents the structure of space and the structure of everything in the universe. So kind of like saying, I've got an eddy in my water and I can see that eddy, and it's just an eddy, but that eddy is ultimately made up of a bunch of molecules underneath. And so one could imagine that sort of a photon. Right now we only think the only thing that matters about a photon is its energy and its momentum. That's the, you know, the only thing that distinguishes one photon for another is its energy and its momentum. There's no labeling of a photon beyond that. And. Well, actually, that's more. Well, okay, that's basically a fair statement. It's a little complicated, which is the photon, and that you'll end up with two of them and they have different effects and so on. And the question is, they exist physically or do they exist in branchial space? And that's the whole story of quantum mechanics and so on. So that gets complicated. But in a first approximation, sort of a photon is a photon, and it's kind of like a black hole. In current theories of physics, it's kind of like a black hole. All that matters is essentially the same thing. Energy, momentum, and there's a few other parameters that, that matter as well, the spin and the charge and so on. But. But to the outside of the black hole, that's all that matters. Inside the black hole, it could be the case there was some whole civilization that was, you know, whose planet was crushed by some collapsing star and so on, and that's inside the black hole. But from the outside, you don't see any of that. And might be the same way with photons. Maybe that's. There's no way of sort of cracking open the nut of the photon, so to speak, and seeing what's inside. In the case of black holes, we think that quantum mechanics in some sense can crack open the nut and see what's inside. And so this question of, okay, you've packaged things in a photon that's going at the speed of light and so on. Is it possible you could sort of crack open that nut and see more detail about it? I mean, it's faintly conceivable, given our models of physics. It's certainly something far away from what we know how to do at present time or even sort of. It's, it's. I mean, it's this question of are all photons really the same? And, you know, right now we don't have any way to distinguish them. And the. But that's. That will be one possibility. That will be one bizarre possibility for sort of additional channel of communication. I don't hold out that much hope for that, but I think that's a conceivable thing. I mean, something people are always commenting on is when it comes to, you know, have we met the aliens? So to speak. And I always think that's an. A very poor question because the details of us humans, if, yes, if there was something in the galaxy that was just another, you know, humanoid, like carbon life form type thing, we meet them and we say, yes, you're a human, carbonoid, humanoid, carbon life form or something, we recognize what it is. But if what it is is something that involves chemical reactions involving manganese and niobium doing this and that and the other and producing very complicated patterns, which to the inner experience of the manganese niobium thingy is like, look, we have this amazing civilization, but we look at it and we say, that's some weird chemical reaction which we don't understand and who cares? Type thing. So I tend to think that the problem of alien finding is a problem that there are many complicated things going on in the universe. They're just in the terms of our physics project. For example, they're not close enough in Rule of Space to us that we can readily interpret them. But in any case, the thing that's often sort of a staple for science fiction and so on is imagine there's a new way to communicate in the universe, which is basically saying, imagine there's a new feature of the universe that we get to bring into our perception. Like, you know, we've had eyes since the trilobites or something. I don't know. Whatever it is, you know, a solid few hundred million years, you know, life on Earth has had eyes, but life on Earth doesn't have neutrino detectors. Life on Earth doesn't have gravitational wave detectors. We built those things as pieces of technology. Our species has built those things. But that's something where we get to see a new feature of the universe when we do that. And one sort of, one might imagine, okay, well, guess what? If only we had managed to tap into the, you know, the gravitational wave sort of Internet, then sort of, then we would discover the aliens. Now, in some sense, things like that are true, because this point that there's a lot of complicated stuff going on in the universe and what you notice depends on sort of how you can do the noticing. And so that's something where you can expect. Yes. Once you know about gravitational waves and you can detect them and so on, you'll be able to see complex processes going on in the universe that you weren't otherwise able to see. And you could say, well, we've detected sort of the alien intelligence. I might say that, you know, the weather on Earth is an alien intelligence. It doesn't happen very close to our intelligence, but it is something that has the same kind of computational complexity that our brains have. Let's see, it's a question from imagination here. Do you think Ray Kurzweil's longevity predictions are likely to happen in our own lifetime? Haven't talked to Ray in years, actually. So I don't know. I don't know how he's doing on the longevity scale. But I remember having lunch with him a number of years ago and was impressed at the number of pills of different kinds that he took along with one piece of lettuce or something. So, you know, who knows, maybe that's a winning solution. I think that the question of what will happen with sort of human longevity is it's interesting one. I mean, you know, we've been in many countries, sort of the life expectancy has been increasing by about a third of a year per year for the last 70 years or so. I think most of that is cardiovascular kinds of things that things which are essentially mechanical people can detect and so on. I tend to think, and it's certainly my own personal sort of approach to this is the first step is detect things. You know, do all those tests make all those measurements? You know, if they all say it's all fine, then, you know, assume it's going to be fine. But by doing all those tests, you get sort of an early warning of what's going on. I mean, it's kind of like how did aeroplanes or general anesthesia, two examples where sort of there have been great advances in sort of their effectiveness and not crashing and not failing type thing over the course of years. And I think the thing that's really been the big driver of those is better sensors, better kind of awareness of what's going on. You know, is the tail of your airplane about to fall off? Well, you know, you might have some sensor that would tell you that rather than just, well, you have to wait and see what happens. So I Think the, you know that, that, well, like, you know, metal fatigue detection or something from, from whatever. But, but different, different kinds of kinds of things. But the point is. So I think one, you know, that that's surely a driver of longevity, but it's not going to solve the problem because that's just saying detect things early, maybe you can patch them, you can fix them. If you detect it earlier. That still is the case apparently, that life eventually wears out. Now at some level, there are plenty of organisms where that organism buds off another piece and the bud kind of keeps going and the organism's gone on for a gazillion years. The organism in some sense you could say, but it's not the same organism. But it isn't the same of us either because, you know, our cells are continually turning over. Okay, there are some cells, like nerve cells that don't turn over, at least not at all quickly. But you know, the. Is it us? Well, you know, our skin cells, our gut cells, whatever else, blood cells, they're all turning over fairly rapidly and being replaced by new cells. And so I think there's this question of. But we think the part of us that is critical to the usness of us is mostly brain type things. Whether that's really the ultimate story is not so clear. I mean, I don't know how much of the usness of us depends on things other than brains. That might depend on the endocrine system, might depend on the immune system, might depend on the gut microbiome. Who's to say, you know, those things you say, well, those aren't really relevant, but they might change. For example, your approach to life might be different depending on sort of, you know, hormone levels that are produced by something that has little to do with the brain as such. So it's not completely clear what the package that you need to sort of maintain the usness actually is. But I think the. When it comes to, you know, maintaining the whole package that, you know, supports the brain and all these other systems, that's the question is, can you do that? And sort of, you know, how will one increase longevity? Well, the first question is why do we age to begin with? And that's not completely clear. I mean it's, you know, there are different sort of things that go wrong. And there are telomeres that form, fall off the ends of DNA, but then they can put them back on again. You know, the enzymes can put them back on again. Then there are sort of oxidative damage things essentially just burning up. And there are, and There are sort of deposits of this or that that get produced, all kinds of things. There are sort of tanglings of the DNA. There are all sorts of different things that. That. That go wrong. Obviously, they can be fixed, because when you go to the next generation, those things are restarted. And it's not obvious that you can't take the things that you have right now and kind of restart them from stem cells, for example. I mean, there's increasingly increasing success in kind of taking sort of. Well, first thing was taking a cell that exists, you know, some random skin cell or something, and reverting it back to something that will be a cell that can differentiate into anything, just like our original, you know, fertilized egg or something. Cell can differentiate into all of us, so to speak. And so that's, you know, it's clear that it is possible to turn the clock back because it happens with every successive generation. So now the question is, can you sort of turn the clock back within an organism? You know, in a sense, can you rebuild the plane even as the plane is flying, so to speak, with us organisms? And we don't know yet. Is it conceivable? Yes, I think it's conceivable. I mean, people. The very fact that was discovered, what is it, 15 years ago, 20 years ago, of how to. Well, 15 years ago, I think of how to go back from a sort of a specific cell to something that is a stem cell that has the potentiality to turn into lots of different kinds of cells. That was only discovered fairly recently. And I know there are experiments going on of can you kind of cure aging by just sort of having cells be fresh cells in some ways. And, you know, there are things where you say, let's, you know, when there's cells that didn't make it, and that one kind of became senescent, you know, when that. When that cell was sort of died off. Let's get rid of those cells, and then maybe there'll be more cells get produced that don't have those same aging characteristics. I think that, you know, but. But it's also the case. It's very strange thing that there is so many clocks in biology. I mean, the thing that always is. Is a little shocking to me is you use a facial feature recognizer like the one we have in Wolfram language, and it'll tell you, you know, it takes a picture of you, and it'll tell you how old you are. And it's like, really, you know, and it gets it right quite accurately. I mean, I. I was Thinking for me for a while that I might have been in the training set for the machine learning classifier that figures out ages. But I don't think that's what's going on. I think what is going on. What's going on is that there are gradual changes in facial structure, just as there are gradual changes, eventually, by the time you're 40 years old or something, the plates of the skull fuse and so on. There are things that happen over the course of many decades and which sort of inexorably happens and which, by the way, the AI can notice, even though we humans don't so readily notice these things. And so that's sort of an inexorable clock through life. Now, that particular thing of moving around facial features and so on and, you know, pieces of cartilage getting bigger and whatever else is probably not relevant to whether you live or not. But it's certainly a kind of a. A sort of a reminder of the fact that there is an intrinsic biological clock. And what is needed to divert that biological clock, we don't know. I mean, and of course, with things like tumors, the. What's happening there is that sort of the usual sort of clocking and limitations on growth and so on that exist in biological organisms. They have been. They have been removed. And we're kind of back to a form of life before we had sort of individual organisms that grew to fixed size. It's just like, just keep growing. And that turns out not to be very compatible with, you know, with organisms that were at our stage in the evolutionary process, so to speak. So, you know, but what it takes to get into something which sort of grows like us, but doesn't go out of control like that, it's almost one of these undecidability problems. You set up a certain set of conditions in your molecular biology. Is it going to be something which sort of does the right thing, or is it going to do something computationally irreducible and unexpected and do the wrong thing and, for example, start sort of growing forever or some other bad thing like that. But so, you know, I think, look, in the end, there will be human immortality. I'll be. I don't think there's any fundamental reason to think that that's impossible. What exactly it will mean and how much is sort of turned over in that process, if it. What it means is you kind of get to have a biological organism that sort of has the same memories and can remember from 300 years ago and so on. My guess is that will happen. And how much of that will have been sort of, oh, and you by the way, have a digital sort of assist that's connected to know, forget having actual organ X, let's just have the, the, the kind of technological version of organ X. Why, why bother to have an actual liver or something? Just have this kind of artificial one that sort of plugs into the. Is plug compatible with the biological one. And, and maybe much of what you have ends up being that, that artificial technological solution. And then you have to ask what does it mean to have human immortality in the case where almost everything has been replaced by something that isn't really terribly human? And of course, that gets even more complicated when you're replacing pieces of brains by sort of the digital assist, so to speak, or the neural net, the artificial neural net that sort of. Oh yes, well, you know, something went wrong with some piece of the hypothalamus, so we replaced it with some artificial hypothalamus. And then it's like, how far does that go? And at what point do you have human immortality versus essentially just. Or I say just, but it might be that this quotes artificial, this technological immortality. I'm certainly hoping, as a person who's getting old myself, I certainly hope that the advances that might happen that would get one to the point where, you know, the life expectancy is increasing at more than one year per year, so to speak, so that it sort of goes to infinity that, that happens sort of soon enough to not, you know, to not sort of exit oneself, it'll be great. I mean, you know, the other opportunity is the whole cryonics story and the ability to sort of have a pause in and sort of freeze everything until. For a better day, so to speak. I think the challenge with all of these things, the challenge with that one is the discontinuity of experience. May, you know what, what. As you know, if one's been around and exposed to the way that technology is evolving and so on, we're not that, you know, one isn't, you know, if, if you took the me from 1965 or something and brought me to today, there'd be lots of things where I just, I have no idea how this works. What the heck is this? It's, you know, the world has changed because of technology primarily a little bit because of social kinds of dynamics as well. But one might sort of say some things that would be terrible today or say some things that will be charming today from, you know, jumping from 1965 or something. But I think that, you know, that dynamic of kind of, you know, if you are continually around and sort of paying attention to the development of the world, you're going to be sort of still in your time. If you kind of were frozen for 100 years and then come back, then you're not in your time. And it's not completely clear how that plays out. Let's see. Comments? Would it make sense for alien AI civilizations to broadcast radio or other signals with information about how to build an AI that can propagate an universe faster than material space travel? Yes. But you know, the thing that's so difficult is it's so hard to go outside of one's current human purpose. You know, the things that have been sort of, you know, bashed into us by our struggle for life in biological evolution, etc. Etc. Etc. It's so hard to kind of go to the, the kind of the abstract purpose of, you know, now, you know, it has to be said that if your, if whatever you do causes you to not exist, then there's nothing more to say. So there is a certain sense in which sort of the propagation of something and making more of it exist is a self fulfilling thing that has to happen. So, you know, I suppose that's an argument for why the. I mean, you know, one could say that. Let's see, what will be an extreme thing to say? One could say that. Look, you know, all of the, what can I say? Something like all of the interstellar grains that have all these particular molecules on them that we can, you know, that we get radio signals that show us that this molecule exists and a slightly weird molecule we never bothered to look at before exists. Gosh, golly, there must be an intergalactic conspiracy, or at least interstellar conspiracy that's putting these particular molecules on these grains that we will detect and that we'll say, gosh, that's an interesting molecule, let's go make that molecule here. Now that isn't our usual interpretation. Our usual interpretation is it's just physics that's putting these molecules on these interstellar grains. But I think that this it's just physics argument is a complicated one because look at what our brains do. It's just physics. It's physics where we say it's packaged in a way that's very special to us, et cetera, et cetera, et cetera. But at some level it's just physics. All right, maybe one more question or comment. A bunch of comments here about. Diminishing returns. Foo says related to AI and machine learning. Hans comments. AI will create this time of Step function upwards in everything. But that may not be the case due to diminishing returns. Yeah, well, I think it's an interesting question to what extent. What we've seen, sort of observationally in the technologies, we've seen the big jump in 2012 with image identification and so on, image processing, slightly less heralded, big jumps in speech to text, for example, big jumps in language generation and so on. It seemed like they keep on being these sort of jumps where something gets to the point where it wasn't possible and now it's possible. And, you know, there's the question of, well, you know, there's a certain set of things we humans do, and we can kind of check them off. Could we do that one? Can we do that one? And so on. And I think the. And then there's the question, well, what about the things we humans don't do right now? And that's where things get challenging because, you know, it's very easy to make a computational system that does a lot of stuff that we humans don't do. Problem is, it's not clear we humans care about it. I mean, in my life, I've spent quite a lot of it studying ruleology, kind of the study of simple rules and what they do, kind of what's out there in the computational universe, your average cellular automaton with all these rules, very simple rules, often very complicated behavior. You look at it and I say, that's pretty interesting, but I can't really humanize it very well. I can't really say. And that's why it makes me have a better lunch type thing. It's much more abstracted from that. And I think this question about, once we know there's certain things we humans do, now there's the question of what else is out there in the computational universe. And there's a lot out there in the computational universe. But the history of civilization, the history of sort of intellectual history, is about making these small moves away from what we've already done, sort of building on the tower we've already built. Then we go to the next step, and there's this sort of vast expanse out there of what's computationally possible. And most of it, we humans, as of today, would look at it and just sort of shrug and say, we don't care. I mean, I think we care because I care about ruliology and studying this computational universe and sort of the science of that. But in general, in term, I would not be able to say that they're sort of immediate, everyday applications of what you see out in the computational universe, except things that are pretty close to where we've already gone so far. So I think this question of sort of can the AIs do they go out into the computational universe? What do they find? I think the limiting factor there is mostly what do we humans care about? That is we can. And that becomes sort of the. What are the humans going to do? Well, the AIs can zoom off in any direction. You know, the AIs are, you know, fast driving cars or something. But it's not particularly useful to have a fast driving car unless you have a, you know, unless there's a human driver who's deciding where the car should go. I mean, the disembodied fast driving car can zoom off in random direction in the desert or something, but it's like the humans are probably going to say, who cares? The thing, the place where the humans fit in is in this. So which direction do we want to drive in? And I think that's something where, when people wonder sort of how do humans fit in to the world of AIs yes, the AIs can provide the fast driving cars, but it's still got to be kind of one's got to have choices which are for us humans, insofar as we are, we care about how we, what we perceive in the world. That has to be determined by what, by kind of what we humans choose, so to speak. So I think, you know, they're both practical technological things about these thresholds associated with different kinds of things you can do. Then there are these kind of. So what does one care about doing that one gets into. And I do think it's still sort of an open question if in the what does one care about doing? We care about, for example, making human language. Let's say there's another level of thing there that is, I don't know, let's say you have sensory information not from just you and your eyes and your fingers and so on, but you have sensory information in detail about the whole world. Now in a sense we already have that because we're looking at the web and so on, and we're getting live streams from all over the place and all that kind of thing. But let's say we were able to kind of have sensory information, sensory input that really was looking at every Internet of things, sensor, everywhere in the world. And maybe there's a different level of, kind of description that we can start to apply. Human language maybe we could say is relevant if what we're talking about is the kinds of things that we as sort of humans walking around and you know, whatever you know about, you know, five and a half feet tall or whatever it is, you know that, that, that we experience. But there's a quite different level of thing that's kind of a super. Not language like but who knows what it is. Some, some way of organizing things that's relevant if you are seeing every, every sensor in the, in the world so to speak. I mean it's kind of, you might say there's certain kinds of things that are worth thinking about if you have eyes and you're getting some visual data. Different things if you only have touch, for example. All right, I better go back to my day job here but thank you for a lot of interesting questions and comments today and got me to think about a bunch of things I haven't thought properly about before. So thanks very much and look forward to chatting again another time. Bye for now.