Future of Science and Technology Q&A (June 16, 2023)

Okay. Hello everyone. Welcome to another episode of Q and A about future of science and technology. I see we have a whole bunch of questions saved up here. So let me try to address a few of these question here from Paula. What's there to say about the future of neural nets? Neural nets are an interesting episode in the history of science. I think I've talked about these a bit in these live streams before, but neural nets, the way they're currently, the way they're kind of conceived, originated in 1943. This idea of having this kind of collection of neurons that each have a value and where the value gets updated by having other neurons connected to it and then having certain weights with which they're connected and having some kind of activation function for each neuron to, to determine how the values will be added up with the weights and then used to determine the new value of the next neuron down the line. That idea is from 1943 and Modern Neural nets pretty much using that idea. Now the thing that's happened is one could at first think about, oh, every neuron is connected to, or which neurons is it connected to? And the brain, which is sort of the inspiration for all of this. A single neuron might be connected to 10,000 or more other neurons. There's sort of a question of how do you organize those connections? And I suppose a couple of the innovations that have happened over the years have been involved with that. So convolutional neural nets led to sort of big advance around, well, the convolutional neural nets have been used earlier, but the sort of big advance that happened around 2012 in image recognition was using convolutional neural nets. Convolutional neural nets say, well, let's imagine that at least some layers of neurons are arranged in some kind of spatial fashion, so that a particular neuron is dealing with a particular part of, let's say the visual field. And the next neuron over is dealing with another part of the visual field arranged in this kind of spatial way. The thing that sort of the big innovation for things like ChatGPT are so called transformers. And the main idea there tends to be in a convolutional neural net, you say there's a neuron here and it's connected to some neighboring neurons on the, on the previous layer. It's, it gets its new value from values of a neighborhood, a geometrical neighborhood of neurons, previous neurons, but it's sort of a limited size of neighborhood. So it's all kind of geometrically local. The idea in transformers is to have something where typically it's one dimensional, to have something where the new value can depend on lots and lots of previous values. But the weighting of those previous values in the sequence is determined in some way, is learned in some way. And that's, it's kind of the notion is you have something which isn't precisely local, but for example, it only looks at things kind of earlier in the sequence and so on. So called causal attention mechanism. So these are, you know, these are sort of particular methods of how to organize the connections of neural nets. In neural nets there have been other innovations, things like normalization layers and pooling layers and other kinds of things like this. There's been lots of kind of thinking about how best to do, how best to organize training and so on. There's been sort of, oh yes, there are slightly better activation functions that you can use. The, the early ones tended to just be if the total value was below zero, make the new value be zero and if it was above zero, make the new value be one. You know, different, different approaches have been, have been used there. So, but I would say that the things that have led to lots of details that have allowed neural nets to be trainable and to not kind of just wander off and not be able to be set up to do the things that one wants them to do. The things that have led to many of the, for example, the ChatGPT advances I think have more to do with, with sort of the how to outside the neural net, how to kind of set things up so that the neural net kind of stays on track and achieves the task you want it to achieve. I think if we say, well, what's the future of all of this? There will be a bunch of engineering details which will be tweaked and understood better. Over the course of the last decade, several decades, there's been sort of steady improvement in kind of noticing, oh, if you do this, if you don't let the weights go out of, if you don't get the values go out of control in this way, you do better if you, if you connect only these things, you'll do better and so on. Engineering details which are important in terms of whether the thing can actually work or not. I think that's one thing. The other thing is tremendous potential in sort of how you actually put that underlying neural net inside the actual task you're trying to do. In terms of what will be kind of the big advances, it's kind of shocking how small the advances in some sense looked at in sort of the big picture have been over the course of decades, it's mostly been more data, better understanding of use cases and so on, and the realization that sort of if you just train the thing and bash it hard enough, you can make it do all these tasks, which one just didn't know before. One didn't know that one could get neural nets to do pretty good image recognition. One didn't know one could get neural nets do pretty good speech to text. One thought one had to break those problems down into lots and lots of carefully engineered sub cases rather than just throwing the whole problem at neural nets and kind of bashing the neural nets hard enough with training data. So I think that's the, you know, there'll be technical advance in how to make neural nets learn faster, do better, et cetera. I suspect there are some bigger things. There are some things where, for example, language models make use of lots of sort of facts about the world. They don't really need the details of those facts about the world inside the neural net model. Those are things that can be factored out, delegated to, well, typically our technology with WolframAlpha and Wolfram language and so on, and the sort of computable knowledge that we have there, that doesn't need to be all mixed in in a way where neural net isn't even very good at dealing with that to the whole structure of the neural net. The neural net can really be dealing just with linguistics and common sense and not with detailed data or computation, which neural nets aren't really directly able to do. Now, people have tried to say, let's add little extra bells and whistles to neural nets, let's add some computation capability to neural nets and so on. Let's make it possible for neural nets, rather than just interpolating their data and finding the output based on that, let's let the neural net actually do computation right there inside the neural net. What seems to be emerging right now is that what neural nets and language models and so on are pretty good at doing is formulating what they want to do. So formulating, okay, we're going to use this computation tool now go off and just use that tool externally rather than trying to burn that into the innards of the neural net. There have been attempts to do that, but there tends to be this big trade off between how capable computationally do you want the neural net to be and how trainable is the neural net. By the time the neural net has more diversity in what it can internally do, it becomes extremely much More difficult to train the neural net. And the various attempts over the last decade particularly or so to make sort of more computation inside the neural net have tended to lead to less trainability and less effectiveness at the tasks that neural nets seem to be good at doing. Now, you know, there are some clues we get from the brain. I mean people used to say, oh, these artificial neural nets are very idealized models of how the brain works. You know, there's, there's a lot different things happening in the brain as these artificial neural nets succeed in doing so many more of these tasks that we thought were very brain like tasks like language generation. It seems much more that we can learn things about brains from artificial neural nets. And that's something that's, that's happening now. But still there are clues from brains about how we could sort of go in different directions with neural nets. One important feature is that in something like ChatGPT, the that's a pure sort of feed forward type network. When it's running, you know, it's generating the next word. How does it do that? Well, it looks at all the previous words, it turns those into numbers, it feeds all those numbers through the neural nets and, and comes out with the probabilities for the next word, then picks the next word and then iterates that whole process. But so just, it just sort of feeds through the network forwards. Now there's the question right now, sort of training is a very separate thing for neural nets from this kind of actual execution of the neural net, actual use of the neural net. In brains we seem to intersperse sort of learning and doing much more than current neural nets do it. And possibly that's because in brains neurons are both places where memory gets stored stored and places where computations get done. Those are sort of, those are happening at the same time in the same place, so to speak. And even the very early models of neural nets, back from the 1940s and so on, had this idea that there would be sort of memory and doing sort of all mixed together. But the actual engineering of neural nets hasn't tended to do that. It's tended to be the case that there's sort of a separate, this is the place where activity is happening, separate from this is the place where we're storing the data. Presumably there will be ways found to merge that and to do some more of the things that are more brain like in that regard. And another thing is that there's this whole question of sort of how the dynamics of the neural networks, whether it's just oh, you feed forward through the network and get your results. And maybe you do that over and over again. But it's kind of a very coarse kind of overall architecture for how information flows through neural nets as opposed to things happening inside the neural net in some much more dynamic way. As I say, the attempt to sort of mix training with those kinds of dynamic things has not been particularly successful. But one suspects that there are probably things which involve sort of more dynamism in neural nets than these kinds of workflows that just say, feed in an image, get out the caption from the image, feed in a caption, get out the image, feed in text, continue that text, and so on. There are probably more dynamic kinds of things that involve more sort of explicit time evolution of neural nets rather than just this sort of neural nets, more like functions where you have input, crunch, crunch, crunch, output type thing. You know, I think that there are lots of things about the neural nets. For example, the fact that neural nets use real numbers to represent the strength of sort of neural excitation at each neuron in the neural net, that's probably not ultimately necessary. People try experiments with sort of less and less precise numbers, and neural nets seem to keep working just fine. It's much more difficult to do training because training tends to involve a incremental changes in those numbers. And if those numbers are discrete, it's much less obvious what it means to make incremental changes. So you might train with something with precise, sort of fairly precise real numbers, floating point numbers on a computer or something, and then try and distill that down to something which has much less precise numbers for when you're actually going to run with that neural net. But there's a sort of big outstanding question I've been interested in for a really long time of whether one can make things like cellular automata where every cell value is discrete, maybe only has two possible values or something. Whether one can do neural net like things and do training kinds of things on systems like that. I investigated this back in the early 1980s, didn't make too much progress with it. But I have to say I suspect it's something where with kind of modern sort of willingness to bash these problems and these training problems with just vast numbers of trillions quadrillions of computer operations, it might be something worth investigating again. And it's something where you would then be directly training in this discrete world which is much more suitable for current electronic computers. So, I mean, those are a few thoughts there. I mean, I tend to think that the Sort of the idea of neural nets. There's a lot of uses of the idea of neural nets which we will yet see. I think in terms of the sort of the core ideas, well, they've been surprisingly sort of just being tweaked over the course of many, many, many decades. I mean, I think it's sort of reminiscent of some of what one saw in the history of things like linear algebra. You know, there's kind of this idea of matrices and there's these operations you can do on them of solving linear equations, systems of linear equations, doing various kinds of decompositions, things like this. In the 1960s, 1970s, sort of computational methods for doing those things really kind of got established and that became this methodology of doing numerical linear algebra that was at first a thing for its own sake, but then found lots of applications and launched lots of areas like computer graphics, for example, which makes very direct use of numerical linear algebra. Neural networks actually make direct use of numerical linear algebra as well. But that's something where that sort of methodology of numerical linear algebra and all the exotic matrix decompositions that we implement in language and so on are things from that time, and they've been sort of incrementally improved. But basically it becomes a method that you can use and apply in lots of different areas. And my feeling is that's what neural nets are becoming. And. And the, you know, the dramatic sort of drama in the core technology, probably not so much drama in the use cases that can be found much more likely. So that's a few thoughts on that. But I do think there are some things that one can imagine that are more kind of structural. Like these things about a discrete version of neural nets, possibly versions of neural nets that don't just involve numbers, that involve something more like symbolic expressions or graphs or something like this for the actual data that the neural net is dealing with. Things where there is a better way found to have combination training and use of the neural net, things like this. I mean, there have been all sorts of adventures in things like reinforcement learning and so on, which is sort of an idea that's been discovered and rediscovered for 60, 70 years or so between control theory and machine learning and so on. The idea where sort of the neural net goes out and probes the world, or probes something that's trying to learn about, rather than you just saying, okay, here's what you need to know, rather it's going off and sort of actively poking at things and so on and seeing what. And one's giving sort of the world is Giving feedback to it. Those are techniques that are useful, but I don't know, I don't think there's something sort of truly transformative. I don't think that's around the corner there. Let's see, Let's see. Luis comments. Neural nets could evolve to be able to be trainers. What are the limits? Trainers of humans, trainers of neural nets. I mean, one of the things that is certainly an interesting kind of thing that builds up a little bit like our civilization builds up, you know, in our civilization, you know, one generation discovers certain kinds of things and then subsequent generations get to make use of that knowledge to build on that knowledge. So it is with neural nets, you know, when, when one has had sort of one neural net kind of has figured out things about image recognition or something, you can build on that with the next generation. There are a couple of different ways you can build on it. One is to use kind of the features that one neural net recognizes and to say, okay, rather than looking at the whole image, just look at these features. When you want to do the next level of processing. That's a place where you can almost do sort of a brain transplant of one neural net. You kind of chop off the, let's say, the last layers of the neural net. You leave these sort of this thing that represents the kind of, where the activity of the neural net represents kind of the features it discovered that you start training from there. That's kind of one way in which sort of a new generation of neural net can learn from the last. The other is something more direct where you say, okay, we're trying to learn, we're trying to generate training data for a new neural net. Let's use the existing neural net to generate sort of boatloads of training data for the new neural nets. And that's something people are interested in doing for large language models. There are other cases, like for learning about, you know, how to do self driving in cars, where you really want the car. In addition to experiencing the real world, you want to be able to make it experience kind of a video game world. And for that, it's useful to think about having that video game world kind of learn from things in the real world and then sort of leverage that up for the next, for the next generation, so to speak. That's one sense of this. The other thing is having neural nets and AI in general, large language models and so on, be used as ways to help train us humans. And I think that's a very, a very sort of productive direction in which to go I mean, I think that there's both much more personalization that one can imagine and there's also much more ability for large language models to learn how it is that we learn and learn for each individual person what the sort of map of their knowledge is and the things that they are not understanding. And if only you told them this, they would understand it. I think it's promising. I don't know yet. I mean, we've been doing a bunch of experiments on this to make it so that you can make a large language model or an AI system in general that will be sort of an optimal teacher for a particular human and will optimize both keeping the attention of that human, keeping that human motivated, and actually figuring out, given the thought patterns of that human, what kind of what you can, you know, what is the right thing to say to that human to help them over the next, over the next hill, so to speak, to, over the next barrier to let them understand a new thing. So I think those are sort of directions. Now an interesting question is just how successfully can we humans be taught? We have our whole system of education. Most people go to sort of schools where there's kind of a broadcast form of education of one teacher and some number of tens of students and so on. Some people, you know, do one on one learn things one on one with tutors or whatever else. And that's probably a more efficient way to learn certain kinds of things. I think, you know, there's, it's complicated because there's also a dynamic of being able to discuss things with other people you're learning with and getting the motivation from other people being around and so on. So it's sort of a mixture of things. But I think this question of just how fast can we humans learn something is an interesting question. And you know, in, in my own life, I've tried to learn lots of kinds of things. I think I gradually over the years have gotten faster at learning things. But that's mostly because I've learned lots of other things before and because new knowledge that I'm trying to get usually fits into existing knowledge. And so that's kind of allows me to, to sort of leverage what I already know to be able to more quickly learn the new thing. But there's sort of a question of if there was a, an AI based teaching system that when I want to learn topic X, it knows kind of what I know and it can tell me the exact five things that I need to understand to really sort of get on top of that new topic I kind of feel like it often is the case that there are a limited number of new ideas that one just has to really grok that new idea. And I might be get it very quickly or I might be like, I don't get it, I don't get it, I don't understand. It kind of bashes up against some other idea, some other thing I knew I don't understand. It seems inconsistent with that. It seems inconsistent with intuition that I have whatever else. But if one had an AI that kind of knew me well, one could imagine it basically saying, look, you know, this seems incompatible with the intuition that you have about this or that, but here's why it isn't, or whatever else, or even just, you know, the reason why you're having trouble with this is because it is incompatible with this piece of intuition you have. You know, I know from my own efforts to try to explain things as I try to do on these live streams, there are all kinds of ways that one can. One learns about sort of explaining things to people. For example, one of the things that is perhaps not obvious is when there is something that's sort of a big idea, something that is a very surprising intuitional point. Something like this. If you just tell people this is how it works, bloomp people won't get it. I wouldn't get it if somebody did that with me either. You really have to kind of sort of signpost what's going on. Say, look, this is surprising. This is a big deal. This is something that's going to be hard to understand. You've got to kind of say there's something difficult coming. And then people have a much better chance to be able to understand that than if you just say, oh by the way, blah, blah, blah. And that's just one of a zillion different examples which certainly good teachers learn how to do these kinds of interactions. And can an AI be better than any teacher, better than any human teacher at being able to kind of inject knowledge into us? I think quite possibly the answer is yes. And that will kind of. It changes the economics and sort of the calculus of education quite a bit. How long does it really take to. To learn advanced topic X? And how much kind of back and forth do you have to have to really cement those ideas in your mind? And so on. Let's see. Desk comments. It seems like every decade scientists keep saying, we just realized that brains, DNA, et cetera are a lot more complicated than we realized. How close are we to actually understanding them? Do I anticipate this Kind of. It's really more complicated than we expected. Continuing. Most of the time that's been the case in biology that people say, oh, my gosh, we turned over another rock and there's just an incredible amount of stuff crawling around underneath. And it's more complicated than we could possibly imagine. There are some cases where that didn't happen. For example, in the. I mean, the two big theories of biology, natural selection and sort of the digital concept of genomics, those are two very simplifying kinds of, kinds of discoveries which sort of went from, oh, there's all these different things going on, and then we pulled it all together in a theory. Then in the case of genetics, there are all these different things going on, but then we pulled it together in a theory. By the way, another case where that happened was when people realized that there are only, I don't know, 30, 40,000 genes that are relevant to the biology of us humans. Oh, it's so few. How could so few programs effectively make something like us? That was kind of, in a sense, I view that as being a little bit like the realization that I've importantly worked with a lot, which is that even very simple programs, you know, computational systems, can do very complicated things. That's kind of biology showing the same kind of phenomenon, that actually the underlying laws are not as complicated, the underlying rules, the underlying genetic programs are not as complicated. They're not as many. They're not as complicated as you might have thought to make something like us. But. So I think the reason that biology keeps on seeming more and more complicated is that we don't have a good theory for it. Now, could it be the case that there is no theory, that it's all kind of computationally irreducible and they're just running these rules and then it happens to come out this way? My guess is that there is a potential for theory in biology. And part of the reason I say that is that biology, at some level, the things that happen in biology make sense to us. That's sort of inevitable. We are. We are that biology. And it's not like we have this sort of random sensory system that senses things. We can't tell what it's sensing. No, we have eyes, and we have a good narrative description of kind of how seeing works and so on. So we've already managed to sort of define biology to be something for which there is at least an overall narrative about what's going on. So now the question is, if we look kind of at the level of molecular biology, can we start Seeing what's going on? Or does it just look like every time we look at something else, it looks more and more and more complicated? I think I'd like perhaps give an analogy to this, which I think is interesting, which is physics. Back in the 1600s, people like Descartes were saying, it won't be long before we have the final fundamental theory of physics. And the. But it didn't pan out that way. Instead, what happened was, you know, looked like things were pretty good at the time of Newton. We've just got these laws of mechanics and it's all going to work, and law of gravity and so on. And then things started getting more complicated and there were more different kinds of laws, et cetera, et cetera, et cetera. And then we got into sort of this level of stuff and, you know, quantum mechanics came along in the 1920s and so on, and it seemed more difficult for us humans to understand. It seemed there was more and more complexity. Then we got particle physics. We got all these different kinds of things in quantum field theory. It seemed like the hope to find sort of to push through all of that and find a truly fundamental theory of physics looked unpromising. I mean, what happened in physics tended to be. We thought we had a theory that worked pretty well, like Newtonian mechanics, but then there were little glitches found, and then that was kind of a sign that actually there was a lot more underneath. And then as we kind of probed that, it seemed like things got more complicated, the mathematical formalism got more complicated and so on. Now, the fact that the mathematical formalism is more complicated to deal with is different from saying it is fundamentally a more complicated structure. And I think much more. What happened is it got more complicated to deal with. Not necessarily that the fundamental structure was more complicated, but as we've learned from our physics project last few years, it looks very, you know, at this point, I consider it kind of certain that in the end, physics and the universe is sort of computational at the bottom, and there really is an end to this process of, oh, you look at another level and then you're going to see something more complicated, and you're going to see an exception to the things you discovered before and so on. No, in physics, it really seems like there is a bottom level. There is an underlying computational machine code, so to speak, to the universe, and we can actually get to it. And sort of exciting that at this time in history, we managed to get to the end of that process. Well, okay, so what's going to happen in biology? Well, I think in biology, we kind of know what the underlying underlying machine code is. It's basically physics, chemistry, and so on. But then the question is, well, how does that get organized to make a thing that is a living system? Well, the thing that has been noticed is people keep on thinking, oh, the narrative we have right now is sufficient. You know, immunology works this way. It's only this number of different types of, I don't know, white blood cells or something, or the brain works this way. It's just this and that is connected and so on. And then you look in more detail and you see more things going on. Actually, I have to say one thing about brains, the very fact that ChatGPT works is a fascinating fact about brains. Basically, it's telling one, this thing of generating language, which has always seemed like a very difficult human thing. Actually, this artificial neural net that's made in this very kind of formal way is pretty good at doing that. So it tells us that, yes, there's a lot of complexity in brains, but a lot of it doesn't really matter to being able to do this operation of generating language. So that's already a very useful fact. Now, if you want to explain something about some particular aspect of how brains react to eating more sugar or something like this, that's quite separate from does it really matter to the operation of brains? Are you able to abstract from. From thinking about the details of brains, the things you need to be able to produce language? I think the sort of, one of the features of, of molecular biology, things like that is. Well, I remember when I was a kid, for example, there were these wall charts of the, the metabolic pathways of the cell and they were, you know, they were wall charts, but they were fairly modest in size. I could sort of imagine, you know, kind of there were the Krebs cycle and this cycle and the. And so on. You could kind of, you know, it was easy to kind of sort of more or less get them all in mind. Now you can get these sort of metabolic signaling pathway charts. They're, they're much bigger, they're much smaller print and they're many, you know, they're multiple panels and so on. There's a lot more that's known. There's a lot more sort of complexity that's been discovered in biology and there isn't a set of organizing principles around that. In a sense, the sort of. The ChatGPT story is telling one, what are the important things about brains to know, what are the important things about biology in general? I don't think we really know those at this point. And I think that when it comes to sort of what do the molecules actually do? Well, they're dancing around, they have all these ways of interacting with other molecules. It's all rather complicated. Is some of that interaction just kind of random? The molecules are sort of bouncing around randomly? Or is it heavily orchestrated where there are specific molecules with specific shapes that specifically transport things in this and that way? I think it's increasingly looking like the whole thing is much more orchestrated. What is the theory of that orchestration? We don't know yet. I mean, I have some ideas myself that actually come out of some of the formalism of our physics project. But thinking about kind of how to think about the general operation of a biological system, what's really going on? Is it just a bunch of molecules bouncing around and having chemical reactions, or is it something where you really want to unroll, in a sense, the space time graph of all the different interactions, all the different events that occur between these molecules. Maybe there's a theory about what happens when you look at this whole space time graph. Maybe that thing has some regularities in it that we can make a narrative about and that are really a story, not so much about this is the molecule that exists and it has this pathway that connects to that, but something more detailed that at first might seem kind of dauntingly complex, but in the end will have some kind of large scale story associated with it. I mean, a little bit like you could have said, oh, DNA, it's a very long molecule. You know, you stretch that molecule out, it's a meter long. If we were to pull a single molecule out and oh, that's just so much stuff, it's so complicated, we'll never have anything to say about it. But in the end we have kind of this important fact about its digital information. Everything is operating digitally and it's being copied and so on. And that allows us to make some pretty general conclusions about biology. And I think, you know, it depends a little bit what we mean by understanding things. So, for example, in the case of AI and brains, I think we can say that even if we don't know, sort of, even if we can't talk about every biological detail of how language generation, let's say, works, we do know a lot at this point about sort of the big picture, the big kind of computational architecture of how that can work. So in the case of the rest of biology, so to speak, it's like, well, what would be a sign that you kind of knew the big architecture of what's going on. I mean, there are things about biology, like aging, for example, sleep, even very fundamental basic things about our experience of the world where it's surprisingly obscure what's actually going on. I mean, there are different theories and there are some features of them that are well known and so on. But you know, is there a big picture? Is there a fundamental theory of, of how aging works? For example, Is there a fundamental theory that proves that you could never reverse aging? Is there something that, you know, is there some computational irreducibility argument that says aging is inevitable and there's nothing you can kind of do about it? Or is that not the case? Is it just some strange evolutionary adaptation to, to winnow out the old f, because they're just a drag on the community, so to speak, and just something where you flip the switch and pretty soon that feature of sort of the degradation of biological systems goes away. We don't really know. But I think there's sort of this question of can you have a theoretical framework in which questions like that can be answered? And that would make it seem like biology wasn't just detail, detail, detail, detail, detail, and a never ending collection of details where we never get to the bottom of things again. I think it's a little different from the physics case where sort of what was happening there in terms of probing further down in sort of the hierarchy of what can be looked at a little different in biology. We know what's at the bottom. It's just atoms and molecules and things, and we know the physics of what goes on there. But this question about sort of, what's the sort of overall picture of the organization is I think, the place that's significant there. I mean, it is shocking. I think I mentioned this before that over the time I've known a little bit about biology, the level of, sort of amount of detail that's been discovered about actual biological systems has so unbelievably dramatically increased through experimental methods, through more emphasis on biomedical research and so on. But the extent to which the theory has expanded is very, very negligible. I mean, there's, there's people, the, the, the, the sort of traditions of the field are all about do an experiment, see what happens, do the statistics of what goes on. Oh, there'll be a bunch of randomness. We'll never be able to say this is the fundamental answer. It's only there's a certain probability for this to be the answer, et cetera, et cetera, et cetera. And so there's been sort of, I would say, a prejudice against the idea of grand theories of biology, because there haven't really been many. I mean, the last big one was kind of digital structure of DNA and that's from 1953. But I think it's an area ripe for big theories. I tend to think that the best hope for the sort of next round of big theory is this idea of multicomputation that's emerged from our physics project. And I hope I'll have a chance to pursue some of this and figure out perhaps some things about biology and sort of these, these big picture answers about biology, which I think have been rather elusive. Let's see. Gosh. Riff. Comments about the dynamics behind crowdsourcing have interested me, they say, for a long time. Can we start seeing its potential when it has the same principles as neural nets and so on? I don't know. Crowdsourcing is one of these things which to me is sometimes a bit of a. It's complicated, it's sort of a. It can be a bit of a wimp out. It can be not as great as it seems. I know, and sometimes it can be quite successful. The general principle that I've seen, and this is more a comment about kind of business affairs than it is perhaps about science and so on. But when there's a very well defined set of things to do, people, sort of a large crowd of people has a much better chance to be able to do it. You just tell people, go, you know, if you've given a framework, people have a much better chance to be able to contribute within that framework. And you know, a lot of things where people say it's the crowd, it's, you know, it's going to. The crowd's going to figure it out. When you really peel back the covers, it's 50 people or it's 20 people or even less. And you know, it's in. One of the things about crowdsourcing can be, if there isn't really leadership about what should be done, it's very hard for it to achieve anything. Once there is sort of a well defined. The leadership has defined. This is the box that every contribution has to fit into, has a better chance to be successful. But still, you know, I haven't looked so much in recent years, but in the past I did look at just how big are those crowds that are contributing? I mean, whether it's bitcoin miners or Wikipedia editors or whatever else, just how big is that pool? And it usually turns out to be a lot smaller than you'd imagine down to tens of people and so on, or tens of organizations. And it's, you know, I think that certain kinds of things people find just fun to do. And so they will sort of do them for fun, just for the fun of the thing itself. Sometimes there's, you know, in successful cases, there's kind of a community spirit that's developed where it's not just. Just fun in and of itself to do independently, but something where it's sort of fun to participate with other people in. I don't know. I. I think I've. I don't know. Maybe there are sort of ways to organize sort of groups of people that we haven't yet discovered that that will be very productive. And I don't know whether there's anything that one learns from neural nets about that. You know, the very traditional view of how to organize people is a hierarchical kind of organization where there's sort of. Because, you know, vision and leadership tends to be something that individual people can do, different people will do differently. But if you say let's put together 10 people to all be the leaders and let's make the average of what those 10 people do, it's usually not as good as what any one of those people might have been able to do by themselves, or as good. If you want to have anything new happen in the world, if you want to sort of simply say, don't let's have anything new happen. Let's just keep the status quo, then having that sort of average might not be a bad thing. But it's. And so, you know, there's the sort of hierarchical organization of things. And then there are experiments people do on sort of very different organizational structures. You know, there was one that was this holacracy idea where you have kind of these intersecting circles of people, I think has not been very successful. I think it's sort of actually kind of disastrous in certain cases where sort of there really isn't anybody in charge and everybody's kind of in intersecting circles of who's. Who is responsible for what. I mean, as a person who's in practice run organizations for much of my life, I, you know, it's hard enough to run an organization where you can kind of define a vision and try and get that executed. But to run an organization where you've got these intersecting circles of. Of thoughts about what to do seems. Seems just setting yourself up for. For absurd difficulty. So I don't know, that's interesting question whether there are forms of sort of human organization that Work better. Now I think one of the things to realize is that history tends to suggest that there are certain sort of natural forms of organization, human organization, that work well with the human condition. There are ones you can scientifically imagine where you sort of say, well, we can just use mathematical principles to optimize how we produce things and you know, use linear programming and optimization to, to work out how everything has to work. History has not been kind to those kinds of innovations. History tends to say that that sort of attempt to fit kind of mathematicization on top of kind of the way people should do things hasn't been terribly successful. Whether that means there's no better thing one can do is. It's certainly not clear to me whether for example, with AIs as intermediaries, they it's possible to have different forms of human organization. I don't know. I mean, it kind of relates to things like, you know, can we have a democracy 2.0 where everybody writes, you know, instead of voting for things, everybody writes a prompt and you feed it all into a big AI and ask it what to do. That doesn't really, if you try and unravel that. It would be interesting to try and do things like that in practice because I think one would learn a bunch of things, a lot of things would go wrong, but one would learn a bunch of things in practice which even in sort of theoretically thinking about it, are not obvious. And quite often in these kinds of, you come up with some scheme and you can say that scheme's not going to work because of this effect and that effect. And then you realize that in practice nobody runs into that pothole. In practice it works just fine. I see that a lot in design of computational language, for example. There can be this sort of pothole there that you might fall into, but it turns out which you might theoretically worry about, but turns out nobody does, so it doesn't really matter. And yet there are other cases where you say it's going to be great, but then you're ignoring something which ends up being this abrasion that eventually makes the whole thing fall apart. And that's what a lot of these kind of sort of schemes for better organizations of human interactions and human governance and so on have, have had a pretty bad historical record. Now of course we have new ingredient now which is sort of better working AI. And perhaps there are new things that can be done there. And that's an interesting thing to start thinking about working out. Let's see, gosh, robot is asking how do you anticipate biotechnology future Shaping the future of biomaterials and tissue engine and thoughts on accessibility and affordability of biotechnology and equitable distribution of benefits. You know, this whole question about what's expensive and what's cheap in the world is kind of interesting. I mean, when I was younger, I was lucky enough to have access to big fancy computers that allowed me to do things that your average person couldn't do because they didn't have access to big fancy computers at this point. Mostly the computers that I use are just the same as the computers that a billion other people use. They're really. There's no sort of computers got cheap enough that they sort of are very widely distributed now. I have to say I just bought a fancy GPU that's a little bit above the. For a particular project I'm doing, it's a little bit above the average. But that's the first time I've had something fancy like that that I remember in, in a couple of decades. The, the thing that, you know, this fact that consumer electronics became cheap was something, or electronics in general became cheap was something. I'm not sure that it was super easy to predict. Could it have been the case that that there was the perfect chip that was really big that could be produced, but there were only very, very few of them? I remember seeing this must have been mid-1980s, seeing large flat panel displays. And it's like, okay, these things exist, they will become common, but it took another 20 years for them to become common. Because when you make a display, you have to have a yield that's good enough that that sort of every pixel on the display works perfectly. If any pixel doesn't work well, it looks really goofy on your. On a television or whatever because there's this kind of this dead spot on your television. And so that wasn't a viable thing you could sell. So there was this sort of surprising extra time. And it was the case that you could buy flat panel displays, but they were really expensive because you had to produce a lot of them to get one that was perfect. So this question of what ends up being expensive, what ends up being cheap, It's a complicated question. In electronics, things have become cheap and become pretty widely accessible as a result of that. Now, in things like drugs, you make a chemical. You have a process for making the chemical. Often that process actually is quite cheap to implement on a per unit basis. But then the issue has been, well, the way the world is set up, kind of the sort of regulatory compliance of setting up the drug and Testing it and so on, that's super expensive. So you know, by the time you spent billions of dollars on that, the drug can't be cheap, at least not for a long time because you're still paying off those billions of dollars. And by the way, you're also even sort of paying forward because you're saying, but we want to develop new drugs and each one of those is going to cost billions of dollars. So we kind of got to make the money to develop those new things by selling the existing things. Actual unit price of making that stuff has gone down to be really low. So that's I think more a question of sort of how the world is set up and the sort of trade offs about testing and versus deployment and so on. It's not an intrinsic expensiveness to those things. When it comes to kind of biomaterials, things like this, the question of, you know, when one has 3D printable organs and so on, which will happen. I've seen a nice example of a 3D printable 3D printed lung. And you know, there'll be things like this. Lungs are actually harder than many other organs but were prioritized for various reasons. The it's, you know, that stuff will come and will it be expensive, Will it be cheap? My guess is that it should be fairly cheap. There's nothing about it that says, oh, you know, it's, you can only mine this number of diamonds type thing or you can only, you know, there are only this number of diamonds to be found. You can, you know, you make, once you've got that 3D printer that can print at sufficiently high resolution, once you've got these processes for feeding in stem cells and so on, you can go into mass production and there should be great economy of scale. Now the question of, of what is it going to take to do these things. There are different approaches to biomaterials. One is the sort of 3D printing additive manufacturer type approach, which seems quite promising. I mean essentially you're making a scaffold and then a scaffold with inorganic materials, so to speak. And then you're adding in various kinds of cells that sort of arrange themselves on the scaffold. And that seems to be a pretty promising way to do lots of kinds of bio sort of tissue construction and so on. I think there are other things one could imagine. One can imagine. Well, another approach is you just let the biology run its course. You start with stem, cel stem cells. You try and figure out how to direct those stem cells to differentiate to, you know, Usually I think for us humans it's maybe 15 levels of differentiation that there's this whole tree of different cell types. There aren't that many cell types, maybe depending on how you count it, hundreds to thousands of cell types. And you know, the question is, can you sort of give that cell the right pokes to get it to differentiate, to become, you know, a pancreatic beta cell versus a heart muscle cell versus whatever else. And you know, can you just use the pure biology to be able to do that? And then if once you've got those cells, can you get them to organize themselves into some structure that's useful as a piece of tissue? Or do you want to, you know, have a sort of non living scaffold onto which you assemble those things? I suspect there are, there are things to know about sort of how you get cells to assemble themselves. Whether you're using, you know, bioelectric type meth, whether you're using some other kind of way of getting the cells to organize themselves, or putting some appropriate chemical gradient in there, I'm not sure. But those are things that one could sort of imagine doing. Then of course there's a question of, okay, you've got this organ you created or this piece of tissue you created, now install it. Well, installation in us humans is pretty hard. And that's sort of a story these days of surgery and so on. And while, you know, there are, there's plenty of sort of enhancement from surgical robots and things like that, it's still a big complicated mess of, you know, you poke holes in the, in, in, in us or cut, cut incisions and so on. And you know, it's not a particularly easy thing to do. Now there will presumably come a time when you will be able to like have sort of self assembled things delivered without sort of sticking them through a hole, so to speak, where you could do something. And this is still science fiction where you're kind of like, okay, just swallow that pill and then it has little nanobots inside it that go off and go and assemble themselves in the right place in the body. And then all the nanobots start assembling themselves and sort of self assemble the thing you need without having and sort of attach themselves to the right place. By the way, one of the surprising things about a bunch of sort of stem cell therapies is that you can just put various cells in and they will find their way to the right place where they fit, so to speak. And maybe biology will help us in doing that. Maybe it's something where we have to invent the sort of active nanobots that can figure out what to do. But I think those are, those are kinds of things one can imagine. As I say, I think it's going to be, I suspect that it's more a question of kind of governance than it is a question of technology, whether these things end up being cheap on a unit by unit basis. And I think that's a complicated societal question, how that's handled, and sort of a mixture of economics and kind of social issues and so on. But I think that's the, You know, that's the direction here. Let's see. Time, Cake says might be better to prompt the body to regenerate its own organs instead of 3D printing them. Given I think you're mentioning work about bioelectricity and so on. Yeah, that's quite possible. I mean, certainly amphibians manage to regenerate limbs. Why can't we regenerate things? Why can't we regenerate limbs, for example? Not clear. And perhaps there is. I mean, the program exists in our DNA, you know, the DNA that makes us from the first, you know, first sort of fertilized egg to the whole of us. The DNA that's being used there is no different throughout our lives. So whatever was being used, you know, in utero to make the first, you know, to make the thing that produced that piece of our brain, that DNA still exists and is still carried by us and will be passed to the next generation and so on. So why can't we just activate that program from our DNA and, and remake some particular thing as amphibians managed to do? I don't know. And it's quite possible that a technique will be found to do that. Just like people used to say. I used to ask for years for amphibians, it was known that one could make clones. And it's like, well, why can't we make clones of mammals? People say, well, it's just very hard, it's very hard. It's never going to be possible. And then when was it? 1990s, kind of dolly the sheep got, got made. And that's a weird process that involves at some moment, you know, putting this electric shock into the, the, the cell and things like this. And it was sort of a technique that was found, a trick that was found that let one do mammalian cloning. And I think that's. Yes, I think it's quite possible that though. I mean, I think in biology there are going to be a bunch of tricks that are findable, so regeneration one Possibility, My guess is cryonics. There will be a trick found that lets one do that. I think aging. I'm less optimistic that there's a trick, so to speak. I think that's more of a kind of computational irreducibility story that has sort of. That has an infinite collection of things you have to solve, unfortunately. But I think. Anyway, I think those are some thoughts about that. Okay, maybe one more thing I can look at here. And then I need to go back to my day job here. DES is asking, is burrowing into an asteroid the logical way to shield people and equipment on voyages to, say, Mars or even out of the solar system? Well, I don't know how hard it is to burrow into an asteroid. I mean, I think there was a recent experiment that was done of an impact on a small asteroid to try and figure out could one sort of push the asteroid to change its course so that if one detects an asteroid that's on a collision course for Earth, you can kind of, you know, thousands of years before it actually collides with us and Earth and wipes us out. Can you push it off course and so on. And I think that experiment worked fairly well. But that was the concept of, of setting up a whole kind of mining operation to burrow into an asteroid. Sounds kind of hard. I. It doesn't. I mean, it is a big issue. How do you shield people and things in, in interplanetary space? Because, you know, near the Earth, we have the van. Well, we have our. The magnetic field of the Earth magnetosphere and so on. And we have sort of protection from cosmic rays and so on. Radiation coming from mostly the sun. If you're out, away from kind of the protective sheath around the Earth, you're exposed to all those kinds of things. A thing I don't know the answer to and should is. I know people talk about using water, for example, as a. As a thing that will shield radiation. I don't know what radiation depth you need. I know that the peak of the energy spectrum for charged particles from the sun is about 1 GeV. And I think that. Oh, let's see, that's around minimum ionizing. It's a good question. How far does that go in different materials, as you go to heavier. As you go to things with higher atomic number, they will tend to scatter those charged particles more. But carrying a bunch of lead on your spacecraft is. Well, it's very heavy. Now, of course, the economics of what it takes to schlep a very heavy thing into orbit are changing fairly rapidly these days, so maybe that's not such an issue. But what kind of sheath you need to protect us humans from all these cosmic ray charged particles? I'm not sure. My suspicion would be that burrowing into an asteroid, that the technology you need to do that is more difficult than what you would need to sort of send the necessary sheath up to just sort of create it on Earth and launch it into space. Or maybe there's a way of mining its components on the moon or something and using that to sort of, you know, it's kind of. It seems a bit kind of old fashioned to say let's build ourselves a stone spacecraft, so to speak. Take lunar regolith or something, make bricks out of it and put it around your spacecraft. It's kind of will be a sort of charming kind of old meets new situation if the, if the spacecraft going to Mars or something ends up with a stone exterior or something from lunar stone. But I'm not sure if that's a viable way to do it. All right, well, I think we should probably wrap up there and I see lots of interesting questions still to be addressed and I will look forward to doing that another time. So thanks very much and bye for now.