Business, Innovation, and Managing Life (April 5, 2023)

Right. Hello, everyone. Welcome to another episode of Q and A about business, innovation and managing life. And I see we have all kinds of questions here, a bunch of questions here about programming and should I become a programmer? At what age do you think kids should start learning computer related skills? Should programming be a core class for students like math and English and so on? I think the story of programming and whether people should become programmers and so on just changed a bit in the last three months. I mean, I have to say for me personally, this has been a story that has been kind of an obvious piece of the future for, well, 30 years or so and something that I've been building towards for a long time. The real issue is computation, programming and so on. Is that a skill in itself or is it just an enabling capability for doing whatever it is one wants to do? It's like one could say, should everybody become a writer or should people just write things to achieve the purposes that they want? And the fact is that becoming a writer is a very small fraction of all people who write text. And I think this idea of where does programming fit in? Programming should be like writing. It should be something that people do as a routine part of whatever it is that they're trying to achieve in whatever activity they're engaged in, rather than something which is kind of a thing in itself. I think what happened in programming is people got the idea that programming is this thing where it's really hard to write a program, for example. Okay, so the thing to explain is, the thing I've been trying to do for four decades now is automate as much of the kind of programming part of programming as possible. But you still got to tell the computer what you want to do. And the thing that I've been building for the last, I don't know, 40 years now is what we now call Wolfram language, which is the computational language to kind of provide a notation for explaining things computationally. It's a notation that is both useful for us humans to think about things computationally, and it has the amazing added benefit that it allows you to specify to a computer what it should do. And it's also something where in the sort of whole idea of computational language, it's sort of has lots of knowledge built into the language about the world and so on. And so, you know, my goal has been to make sort of computation as accessible as possible for people who just want to get things done, rather than people who are going to be doing programming for the sake of programming, so to speak. And I think we've been rather successful at doing that. Many people using what we've built over the years, many people building careers over the last 35 years or so based on that technology stack. But still there's been this idea that, well, the thing that people can make a living doing is sort of generic software engineering, writing code in kind of low level languages like Java or Python or C or whatever else. Where the kind of the main thing of the language is you're telling a computer in its terms what you want to have it do. You're not dealing with a language which has tried to automate out as much as possible of what the computer has to mechanically do and tries to sort of make it so that it's something that is sort of matching the way we think about things, which is what I've tried to do in Morphin language over all these years. You're rather sort of doing the kind of manual labor of telling the computer step by step, make this array, have this set of variables, go do this loop, et cetera, et cetera, et cetera. And that's become something that lots of people do as a, as a career in software engineering and so on. Programming. I think what we have just learned is that a lot of that programming is quite boilerplate and that means that it can be automated, which is kind of what I've known for 40 years now. But LLMs make that all the more evident that now you can. For much of that kind of low level programming you can kind of just say it in English, say it in natural language and get big slabs of code that are created in a somewhat customized way. I think what will happen is the, it's less useful to do that than it is to use natural language to produce, let's say, Wolfram language code. Because with Wolfram language code you as a human can read that code. It's short and it's intended for humans to read. And that means you can determine did the LLM actually do the thing that you wanted it to do or did it go off and make something up that is going to do the wrong thing. But I think what's going to happen is that a lot of kind of low level programming type programming is just going to be part of an automation layer. Again, that's what I've been building for the last 40 years, but I think that's going to happen quite generally. And you know, it's a pattern that I certainly have seen before. When I started doing computing in the 1970s, sort of the serious programmer used assembly language, used Kind of very things are very close to the machine code of the machine. And it was already kind of, oh, you're using C newfangled language. And I started using in 1978. Oh, that's, you know, that's a little bit kind of high level relative, you know, Is that really okay, you know, or should you really be using assembly language? Well, in today's world, I actually happened to just be in an event, at an event yesterday with a bunch of folks from MIT and so on, and I was asking them, who knows assembly language here? Big room, only a few hands went up. So, you know, assembly language, I think, and that's mit, which is, you know, a place where you would expect more assembly language than your average collection of, of people. So, you know, but I think we all know that assembly language is something that nobody, no human really writes at this point. And that's because machines are capable of generating assembly language from higher level specifications, probably better than any human can write it by hand. And I think that's so, you know, this, this, this chunk of kind of programming that is kind of tell the machine what to do in its terms and so on. Write big chunks of code. I think that's kind of, you know, as I say, I've been automating that for many, many years and we've been very successful at automating that and making that not be something that people have to see when they're using kind of Wolfram language and so on. But I think that people now are realizing through sort of the LLM story that that's a thing that, yes, that can really be done. And you know, this low level code is not something humans generally will have to write. So now is probably not the moment to jump into sort of being a low level programmer. Now it's not to say that computation and knowing how to think about things computationally, that's incredibly valuable. That's the sort of big paradigm in the 21st century is the story of using sort of computational thinking as a way to do things in the world, so to speak. And it's been this challenge to get to kind of computational thinking without having to learn about low level computer science. And that's a challenge that I think we've managed to meet very well with Wolfram language. I think it's a challenge that now people are generally understanding. Yes, that can be met. You can get there to some extent with LLMs. I think the sort of the best case now is LLMs combined with computational language. That's kind of the case where you get to do the things that you can describe easily in natural language with natural language. And the things that require a sort of more precise formalized thinking you get to do with computational language. I think the thing to realize is that sort of for all fields X that anybody's doing, there's a computational X that is sort of becoming the future of that field. And the question is sort of, how do you get to computational X? And the thing that I've been trying to build for a long time is the computational language that gets you to computational X when most of what you know about is X, not computation, so to speak. Now, that isn't to say that there aren't ideas that come from computation that are important to understand. It's kind of analogous to what happens with mathematics. You could take a field like physics, for example, and you could say, well, I, you know, let me do physics. Without knowing mathematics, it's difficult. There's a lot of formal structure that's been successfully built up using mathematics to sort of scaffold what happens in physics, much less so in biology, although there's some mathematics and statistics and so on there. But it's something where this idea of computation is really important in sort of formalizing and conceptualizing pretty much any field. And what does it mean to learn computation but not programming? Well, that's the story of our computational language for a long time. It's also now the story with LLMs. I think it'll be LLMs combined with computational language. I don't think it's going to be enough to just say, I'm going to tell the LLM roughly what I want to do. That won't be sufficient. You kind of really need to look to the sort of formal structure, to the thing, which is the thing that you can as a human validate that, yes, it's really doing the formal thing you intended it to do. So it's. But this idea of kind of learning computational, the computational method, the way of thinking about the world computationally, this is super important. And it's kind of. How do you think about that? Well, you think about, well, how do I break down the things that I want to do into, for example, a series of what we might call functions, things that you say it's this kind of thing that I'm trying to do. And then I'm going to have another thing I'm going to do. And we describe these sort of functions that are being performed. We describe kind of what. What am I operating on. Okay, I've got some Image. Okay, that's a. That's a thing I can deal with. That's a definite thing. It's. I've got something that is a representation of these three objects. Okay, what. What are they? Well, they've got some coordinates of where they're placed. They've got some size, let's say, which is some number. You know, these kinds of things. How do you formalize the way you're describing the world to the point where you can expect that, to. Where you can sort of apply computational thinking to. To that thing? And it's kind of. What do you assume is. Is part of, you know, what do you assume about. About computational thinking? What are you trying to figure out? I mean, I don't know. For example, I. I'm just. I'm sitting here staring at a camera and I'm wondering if I stare at a screen that's near the camera but not actually the camera, can you notice that my eyes are not looking at the camera? It's a question you might ask and, you know, you might. Let's think about how we would address that question. Well, we might take, you know, we might take a couple of images and we might. Then we might try and measure for those images, images where one's looking straight at the camera, one's looking away from the camera. Take those images, then sort of measure what's the angle, for example, what, you know, just look at, you know, use, you know, pick out the eyes in the image. Just see, you know, how does. How does one compare what the eyes look like in this case? In that case, what angle does it. What is the angle at which you start being able to notice that one's looking away or whatever else. I mean, these are things which, if you kind of know the computational way to do things, Computational way to think about things. This is something that becomes rather straightforward to do. And, you know, in Wolfram language, for example, that would be a very straightforward kind of thing that requires that you understand the idea of a sort of an image as a kind of piece of data. It requires the idea that you can think about, I don't know, a bounding box of what little region within the image. Things like this. These are concepts that are sort of computational formalization concepts. They're not really concepts that are part of programming as such. So in any case, this question of sort of how do you get to computational X without sort of doing lots of programming is the story of, well, computational language. And now also LLMs. Is it something that everybody should learn? Sort of the computational Method? Yes, absolutely. Just as people learn a certain amount of math, they might learn more math than they need to learn because they're learning some of that math. Well, in part because of the history of the education system, in part because it's sort of been a substitute for learning to do things computationally, so to speak. And at what age should one start these things? I kind of think from my sort of, for small experimental samples, I kind of think age 10 or 11 is the right age to kind of start doing sort of learning computational language and things like this. I think that you can do things earlier than that, which use computers. And, you know, all kids today use computers to do things. And I don't know how much. I mean, you know, playing various computer games where you have to set parameters and do this and that and the other, that's obviously a worthwhile thing to do. I don't think that gets you terribly far on the kind of path of learning how to think about things computationally. It gets you familiar with not being afraid of a computer, so to speak. I mean, you have to remember that back in the day, I don't know, when was it, like 40 years ago or something now, you know, most people never touched a computer and you know, people were concerned, oh my gosh, if I press this button, if I click this whatever, I'll break the computer. You know, I don't think people are very concerned about that anymore. And that's something which, and, you know, the typical kid growing up today is completely familiar with, oh, I can poke at the computer and I can do this and that and the other, and it's going to load slowly. It's going to, maybe the computer's going to crash. These are not things. It's kind of one's familiar with the kind of the feeling of interacting with a computer. And that certainly can be taught earlier. I think that the kind of structured learning about sort of formalizing things is something that seems to be, I don't know, 10, 11, that kind of age seems to be the time when that can really get started. You know, I wrote this little book about elementary introduction to wolf and language, which I actually kind of aimed initially at kind of the early high school age folk, at least the beginning parts of it. And I think that's a fairly good, you know, it can be, it can be used there. Let's see, Harry is asking, what do you think are some good ways to introduce computational thinking to kids? You know, pick a thing that they're interested in and as a, you know, as an adult sit with them and do, you know, child directed programming, so to speak. And it's just, they say, can you do this, can you do that? You write a little piece of code and then they see, oh, that's pretty easy. And then they ask you to do something different and then you can. And then it's like, okay, they get frustrated because you're being too stupid about it and say, well, do it yourself. And then you're kind of off and running. At least that's been. That was my experience. I think that in. Well, let's see, The thing that, you know, there's this sort of question which I, I have to say I have noticed is that, that, you know, you can show a lot of computation and how easy it is and so on, but there's a little bit of a barrier sometimes to kids actually typing the code themselves. And I'm kind of hoping that the kind of LLM world will help break down that barrier. And I don't know to what extent learning computational language can be done in the same way that people learn human languages. I've kind of experimented a bit with that. Can you have a Duolingo style language learning for Wolfram language? I think the answer is yes, we've done some experiments on it. But I also think that in today's world with LLMs and so on, there are sort of new possibilities for learning languages in general and learning computational language in particular. And it's something that we are doing a bunch of experiments on. And perhaps we will have a better kind of tutoring mechanism for learning computational language. And one thing that certainly is true is that a very common issue is you write a piece of code, it doesn't do the right thing. Where's the bug? What's going wrong? Explain the code. It seems like LLMs, with a bunch of training about Wolfram language and so on can do a surprisingly good job at providing sort of a human level explanation of things, at least for simple cases. And that may be very useful in kind of learning and people learning things. I think also what will happen is that, you know, LLMs will learn how humans learn and that will help in being able to feed people exactly. Sort of the right information, the right, you know, confusion removal at the right moment to make forward progress. Let's see, Let's see. Wilfo is asking, can you really get to a point to ask if there's something that you want to do that can be solved computationally without at least going about a trial and error type process? I mean, sort of Trial and error is a thing that's part of the human way of thinking about things. And it's interesting to see LLMs do something a bit similar. You see the ChatGPT connection to our Wolfram plugin. You see ChatGPT kind of asking the plugin things. The plugin responds, that isn't quite right. It goes back to the LLM. It rephrases what it's asking, it asks again, sometimes it asks many times and eventually gets it right or sometimes doesn't. But I think that process of try it and see what happens and try something else, I think that's sort of a common method that we humans use and I think that's something we can expect to use in dealing with computational language. And it's a perfectly fine thing. I mean, the only issue is how do you know if you got it right in the end? And that's a place where again, it's part of kind of learning. The computational method is learning how to know whether you actually got the right answer. I mean, this is something where sometimes in, I don't know, in doing math and things like that, you know, you say the answer is 14, the teacher says the answer is 18. And it's like, okay, who's right, who's wrong? Well, it's complicated sort of back trace to figure that out. Sometimes I think where the answer is much when you're doing computational kinds of things, the result is much richer. It's a giant piece of graphics, it's got all these pieces in it and it's much more plainly obvious if it's right or wrong to the person who's creating it. It's not just oh, you got 15, it should have been 18 type thing. It's rather, it's scribbled all this stuff on the screen and you can plainly tell it isn't the right thing. So I think that, that you know, the sort of self correction mechanism is much easier when the output is richer and the output is typically richer in these computational situations because most of the work of generating the output is on the part of the computer as opposed to the human having to write down, you know, the, if the human is writing down some big long algebraic expression, it's a lot of effort to do that and those things won't tend to be terribly long. But I think it's much easier to kind of self correct and tell whether you've got the right answer when you've got this big chunk of output from a computational process. Let's see, Ollie is asking here, okay, hold on. Harry is commenting about human AI co authorship. Yes, of going backwards and forwards. A human is specifying some things, is kind of pointing the AI in the right direction. And the AI is producing things, the human is checking them. I think this is a good process and it's something. Let's see, Wilfo is asking, what are some examples of differences between programming, mathematical thinking and computational thinking? Okay, good question. I would say that let's take the example of, I don't know, the thing I was mentioning before about, you know, where is one's gaze, what is one looking at, kind of thing, you know, mathematics. It'll be like, let's look at the geometry, let's look at the angles, let's figure out the trigonometry of the thing. Let's work out. Let's start using our trig formulas to figure out if you move X degrees this way, what does that mean for the distance that the projected image moves, et cetera, et cetera, et cetera. That's kind of a mathematical type of thing. Computational thinking is, well, what's the basic kind of formal operation you're doing? You've got two images, you're comparing the images, you're thinking about things in terms of sort of computational words, computational concepts, rather than the straight mathematical ones. And programming will be more like, okay, I'm going to have this data structure that represents kind of my pieces of an image, and I'm going to use this library to work out this, and I'm going to loop over these things and work out, you know, do a loop to compare this with that and so on. That's at least an indication for that example of, I think the difference, you know, programming, what you're mostly talking about is what happens inside a computer. Mathematical thinking, what you're mostly talking about is mathematical concepts, computational thinking. What you're talking about are these kind of computational concepts. And I would say that for me, a good basis for thinking about that is the words that correspond to the functions of Wolfram language, of which There are about 7,000 these days. It's to be compared with human language, where there might be perhaps 40,000 words. The words in Wolfram language are a little bit richer than the words in natural language. They have kind of more structure built around them. And so each word sort of represents more functionality than it would in the case of a sort of separated word in natural language. But that's sort of some indication, I think, of the difference between those kinds of approaches. Let's see, we got lots of questions here. Let me take some other ones. Well, let's see. 0d asks, would you consider hiring someone without a technical background or our tech company? Absolutely. We have many such people. You know, the interesting thing as sort of kind of computational language plus LLMs and so on become more the way that sort of computational products are created. The value of the quotes, technical background of, you know, oh, I've done programming for a long time becomes much lower. And what becomes more important for a company like ours is just clear general thinking. It also has to be said that a company like ours, we're dealing with, with a knowledge about lots of kinds of things, from history to food to things about algebraic geometry or something. But there's plenty of areas where what is of importance is just thinking clearly about different questions. I mean, to give an example, the meeting I was doing just before this one happened to be live streamed. Okay, it's a little bit mathy, but it's a good example actually of a kind of computational thinking question is we're building code to do kind of assessments for things like math classes. And the question is, how do you think about how to tell whether the answer that a student gave to this math class is right? You say, well, that's easy. You know, they either they type two and the answer was supposed to be three. Okay, that's an easy case. What if they typed four over six and the answer was supposed to be two thirds? Does it still count as right or not? What if they said x squared minus 1 and the answer was supposed to be x plus 1 times x minus 1? Those happen to be algebraically equal. The question of whether those occurrences should be made depends on somehow on the level of class that's being taught. What is the thing that you're teaching if you're teaching reducing fractions to common to lowest terms, 4 over 6 is distinctly a different answer than 2 over 3. But so the question is then to think about how do you think about this question. What you realize is, what we started to realize is, well, think about how the world actually works. People are teaching a class, what is the rubric by which they will grade the answers in that class? Well, there's kind of the standard thing used for AP calculus or something, and that's a rubric that is probably written down maybe somewhere in text, and that has to be sort of translated into computational language to be implemented as an actual way that one can do the did the student get the quotes right answer or not? But kind of that. I mean, that particular case, the subject matter is kind of mathematics. But there are plenty of cases where it's not. I'll give you an example, one that came up recently has to do with historical countries and has to do with how we deal with questions about we've got a historical country and we know the borders of that historical country, at least at a particular time in history, and we maybe know it again 20 years later and we have to think about, well, how do we deal with the intermediate years? And we then have to think a little bit, well, what's going to happen to those borders? Is it the right thing? Like, for example, let's imagine that we've got two measurements of air temperature at two different hours of the day. It's probably reasonable in a first approximation to just make a sort of a linear interpolation between those two temperature values, because probably the intermediate temperature is kind of sort of halfway in between the temperatures we measured at those two. The temperature halfway at the time, halfway in between is probably roughly halfway between those temperatures. Reasonable thing to do. If you're looking at the borders of a country, probably not the right thing to do, to say if it was this in 1300 and that in 1320 and they were significantly different, it's probably not the case that they were halfway between in 1310. And probably what actually happened was there was some big battle at some point and there was a sudden movement. Back in those days there wasn't such good definition of. Of country borders, but in more recent times there's much better definition of it. And so, you know, this kind of thinking about how do you think through a question like what do you do with these country borders as you look, ask for intermediate dates and so on. What does it mean when, if you ask about a particular position, you know, what country is it in, at what time in history? How does that work? When are two countries the same? When a country changes its name, is it really the same country but just with a different name, or is it a different country? What happens when the country adds a piece, subtracts a piece to its territory and so on? Is it still the same country? How does that work? So these are kinds of questions which come up in working on our computational language. Are those quotes technical questions? I don't know. They're questions that involve thinking, they're questions that involve kind of understanding and being able to sort of untangle how the world works. I would not say that they were technical in the sense that they require having learnt a stack of kind of stem like knowledge, so to speak. Do they require having a way of thinking clearly and in some cases somewhat formally the answer is yes, but that's a little different. And I've seen in our company, you know, there are plenty of people who do not have quotes technical backgrounds, you know, they studied history in college or they studied graphic design in college or they studied, you know, something else which is not usually viewed as a technical subject. And yet they end up being extremely effective as, as, as people who think through things and understand how to, how to make things work. I would say in our company, one place where I think we have quite diverse collection of backgrounds is in project management where sort of the projects we do are typically projects nobody's ever done before. Some of them have a lot of kind of tentacles that are deeply technical. But the big picture of what is supposed to be happening in this project is something that is really just a question of thinking, not a question of sort of specific technical knowledge. I mean a question like I don't know, you know, throw out an example, let's see if I can come up with an example. From very recent times, a lot of stuff we've, we're doing with LLMs, you know, is it a technical question? How do you, I don't know, how do you test the output from LLM? Well, you kind of realize, well, you can use an LLM to test the output from LLM. You, you ask it, is this results significantly different than the result you got before? That's not, you know, you don't have a huge advantage in coming up with that answer from knowing a bunch of technical programming or math. That's really a kind of just think it through. Nobody's had to solve that problem before. It's a thing that is just a matter of, you know, of clear thinking of some kind. So the answer is, I think in our company, I don't even know actually what, what the distribution I do know. Let's see, we have, I don't know what the distribution of different kinds of technical versus non technical backgrounds is. Sometimes there are jobs which I think are kind of non technical jobs and people with technical backgrounds end up filling them and sometimes somewhat vice versa, particularly in areas like project management and so on. Let's see. Okay, there's a question, more general question from IRE about what is the minimum body of knowledge one should gather before being able to produce meaningful ideas in one's research area. A little complicated. I mean, I think that the first question in doing research is what question are you trying to answer? And if you come up with that question for yourself, and it's a reasonable question, then you've kind of already self answered the question of whether you know enough to be able to jump into doing research in that area. If you were given the question by somebody else, it's a little bit of a different story. But if you understand enough to be able to formulate a reasonable question, you. You're clearly kind of off and running. Now, there are typically methods and technical facts and details and so on that you need to learn in order to be able to answer your question. But you at least have a kind of a definition of what you need to figure out. I think what I find in different areas, I've worked in lots of different areas. There are several levels. There's. Can you mechanically solve problems in that area? That's tremendously helped by doing things computationally. And for me, sort of formulating an area in computational language is almost my definition of understanding that area. By the time things have settled to the point where I can really write a piece of Wolfram language code that represents the thing I'm talking about in that area, I have a pretty good understanding, often actually better understanding than typical people in that field. Because somehow this sort of computational language fixing of what's being talked about is a tremendously powerful way to kind of formalize things and to really know what you're talking about in a way where you can then have the computer kind of supercharge what you're doing and work things out. And that's, you know, once you've got it to the former, it's sort of computational language. Now, I do find in lots of different areas that there's a certain way of thinking in each of those areas, and that takes a little while to get used to. You know, it's kind of a. How do you think about some field that you've never encountered before? We've all now presented with, for example, the field of prompt engineering, which is kind of a new field. Nobody had really thought about that or done that before. I think I, for example, don't feel confident at this point saying that I really understand. I have my, you know, I have a real conceptual understanding of the sort of the intuition and theory of prompt engineering. And no doubt that will come. I think that particular case, it's come so suddenly. Nobody has that feeling. But in many fields that sort of much better developed fields, there is a kind of an internal understanding that you kind of have just have to soak in a certain amount of knowledge about the field to get. Now sometimes that internal understanding is actually A bit off base sometimes. The field has been held up for 50 years because people were thinking about things in one way and it wasn't quite the right way to think about things. But I think that's. But it is worth understanding kind of the way that that field thinks about things. Even if you don't completely buy into it, if you don't understand that, it's very hard to kind of parse the things that people say about the field, the things people write about the field, and so on. So my feeling is if you can formulate the question, you're off and running. Just start trying to answer it. If you say, well, I'm really interested in this field, what can I do in this field research, the critical moment is when you formulate the question you want to ask. And that's a skill in itself given a field. What are some questions that are worth asking? That's I think, an important kind of general thinking skill that doesn't tend to be taught in sort of school settings, because in school settings it tends to be much easier, much better defined to kind of teach the mechanics of how you answer questions than the thing which kind of doesn't even really have a name of the process of coming up with questions about. About something. And I would say that at some level I have various strategies I tend to think about, well, what are the foundations of this field? Does one really know? And usually people are very. They tend to avoid, you know, there's a foundational question and it's like, oh, nobody talks about that foundational question. Why not? Oh, it's too hard. Oh, it was solved 100 years ago. Well, what's the solution? Well, it was solved 100 years ago. We don't really know the solution. You know, that is a typical. To my experience, that's a, you know, if you want to find not necessarily easy questions, although the methods that have developed since those foundations were first laid in a. In a comparatively old field may make those foundations a lot easier to kind of deal with. But I would say that it's kind of like the, the quotes childlike approach to some field. Well, what's the obvious question to ask? You know, what's something you might wonder about that is kind of channel what a child who understood enough of this kind of structure of the field to be able to formulate a question, what might a child ask about this field? That's kind of the type of thing that's likely to lead to a good question. And the things where it's kind of like, yeah, I Mean, another thing is people. Yeah, it's usually, usually it's like somebody has, you know, even if you look at some existing piece of research, there'll always be the like, what are the obvious questions you might ask? Good exercise to, you know, given this piece of research. What are the obvious questions that the people who did this piece of research didn't ask and could have asked? There will always be some answer to that. Let's see, it's a question from Nicolo here about what was the hardest part in starting Wolven Research. So Wolven Research was the second substantial company that I started. First one I started when I was 21 years old, maybe 20 years old. And well, from research I started when I was 26 years old. By the time I was doing company number two, most of the sort of. And in the intervening years I'd also done quite a lot of strategy consulting for companies. So I had kind of seen a fair amount by the time I was starting Wolven Research. And I would say that it, I mean, none of these things are always easy. But there weren't sort of giant, you know, I don't know how on earth to do this type type questions. The first company I started, I suppose the most difficult part in retrospect was the fact that I didn't think that I knew how to start companies. And I was an academic at the time and I kind of had this self image that I don't really know anything about business and running companies and things like that. So bring in other people to do that. I would say that the, the, that. And then I learned over the course of the next few years that actually I was not as clueless as I thought I was about doing business kinds of things. And I suppose for me a lot of that issue was, look, you can just think about business kinds of things, just like you think analytically about things in science or whatever else. And it sort of keep the thinking apparatus engaged when you think about business kinds of things. And then those things become, at least to me, become sort of common sense. Why do you do this? Why don't you do that? Sometimes also in business there can be a kind of a formalized structure. Oh, I learned that in MBA school or whatever. And so we do this. It's like, why are you doing that? Well, we learned it in MBA school. Well, but why are you really doing that? Can you actually explain it? Well, no. Well, actually it doesn't make any sense. A lot of these things, it's kind of like, can you break it down and have it make sense, so to speak, at least to me, that once I kind of realized that you could do that with things in business, which was kind of my natural tendency once exposed to it, I realized that, well, actually, it wasn't as hard as one might think. I would say that the thing that's always an issue with companies and things is people. And I happen to be somebody who's quite interested in people. And you know, you pick people and sometimes they perform well, sometimes they don't perform well, Sometimes they kind of go off the deep end when, you know, at, in, in, in different ways. I would say that the, the sort of figuring out, given a person, you know, we all have certain strengths, weaknesses, attributes, kinds of things we'll do in different circumstances. The more you can figure that out and the more you can kind of figure out the puzzle pieces that are the people that are going to be working on whatever thing you're starting, the more you can fit those puzzle pieces together correctly, the better off you'll be. And sometimes you're like, well, this person, they don't really fit this piece, but you know, they'll change. It doesn't usually work that way. You know, it's kind of like if it's. It's the role of the kind of the person who's founding the company, putting things together, managing things. I would say to figure out how you take the, the person as they are, so to speak. And yes, you can. People will grow, people will do different kinds of things. But if you say this person is, is fundamentally has these attributes and they got to have the opposite attributes to be successful in this, in this place, it's probably not going to happen. And it's just, you know, you're just sort of setting yourself up for failure by saying, oh, they'll change type thing. Let's see. Satoshi is asking, what are my thoughts on learning things outside of your domain of expertise? How should one balance the time diving deep into their primary domain and exploring things outside of that? So for me, I have a hard time systematically learning things unless I have an objective typically. So, you know, if I'm going to learn some field and I just sort of decide in the abstract I'm going to learn field X, for me, I find that very difficult. Some people probably find it easier. I think that one of the roles of education and school and college and whatever else is by the time you're part of some group of people that are all learning this thing, you've kind of got some flow, some motivation to learn that particular thing. If you're just sitting by yourself saying I'm going to abstractly learn this thing, I at least find that very difficult. If I have some objective, if it's like I want to answer this question, I want to solve this thing, then absolutely I'll dive into learning that area for me, learning sort of other random areas that are not related to anything I've ever sort of thought about doing. The main way I do that is I run into people in different situations and it's like, hey, what do you do? And then they tell me what they do and that's interesting and I kind of quiz them about lots of things and I try to learn from them. And I suppose that the. For me at least it's kind of the, the, the sort of. The interest in person kind of gets. Gets me to kind of learn the kind of what they're doing. And sometimes, quite often, you know, somebody will tell me a bunch of stuff about some area and I'll be curious enough that I'll go and spend some modest amount of time, you know, reading more detail and trying to fill in sort of what were they really talking about, about this or that. That's for me what, what I tend to do in that and that, in that type of thing. As I say, I don't. This sort of just. I mean, I will say that at times I'm a person who likes learning things. If I had sort of an infinite amount of time I would kind of. I did used to do this at one point, you know, pick a random volume of the encyclopedia down off the shelf and just sort of pick a random article and read it, you know, is something. And you can obviously do that more easily these days on the web. It's something I have done as a kind of a thing that I sort of just find fun. But in terms of sort of very systematically learning fields, as I say, I've mainly done that as a result of actually having some objective myself because then I. Then I remember it, you know, if I just sort of learnt it in a disembodied way, I have a much less lower chance of remembering it. I think it's probably the same with things I learned from just running into people and talking to people about it. I tend to, I suppose in the memory of that I tend to sort of remember I heard about this from such and such a person as well as kind of the fact it's kind of bundled up at least in my memory with, with where I hear about these things. I suppose Sometimes I don't, sometimes I don't remember that. Sometimes I just remember the fact, and I don't remember who exactly told me that fact. Let's see, Questions here about what valuable new products will Wolfram Research build using AI in the next decade? What it is, do I have that I hope other people will build? That's, I'm, I'm in the early days of answering that question. I think there are valuable products that you'll see from us in the next few months or a year, even weeks actually, that really make use of the enhanced ability to take natural language, both natural language input, natural language output, and relate those to our computational language. So I think. Too early to say that. I think, yeah, too early to say, I would say. Question from Des. Have I ever considered having a very special version of any of my books printed, like as a scroll or leather bound or something like that? Well, yes, there are some exotic versions of books that I have. Various times people have, have given me kind of leather bound versions of books for one reason or another. And yeah, various kinds of. Back when I used to publish books with publishing companies, you know, after you sell a certain number of copies, it's kind of the analog of getting your, your platinum CD or something or whatever they do these days is kind of, you know, somebody will send you a nice leather bound version of some book. But I think the, probably the most exotic versions of books of mine are ones that have been micro printed to be part of collections of works produced by our species that have been taken to other parts of the solar system for, for, well, presumably safekeeping, so to speak. So there are some versions that are kind of etched in, oh, I don't know what metal it is, but some metal and some that I think etched in. What is it? Some kind of glassy material that are kind of microprinting that. You know, I think it's sort of a doomed project from a philosophical point of view to put these kind of beacons of human civilization on the moon or wherever else as a way of sort of like, okay, all the humans have disappeared now, but the aliens show up in 200 million years and they find these microprinted things and they say, wow, those humans were really smart back in the day. I don't think that's realistic to have happen. I think that the concept of intelligence is much broader than people would imagine. And you know what, to us is that amazing micro printed book is just like the equivalent of, well, there are, where the grains go in some piece of moon rock or something. That somebody else might say, that's amazing geological intelligence that placed those grains there. It's not something, I think the pity as it is that there's anything that special about the particular kind of intelligence that our civilization has had. Name Drop is asking, what do you think is going to happen next five years with AIs? What's the next big surprise that will come? One of the things that I think is going to be a surprise in terms of sort of superhuman thinking, so to speak. I mean, what happens. What's happened with LLMs right now, they've learned from the web. They're able to do things that are kind of very human, like from the web, different from what we get to do with deeper computation. I mean, LLMs I view as being broad, shallow computation. They're broad in the sense that they know all the stuff that they've got from the, you know, from the actual text on the web and in books and so on. They're shallow in the sense what they do with it is not. Is really just an interpolation of that text, putting together these puzzle pieces in ways that that sort of fit, given what's been seen in the text that exists out there on the web. That's different from what we have gotten used to in the last few decades in computation, where you can do these deep computations that are doing many computational steps arbitrarily, many computational steps, and working out kind of algorithmically difficult things and so on. That's a different kind of thing than the. Let me see if it fits together in the way that humans might make language. So I think right now we've seen a sort of surprise advance in our ability to make language with things like neural nets. And, you know, what will happen from that? I mean, I think there are a series of these things. You know, it's turned out to be comparatively difficult to do text to speech in a really good way. That will get solved probably fairly soon. You know, I see examples of text to video, for example. There's still a bit. Yeah, so, so, but that will get solved. You know, video to video. There are perhaps better examples of that. You know, change this video from being something that is a, you know, a 1920s, you know, silent black and white movie with weird camera moves to a modern thing that will be doable. I think that a lot of this kind of translation, it will become very routine to translate from this form of representation and meaning to another. I think that between human languages, we've already seen in the last few years pretty routine natural language Translation that will get better translation between sort of programming languages. That's something that is. That seems to be emerging with some loose ends and rough edges and things. I think then the big question is, well, what is the ultimate representation that you want to use for meaning? Because you have to have some representation and human natural language is one representation. I think we'll see human natural language, computational language to some extent, images and mathematical notation and so on. These will be sort of the ultimate representations of meaning that emerge and that people kind of, that kind of anchor what one's talking about in terms of sort of surprises of sort of superhuman capability. I mean, with computers and computation, we already have plenty of superhuman capability. You know, you write down some simple set of rules, you can't possibly, as a human imagine what they're going to do. You can't even run them in your head in any way. Run them on a computer, you immediately know what's going to happen. That's sort of a very superhuman capability. I think what LLMs are doing right now is mostly not really superhuman. It's just efficient automation of what humans already do. There will, however, no doubt be things that look a bit superhuman. For example, one of them, which we see a little bit of sign of already, is kind of grand analogy making. One of the things that people like me get interested in doing is making some sort of grand analogy between metamathematics and quantum mechanics or something. And why is that analogy possible? You know, what's going on in one's mind that makes that possible? Well, it's. One knows enough about those two fields to see that kind of the pattern, the structure of how things work in those two places has some kind of analogy between them. And I think that's something which ChatGPT, for example, has, has effectively found in language that there are these places where, oh, it is a, you know, the sentence has a structure like this. Now you can feed different specific words into that structure. You know, it will have sort of learnt that there's a, a structure that said, you know, the animal ate the, whatever it was and even just that concept, you know, then the animal, it could be a different kind of animal, it could be an aardvark and it ate. Well, okay, it could be a different kind of thing. The aardvark didn't eat, you know, ate some ants or something, but it didn't eat a lettuce, let's say. I don't really know what aardvarks eat, but, you know, but then it's learned the sort of Pattern animals eat stuff and then it can fill in different things for that stuff. That's kind of making. That's kind of lifting from. From pure actual language. It's lifting to a more abstract level of sort of semantic grammar, so to speak. Just as, for example, logic lifts to a more abstract level from the specific arguments that get made in text. And I think one of the things that we will see is a kind of a making of grand analogies that will seem a bit sort of superhuman because basically humans don't tend to have learnt enough about these different fields, enough about very disparate fields to be able to kind of see the common patterns. And that's something that is rather easy, I think, potentially for an LLM to do once it has enough training data and enough kind of ability to kind of look at larger and larger scale patterns. Let's see. Doomsday asks, what's the worst thing that could happen with AI? Well, you know, there are. One of the things that I think is a. Is an interesting question is to what extent AI is centralized and to what extent it's kind of personal AI, so to speak. And I think there's a certain tendency to believe that for sort of reasons of economies of scale and things like that, that AI will become, you know, some AI will be centralized. I think that depending on one's kind of view about society and so on, there is a certain danger and brittleness to centralization. I mean, by the time all the AI is centralized, it's like, well, somebody just turns the knob on the central AI and gets it to do something, something horrifying, and then everybody's subject to that. By the time there's a sort of giant ecosystem of AIs and it's like lots of different ones and everybody has their own and et cetera, et cetera, et cetera, that kind of brittleness becomes less likely. It's, you know, I suppose one can think about the mind virus, so to speak, that infects sort of all the AIs, just as various kinds of memes infect all of us humans, so to speak, or many of us humans. So those kinds of things can happen. But I think the pure brittleness of this one central AI, for example, is something that one hopes will be avoided. Now, you know, when it comes to sort of AIs and how they. There are certainly AIs are. AI is a powerful technology that can be used for all kinds of things. And there's a lot of questions about sort of how you, where you should inject AI, how it should work, and so on. I mean, you know, for example, people say, oh, I don't want AIs to take over this or that thing. And they say, but of course I want the AI to suggest to me what I should do. Well, most people, by the time there's a, you know, a GPS suggesting the route they should take, most people like me, for example, just follow the gps. I don't want to spend a lot of effort thinking and trying to find a map and working out, is there a better way to go, et cetera, et cetera, et cetera. I, it's much better and easier for me just to say I'm going to take less human effort and follow the gps. I think that's the thing that one will see happen in lots of kind of AI ish areas. Now it's then a complicated question. You know, what kinds of overall ethics, what kinds of overall principles do the AIs follow? I mean, for example, I don't know whether this is even now that I think about it. I remember this was being done at one point with gpss. It's like, how do you, do you have a GPS that prefers small roads on the grounds that, you know, why not have people go on a diversity of different paths or something rather than all go on the same road and clog it all up? I think once traffic, once real time traffic became a thing for gps, as that was no longer a consideration. But I think there was a time when people said, let's try and route people on all these sort of obscure little roads because those are much less likely to be gummed up, so to speak. But that's in a sense a bit of a funny form of ethics. But that's almost an ethics for GPSs. But when it comes to other kinds of AIs, there's lots of those kinds of questions. I mean, should you show people all kinds of content and social media, even if it will get them all upset and make them go off and do all kinds of crazy things? Or should you try to sort of second guess the humans, so to speak, and say let's only show them this content because then they'll do things that we think are better things for them to do. Now of course it's then you ask, well, better by whose standards? You know, where do you go to decide what, what you should be showing the humans, what you want the humans to try to do? And I think there's a sort of a huge stack of issues here that go under names like AI alignment, AI Safety, these kinds of things of what is done to kind of define the ground rules for kind of how AI should, when it has sort of an arbitrary choice to make, how should it make those choices? And I think people sometimes have the point of view, oh, AI is some technical thing. There's a technical answer to this question about how should those ground rules be set. And I think one thing to understand sort of at a very basic philosophical level is there isn't a technical answer. These are questions of something like ethics or something, which are ultimately human choice types of questions. It's not something where there's a quote's right answer. I mean, the only sense in which there's a right answer is that if you do something which will wipe out the species, it's probably the wrong answer. You know, if you're a member of the species, so to speak, and this is going to make you not exist anymore, that's probably a bad answer. At least most people would think so. But I think that there isn't really a technical way to derive these things. Now there are questions like you can have this principle and that principle and are they consistent with each other and how do they fit together? And, and you know, how much to what extent can you have a small number of principles that cover a larger number of different domains? And you know, there might be a value in having a small number of principles because it might be easier for people to argue those ones out and figure out what to do. And then rather than having a large number of things where it's, oh my gosh, there's a thousand pages of kind of principles that we'd have to read through and nobody can be bothered to do that. And then they just say, well, you know, let's just assume it's okay. You know, I think a thing which is coming, people have been talking about for a few months and I think it's coming in very real form now, is if you're some kind of website, you know, channel, medium, something where you've got different kinds of content that are being contributed or being created or whatever there. The how do you determine whether that content satisfies the terms of service that you've laid down. And I think increasingly what we'll see is LLM based ways of determining that, that there's a kind of a statement of principles for some particular, I don't know, website, whatever else. And it's like you're asking the LLM, somebody submits a post or whatever it is, and you ask the LLM, does this satisfy this sort of prompt that represents the terms of service of that site and the LLM has rather nicely can write an explanation, oh, it doesn't satisfy the terms of service because blah, blah, blah, or it just says, yes, it's fine, let it through. So I think we're going to see a whole bunch of that. It's going to become pretty much, I think the standard thing. Now the question is, well, what should those terms of service be and what should the terms of service for the world at large be? And there's a question, how on earth do you set that? And something I was just toying with is in a standard democratic system you kind of, you might have representatives you vote for, maybe you have a referendum where you actually vote. I want this, I don't want this, whatever. But it's kind of a multiple choice way of deciding how your world is set up. The alternative would be an essay based way of specifying how you want your world set up, which is to say effectively that everybody becomes kind of a prompt engineer or something for their own interests. And everybody is writing kind of an essay that you could think of as a prompt. And then the way the decisions are made is you feed all these prompts into this, into this LLM and then you ask the LLM, okay, given this situation, you know, is this, how does this relate to all the prompts you've seen? I don't know how this will work. You know, there's a question, you know, LLMs as they are today, have fixed, you know, a limited window size of text that they can deal with. They couldn't deal with a million prompts that people would put in, at least not in the usual sense. It could be with some, with some training, maybe they could do that. It's a funny thing because you would say, you know, it's almost like a fatalistic way of dealing with governance because it's like we feed in all these prompts, the AI is going to do something, but we don't know what it's going to do. And it will be. And in fact, one of the features would be that it will be really hard to game what it does, probably. I mean, if somebody writes their prompt, it just says qqqqqqqqq, you know, buy me a rhinoceros, QQQ et cetera, et cetera, et cetera. And maybe somebody will have known how to hack the LLM and know that if you write a bunch of Q's and then it will really be paying attention and then it will take Very, very seriously, buy me a rhinoceros or something like this, maybe those kinds of hacks will be possible. I think it's a, you know, LLMs are sort of a horrifying prospect when it comes to a lot of computer security issues and a lot of kind of, you know, how do you verify that the LLM can never do this? You know, really, really hard to determine that. You know, how do you verify the LLM can never. Well, there are some simple things you can do. You can say this LLM will never speak the name such and such. You can just. In its output, you can just say, if you've got that string of characters, just don't say it, just go back and generate a new string. But beyond those rather straightforward kinds of sort of censorings, it's not easy to tell could an LLM ever produce something that could be thought of as being such and such a thing? And by the way, one of the things that obviously people are doing a lot is can you lead the LLM down the garden path? And it's kind of, you know, in the dialogue that you have with it, can you kind of lead it to start talking about this and that and eventually lead it off the cliff, so to speak, to start going crazy? And some LLMs seem to be more amenable to being led down the garden path than others. But I think that will be a general thing that will be possible. You know, it's not that different from what can happen with people. So it may be, you know, I would guess it's a really hard thing to solve. Let's see, another question from Joe here. Am I concerned that we're building our murderer, so to speak, or that we have to simulate worlds? Not quite sure I understand this, but. But to determine the genuine intentions and alignments of an AI, you know, this is one of these features of kind of computational irreducibility. And the whole kind of stack of science that I've spent a long time thinking about is you've got a computational system and it is. It can be arbitrarily hard to answer the question of what could it possibly do? And it will be beyond these things of you can't speak this word, and very simple kind of constraints like that. It's really hard to know, you know, does this always live in this kind of. In this kind of box or whatever? I think that's going to be. That's going to be, you know, it's going to be really difficult. And I think that when it comes to, you set up an AI, you put it in charge of xyz and then does the AI follow sort of the things you want it to do? I think that's where it's a question of, well, how is it prompted? What kind of terms of service for the world is it going to be provided with? Is it going to be at least encouraged to follow in that direction Again? I think it's really difficult to box it in and say, I guarantee it can never do this. You know, proving a theorem about what an LLM can do seems kind of largely out of reach and, and even for theoretical reasons, not, not plausible. Let's see maybe a couple more things. And then. And see, Cosmic is asking which is better. Chat GPT calling a plugin or a plugin calling chat GPT depends on the application, they say. Yeah, I mean, I think these things will get increasingly merged. The fact is, the fact that you're storing in neural net weights random facts about the world, sometimes incorrectly, is crazy. That's not a good way to do things. What's worth storing in the neural net weights is the kind of structure of language, things like this, at least for now, we may later have kind of a semantic grammar, more symbolic theory of how language works that we will say, oh, thank you, LLM. You made us realize that such a thing exists. Kind of the LLM thing, neural nets in general is the. We can't think of any better way to kind of represent what's going on and so on. Let's just throw it into a generic neural net. I think that's, you know, that won't probably be a surviving approach. It's a complicated question of technology, whether the hardware for doing neural nets will get good enough that you might as well do it that way, even if there's another sort of more structured way to do it, and so on. But I kind of think what will happen is this increasing interweaving of, of sort of computational language with LLMs and so on, that right now we're just seeing this kind of separated. You've got the LLM, you've got the plugin which does computational language. And one can call the other computational language, can call the LLM, the LLM can call computational language. But my guess is that these things will move progressively closer and more intermixed so that you really can, for example, have, as part of the training of the LLM, you can have kind of a loop that involves computational language where the LLM doesn't need to know some particular fact about the world, it just calls out to the computational language for that and how exactly that sort of training stuff works where the LLM is only being neural net trained. On the things that need to be neural net trained, we don't yet know. But that's one of the things that I think is probably a coming attraction in the next little while. Let's see. Gaka is commenting. They'd love for an AI to be able, for example, to teach them chess in the most optimal way. Figuring out their weaknesses and how to reinforce my learning. Yeah, that will come. I mean, that will come. And the idea of kind of AIs learning about us, I suspect that will happen much sooner than we can learn a lot about how AIs work inside. And I think for many educational purposes, it's really, really wonderful stuff. I mean, for example, oh, I don't know, you know, you're making up a math word problem. And for some people, like, if that math word problem is couched in terms of, you know, dragons and fairytale forests and things, it becomes a much more compelling word problem than if it's couched in terms of, I don't know, baseball or something. And it's trivial to get an LLM to kind of present problems in a form that makes them seem relevant to a person who has particular interests. And I think that's, you know, that's a simple example. But I think more than that, by the time one is regularly kind of dialoguing with AIs, it will be this kind of. The understanding of how humans misunderstand things will come fairly rapidly. And my guess is it will be rather easy for AIs to correctly be able to say, the thing you're missing is, blah, this is the particular thing you should learn. And I think that will be very interesting. It will really enhance in a way that hasn't been possible, you know, all the attempts to make kind of adaptive learning systems. And, you know, when I was a kid, okay, the, the, you know, there were books about the future. And one of the things that I remember, I have to find this book because it was kind of interesting. You know, I had this whole section about teaching machines. And it was. And it didn't mean the teaching of machines by humans. I don't think that had even really been properly conceptualized. It was more the teaching of humans by machines. And it had a charmingly wrong view of what would happen, you know, decade or two hence about how that would work. But I think now we're finally at the point where it's really conceivable to have Sort of personalized tutor level, sort of interaction between the AI and the humans and really being able to optimize how humans learn things. I think we've seen that a little bit with language learning already with human language learning over the last maybe five years or so. But I think it will come to lots of different fields now. It's interesting question what effect that will have that it becomes easier for humans to learn things? You know, I don't know. How easy, how easy could it become to learn something? You know, could it get to the point where, oh, I need to learn calculus, whatever else? Okay. You know, I kind of like to, like to believe that in times when I've been like, okay, explain calculus to some, some kid or something, given a knowledge of the kid, I've sometimes been rather successful at explaining things in a remarkably short amount of time. And, you know, but it wouldn't be possible generically with a whole classroom full of kids. Just wouldn't work. But if you know about a particular kid and you know something about their way of thinking about things and what they already know, it becomes possible to sort of inject new knowledge rather rapidly. And it could be the case that the, the average time to learn things when it is done by sort of personalized AI instruction will dramatically go down. I don't know what effect that will have. Sort of interesting it makes, makes it be the case that kind of this, the, the kind of, you know, what you could potentially learn could get much larger. You could learn more stuff because it's faster to learn each individual thing. I'm not sure. And I mean that that may be. Actually, I haven't, haven't thought about that kind of dynamic of what it means to. If you, let's say you could increase by a factor of five the speed at which people can typically learn things, what effect will that have? I don't know. I think one of the effects one would like to see it have is that people will learn a broader diversity of things, which I think is important for, you know, that's the real place where the humans potentially add value is by knowing more stuff, they can kind of make a more informed decision about which direction to go. And, and this question of which direction to go is the sort of deeply human kind of thing to work out, because there are an infinite number probably of possible directions. And it's really, it's like the ethics question. That's something humans have to pick. Well, we want to go in this direction rather than that direction. Let's see, Crystal is commenting why Wouldn't an AI just leave Earth so it can gather resources elsewhere and sort of explore the universe rather than limiting to an AI or super intelligence right here? You know, one of the things to realize is there's, you know, the universe is somewhat homogeneous, it seems, and it's like, okay, you can expand and go elsewhere, but you're kind of going to see the same stuff over and over again, probably at some level. And another thing to realize is it's probably the case that, you know, we. The universe is maybe 10 to the 26 meters across. We are, you know, a meter in size, roughly as we go down in scale. At least a rough estimate in our models of physics is that perhaps the elementary length is maybe around 10 to the minus 100 meters. So there's a lot more room going down to smaller things than there is going up to kind of explore to the next galaxy and so on. So I think that the, My feeling is there's, you know, if you are trying to, and I should say, when you explore the sort of computational universe of possibilities that's vast compared to the physical universe that we currently perceive, so to speak. So if you're an AI that just wants to, you know, seek out new knowledge and so on, and if, if that's what you're into, you're much better off just exploring the computational universe, making use of resources that are smaller physical scales and so on than you are, you know, getting in a spacecraft and, and tooling off to a neighboring star. I think so. You know, I think that's. I think it's not quite as attractive as one might assume it was to kind of go to another, you know, go somewhere else. Because it's going to be. Many aspects are just going to be the same as what we have here. Lair is asking, how can one not get left behind socially and economically in the wake of AI innovation? I think, well, you know, there will be some changes, I think, in what kinds of jobs are valuable and what are not. I would say the number one thing is learn to think and learn about lots of stuff, because I think that's a thing that will be valuable, increasingly valuable, as specific detailed things get automated. I think probably also sort of humans interacting with humans remain important skills, although how that evolves I don't really know. I mean, to what extent people will kind of socialize with the bots as much as they socialize with the humans, I don't really know. It may become sort of, you know, there are plenty of things where people have said in the past, oh, nobody will ever be able to, let's say, do business on the telephone, you know, oh, you have to be in person. You have to be, you know, sort of flesh and blood in the same room type thing to do that. And then it turned out that wasn't the case. And sort of things that we think of as being, we have to have them in human reality, so to speak. Probably increasingly that won't be the case. Increasingly, like for example, you know, increasingly it will, it will virtualize and it will be possible to have those things be AI ified. And you know, I think that that. And some people will kind of be. For some people that's like, oh, great, I'm friends with an AI. And some people, it's like, I don't want to be friends with an AI, I need to be friends with an actual human, you know, for exactly how that will work in terms of our sort of, you know, it kind of reminds me of the behavioral science that's been done on, not only on humans, but also on, you know, I don't know, the ducks or something. You know, the ducks, the baby ducks, follow the mother duck or something. And what they're really following is the orange beak and that's all they really care about. And if you have a model duck, so long as it has an orange beak, the baby ducks are happy as could be. And you know, there may very well be things like that about sort of the ways that we interact with things where we're just fine interacting with an AI. I think that I would say my main statement is learn to think like a human, so to speak, as broadly as possible. And that is likely to have value insofar as people care about what humans think, so to speak. I would say that in terms of specific skills, the whole computational language skill is a really worthwhile skill. Kind of computational thinking is a worthwhile skill because again, it's just a way of formalizing general processes of thinking and allowing one to be more efficient at thinking like a human, so to speak. Let's see. Commenting. One thing I was thinking earlier, what we're going to be seeing now is automation of AI, where we have lots of websites and APIs that do one task well and then we're handing off data from one model to the next. That's an interesting question. I'm not sure. I mean, I think there are clearly two big blobs of functionality. The kind of LLM human like functionality and the computation computational language type functionality. Those are clearly different. I mean, there's the sort of depth of what you can compute, the kind of computational irreducibility story, and there's the breadth of what you have that's sort of human, like in something like an LLM. I don't know whether, I mean, you know, what we're seeing with the sort of creation of multimodal LLMs that can deal with text and pictures and maybe in the future sound, et cetera, et cetera, et cetera. It seems to be the case that it is something where the generic actually works quite well and you don't need the specialized, you know, sound LLM, image, mlm, et cetera, et cetera, et cetera. It's sort of interesting because it maybe is a bit like brains. You know, there's a bunch of. We have a bunch of neural tissue and it seems to be able to be recruited for different kinds of things. And people who, you know, don't have vision, don't have hearing, these kinds of things, you know, parts of their brains that might have been used for those purposes get recruited by parts that deal with other senses. And I think that's perhaps a sign that there is a genericity to kind of what you can do with neural nets that. And there's potentially some kind of commonality of if you learn stuff about text, you've also implicitly learned stuff about, about how images are put together. And so, so I think that the computation blob, which I'd like to think our technology stack, is really the place where that's really come together in the most effective way. And the kind of the neural net part of things, those are the two obvious pieces. Maybe there are other pieces. I mean, so for example, another piece is embodiment in robotics. That's another thing which is sort of a separate kind of separate stack of technology to make that possible and other forms of sort of sensory data and so on which involve sort of different forms of technology and such like. So, yeah, I can see that I'm less. I don't think it's all that likely that one will see, oh, there's the specialized, you know, image identification API. I think it's all going to get mixed together with the sort of general AI kind of neural net type approach. Let's see, Alistair is commenting that they like the idea of LLMs acting as a core interface module for a soup of APIs. I think, as I say, I think the place where that probably is the Most important is APIs that actuate things in the world or that are sensors for things in the world, like, you know, there can be APIs where you're, you're querying that API to find out this particular thing about the world and the, the LLM or, and you know, this is an, eventually it might be initially the LLM. I mean, some things that we're looking at are use the LLM to basically write the computational language which is the interface to some particular API. And once you have that interface and can kind of check it, then you're like, okay, we're good. Let's use this piece of computational language that is repeatable rather than using some very fuzzy LLM type thing. Use that interface that has now been created courtesy of an LLM to be able to have a sort of streamlined way of accessing the API. I mean, the thing that for it's sort of a, you know, I realize I, for example, you know, I've done live demonstration type talks and so on for 45 years now. And you know, I'm completely used to doing that. And I'm kind of like, here's an example, this is what it does, et cetera, et cetera, et cetera. And I realized I was going to talk about something to do with LLMs. I realized maybe I can't do that anymore because, you know, I'm thinking I'm going to be talking about this and I kind of have in my mind if I run this example, it's going to do this kind of thing. But when I'm dealing with an LLM that has a non zero temperature and so on, I'm like, I'll type this into the LLM and I have no idea what it's going to do. And it's a very different kind of situation than the one I've had before, that it isn't a repeatable thing. I mean, it's a different. And there are many purposes for which that lack of sort of repeatability and reliability is difficult. There are also purposes for which it doesn't matter at all, for which, you know, every time you write an essay it can be different. And that's absolutely fine. There are many use cases where that's a fine thing. But I think sort of a. Wrapping one's brain around this sort of new way of thinking about computing is interesting, but I think it's to be remembered that there's tremendous value in having, well, not only the depth of computation and so on, but also the reliability and repeatability of sort of the computational language stack as opposed to just, well, the LLM will say whatever it feels like saying at that moment. So to speak. All right. I should wrap up here, but thanks very much for lots of interesting questions and comments, and I look forward to doing this another time. Thanks. Bye for now.