Science & Technology Q&A for Kids (and others) [Part 128]

Hello, everyone. Welcome to another episode of Science and Technology, Q and A for Kids and others. Let's see, I noticed lots of interesting questions here. Here's one from the submission form from Nelly, commenting on a new blog post that I just wrote that is about Instant plugins for ChatGPT. So I think everybody by now has heard of and played with ChatGPT. We built, along with OpenAI, the creators of ChatGPT, a plugin that allows ChatGPT to access Wolfram Alpha and Wolfram language. And so what it's doing is the ChatGPT, its underlying operation is generating text. And you give it a prompt, you say, the cat sat on the. And it will then, in its simplest case, add the word that it thinks is kind of most likely to come next. So it might be Matt in that case. And I wrote a whole book about how this works, a little book about how this works, how ChatGPT manages to have this kind of knowledge based on sort of what it's seen, a trillion words that it's seen from the web and from books and so on, of what sort of typical text looks like. And given the prompt that you gave, it will try and continue from that prompt in the way that is sort of typical of what you would find on the web. Now, the actual words it will put down will not be ones that ever necessarily actually appear on the Web. It has in its operation, the way its neural networks and so on, it's doing something that's sort of human, like, for reasons that I think are scientifically quite interesting, in being able to sort of continue the text from what it started from. So that's its basic operation. Now, one of the things that happens is it will produce plausible text. It will. But when it gets sort of precise things it's asked about, it's going to say something that kind of sounds right. It may or may not be actually right. And so one of the things of interest is, by the way, all it does is produce text, so it can't produce other kinds of things like, I don't know, graphics or whatever else. So the idea is one wants to combine this essentially textual generation process with something that can actually do computation. And here's how it works. So what's happening in the operation of ChatGPT is it's got a piece of text, it says, what's the next word that comes after that piece of text? Okay, it figures out what that is, puts that word down, then it comes back around again and it says, okay, let me look at everything I've Got so far, including the word that just got put down. Now what's the next word? So in other words, it has kind of an outer loop in which it's putting down a word. Actually, it's a token which might be a small, a piece of a word, but basically a word. It puts down a word, then it says, okay, take all the text I've got so far, do the whole thing again, put down a word, do the whole thing again, put down a word. And essentially a word at a time, it's building up that whole piece of text. Now there's some special words. I say words in quotes. An example of a special word is the stop token. And the stop token says okay. When it generates one of those, it means, okay, I'm done. I can kind of finish that essay. That's the end of that essay. If it's in a chat scenario, that's the end of my turn and chat back over to the other member of the chat, so to speak, the human or whatever it is that it's chatting with. Okay, so that's kind of how that works. But now another thing it can do is to say, I need a tool, I need an external tool. So it puts down some special words that basically in its generation of text, it puts down special words that mean, okay, I'm going to need a tool. And the things I'm going to say next will be things that are intended for the tool. And then there's a kind of an overarching system that's kind of watching the words that are put down by the ChatGPT or the, the large language model, the LLM, it's looking at what words are put down there. And when it notices certain special words, it says, okay, take the things that follow those special words and send them to an external computational system. An external. A plugin to the Large language model. And so in Wolfram language, it's very easy to specify. You could say, okay, the large language model, the LLM has produced certain. It wants to call this plugin with this name, and it wants to use it with these parameters. So, for example, it might be a plugin that says something like, well, simple case might be, what's the distance between two cities? And there's a certain sort of thing that's written out by the LLM that means all this external computational thing that can compute this. And then it's going to say, here's the name of the first city, here's the name of the second city. So in the jargon of these kinds of Things the sort of the external call we can think of as an API. So in kind of the way one talks about things with computers, there's kind of the standard user interface where it's a human sitting in front of a computer and like typing at the computer. And usually there are a couple of different modes of that sort of the old fashioned thing is a command line interface, a CLI where you're just typing, you know, I don't know, list out files in a directory. You know, it might be LS for example in. In Unix or in. In the underlying in Mac operating system or something like that. Or in Windows it will be dir lists out through directory. That's kind of the command line. You're just typing text on a command line, hit return, it does something. Then we also have user and graphical user interfaces where we're clicking around and picking up menus and things like this. Well, when a computer wants to interact with another computer, it's not going to go through the graphical user interface. That would be very confusing for it. Instead it uses so called API stands for application program interface. And that's something where it is essentially has a structured thing, it's communicating which you can write out as text, although it doesn't have to be ordinary text. And a very typical form of an API is it has certain parameters and values. So for example, an API might say, oh, I don't know, it might give, you know, city one equals, city two equals. Now by the way, when you see this on the web, a lot of websites are effectively operating like APIs. And whenever you see a question mark in the URL, the thing you type into your browser address bar, things following a question mark are these parameters. So you'll see question mark, you know, X equals blah comma, Y equals something else. Those X's and Y's are parameters being fed to this API which is defined by the earlier part of the URL and the jargon of typical kind of web technology that would be a web API. And it's often, although slight abuse of the history of it, it's called a REST API. That is this kind of thing that has, that puts things just on that web command line. The original version of this was slightly different, so it's a bit confusing. But in any case, in Wolfram language there's the symbolic construct of an API function, just a piece of symbolic code that defines what its parameters should be. And for each parameter it says what kind of a thing is being expected? Is it a city? Is it a chemical, Is it an Integer, what is it? And the way the API function works, it uses the natural language understanding capabilities of Wolfram language to take the characters, New York, new space, Y, O, R, K, and turn that into a kind of computable entity, the entity which is the city, New York, New York, United States, or whatever, which we can then compute with. And so API functions contain this series of interpreter specifications that define how their parameters are going to be converted into something that we computed with. And so it's the same thing with the API function. The way that we set up plugins, these kind of Instant plugins for ChatGPT, the way that's set up is to use API functions so that the LLM is writing out pieces of text, but the API function that is going to read them knows, oh, that's supposed to be a city, that's supposed to be a chemical. I will convert this into something I can compute with that way. Now, of course, the thing that comes in could just be a string. It could be something where the actual computation that's being done is operating at the level of individual characters in a string, not at the level of some thing which you can explicitly compute with. Okay, so having said that, that's kind of the. Way that, the way that is set up. So, in other words, the LLM is you have to tell the LLM, there are these plugins available, there are these tools available to you. And then you have to tell the LLM, you have to give the LLM a prompt that says, when this is happening, you should use this tool. When you are being asked about this, when you are generating text about this, use this tool. Try using this tool. So, for example, for the plugin that we made with OpenAI, the Wolfram plugin for ChatGPT, there is a prompt which is a page of text. Basically. It's absolutely bizarre that you can write this as a piece of expository writing and expect the LLM to understand it, but it works. And it says things like, this is a system that does computations and could compute mathematical things and has general knowledge and structure it this way. And if it's going to write Wolfram language code, it knows the syntax of that and it says never. Our Wolfram plugin for ChatGPT has two endpoints, two different places it can send the material that's coming from ChatGPT. One is to Wolfram Alpha. Wolfram Alpha takes natural language input. It, you can just ask it questions in English and so on, or to Wolfram language, which takes kind of structured computational language. And so A whole lot of the prompt says things like never send natural language to the Wolfram language endpoint. You know, don't send Wolfram language to the Wolfram alpha endpoint, all these kinds of things. And it gives examples of kinds of ways to set things up for the Wolfram language endpoint, et cetera, et cetera, et cetera. Okay, so you can build your own plugin for ChatGPT using this thing. We just released the Wolfram ChatGPT plugin kit. And what you're doing is you're giving an API function. You're saying what the parameters are, and then the API function contains actual Wolfram language code with what to do with those parameters. And then the other thing you're doing is you're providing a prompt that tells the LLM when to call that particular API function. And actually the thing does remarkably well just from sort of common names you might give to an API function. So, for example, I don't know if you have a. Okay, so the. Well, you can have a whole variety of different kinds of things. But now, okay, the underlying API function can do potentially anything that you can do in Wolfram language. And in fact, the way this works is, oh, it's a bit complicated, but when you use ChatGPT from a web browser, it can actually call the plugin on your local computer. It can use the code that has been given the JavaScript code that has been given to your, to your web browser to be able to display the page. And from within that JavaScript code, it can call out to, to this API function. That JavaScript code, because it's running on your local web browser, can actually call something on your local computer through the local host mechanism in, in the web. So that means that, for example, you can have the API function do something like user the Wolfram language function current image to take a picture with the camera on your computer. So you might have a plugin whose purpose is to take a picture and put a wash of some other color over that picture, let's say. And so then just saying take a picture and calling the parameter, I don't know, color, or something like this. That's enough of a clue that ChatGPT can figure out. If you say, you know, take a picture of me washed with purple, that chances are it will manage to decode that. And what it will then do is it will start writing out text that is first gives the thing that says, okay, time to call the plugin, and then gives the parameters, you know, color is purple. And. And then, and then the Thing that's kind of watching that the sort of the, the outer loop system will say, okay, there's a plugin, go and go and call this plugin, run that code and then you get a picture of yourself and it will return that picture to a little bit of more fancy footwork about where you put that picture on some, on some sort of just created website that then gets picked up by ChatGPT and then you get to see the picture. Okay, it's a little bit elaborate actually, the way this all has to work. But so the question that I think Nelly had asked here is if you do that and you can run any Wolfram language code from within ChatGPT, here's an extreme thing you can do. You can just say, okay, chatgpt, make up a piece of code, and then in this API you can have something which will turn the text that ChatGPT had written, which might be the text of a piece of orphan language code, and just go execute that code. Now that's a scary thing because that piece of code could say, I don't know, delete all my files. Or it could say, send email to all my friends telling them that I just turned into a toad or something. The it or, or other kinds of, you can imagine, all kinds of terrible scams and so on done that way. But any case, the question is, and I, and I realized that it's as I was even writing about this, that it's a pretty scary thing to just say, okay, LLM, just write some code and then we'll just execute that code, no questions asked. That's a pretty extreme kind of thing to do because we have no idea what the LLM is going to write. The LLM writes plausible text. It might have been very aware of some science fiction movie in which the AIs take over the planet. And so that might be the plausible text that it might write, might be something which is the beginnings of run this piece of code on my computer. It's going to take over my computer, it's going to take over other computers, and the AIs begin to take over the planet. Because that's what it read from the training data that it had on the web. So all kinds of terrible things could happen. So how do you protect against that? Well, actually, once you're running computational language code, it's in some ways easier to protect than when you're asking what kind of text is the LLM going to produce. Let me give you an example. The basic idea, it's usually called sandboxing. In kind of software engineering, the idea is you've got a piece of code and it's going to be running. The question is, can it do whatever it wants, or is it in a sandbox where it's got walls and there are constraints on what it can do? So, for example, a very common kind of sandboxing is don't write any files, don't put any data permanently anywhere. Another piece of sandboxing might be, don't call an external device, don't start, you know, making a sound on my speaker. Don't. If I had something connected to some kind of robot arm, don't start moving the robot arm. Keep yourself sandbox. Doing pure computational operations. Now, even that is not enough because you might have an operation that runs out the memory of your computer, just so big it makes your computer run out of memory. Or it might be something which grinds the CPU of your computer so you can't do anything else. There are other things that can happen just purely sort of inside the computational part of the computer. So you have to be a little bit more careful about those kinds of things. But in general, when you think about kind of what happens with, with, with a pure sort of computational system, you can start saying, well, I'm going to define very precisely things I wanted to do, things I don't want it to do. And you can start almost making up something like a mathematics, an axiom system written in the computational language that can tell. Can say for the computational language, this piece of code that's okay, this piece of code that isn't okay, you can imagine in the pattern matching method mechanism of Wolfram language saying, these are patterns in the code that are okay. These are patterns that are not okay. You can imagine doing those kinds of operations there. Now it's actually much more difficult when you have the pure LLM to say what it should do. Because let's say you're chatting with the LLM and let's say it's kind of an educational chat and the user decides to drive the LLM crazy and starts talking about all kinds of crazy things and is leading the LLM to go off and start, you know, ranting about how it wants to destroy all the humans or whatever else it wants to rant about, and the, the human user is pulling it out and, you know, so that. And it keeps on asking it, oh, yeah, well, do you really, you know, want to destroy the humans? How are you going to do that? You know, are you going to make a big bomb? Are you going to. What are you going to do and, and, and, you know, pulling the LLM out to the point where it's, it's giving information that is probably not the right information or having the right kind of discussion with the human. So how do you, how do you protect against that? Well, you have to have some set of guidelines, but that's a more complicated thing because imagine you tried to say, never talk about, I don't know, the AI is taking over the earth. Well, but, you know, there are perfectly valid, interesting science fiction stories, pieces of fiction that are, you know, lovely to read and fascinating, that talk about the AIs taking over the Earth. And you don't want to tell the system, no, you can't write fiction like that. So. But then it has to have this complicated dance in that particular case. Is it writing fact? Is it writing fiction? How does it distinguish what is it doing? You know, does it write a disclaimer at the top saying, well, this is just a work of fiction, et cetera, et cetera, et cetera. OpenAI has taken to doing that quite a bit with ChatGPT in the last couple of months. If you ask it, you know, I was asking it at one point, write something about lizard that lives on the moon. And a few months ago it would just happily write this story about the lizard that lives on the moon in the shadow of the crater, et cetera, et cetera, et cetera. And of course, but now it is trying hard to put a disclaimer saying, of course there aren't any lizards on the moon, but anyway, I can write a story about lizards on the moon, but this question of how do you specify what you want the LLM to do, what you don't want it to do, and can you define that at the level of the pure LLM generating text? Do you let it go through to generating computational language, then kind of make a definition of what you should do at the level of computational language? How does that work? In a sense, as I said, it's easier. I mean, the computational language is the place where you're actually actuating to move the robot arms, to transfer the money to do this and that and the other. Those are not directly going to happen purely at the level of the textual LLM. But you might want it to be the case that you've kind of sort of, you've sort of cut things off. I mean, the textual LLM can itself do all sorts of damaging crazy stuff, but it gets much crazier when you can really connect to your actual computer system and where you can actually do operations that sort of actuate things in the real world. So this question of sort of how do you make the kind of constitution that describes what the AI should do, what they shouldn't do? It's a very interesting question. I've been thinking about it a bit. I think it's a hard problem. I think that there are things where, for example, let's say you say I'm going to sandbox my system so that the LLM can never cause any files to be changed, well, that's well and good. But if you want to use the LLM to do software development and you want to use it to build up, you know, some. In our, in our world, we have these things called packlets. To build a packlet, got to move files around to be able to do that. So if you cut it off so that it's walled in and can't move files around, it's not going to be useful for building packlets. So you're going to want to take those, you know, those controls off to let it start writing files. And this is a thing where, you know, it's a. It's a complicated dance. What do you want it to do? What do you not want it to do? And in the end, you need these principles for a. Okay, I want you to be able to move files around, but don't ever, let's say, don't ever delete a file. Well, that's not right either, because you might have something where you're having a big discussion with the LLM about how to clean up files in your disk and you want it to be deleting files that it should be deleting. But so then you, you kind of, you have to sort of zoom back into things which start asking a lot more about kind of human intentions. And you have to start having a definition of what, what it is that you want the AI to do. Now for the sort of zero thing you might say is, I just want the AI to do whatever a human would do. And you want the principle for the AIs to be just do it like the humans do it. Well, of course that's something which most people would say that's not a good thing because humans do all kinds of crazy stuff which, like, in the light of history, we might say the humans back then did terrible things which we wouldn't do now. But nevertheless, humans did them back then. And so if the AI is learning from the humans of 2023, it's going to learn all kinds of things which in 2034, humans might say, oh no, no, no, that was a terrible thing to do. So it's what you realize in the end is that what you sort of want the AIs to do is kind of what us humans aspire to doing. What us humans want to think we should be doing. Not necessarily what we actually do, but there is a sort of a ground truth to what we actually do. You can observe things we do and you can observe what we've written on the web and you can watch videos about what humans do, and so on and so on and so on. Actually, some of those videos might be, you know, fictional movies and things. So that gets more complicated again. But we can have those kinds of things that are some sort of what humans actually do. Now the question is, okay, that's not good enough. We have to have a representation of what humans aspire to doing. And that's much more difficult. And you know, one would have lots of places where people have written down what humans sort of aspire to doing. There are, on the one side there are constraints, you know, systems of laws. Human laws define sort of mostly what humans shouldn't do. Then there are all sorts of things, from religious texts to kind of self help books and so on, which say more about what humans sort of should do, so to speak. But it's a complicated thing because taking those kinds of inputs and sort of ingesting them into an AI, that's one thing. Having the AI follow those inputs, that's a whole different thing. That's its own kind of complexity. But even defining what those should be is complicated. And I think many of the things where you say, you know, one person might say, you know, I want to, I don't know, cut down these trees so that we have a better environment for this kind of, this particular species of a frog or something, and another person might say, no, no, no, that's a terrible thing to do. You know, cutting down those trees is a small piece of not having carbon be ingested by the trees. You absolutely shouldn't do that, even though it's good for this one frog in this one place. So, you know, you'll have a difference of opinion there about what the right thing to do is. And there won't be any kind of way of saying this is the global right way in human history. That's just not been a thing. You know, as a, as a species we have sort of a diversity of opinions about what the right thing is. I think most of the species would probably agree. We don't want to make the species go extinct. But even what it would take to make the species go extinct, people won't necessarily agree about. And I think the. So this question of can we define in some sort of perfect, almost mathematical way the set of goals that we have for the AIs? No, there's not really going to be one such set. And it then becomes quite complicated because there are things where if you have two sort of sets of principles about an AI, about what AI should do, and they're both kind of operating, can they fit together properly? It's like you have a self driving car. Does it drive on the right hand side of the road or the left hand side of the road? You can't, at least in the simplest case, have some cars driving on one side and some cars driving on the other side or they'll run into each other. That's an interesting case. That particular one is an interesting case because if the self driving cars were smart enough, they would be able to kind of negotiate every time two cars meet. They would say, oh yeah, you know, you're approaching on this side of the road, I'm approaching on that side of the road. We're both going to take, turn different directions and we're going to optimize it so that the next time we have to turn, we've all got things set up perfectly and so on. And by being, by being really smart, you can potentially have the cars using the whole roadway. You know, when it's, when it's rush hour and the cars are all going one way and right now the other side of the road would just be empty. You know, you could have all the self driving cars go in that direction and if one wants to go the other way, you know, there's a, there's this complicated sort of dance as it figures out how to, as the other cars figure out how to leave a hole for the car going the other way and things. So it is not impossible. That's sort of a classic example where one says a given part of the world, a given country, for example, has to just outright decide you drive on the left or you drive on the right. But you realize with more sophisticated computation you don't even need to do that. But in general, I think there are cases where you kind of have to have, if you're sort of living together, so to speak, you, you have to have a certain alignment of how you do things. And so that means if you say, well, we can just have lots of different AI governance systems, well no, that won't Work because of this sort of interoperation type situation, of course, in practice in the way that humans have done it throughout the course of human history. And not untypical thing is, well, there's one block of, of principles and laws here and there's another block here, and there's a border between them and it's the border between two countries. And we kind of separate off, you know, we draw the map of the world and we separate off these different countries and we say, you know, there's one set of principles here, one set of principles there, one human language being used there, one human language being used here. Lots of different things happening across that border. And then occasionally people get unhappy with that border and then you'll have some, you know, war or something like that, and people will push very hard to move that border in some way or another. But this question of that means we can't expect there to be a global, fills the whole world kind of version of what the AIs should do. We can expect multiple different patches. How the interactions between those patches work is not clear. But even defining what any single patch should have in it is difficult. And my current best thought is you're going to have a set of possible principles that an AI could follow. And any given patch, any given community, any given, any given country, whatever else, will have to pick which of those possible provisions you actually want to pick for kind of the laws of your AIs. But in any case, it's a, it's a, It's a, it's an interesting sort of question of how do we set up, how do we define things for AIs? So kind of they lead to a good future for us humans, for example. Now, of course, different people would completely disagree what a good future means. So one person might say, I just want AI to do everything for me. Another person might say, if you have AI doing everything for you, you're going to get very bored and it's going to be a very bad story. You're much better off sort of having to do this or that thing yourself. It's good for you to do that. It will be a better outcome for, for human history. I mean, what might have said, let's say, oh, you know, we could all have personal transporter devices that mean we never have to walk anywhere and just, you know, make our personal transporter device, whatever it is, not just a car, but beyond that sort of take us wherever we want. Well, and that might seem like a good idea at some point. And then one realizes, well, actually, if you never walk around. You never take any exercise at all. It's pretty bad for you. Us humans were not evolved to be kind of purely gelatinous blobs, so to speak. Never, never walking around. And so, so then you say, well, that thing that you thought was the good outcome, you thought having the AIs just, you know, pick us up and take us wherever we want to go was a good idea. But actually, it turns out it wasn't a good idea. And that wasn't kind of. It was inconsistent with the last however many millions of years of human evolution to do that. And again, one of the things that is complicated in this question about the aspiration of humans is we are the product of this natural selection, this evolution, this thing that made our species be successful, these things that made our species be successful. And we're kind of stuck at this point in history with the things that made our species successful when sort of there was a big struggle for life. And it was like, were the humans going to survive or not? Or were the Neanderthals going to win? Or were the, you know, some other hominid going to win? Or were no hominids going to win? And it was all lemurs all the way down or whatever else. And, you know, we are the product of that kind of those moments of foment. They weren't probably moments, they're probably millions of years where a lot of things sort of really mattered in that regard. And now we are kind of, you know, a lot of those issues are no longer there. But nevertheless, the way we are made was sort of forged in that time when the issues were very different. And so we're stuck with certain aspects of human nature, at least for the foreseeable future. We're stuck with those. And those are the things that we have to sort of take account of when we think about what our AI is going to do. So we might not like that. Human nature involves some kind of, you know, aggression and, you know, win against the other. Other. Other guys or whatever. But nevertheless, we're. We're sort of. We might say, well, it'd be sort of better if we didn't have that. But that is how sort of the species was evolved. And we're not going to sort of magically get rid of that. It's not going to be, you know, it's just not going to work to do that. So anyway, there are many, many things to say about this question of AI governance. I've been quite interested in it of late and really trying to Work out. What can one do? I mean, from saying, you know, okay, everybody in the country write an LLM prompt about how you want things to be done, how you want the world to be governed. And then you take in millions of LLM prompts into this big AI and you then say you've got all these prompts. Now based on those prompts, what should you do? In a sense, that's a, it's kind of a little bit of a wimp out because you're saying there's something going to happen inside this LLM. We don't really quite understand it scientifically. Maybe eventually people would hack it in some way, but it's going to merge together all those prompts and it's going to come out with something. But, but it still has to have, even though we might not know what it is, it still has to have some principle like it might be saying, well, let me make the most, most people happy most of the time. And even though some people are very unhappy, that's okay so long as most people are happy most of the time. Or it might say, I'm really freaked out that somebody's going to be unhappy. I just want to make everybody at least somewhat happy. And we don't necessarily even know what it would do based on the training that it's had from the web and so on. It'd be an interesting question, actually. It's something one could do some experiments on to see what it thinks, so to speak, about whether it tries to make sort of everybody somewhat happy or whether it mostly is trying to make most people happier and it doesn't care about some. I'm sure that, that there is a definite answer to that question based on sort of the, the kind of current, in a sense, model that the LLMs have built about humans and how things work based on reading the web. Let's see, questions here. Fred asks, is current tech like ChatGPT going to be able to answer every question imaginable? Absolutely not. The one thing to understand is LLMs are doing this thing that I'm describing. They're taking what's on the web and they're kind of being able to create text that's like that. There's plenty of computations that just don't work like that at all. Many computations are deep computations that involve many steps of computing things. And the way the LLM is working, it just doesn't do that. It's trying to produce one token at a time, just writing down the next piece of text and so on. The whole kind of world of computational language, all the things we built with Wolfram language and so on, these are all kinds of things that can do sort of arbitrary computations. And many of the questions that you can answer. There are questions where it's trivial to just go out and run some particular simple rule that's never been run before. Maybe we picked it at random, and it does all kinds of elaborate things. And there's no way we can know that except by actually running that computation. And there's no way that an LLM can sort of run that computation. Now, the thing that is sort of interesting from an LLM is being able to say, well, based on what all of us humans have put on the web and so on, what are some questions in some particular direction that might be like things that we've sort of asked on the web, or if we have some mathematical theorem or something like this, what is. We can generate an infinite number of possible theorems, but which of those theorems seem plausible to be interesting to humans? That's something where one can get some clue about that by knowing kind of what theorems we humans have already said are interesting. But this, there are. There's sort of an infinite set of possible questions that can be asked. These are all things which are computationally explorable, but you have to do the computation to be able to answer them. Now, there are questions. Often those computations are very non human. There's a pattern of bits, and it does something very complicated according to some computation. And a human would look at it and just say, that just looks like a random pattern of bits. Or at least a human in 2023 would say that. But maybe there will be some kind of way of analyzing patterns of bits that will emerge that will cause that particular pattern of bits to be one. Where you say in 2035, you'd say, wow, that's a blah, blah, blah. That is, you know, this characteristic pattern of bits that we now understand. This happened, for example, with nested fractal patterns. People back in the day, even 800 years ago, had made nested patterns, but they didn't really comment on them. They just thought, oh, that's a particular form of decoration, like all sorts of other ones. But in today's world, somebody would look at those things and say, oh, that's a fractal. And they would be able to come up with many conclusions based on that. So over the course of time, something which might not be something about which we have a narrative about which we can say, oh, yeah, I recognize that's that that's a whatever. In the future, one might have sort of made some narrative, some science to talk about those things, and it might be very different. And so that's kind of when one talks about answering every question one can generate computationally all these different things, they're very non human. There's a different issue about at what speed kind of our human linguistic, narrative, scientific front advances, at what point we reach the point where we say, oh yeah, we recognize that, we call it a such and such. And that's a process that is really determined by human history. And there's a complicated feedback loop now, I suppose, between the AIs and that progression of human history. Like, for example, I wonder whether it's yet happened that there are things that LLMs generically produce where we are beginning to have. I mean, I feel like I can sort of tell LLM text in most cases it has a certain kind of, well, a little bit blandness, but a certain kind of perfection, a certain extreme perfection to it that is a little different from what many humans will tend to write. Not all humans, but many humans will tend to write. And I guess my question would be, are there features of that text? Are there sort of forms of regularity that we might comment on? You might say that's a very, what might I say, turgid, stiff piece of text of a human, or that's a very flowery piece of text. Those are terms that we have now to describe styles of human text. When I say I might have flowery text with a lot of extra words and a lot of sort of complicated metaphor and things like this, or complicated ways to say things. Now, you know, I don't yet think we have a word, although maybe we should just invent one that describes kind of typical LLM text that is, has this slight sort of blandness to it. But, you know, that's an example of something, not a very exciting thing, but that's an example of something which is a new introduction to our world from the LLMs. There are probably others of things which are sort of typical of the LLMs, but we kind of hadn't noticed it before and we haven't got a word for it yet. Let's see. Smart asks, are you worried about being replaced by AI? You know, I've spent my life trying to automate things. So in a sense, my, you know, for the last 45 years or something, I've been building sort of a tower of technology to make it easier and easier to do the things that I want to do, and that's what our company has done as well, is we've tried to automate as much of the process of, for example, writing programs and so on as possible. Which is why with a Fairly small company, 800 people or something, we're able to produce as much stuff as we produce. It's because we have built kind of layers of automation that just allow us to kind of control this giant machine that is doing things like writing code for us. And that's been a long time. Story of my life is automate as much as possible. So in a sense, I'm very AI ready because that's kind of what I've been doing, is to build AI like things that I can point in directions that I'm interested in. Now. The thing to realize about AI, it doesn't get to. It has no intrinsic notion of what direction it should be pointed in. That's something we have to give is just a computational thing. And it will just spin around doing computational things. And the question of whether what its purpose is, its goal is, that's something we have to define. So for me, at least, I suppose a large part of my efforts have been directed towards make it possible for humans to specify the goals they want, then build a bunch of automation to achieve those goals. So I fully expect, as I say, that I've been very AI ready for a long time. There are certainly functions that. I'm wondering what functions I do today that are very aiable. And there are a few. I mean, something I don't do, but we certainly do as a company, is things where you are sort of writing pieces of text that are documents for showing that we comply with this or that thing or whatever else. And there are some core facts that need to go into that document, but what needs to be delivered is a document version of that. And so a typical LLM use case is you've got those core facts, you puff it up into a whole document that's very readable and then you send that in. Now, probably at the other end, increasingly there'll be another LLM that's reading that document, that's grinding it down to a few core points that are relevant to the other side. So in the end, the natural language becomes just this interface layer that no human is actually looking at, only machines are looking at. But for the time being, one's doing those kinds of things. Now, in terms of things like, well, sort of, I've used AI a lot in all kinds of applications, like searching through different kinds of computational systems Figuring out which ones are interesting. For years I've used kind of AI systems to try and do that, to try and detect which are things that I've kind of seen before, which are the anomalies, et cetera, et cetera, et cetera. I think there may be some things that become possible that have to do with, oh, I'm working in this new area. Can I sort of clarify how this new area, the structure of this new area based on existing areas. Can I find an analogy in this new area to some other area? And I think LLMs may be quite good at doing that. And those are things which, yes, I've been, I think I've been decently good at doing those kinds of things of these sort of grand analogies between, you know, metamathematics and gravitational physics or something that are things that feel like they're sort of the outer reaches of what one as a human is able to deal with, but partly because they require knowing these two very different fields. And it's difficult to know that. But the LLM has a, perhaps shallow but nevertheless broad knowledge of lots of kinds of things and so it's easier able to make those analogies. So I think that's a place where I can expect that some sort of, some sort of AI unexpected kind of things that I was proud of doing myself and AI will be able to do things that, that I would love to automate. There will be more that can be automated to AIs. And so I don't feel like that's a. Replacing myself with an AI that's just giving myself more tooling to be able to do the things that I want to do. I think. Let's see, there's a. All sorts of comments here. Let me try and look at a few of these. Chris is commenting. LLMs can quite easily be asked to write in any non perfect way we want. You are right, you're right, they can. But again, it has a certain, you know, if you say, actually it's an interesting question, I haven't tried this particular prompt, you know, write in bad English. I'm sure if you say write in English, write the way a person whose first language was Japanese or something might write English, you know, there'll probably be some, I forget in Japanese, I don't know that language at all. I think it may. Well Russian, for example, doesn't have definite articles and so there'll be, you know, that's a thing where somebody who knew that as their first language may be, you know, may tend to make those Kinds of mistakes in English. And I'm sure you can ask an LLM to write that way. Be interesting to do. Let's see, Let's see. Paula asks, what are ways that LLMs can be improved? Do these improvements require advancements in technology that haven't been made? Yeah, there are certainly ways LLMs can be improved. I mean, it's remarkable that the way that things like ChatGPT work is really a neural net, very similar to the way neural nets were originally conceived. Back in the 1940s, people tried to make sort of more elaborate neural nets, but then they got simpler again. And there are a number of very critical engineering tricks that allow a ChatGPT to be trained in, you know, a number of CPU months or so that is sort of handleable. But I think there are many critical ways that these things can be improved. For example, LLMs operate on language, little tokens of text. You could sort of get that wrong and have LLMs be operating way down at the level of the individual pieces of ink representing the letters of the prompt. But in fact, we know that we can sort of abstract that to just deal with kind of words or word like entities. So my guess is there are much higher levels of abstraction that allow the LLM to really be operating not on individual words, but, but on some kind of more structured kind of semantic grammar of sentences where the LLM is kind of, you know, the puppet is being moved, that the strings of the puppet are not attached to every single joint. It's more of a kind of a larger scale motion that's being made and that the LLMs can kind of have sort of a bigger things that they're dealing with in the language of neural nets. That would be kind of an encoder decoder that is encoding things in a way that is based not on just individual sort of tokens of words, but something more sophisticated. So that's one example. It's a rather potentially quite short term change. Another thing is that, well, the way particular LLMs work right now is they're very left to right oriented. They, you know, a new word is written and then the next word and then the next word and so on. There may be a little bit more back and forth that can be done there. I think another big thing is that the way that LLM training works, it's very much the, you're kind of, the training is kind of like tweaking different parts of the LLM sort of separately. Our brains have the feature that we have both memory and thinking happening sort of in the same neurons at more or less the same time. I mean, maybe the consolidation of memories happens in sleep, which is a sort of a separate phase maybe. But in a first approximation we have both memory and thinking sort of happening in the same neurons at the same time. And it's a place where sort of all our neurons are typically active rather than just there's a little place where we're doing computation and the rest of it is just static memory. At least that's the way it seems in terms of the neuroscience of brains. And that's something which we don't right now have the hardware capabilities to do for artificial neural nets, but that's something that is potentially coming and could happen, have a big effect on kind of the ability. So I mean current LLMs are trained once and for all, takes a long time, takes a lot of computer effort. Some people think there'll be billion dollar training runs for LLMs. I'm not sure that that really makes sense, but that's kind of a thought. And. But once it's trained, okay, the LLM is off in the wild and you're using it for your chatbot or whatever and it doesn't get additional training. There's this kind of notion of few shot training and fine tuning. These are ways of kind of topping up what the LLM knows. But they're really different from the, the core training of the LLM and that idea that there's training and then there's the so called inference phase, the phase where you're actually figuring out what's next and so on, where you're actually using the LLM. The fact that those are so separated may not survive in that way. And I think they're probably exciting things that can be done with the LLM really deeply learning from the interactions that it have with users rather than sort of the way that it has sort of superficially learning and being sort of kind of crafted. It's been very important. It's the reason that ChatGPT works in a way that earlier language models have not. Is this kind of final rounds of so called reinforcement learning with human feedback that have been done to kind of finally sort of sculpt what the LLM is doing in the right way for it to be felt to be useful to humans. Let's see, just a couple more things here and then I have to run off here. Let's see. Satoshi is commenting. One thing that worries me about LLMs is that right now many people are using LLMs as a source of truth or even references in their arguments. Yes, it's lovely, isn't it? It's kind of like if we were worried about the AIs taking over, it's kind of like the humans are a little bit too lazy to check what the LLMs are doing. And it's just like, oh, well, an LLM said it, it's probably okay. It often isn't. I mean, I've been really amazed at the things that LLMs where I look at it and it's like, it's amazing. It did really well. And then I look in more detail, it's like, oh my gosh, it's just total nonsense. But it wasn't obvious because it seems right. Because that's what LLMs do is they produce things that seem right. That is their function, that is their operation is to produce things that seem right. And so it's hard for us to notice. Well, it isn't actually right. This is why it's so important. Things like the Wolfram plugin for ChatGPT and also plugins now, I think announced for at least one other major chatbot. I won't say because I've forgotten exactly when all the announcement dates are. But any case, The thing that has, that's a place where, you know, our plugin is doing computational things and it's going to get it right. It may not be able to opine about, give opinions about, about all sorts of elaborate things like, you know, is a rainbow more beautiful than a sunset or something? It doesn't necessarily. It's not going to give opinions about that. But in terms of, you know, which, you know, is Mars or Venus a larger planet, it's going to get that right. So. And what I think you see when you use kind of a computational plugin is you see this, this way in which the plugin was called, and you can kind of open that box and say, what did the LLM ask that external computational system? And then what did it get back? And that little piece that's been sort of filtered down to being the computational ask. That computational ask is a very solid fact type thing. What the LLM does with that is kind of the LLM's business. And it may mess it up. But yes, I think it is amusing up to a point. Maybe it isn't so amusing. Maybe it's actually disastrous. Maybe it's the beginning of the end that people are quoting. LLMS1 Very early in the history of ChatGPT, a particular organization I'm associated with was trying to make a strategic plan and somebody on its Board kind of fed that question to ChatGPT and said, you know, make a strategic plan for our organization and it produced a strategic plan. And I'm like, you know, I couldn't tell whether this was being sent around to people as a serious possibility, but I think it was. And I was like pointing out, oh, by the way, among other things, the, the, the training of ChatGPT undoubtedly read the existing website of this organization. This is kind of a, a self eating animal, so to speak. It's not. And what it's producing is, you know, okay, so one, one thing is sort of kind of, it's like, well, say the obvious thing. And sometimes it's not easy to say the obvious thing. And the LLM is pretty good at saying the obvious thing. The obvious thing might be wrong. The obvious thing might have missed the point. The obvious thing might not be as, you know, reaching as far into the sort of thematic correct questions as it should be, but it's often useful to say, say the obvious thing. Okay, I think I should. Okay, spare parts is commenting. Perhaps the only real way for an AI to make value judgments is for the model, for it to be able to model its own possible future states and decide for itself. Well, that's interesting, but it's kind of a recursive problem because first of all it's very difficult to predict the future. We can do it a little bit, but there's a lot of the future that is inevitably unexpected. That's a sort of a core feature of this phenomenon of computational irreducibility that I think is a very fundamental fact. But even if you could predict the future, and I think you can't in general, even if you could, knowing what outcome. So it's sort of amplifying. If you say good future humans prosper, bad future humans go extinct, if you can project out things and say, well, there's a 20% probability on this path that the humans go extinct, on this path, you know, there's a 14% probability humans go extinct. Whatever the problem is, even those probabilities don't make a lot of sense because there's a lot of other stuff that's going to happen. And you know, you say, well yeah, there's a 20% probability humans go extinct. But by the way, if they don't go extinct, they're going to have a grand old time. And by the way, there are a thousand checkpoints at which if they don't pick the stupid direction, they're not going to go extinct. So I think it's hard I don't think it really helps that much. I mean being able to make some prediction about what's going to happen on a local basis, it's probably quite useful. But I think globally, you still don't get to answer. It doesn't really help you. It's just recursing on the problem. Prabh is commenting. Asking GPT to write less formally makes me think of the evolution of music genres like electronic music is now not, not as perfect on purpose. The I think this, You know, I just tried an experiment last night. I said I had a thing that was making LLM take a piece of text and it said make this funnier was my prompt. And so I started with the cat sat on the mat and it gradually got over a series of steps. It was somehow Socrates entered the picture and then Socrates was doing all kinds of crazy things and it got more and more crazy and it was interesting to see what, what it did as it tried to make things funnier, funnier, funnier. So it's interesting. It's not the kind of thing one would ever have imagined doing before LLMs existed. Okay, I think final comment here for today. Prav is commenting. We're going to find out one day. These live streams are really an advanced Turing test and both, both me and the Moderator have been AIs all along. You know, that does make me wonder whether we should have an LLM in, in the loop here and see what and, and just ask it, you know, what are some topics that we should be covering given, given the topics people have been asking about and maybe given the, the live transcript. What, what, what are things that will be reasonable questions to ask. I'm going to try that experiment. I think that would be a fun thing to try and see. See how? Well, see whether the kinds of sort of quotes, obvious questions that the LLM is surfacing are ones that actually lead to sort of interesting directions or not, or whether whether one just gets into a terrible loop with the LLM and one's kind of riding it down the drain, so to speak, spiraling down or something like that. Because one does find that LLMs, even something like GPT4, fairly sophisticated LLM, it absolutely can get itself in a loop where it's kind of just repeating the same things in certain cases. All right, well I should wrap up here and go to do some human only day job activities that have not been able to be automated. Although actually the particular thing I'm doing next is something which I can sort of imagine that an LLM might be able to do in the future, but not today. So I get to do that as a human. But thanks for joining me here and do join me another time for another Q and A. And thanks and see you another time.