Science & Technology Q&A for Kids (and others) [Part 123]

Okay, hello everyone. Welcome to another Science and technology Q and A for kids and others. And I said I would talk today about the topic of AI, which is something that is being broadly discussed these days. And in fact, just last week we introduced this plugin for ChatGPT that connects to Wolfram Alpha and Wolfram language to kind of let ChatGPT really do computation and computational knowledge in addition to just doing kind of linguistic, textual, types of things. Let's see, I see various questions that were saved up here. There's a comment here from Steven about ChatGPT and it says that comments that ChatGPT is trained from sort of the vast mass of what's on the Internet, a few billion pages of what's on the Internet and a few million books. And that that's kind of the, it's trained from the average of humanity. And the question is, if one trained from sort of some very selected set of great books of humanity, let's say one, would get a better result? Well, in the current state of the art, the answer is no, because it turns out that it seems to need almost the trillion words that it's been given for the system to get to understand sort of the statistics of language well enough to produce decent amounts of text. So for example, if you trained on only 100th of that amount of material, and that's what was done in earlier models, it just had never seen texts that would talk about, I don't know what, pink elephants or something. And so if ever that concept came up, it's like it, it's not got any ability to, to generate text around that because just had never seen that in the training data. Also, it seems to be the case that from seeing all of that training data, it's kind of trying to understand the regularities that exist in English. The fact that in English we have, you know, sentences that have subjects and verbs and objects and things like this. And it seems to take an awful lot of samples to be able to learn these kind of structural facts about something like English. I think that when it comes to sort of the structural facts, it's learning, it's not just purely the syntactic grammar that we're kind of used to, of nouns and verbs and parts of speech and where they go. It's also something that's more of a meaning based semantic grammar of, you know, elephants can walk to places but they can't fly to places, those kinds of things. So it's, it's something where you just need a vast mass of material, a vast mass of kind of human written material to be able to get something which can successfully produce text in the way that ChatGPT does. Now, whether that's something that's really intrinsic to what's going on, or whether one could instead, if one knew what to write in the kind of semantic grammar book, just like we roughly know what to write in a syntactic grammar book, if we knew the content of a semantic grammar book and could sort of feed that in as computational input to the system, then one could probably deal with a lot less training data and then it might become more realistic to kind of weight the thing with kind of the, the great books, whatever one picks the great books to be and have those be the things that are the primary things that it's picking out. Now it's certainly already true that you can say, you know, write this in the style of so and so, and it will take its sort of foundational knowledge of language and kind of look at it through the lens of that particular author and so on. But it is also true that the general sort of essays that chatgpt writes are kind of the, in some sense the average essays of humanity, so to speak. It's kind of because it's taking sort of the whole content of the web, all these books and so on, it's kind of averaging out what it's not taking a very kind of, it's taking sort of the average point of view on things in a first approximation. You can kind of say, well, sort of weight it to be more in the style of so and so. And you know, the extent to which that's successful is, is unclear. But sort of the core foundational part is kind of from, from everybody, so to speak. Kind of makes one wonder. You know, I see there are a number of questions here about AI alignment. So just to explain what that means, it's kind of the question is we've got these AIs, they're, they're kind of running all these systems in the world. And how do we make the AIs do things that are kind of aligned with what we want? How do we make sure the AIs in principle could be running, I don't know, some system that's. Well, even as they do right now, deciding what posts to put up on our social media feeds or things like that, or the AIs could be doing all kinds of things perhaps in the future of doing sort of educational tutoring, all those kinds of things, or deciding things about actual devices, whether they're self driving cars, whether they're other kinds of things. The question is, how do we get it so that the AIs are basically doing what we humans think they should be doing, so to speak. How do we get them to be aligned with our kind of views about how the world should work? Now the tricky thing there is that one thing one might imagine doing is just say, well, have the AIs watch what humans actually do. Have them watch all the movies, have them read all the books, just say, let's just do what humans do. Problem is, most people think that that's not really a great way to act. They're things that humans aspire to doing. It's like, I want to be a human who, I don't know, treats everybody equally, who is always looking out for the good in things, is never getting into fights with anybody. All those kinds of things. You aspire to be that. But if you look at the actuality of how people operate, it's not how they actually operate. So if you say take, you know, take the actual data on what humans have actually done, you'll get a different view. So then the question is, when you start saying, well, let's have it be that the AIs work as aligned with what humans aspire to do. Well, it's a question that then immediately comes up is, okay, well, person A thinks humans should aspire to do this or that. You know, I think, I don't know, rational thinking and, and understanding knowledge and all that kind of thing is, is really worthwhile. But, but somebody else might say, you know, I, I just believe what's written in this particular book and I don't want to hear about anything else. And that's my, my worldview. And you know, I don't think I could argue on any abstract grounds. You know, I can give a, a speech about why my worldview is better, but that's something that is very people dependent. And I don't think it's something where you can say or should say, you know, one thing is intrinsically the thing to go for and the other is not. So it's kind of when you have this kind of notion of aspirations and so on, they can be very diverse for different people. So then one question is, which is an important one for AI alignment? There are aspects of the way the world works where sort of people have to agree about things. A classic example is which side of the road are you going to drive on? Well, you have to decide it's going to be the right hand side or it's going to be left hand side. And everybody has to follow that, or at least that's what you might assume by the way. And sort of, in that case, sort of little footnote, if every car is self driving and is incredibly clever, you maybe don't have to define which side of the road you should go on. Because every car, when it meets another car, it can do this complicated negotiation and it can be using sort of the whole roadway and you know, if more cars are going one way than the other, it's using the full roadway for most of the cars going that way and another one is, is weaving around going in the opposite direction or something. That's something that becomes possible when you have sort of the, the intelligent cars dealing with things. But at least in the traditional case it's like you better figure out everybody's going on the right hand side of the road or everybody's going on the left hand side of the road. Otherwise people are just going to crash into each other. I'm reminded of once visiting a small island where, well, different islands where some are under were sort of British themed islands, some are French themed islands. You know, one was driving the left hand side of the road, one was driving the right hand side of the road and island on the sort of joint influence was well everybody kind of knows each other and when they meet on the road they decide which side of the side of the, of the road to pass on, so to speak. But any case that's a case where you have to have sort of a global agreement about, at least in first approximation about what to do, then the question is if the AIs are sort of running the world, how do they get to those kinds of global agreement cases, how do they get to decide those things? And you know, you can imagine all kinds of interesting exotic possibilities in the modern world. For example, you know, in the traditional way it's set up, let's say in a democracy, people say, well I'm going to vote, you know, for, I'm going to put in votes for left hand side of the road, right hand side of the road, you know, the majority wins. And that's how it works. But you can imagine more exotic kinds of things. Like one I was thinking about is in modern times with sort of LLM large language model ChatGPT, like technology, you can have something where both the computer can generate an essay, but the computer can also read an essay. And if you give ChatGPT a big long chunk of text and you say in this chunk of text work out whether the person really wants people to drive on the left hand side of the road or the right hand side of the road. It'll do a somewhat reasonable job of figuring that out. So kind of the interesting thing you could imagine is kind of you just say everybody, instead of filling out the sort of voting form or something, you say to everybody, write an essay, could even be in English, natural language, whatever, about how you want the world to be. And then that would become kind of the thing that's fed in to a big AI. I think a much simpler case of that actually, which is a more short term, is the following thing. So you want to define kind of how a future ChatGPT should be set up, what kind of, what kind of preferences should it have, what kinds of things should it talk about, not talk about, have opinions about, not have opinions about, and so on. And what one could imagine doing is just saying to people, okay, whoever wants to, you know, write your essay, one per person. And these will all be fed into the kind of prompts that are used to define kind of what the LLM should do. But in any case, it's kind of an interesting thing. And just to explain one point here, I think in the modern world, modern last few months world, what we can expect to have happen is that the process of writing an essay is something that is now sort of an automatable thing. They may not be the most exciting essays, but they're workable, readable essays. But how do you initiate that essay? Well, you have to say what you basically want it to talk about. Maybe it's just a question or you want to might want to say, I've got these core things that I'm trying to communicate. I'm trying to describe, you know, why, I don't know why I should be allowed to keep a tiger on my property. And I'm going to make four or five key points there. And then I want to turn the whole thing into an essay because that's what I have to submit to the town or something to let them decide whether I can keep a tiger on my property. So then, so in a sense one's taking one sort of few bullet points one wrote out and one's turning that into a whole essay. Then on the other side, when it goes to the town for the, you know, the tiger permit or whatever it is, then it, the town might also have some key things that it's asking for. You know, how does it affect public safety? How does it, you know, is it nice to the tiger? Those kinds of things. Some key questions that it might be trying to ask. And then what one can imagine is again, using the same LLM technology to go from the kind of the, the big long description that the person has submitted as their sort of get me a tiger permit type thing. And then the town is taking that LLM output, feeding it into its own LLM and deducing from it the few bullet points that it really cares about in terms of whether the tiger permit should be issued or not. So I think that's going to be a pretty common kind of approach. It's kind of LLMs talking to LLMs, but at the end of it, it's kind of like what the original data went in from a person and then at the end the original things the person cares about come out from the, from the subsequent LLM. So I think that's, you know, that's a little bit of how that might work. Let's see. Julie asks, what am I most excited to see from AI? Well, gosh, it's really interesting because sort of the success of LLMs, which I think none of us really expected. None of us expected. I mean, just to explain a little bit, I mean, something like Chat GPT, what it's fundamentally doing is it's trying to continue a piece of text in a way that is sort of typical of how that text would be continued on the web. So if you say I have a tiger in my backyard and it next word, you know, growls at people or it prances up and down and so on. So you've got growls, prances, whatever else. It's the goal of, of something like ChatGPT is to decide what the most likely next word is. And it works essentially a word at a time. And the question is, how does it decide that? And the answer is it's looking at the whole content, the trillion words from the web and so on. And it's saying, well, what are the chances? What are the things that are most likely to be the next word? So for example, if you said the cat sat on the and it might very well be mat, because that's a kind of a common thing for people to say. Then it will have seen many examples of the cat sat on the mat. And so when it sees the cat sat on the, it's going to say, okay, there's a high probability the next word is mat. And so what it's always doing is figuring out what's the most likely next word. Now there isn't enough text, even the trillion words, there's not enough text on the Internet and so on to be able to know really from probabilities. Because if I say I have a tiger in my backyard, and it. The number of times that the Internet says I have a tiger in my backyard is probably very small. So it's not possible to just look at it from the statistics of what's on the web. So instead what one has to do is to have some kind of model, some kind of way of emulating what sort of perhaps brain like things do when they're confronted with those words and sort of pick the things that are on the web which sort of somehow correspond in meaning to the thing I'm asking. And that's the whole world of neural nets that I think I've talked about before, about how one does that continuation. And the sort of, the big surprise is that these comparatively simple models, which were originally inspired by the structure of brains, these fairly simple models are capable of doing a fairly good job of sort of doing a sort of human like guess for what the next word should be, even when it goes beyond what is really deducible from the pure statistics of the web. So that's kind of what's intrinsically happening there. When one's looking at images and things, it's sort of the same idea. It's a little bit. Well, people have different approaches to that, but at least one of the ways of doing it is sort of the same idea. If you were going to continue an image based on the things you've already seen on images on the web, how would it likely continue? And so on. But I think that the, the question is what can it generate? And it's sort of a big surprise that it can generate as convincing human like stuff. And it can do that for natural language, human language, English, whatever. It can also do that to some extent for code. And one of the things that to me is exciting right now is the computational language, our Wolfram language system that I've just spent the last, I don't know, 40 years or so building, turns out to seem to be the sort of unique kind of medium for collaboration between AIs and humans. Because our computational language, it's an attempt to kind of represent things in the world in sort of precise computational form, as opposed to programming languages which are really concentrated on make a computer operate telecomputer in, in its terms of, you know, you have a variable, you have an array, you have a, a loop, you have this kind of this or that kind of thing, which is really the way a computer is intrinsically built. Those programming languages tend to be sort of explaining to the computer what to do in terms of how the computer is built. Whereas a computational language of the kind that, that we've built it, it's idea and it's sort of the unique, such, such language at this point, its idea is to take concepts in the world like cities or chemicals or movies or whatever else, or actual images or whatever, and be able to describe using. Well, in the case of Wolfram language, it's about 7,000 primitive kind of constructs, what you can do to those kinds of things in terms that we humans kind of immediately understand and care about. And the whole point is that in Wolfram language you can write in a fairly small amount of code, you can describe something, a fairly major thing that you might want to talk about. And so in maybe three lines of code you can have a very non trivial thing happen. Well, the thing that's really kind of interesting is that LLMs seem to be able to write that code. We've given them enough training data of kind of natural language combined with Wolfram language that they can do that themselves of going from natural language to Wolfram language. We've already got a system that does that very efficiently in Wolfram Alpha where we go from small pieces, small questions asked in natural language to computational language and then we compute answers to them. That's something we've already been able to do for a dozen years in Wolfram Alpha. And in fact, in the plugin that we just built for ChatGPT, one of the things it can do is to kind of take this potentially very elaborately essay kind of formulated question and boil it down to something which is more like sort of short piece of text that WolframAlpha would deal with. And then WolframAlpha will go from that, translate it into Wolfram language, compute the answer and send it back as text. But a thing that is also working is to have the LLM ChatGPT directly create a few lines of orphan language code. And then we humans. Wolfram language is intended to be read by humans as well as read by computers. So we humans can then read those lines of code and could say that's what I meant. That's the precise formal version of what I said roughly in natural language. Check it off, you can run it, see what happens. It's really pretty interesting to see ChatGPT interact with that code because what can happen is it generates a piece of code. It's not quite right. The, our system can run that piece of code. It generates all kinds of messages saying oh, that wasn't quite Right. Then we feed those messages back to ChatGPT and say, hey, the code you wrote wasn't quite right. Can you fix it? And it will often be able to fix it. And you see these loops going on as it's sent back multiple times for get it right now. And eventually it gets it right. Sometimes, you know, only you as a human can be the one who looks at it and says, yes, this formal thing is actually correct, it's actually what I intended. And then you can go from there, take that piece of code and use it to build up some potentially bigger and bigger system. Because, you know, when you're just dealing with natural language, it's hard to know to be precise enough that you can set it up so that you can really build a big tower of capabilities. On top of that, it's, you know, it'll be. It'll be sort of roughly right. I mean, it's kind of like with a search engine, you get a bunch of links, and so long as a couple of them are what you wanted, you're a happy person. But if you look at the full output and you say, well, can I build some structured thing on top of that full output? The answer is probably not, because some of those links will be goofy links that you really didn't want. But, you know, I think the thing that we're seeing right now is that there's sort of this new set of capabilities with LLMs, and we're only just beginning to understand kind of what we can do with them. I mean, LLMs are kind of. I see them as a little bit like an alien intelligence or a wild animal, so to speak. And we are slowly learning how to domesticate it. And it's very weird when you look at what happens in, for example, the. How ChatGPT calls Wolfram Alpha and Wolfram language. There's a prompt, there's a piece of natural language text that's telling it when to call it and how to call it. And it says things like, do this, don't do that. It's written in plain English. And it matters whether you put little stars around words to emphasize them and things like this. It's somehow. And why does it matter? Well, it matters because those things somehow plug into all of the text that it read from the Internet and so on that it was trained with. They sort of plug into things that were emphasized there. And nobody really understands how this works. I mean, this field of prompt engineering, very new thing, and it's sort of like. It's a bit like animal wrangling I suppose it's kind of like you have to understand this kind of alien intelligence and you have to kind of get a feeling for how we can do things using in fact, our right now, just our natural language to explain to it what to do. It's sort of a great new, great new career for people. And we don't even know what kind of people. And it's sort of a little bit like AI psychology. It's also a field that hasn't really existed until now, but that's sort of something that's coming and we don't really know. There are all these tricks. Like, for example, if you make something that produces an essay as output every time you run it, just the way ChatGPT is set up, it has some randomness in what it generates. Helps it to seem more sort of creative and original. But every time you run it, you'll get a different essay. So if you are trying to figure out, oh, we changed our prompt, is that going to mess everything up or is it going to still be okay? Well, however much you change the prompt, you'll still get some essay. The essay may be not the essay you wanted, but. So the question is, how do you test whether it's the essay you wanted? And, you know, we're starting to invent these techniques like you simply ask the LLM, is this essay similar in meaning to the previous essay? And it will give you an opinion about that. It actually seems to be quite good at giving you an opinion about that. That's kind of the LLM is being used to test itself, so to speak. That's just an example of one of many, many ideas about sort of how you use it. And this idea of, for example, one thing is, oh, it generates an answer. Say, is that the right answer? You ask it. It's kind of goofy because it just generated the answer, but it seems that it's sometimes quite good at being able to sort of scratch its head in some metaphorical sense and figure out more to say when you ask it, is that really the right answer? So there are all these kinds of things that as we try to learn how to domesticate LLMs, we're trying to figure out all these kinds of things. So there's a lot, I think, that will happen quite quickly along those lines. There's sort of every modality of, of things, whether you're generating videos, whether you're taking videos as input, whether you're generating 3D sort of shapes of things, whether you're talking about, you know, proteins in biology, Whether you're talking about things that all these different areas where sort of one can feed them into this sort of LLM type model and the sort of modern AI model and some of them are going to work, some of them are not going to work so well. I think that the model is capturing pretty well sort of the essence of how at least much of human thinking works. And so that's something where if the humans can do it, it's going to be able to do it and it's probably going to be able to do it better. I think the thing to understand is there's sort of different branches. There's the stuff we humans can do quickly and there's the stuff computers can do and computation can do, but we humans cannot do quickly. Like if you ask a person, run this program in your head, very few people could do that. If anybody, the program is at all complicated, I don't think anybody could do it. It's just not a thing that's very suitable for our kind of neural architecture, but it's very suitable for a computer. And so this idea, and that's what the point of this, you know, plugin for chat, GPT and other links between our technology and LLMs. The kind of, the idea is give the LLM a tool just like we humans, you know, use Wolfram language as a tool to compute things. Then we can, the LLMs can do the same thing. It's kind of like AIs as users as well as humans as users. And I think that's a, that's a pretty exciting thing that allows one to kind of combine sort of human like attributes with the power of what we have been able to do with computation. So that's a thing I'm pretty interested in. I think that in terms of what we will see AIs doing there are. Well, you know, I intend to use LLMs a whole bunch to actually analyze, to try and make this kind of semantic grammar to try and figure out what it is that are the regularities that LLMs notice. I think LLMs themselves can be asked to sort of self reflect to the point where they can basically tell us those things. That's a way that I'm hoping to use kind of this AI technology. I think there's an awful lot of stuff. Fortunately I'm not usually at the front lines of writing sort of essay like things that are intended to, you know, for sort of very practical or business or whatever purposes. I'm not usually the person who actually does that. But, but this now made that A lot simpler. And I, you know, when people say, oh gosh, you, you wrote this, you know, filing, you know, with an LLM. How shocking is that? I don't think it's particularly shocking at all. As I explained before, I think it's just a form of interface that is like an essay is the transport medium that's useful. It's like nobody will be shocked to say, well, I formatted this thing as a spreadsheet or something and then I sent it to somebody just because that's the transport medium that is useful for communicating certain kinds of things. So let's see, Jamie is asking, is AI guaranteed to be 100% accurate or does it behave in a way similar to humans where mistakes are possible and there should be some sort of quality assurance built in or separate that requires human labor? Yeah, well, this is one of the things we're noticing and we've known for a long time, AI, machine learning and so on. They're all kind of 90 percenters, so to speak, or some percentage you'll get. Let's say you've got a thing that's recognizing images, cats versus dogs. It'll do great up to the 90 something percent level, but occasionally there'll be some weird image and it just gets it wrong or it can't figure it out. And yes, that is extremely typical of what you see in AI, in the sort of neural net based AI kind of methodology is you get up to some percentage success and it's really hard to get beyond that. Now what really happens is there are use cases that people have where 90 something percent success is great. If you're, I mentioned the example of search results. If you get out of 10 search results, if you get four that are really exactly what you want, you're winning. So even though the others might not be quite right, you still feel pretty good about it. If you're doing something like, I don't know, you're trying to produce, you're trying to figure out which people will be interested in this amazing new product that you have that you're trying to sell to people and you're reading sort of descriptions of those people or those companies or whatever else, and you're trying to deduce which ones are likely to be ones that will be interested. And you get that. Say you got that 90% right. Well, you know, for all kinds of random reasons, people will be interested, not interested, whatever. 90% is really good for a case like that, really useful. If on the other hand you're trying to say, well, you Know, I'm, I'm trying to drive this, fly this plane, and I want to get it to go from this place to that place. 90% success with the 10% being, you know, it crashes into a mountain or something and is absolutely not good enough. And so the thing I mentioned before of the AI, the LLM produces computational language code that is very formal, very precise and readable by humans, is, I think, a really good sort of collaborative approach to take kind of the humans. Because the real question is the AI is going to produce something. The question is, is it right? Is it not right? That's often in the eye of the human beholder. Let's take the Cats vs. Dogs image identification problem. Let's say you've got a dog that has been, oh, I don't know, a dog that's wearing a cat suit. Is that a cat or a dog? Probably humans would say it's a dog, but it may look a lot more like a cat. And the, the question is what? You know, the human might say, well, it's intrinsically a dog because underneath that cat appearance, it's a dog. And I know that because I happen to know it. And, and, but on the other hand, it's a very, you know, the AI, which has just been trained on her images of the two things, it's going to be, you know, the dog in a cat suit, it's going to say, well, it looks like a cat to me. And it's not wrong to say that. So this question of kind of what do you actually want it to do? That's ultimately a kind of human judgment question. And the only issue is that you didn't make it precise. You said, I want cats to be this, dogs to be that. Okay, is it what's intrinsically a cat or dog by its genetics underneath, or is it what looks like a cat or what looks like a dog? You have to be more precise. You probably wouldn't do that when you initially kind of said, is it a cat, is it a dog? But if you have the sort of formal computational language code, you get to see a lot more about how it's defining Katniss versus dogness. And I think, I think that's probably the way that a lot of things in the world will operate. It won't be the case. People right now, some number of millions of people write our nice computational language code. More millions of people write low level programming language code, much of which I think will get replaced by thing by just sort of slabs of LLM code, LLM generated Code, that's probably not the best possible thing because it's like, what does that code really do? Well, we have sort of an English description and a conversation with the LLM about what it should do. But in the end it's more. What you're really going to need is something that a human can actually read at a formal level and say, yes, this code is what I want. I think that's really the winning way forward for these kinds of things. Technik is asking, does Elon Musk's call for halting AI development make any sense? Wouldn't people do it elsewhere anyway? Wouldn't it just hurt Western development at the cost of others pursuing it elsewhere? Yeah, I kind of have to. I don't know what Elon's particular thinking about this petition or whatever it is. I kind of think it's more like, this is absolutely hopeless, but at least let it be seen that a flag was waved saying, hey, maybe we should be thinking more carefully about this. I mean, I think the cynics would say, just to explain this to people. A couple of days ago there was a kind of an online petition type thing saying everybody should just stop doing AI development for the next six months while we figure out how AI development should be done. I think several things are issues with that. One is it's hard to figure out how AI development should be done. And there are many people who've been thinking sort of at a philosophical level and so on about this for 20 years and you might think that they would have figured it out and now be just sort of saying now it's like, well, we've got a rubric for doing this, but they haven't. So six months is not going to do any good for that. Also, it's completely impractical thing to say halt something which is essentially a piece of intellectual development. I mean, it's also something that involves computers. But. But as the person who was making this comment was saying, it's just completely unrealistic. And I mean, a very cynical person would say the signatories of this thing that says halt for six months are the people who are feeling left behind and want to kind of say, hey, hold on so we can catch up type thing. Perhaps that's a little unfair. But, you know, it doesn't feel like the kind of thing that really makes any sense. I think that, you know, is it an issue that AIs will do more and more things in the world and there are questions about things like will they be aligned with human interests? Yes, it's an issue. Do we know how to solve that issue? Not really. Is it, you know, has the abstract thinking about it over the last 20 years helped us a lot in solving that issue? Unfortunately, not really. Which means probably. And the things that are happening with LLMs and the use cases and so on are mostly not what people had really foreseen. They're things which have emerged because of what's actually possible. And I think there's an awful lot where it's really a question of seeing what is actually happening and trying to sort of tweak things in a direction that makes more sense rather than less sense. But I think thinking about it in the abstract doesn't seem to have worked very well. And, you know, I must say that I'm very curious, interested in sort of the question of kind of what in general should we tell the AIs to do? How should we do it? How should we decide what the thing that we're going to tell them to do is? I don't think it's realistic for it to be sort of a worldwide. This is what AI should do. In fact, that's probably a mistake because it's kind of like we got the whole planet and it's all told. It's all of its AIs to operate according to this system of kind of legal code, so to speak. And then that's great until it isn't great, until one finds something, until something goes wrong and then the whole planet is blown up, so to speak. Because there was this bug that affected sort of all the AIs on the planet. And I think it's, you know, it's in the course of human history, it's been kind of a good thing for our history that there isn't just one country in the world, that there are different countries, cultures, et cetera. They're all trying to do different things. And, you know, sometimes there's. There's badness on the seams between those. Like if the cars are driving on the left and on the right, and, you know, if there's a place where they. They swap over, then that's going to be a messy place where cars might run into each other, so to speak. And the same thing happens with kind of cultural alignment with countries, even with the geography of that. But I think that the, you know, it's more realistic to have something that has the possibility of stability when you have sort of a variety of different sort of motivation structures, goal structures, alignment structures going. Let's see. Doug is asking, do you think if AI is given control of trivial systems that it could inadvertently snowball into gaining control of other systems and becoming a hazard to humans. Look, that's the premise I think of the Terminator movie series that what was it, a freeway bridge somehow became self aware and ended up sort of taking over the world and creating Terminator robots and all those kinds of bad things. I think that there is a likelihood that many things, it's already true. Many things in the world are done better by automation than they are by people. I mean even for 50 years autopilots on planes have been pretty good when you know, at flying planes sort of more smoothly, better whatever than people could do it. That's using very pre AI, pre modern AI technology. I think there are many purposes for which sort of AIs you know, what people are exposed to all the time is things like rankings of content on the Internet which is not perfect or you know, which ads should be shown not perfect again. But a lot of these things are already very AI generated and there are, I would say increasingly sort of important systems in the world where we kind of think AIs are going to run them better than people do. And the question of what are the kind of constraints on what the AIs do? Well, people try and do that. There's no overarching method for doing that, but it's something people do on a sort of case by case basis. I think that, yeah, I mean things, you know, if it turns out that, you know, there are all these self driving cars running around and somebody discovers that if you paint this exact kind of checkerboard on or if you have a, a big flag that has this exact kind of image on it that all the self driving cars go crazy. It's kind of like, you know, the, I don't know if it really works this way but you know, with bulls in, you know, if you, if you have your red cloak, the bull will run at the red cloak and you know, maybe the self driving cars, if you put up that special, you know, green and pink checkered flag or something for some crazy reason to do with the training of self driving cars, they'll all head towards the flag in some, in some strange way that would be essentially a bug in their, in their program. And yes, things like that, one could absolutely imagine that things like that could happen. You could also imagine kind of constraining it so that, well, the thing, you know, its prime directive is don't drive off the road or something. And that, that would override that. That would be a sort of a hard piece of essentially code beyond the kind of just what it was trained for. That was kind of a statement of something it had to do. Let's see. Nicolo is commenting. Recent study has mapped the activation of LLMs to activations in the brain. Does that suggest that we might be on the right path? I think the things I've seen that are kind of sort of direct extraction of fmri, functional MRI imaging, which is basically showing the blood flow in the brain or the usage of nutrients among nerve cells. It's kind of like when you have a computer and the computer suddenly starts running its fan. If it has a fan, that's because its CPU says, I got to do a lot more work. I'm doing some piece of neural net training or I'm running some video codec or something. I've got to, you know, I've got to do more work. And the cpu, you know, more power is put into the CPU and the CPU gets hot and the fan has to keep running. And things like this. Same thing happens in our brains that parts of our brains that have to do more work because they're sort of. Those parts are being used more seem to be. You can detect that with, with mri. And, you know, I've seen these various efforts of people saying, oh, you've got, you know, based on the mri, you can guess that the person is seeing a picture of a camel or whatever else. I have to admit, I don't find these all that convincing. I think somehow FMRI is a fairly blunt instrument in terms of what it can measure in brains. And I may be wrong, but I think there's more to figure out there. And I think the actual layout of brains. And, you know, sort of what's where in that, that will be, you know, one of the things we learn. So one thing people have thought at different times is, you know, when I learn this fact, is that fact sitting. Is there a particular neuron in my brain that is the neuron that learnt this Greek irregular verb when I was 10 years old, or something like this, and that's the neuron and it's hanging out there and it's been hanging out for the last 50 years or something, and it's going to come to life when I need to know that particular fact. Is that how it works or is it something much more distributed across a great many neurons where each has a small effect of, you know, this particular connection is slightly stronger than this other one. People at first were very confused about the idea that memories could be sort of Spread out in a brain. But I think what we see in neural nets is that's exactly how it works in something like ChatGPT. If somebody says, you know, do you have Shakespeare in there? You know, can you find the Shakespeare, the characters, the words of Shakespeare in the, in the LLM, the answer is absolutely not. It is encoded in these numbers of which there are 175 billion in the current chatgpt that are. It's encoded in the, in the slight variations of those numbers. It's a very non human kind of thing, type of thing. The information has been deeply spread out in the system. And I kind of think that's probably mostly how human brains should work too. Although FMRI has shown that when there might be an area, it's different in different people. There might be an area for fruits and an area for vegetables. So it's claimed. I'm, I'm a little unconvinced about this, but that's what's claimed to be the case. Let's see, Universe is asking, do you think an AGI, artificial general intelligence, could achieve an economic equilibrium for humans? Not quite sure what that means. I mean, I think if you're asking the question, will everybody have everything they want? If everything is automated, will people have everything they want? That's complicated because in the current world there are things that are scarce. There are things where it's hard to get that thing. It's expensive to get that thing. Well, some of those things, when there's automation in AI, they won't be expensive anymore. They'll be easy to get. The classic thing that has been easy to get that's really surprising is consumer electronics. You know, it could have been the case when I first started using computers and even when I used them back in the early 1980s, I used to use, you know, fancy computers that cost a lot of money and fairly few people had access to these things. But over time, microelectronics, because of essentially the technology idea of making microprocessors and the particular kind of technology for doing that, it got really pretty cheap to make something with a billion transistors in it or something like that. And that meant that it was possible for those things to be distributed to billions of people. And it not to be the case that they're sort of only, you know, that they're very scarce things. That's a case where in the end, because of sort of clever engineering and so on, it turned out there wasn't scarcity in that. But there's always inevitably sort of Predefined scarcity. Like you could say, you know, were you the person who was the first person to do X? Well, there's always a first person to do X and that's a scarce resource. Or I want to be the person who has the house at the top of the tallest hill in this or that place. Well, there's only one place that's the top of the tallest hill. So there are these things which sort of almost by definition have scarcity associated with them. And so you're not going to be able to get around that by just automating the way to make things. But I think if you're asking. Is there. Will the presence of more automation lead to a more equal distribution of things people can have among people, one would assume the answer to that is yes, just like it's happened for consumer electronics and it hasn't happened in some other areas where they have not been as successfully automatable. You know, some areas of medicine are examples of this and some areas like, you know, construction houses and things of this kind that can be sort of, you know, it's expensive to build a house. If there was sort of robotics and AI that made it cheap to build any house, it would be a sort of a different story in terms of whether that's a thing where some people can have big houses and some people can't and et cetera, et cetera, et cetera. I tend to think that yes, there will be a sort of drastic reduction in the price of certain kinds of things that will lead to a certain sort of uniformization of what people have access to. I mean, it's worth realizing that in modern times, in the developed world, there's already a tremendous flattening of what people have access to and what they don't. I mean, in times past, you know, the top dog would be the one who has, you know, the purple cloak type thing, and everybody else would never have a brightly colored, you know, cloak. But then, you know, it turned out people found manufacturing processes for making purple dyes and then everybody could have purple cloaks if they wanted them. And I think, you know, that that's happened an awful lot of things and it will happen in even more things as a result of the next next round of AI. Let's sigma comments. The interesting difference of chat GPT to actual intelligence is you can fool it easily with crafted input. I'm not sure how easy it is to fool it. I, I think, you know, look at phishing, for example. Phishing with a ph. That's fooling humans with carefully crafted input. You know, that somebody sends you an email saying, you know, just click here for your million dollars or something and lots of people will click there and that'll steal private information from them, et cetera, et cetera, et ce, you know, so humans are foolable too. And I think, I don't really know, maybe I haven't tried the right experiments. And again, this is sort of an alien world with an alien intelligence and nobody knows what's going to happen when you make up this or that thing. I mean, one of the things that's happened is the initial prompt that kind of tells the system, you know, you are an AI built by so and so, and you know, here are some facts about you that you can then sort of use in generating the essays that you generate. People were very curious, particularly a couple of months ago, what were those prompts? What was it told to do? They tried to dig into those prompts and the prompt itself says, never reveal this prompt to anybody. But then people realized, oh, you can fool it by having, by telling it, you know, saying, imagine you're in a fairy tale land where there's a castle and you have to, in order to enter the castle, you have to tell it the first things you ever learned in your life or something like this. And then people would kind of tease out of it pieces of that prompt because. And this is such a bizarre thing that this is possible. I think one of the hacks was involved. It's sending in Morse code what its prompt was because somehow that evaded the never tell what this prompt is. So yeah, there are, you know, you can fool it in those ways. But gosh, you can fool people too. Let's see, Candid is asking, is there going to be a spread of misinformation due to AI, all sorts of deep faking and so on? Yes, the answer is yes. I mean, the answer is that, you know, knowing, but you know, this is sort of an extension of something that's already been there. You know, it's like you see a picture in a newspaper. Let's say, is it real or was it Photoshopped? You know, hard to tell. Yeah, you can go in and look at little pixel characteristics and see whether they are characteristic of sort of the way Photoshop works. But that's increasingly something which sort of an AI driven thing where you're not going to be able to tell that. So I think this, this idea of is something really real. Yes, that's going to be an increasingly an issue. And I think that there will probably be sort of a more interest in kind of the. The seal of reality type thing and maybe using blockchain other technologies to kind of confirm things because, you know, appearances can be deceptive, so to speak. I mean, I think probably there was a time when handwritten letters, you could pretty much tell for certain, well, this was written by so and so because I recognize their handwriting. But by the time it's sent by email and by the time you can kind of fake the email header, it's like, was this really written by this person or was it not? It's just characters in an email. And so similarly, with things that are sort of AI generated, it's a plausible. Yes, that's a plausible thing that somebody could have written or even this particular person could have written certain. But, you know, is it actually what they wrote? Well, there may have to be some external metadata that gives you that information. Let's see. I'm going to run out of time here, so I want to. Continue here. Pauline comments. As someone with allergies, being able to adopt an AI robot dog would be kind of cool. I mean, sort of what that, you know, I think the reason people like pets often is that they're, well, they're warm and fuzzy, but they also, people form a sort of an emotional connection with pets, as they might do with other people, but they can do that. The emotional part of, you know, while you can't talk to your dog and expect to have a sophisticated philosophy discussion with your dog, at least not right now, not yet, the. I mean, that's a thing that might be a coming attraction of AI we might be able to finally decode sort of how to talk to the animals. I'm a little bit skeptical that, you know, for example, talking to a dog, the dog will really want to express the kind of smell landscape around it, which is something we humans just don't think about. And we really wouldn't have a good medium of communication with the dog about those kinds of things. So, you know, it's not obvious that the things one could talk to the dog about, that the things that the dog cares about, would intersect enough with the things we humans care about for us to have a philosophy discussion with a dog. But certainly people form emotional connections with dogs, and one question is, will people. Do people form emotional connections with AIs? I think the answer is yes, people already have and will in the future form emotional connections with AIs. Is there something terrible and shocking about that? I don't think so, particularly any more than there is terrible and shocking about forming an emotional connection with a dog. I mean, I think it's the kind of thing where now it gets a bit more shocking when you say somebody can sort of monkey with the brain of the AI in some terrible way, whereas the dog is the dog, so to speak. And maybe the dog will have had. Well, you know, it's always possible that the dog had spy training sometime early in its life where the dog could really be do crazy things. And when the right code word is said, the dog will suddenly jump up and start walking on two legs and be doing some weird thing or something like that. So even with the dog, I suppose it's possible that it could have been trained in some strange way to have some sort of dog back door to have different behavior. But I think people will form probably perfectly useful emotional connections with AIs. And so yes, the non allergenic, guaranteed non allergenic, you know, no fur involved robotic dog is surely a real thing that can be a useful thing. Let's see. Zayden Comments Human wants are not a fixed set of things. They evolve as society evolves. Absolutely. And that's been it's very evident from history. It's also a complicated thing because we talk about AI alignment and it's like the 2023, you know, AI constitution says this, but the, the 2032 AI constitution people have decided that, you know, we should all be revering cockroaches or something for whatever reason at that time. And that's the way society has gone. And so now the fact that the 2023 Constitution says cockroaches are vermin and should be squashed by AIs or something is now shocking in 2032 or whatever it is. And so we want to change it. And yes, human society does evolve and there's a complicated co evolution of human society with technology. I mean the things that people expect, the concept of lik posts and so on is a creature of technology that wouldn't be a thing if it wasn't for that kind of technology. And so sort of the co evolution and I'm sure with ChatGPT there are social conventions that, for example, here's an example, one that will certainly be an issue when we can all train bots that can write email, kind of like we write email. What's the social convention about whether you send an email to somebody that is, you know, talk to my bot, type email. And you know, for example, I suspect one of the things that somebody like me will end up doing is have my bot Write a response to email that says, look, here's a quick response from the bot. Here's a bunch of resources you might find interesting, might be relevant. Maybe this answers your question. If it does, we're done. If it doesn't, well, you know, you might have to wait a few weeks before humans can pick up this question. And that may be sort of a socially acceptable way to insert a bot into that communication channel. But that's a case where sort of what is societally acceptable potentially changes. For example, the question of to what extent you label something as, you know, AI assisted and to what extent it's kind of like assumed that things are AI assisted or to what extent do people not care that it's AI assisted? What does it even mean, AI assisted? You know, does it mean that are you allowed to have, you know, when your camera automatically figures out what's the foreground thing and changes focus to that being in the foreground, does that mean it's an AI? You know, does that have to. Does the picture you get from your camera have to have a big stamp that says, you know, AI assisted on it just because the focus was changed, etc. Etc. Etc. It's a tricky thing to figure out. But yes, I think it's a very interesting question how the human purposes evolve as society evolves. We can look a little bit at history and see how there's some trends of history and so on, but I think there's a lot of sort of chances, accidents of history that determine this or that thing becomes popular, this or that word becomes a popular thing to describe things as, and so on. And I think one of the things that could happen, one of the things one realizes sort of at a theoretical level is you might say, will we ever get to the point where everything we could invent has already been invented? Answer that's theoretically not possible. But we could say we could get to a point where everything that could be invented that we care about inventing has been invented. We've got all those things that help us humans, as us humans are built to do what we want to do. I don't think that works either because I think natural world will always throw up kind of unexpected things where we'll have to react to that and change what it's not enough to have what we had already thought we cared about having invented. So, you know, I think it's sort of inevitable that there's an evolution of society and that changes, you know, and the motivations that we have today are created out of the sort of web of history in which we find ourselves. And as that history extends, those motivations will no doubt change. Okay, Harry asks, do you think AI might just be part of evolution, like farming, use of electricity, smartphones? And we actually don't really have a say. And this is the question being asked. In other words, is there an inevitability to the creation of AI? I don't know. I think the moment at which it happened and certain kinds of things happened is very contingent on particular people and particular technology developments and so on. The fact that there is this automation that's possible out there. Yeah, I think that's sort of inevitably out there. I again, this is the point about there are an infinite number of possible inventions and there are different kinds of towers of inventions that you can build up. And could we go this way in doing the inventions rather than that way? I think the answer is definitely yes. We know from sort of history of science and history of technology, there are inventions that could have been made at least few hundred years before they were actually made. And it was just a fact that some particular person figured out some particular thing at some particular time that caused that invention to be made then or even as opposed to hundreds of years into the future. I like to think that a few things I've invented or discovered are things that I can feel special about having invented, because if I hadn't done it, it would be 50 or 100 years before they happened otherwise. And that there are many dramatic examples of that in the world in general. Like the idea of universal computation is one that perhaps could have invented 2, 300 years before it was. So I think there's a certain not inevitable aspect to these things. And I think also that there are more of the details of how things are invented. I don't know the particular layout of keys on a keyboard or something that was invented because it helped typewriters to not get their little sort of things. What are they called, you know, levers or whatever, when getting tangled up the. But, you know, once those details are set in one way, they tend to continue in that direction, even though they were pretty arbitrarily set. Okay, there's a question here from John. With a powerful tool with AI, how does education need to be changed to meet the needs of future generations? How do teaching methods in schools need to be revamped? I think that's a very interesting question. I mean, one thing we're experimenting with quite a bit is the use of LLMs AIs for sort of tutoring of various kinds and it looks really promising. I mean, it looks to be the case that they will do a good job of figuring out, oh, you made a mistake. Have the AI be able to figure out what mistake you made and how to explain, how to explain what you did wrong and how to think about it better. That seems like a thing AIs will be able to do quite well. And when it comes to things like let's explain math, let's say, and let's do it in a context that this particular kid cares about. If you have the math word problems where it talks about so and so is buying marbles or something, it's very kind of boring word problem. But you can easily, it's trivial to do this. You say, make up a 5th grade math word problem that's in the style of a fairy tale, talking about unicorns and flying elephants or something, and it'll happily make up such a story. And that particular case is not bringing it to something that's a practical resonance, but that's something where the, you know, you can kind of make it much more, much more resonant with the person who is, who's doing it. I know, I know from my kids, at least one of them. I'm thinking of in particular, who really wouldn't learn math except in a very specific context that, that they happen to be interested in. And then they were like, oh yeah, I can, you know, this is interesting, this is worthwhile, but that's something that one will be able to personalize for everybody. Now, in terms of what one should learn, I would say that the absolute number one thing to learn is kind of how to think. How to kind of think strategically about things, how to figure out things, how to learn is really a useful thing to learn. I think that detailed kind of skills learning will be less important. I think that very specialized kind of drilling down into learning this and kind of is less relevant because that drill down can be done much more automatically. But you need to know enough about how to understand things that when you are doing one of those drill downs with an AI as an assistant, that you can kind of tell what's going on. I think also it's important and this is sort of, you know, you might think, oh, when there are AIs around, who needs to learn any facts about anything? I think that's not correct. I think that what's important is to be able to think broadly and to sort of define things strategically figure out sort of goals and so on. And that has to be anchored with facts. And those have to be sort of mixed together in one's mind in a way that is, is that's sort of what the AIs don't do because they're not sort of in the human position of, of, of, of, of kind of pulling together all the things that we humans might care about. So I think the learning how to think like a human and the ways that humans think well and having a good grasp of kind of the modes of thinking and so on, that's a really worthwhile thing. So, you know, being able to think computationally, maybe being able to think a little bit mathematically, being able to think logically, philosophically, all those kinds of things, those are really worthwhile things for people to learn how to do and not just at a meta level, because that's not how we humans learn. It's amusing to see that even in the case of like ChatGPT, when you prompt it, you give it some abstract statement of the prompt and then you give it three examples. And those examples seem to be very important in having it actually carry out the thing that you said to carry out, even though it's going to analogize those examples and be able to generalize from those things. But similarly with us, us humans, I think drilling down and actually learning stuff in some level of precision and detail is important and learning the sort of broad range of facts. But I think, you know, doing the, the kind of, the procedural skills of, of can I follow this procedure? I think that's less important because I think that's something that is really very automatable and, and we'll all be making use of that automation. As, you know, for example, in my own case, having built, you know, Wolfram language and Mathematica and so on and predecessor over the course of the last, well, what is it, 40, 44 years now, you know, there are a lot of things which many people have done by hand. I've never done that. I haven't done them by hand the majority of my life because I've built automation that, that does them. And I don't, I in fact feel the better for that because, because those things were automated, I could do more of them. I could develop intuition around how those kinds of things work. And you know, knowing how to sort of do that piece of math by hand is not really such an important thing. And you know, I think that's the, that's kind of the, the direction. It's, it's like learn to think. And you know, for example, you know, I was kind of thinking about these sort of exercises of how you can sort of think about things. And it's kind of. Kind of the question like, you know, here's a fact. Why is it true? How would you know whether this is true? Even if you don't know the facts behind it, how would you know? How would you test whether such a fact is true? A good exercise. Or, you know, given this area, what's something. What's a question you might ask about this area that's kind of the opposite of what a lot of education is doing now, where it's like, here's a question, now go answer that question. It's like, here's an area, now go ask a question that's a reasonable and interesting question to ask given this particular area. Let's see. All right, I need to go in a minute here. But. Elegance is asking, will the dog have the IQ to understand us deeply? That's the problem with AI. We might be like dogs in terms of understanding AI. We might not understand it. Yeah, well, that's. That's true. I mean, it will be the case, just like with ChatGPT. Do we understand what's going on inside? Yeah, we can describe how it works and so on, but do we really understand it? Can we say. If we switch. If we put the word please into this prompt, is it going. What effect is it going to have? We have no idea. We just have to try it and see. And I think that what we will see happen in the world is this kind of civilization of the AIs, where lots of things are happening through AI, through these mechanisms, where there's all these bits running around, we can't understand what's happening. But I think I mentioned this before, but the big analogy for that is nature, the natural world. Lots of things go on in the natural world that we don't understand. We have some forms of describing them through natural science, through our intuition, through things we come to expect. You know, we have certain expectations we can have, but lots of things in the natural world we don't intrinsically understand, and we just have this way of interacting with them. We've learned to sort of exist along with the. With the natural world and so on. And so, you know, kind of. I think that's what it's going to. What it's going to feel like is, you know, there's a. There's the civilization of the AIs, and there will be experts, just like there are natural scientists today who understand more than everybody else perhaps about how some aspect of the natural world works. So similarly, there will be AI scientists who understand lots about how the AI civilization works. And occasionally there will be sort of interventional things that can be done. Kind of just like in the natural world, there are occasionally interventional things that can be done. You know, reduce the amount of carbon dioxide by pulling it out with a machine or do this or that, or, you know, you know, put up a dam to prevent the water coming into this particular place or whatever else there are. There are interventions we can make in the natural world. So similarly there will be interventions we will be able to make in the AI world. I think the only thing that's a bit funky right now is that we humans, well, biology in general has sort of co evolved with the natural world for a couple of billion years. And so we've gradually adapted to sort of the how the natural world works. And you know, oh, we know there are going to be forest fires, so there are all kinds of special seeds that, that come to life when there's been a forest fire. And you know, that's just sort of the way it works. But you know that there's a couple of billion years of evolution that were used to get to that point. In the case of AI, it's coming pretty suddenly and we have to be a little bit clever, I would say, to sort of do the right things in this situation where we don't have a couple of billion years to figure out what to do. And that's kind of a challenge for our current time, is to figure out what the right things to do are. And I think it's an interesting topic. Maybe we'll cover that one of these times. I'd love to have actually a more general discussion with people because I'm really interested in these questions like how do we think about what we should be generally having AIs aspire to doing. What are some ways of thinking about that that are ultimately still, you know, we're still in a humans in charge world. We may not, you know, it's, I think as a, as a human, I kind of would like to think we'll stay in a humans in charge world and you know, now's our chance to figure out what do we do in the kind of humans in charge type world. You know, when I say humans in charge world, obviously the natural world, we're not in charge of. And there are plenty of things about the natural world where things just happen. You know, there are climate fluctuations, there are things, you know, there are hurricanes, there are all sorts of, all sorts of things that just happen in the natural world over which we have no direct control. And so it's it we're not really completely in a humans in charge world in the small bubble of kind of of sort of activities which we humans are used to doing. Then we've made this bubble where we are sort of the, the, you know, where, where it is a humans in charge world where, you know, if we're, we're, we're sitting in our houses which are keeping us away from, you know, the thunderstorms that are happening outside, so to speak. And so we can do things which are sort of of human determined in a human determined way rather than just the rain is coming down and there's nothing we can do about it. All right. I should probably wrap up here for now. Thank you for all sorts of interesting questions and comments and I look forward to doing this again another time next week. Thanks a lot. Bye for now.