Future of Science and Technology Q&A (August 16, 2024)

Hello there. Welcome to another episode of Q and A about future of science and technology. And I see a bunch of questions queued up here. Let's see, There's a question from Michael here. What do you view as the best strategies for reducing or eliminating hallucination and confabulation, presumably in LLMs right now, is there any chance that we'll be able to get something like confidence levels along with the responses we get from large language models? I think it's pretty difficult. Here's the thing. Well, first thing to say is if we think about what people do, people kind of make responses to questions kind of off the top of their head. And there's certain kinds of things you can do when you do that. You can kind of put together coherent sentences, you can kind of figure things out based on what you remember, you can do those kinds of things. Then there's what we do in a sort of more formal way, what we can do with our computers, what we can do with sort of logical structure, with mathematical structure, things like this. These are in a sense rather non human activities. Nobody can run a program of any significant complexity in their brains. You run a program with a computer. And the way that neural nets and large language models are built these days is something that is sort of a rough approximation to probably roughly how brains work. And it involves layers of neural nets, layers of neurons and so on. Qualitatively, like the way brains work and systems like that are good at doing the things brains are good at doing and not very good at doing the things that brains are not good at doing, but computers are potentially good at doing. So it's kind of a situation where there are things where you really shouldn't be asking. So if you ask a person, do this seven digit multiplication in your head, most people won't get that right. A computer would have an easy time getting that right. I mean, you know, and so it's sort of the wrong thing to ask a thing that's sort of brain like to do that. So one of the challenges is how do you get the best sort of neural implants into the LLMs to let them sort of do computation when they can do computation. This is something we've worked on since the very sort of early days of LLMs. Back at the end of 2022, we had this very convenient situation where WolframAlpha was built to use computation and systematic knowledge, but to have an interface that was based on natural language. So it was kind of an ideal situation where the LLM could say, here's a piece of natural language. Okay, WolframAlpha, go use computational knowledge to answer that question and then give the answers back to the LLM and have the LLM kind of sort of develop the wording around it and feedback the answer. Turned out LLMs were better at writing Wolfram language code than we expected. And they've gotten even better partly because we provided a bunch of training sets about how to write Wolfram language code. It probably helps a lot that Wolfram language, the idea different from programming languages, the idea is to kind of have a computational representation of everything in the world, which means there are a lot of words in Wolfram language which are like words that you expect to see in kind of plain text and ordinary language. And so that's something that the LLM as it's trained can kind of latch onto to understand what to do. So the thing that seems to work well is you use the LLM as kind of the linguistic interface. Then you're calling actual computation and knowledge as a tool, so to speak, through our technology stack, and then getting results back either directly in the form of this is the computation, this is the computational knowledge that you asked for, or perhaps paraphrased by the LLM. Sometimes the LLM will mess it up when it paraphrases it. Something that we've been working on, you'll see as products soon are things we call Wolfram boxes where they're just within the the narrative discussion with the LLM there'll be a box which is something where you can see the input. It's a piece of Wolfram alpha input, or it's a piece of orphan language input. You see the output and you know that everything inside that box kind of makes sense. So that's kind of something which is kind of a sort of a guaranteed successful approach to this question of well, do the LLMs. You're using the LLMs for what they're good at, which is the kind of conversational aspect the linguistic user interface. You're using the computational language, computational knowledge for what it's good at, which is getting the right answer and sort of knowing what it's talking about, so to speak. Now there's a big question of how fine grained that interaction between kind of the computational language and the LLM can be. It is kind of crazy if one is training an LLM and feeding it, you know, hundreds of billions of parameters and so on, where the sixth digit of some number somewhere is the thing that represents the third digit of the population of New York City or something Nobody knows how that sort of grinding up of data really quite works. But the very idea that one should be teaching the LLM, the third digit in the population of New York City or something doesn't make a lot of sense because it's not going to be good at dealing with that. It's something where the computation system will be very good at dealing with it, but the neural net will not be good at dealing with it. So the issue is, how do you get the neural net in a very fine grained way to say, hey, this is something where I want to ask my neural implant. It's by the way, an interesting question. If we do one day get kind of neural implants for actual brains, it's kind of a question of what does that interface look like? How fine grained can it be? Can it be at the moment where you have that one thought, where you need this, where you need to consult the computational system that you can do it? Or do you have to package your thoughts to in some form where you have a sort of a much coarser interface to the kind of implant, so to speak. And sort of our one experience of this kind of thing is human language where the thoughts we have inside get very much sort of concentrated down and packaged into the stream of words we say that allows us to communicate those thoughts. The raw thoughts are probably considerably more complicated and not things that we have any kind of transport layer to communicate with. So I think one of the challenges is can you fine grain kind of connect? What's there to know when to be computational, so to speak. Now there are a number of other tricks you can pull. So for example, the whole technology of rags retrieval, augmented generation technology we just introduced in version 14.1 of Wolfram language a built in mechanism for doing rags. So what is a rag? The idea of a rag is all based on these things called vector databases. And kind of the idea is grind up your text, let's say, sentence by sentence, and to every sentence associate some vector of numbers, some collection of numbers that represent in some sense the meaning of that sentence. The fact that anything like that is possible is not obvious. We can talk about why that might be why it's plausible that that's possible. But anyway, so for every sentence you've got this vector of numbers. And then if you want to know, well, is this thing you're asking about related to any sentence in this piece of text? What you do is you have the vector of numbers associated with what you're asking about and you, you compare that vector of numbers with the vector of numbers for every sentence in the piece of text, and you say which sentences were closest to that vector of numbers. Now, those sentences, some of them may be real wins, some of them may be totally off topic. But if you've just got plausible candidates, then the LLM has a serious chance of being able to say, yeah, that one is really the hit. It's relevant. These other ones are irrelevant. But just priming it with these are the things that might be plausible. Now, for example, when we get an LLM to write Wolfram language code, one of the things that really helps it is to prime it with a rag that's been trained from our documentation. And other things where it's like, oh, this is what you want to do. Here are a hint. Here are five functions that might be useful. Then the LLM can go read the documentation for those, compare with what it's being asked, and have a much better chance of being successful. So one can do things where one kind of tries to say, okay, this is what the LLM said. Let's try and reverse engineer what it said based on, and use a rag to connect that back to pieces of training data. That's one way that you can kind of say, hey, this is where this came from. What one doesn't know how to do is to take the tag, this is the reference for this thing, and say, well, this. When you train an LLM, you train a neural net in general, you're just progressively tweaking all these weights in the LLM and you don't really know what tweak came from exactly where. So it's hard to pull that through. It's sort of a separate thing. Where did it come from? As it is, what is the thing that is relevant for knowing how best to complete these sentences or whatever. So I think now, in fact, my view of this has evolved a bit very recently because I happen to have been working on kind of the foundations of machine learning and trying to understand how it is that machine learning even manages to work. What are you doing when you're doing machine learning? When you're using a neural net, for example, it's been very difficult to know, partly because neural nets have been very difficult to visualize. Very difficult to visualize what's going on because you have all these connections and they're all on top of each other and so on, and they've got all these. All these real numbers that are sort of. You can kind of color the connections different colors, but they're all on top of each other. And you can't really see what's going on. So the question is, can you kind of make a simplified version of neural nets where you can more see what's going on? And I managed to do that using ideas that I should probably have followed up on in the mid-1980s, but didn't. But in any case, so have sort of a simplified version of neural nets where you can sort of see what's going on. Well, when you see what's going on, what you realize is that what's going on is complicated. In particular, a lot of what's happening is inside the neural net. There's some sort of complicated computation happening. Now, when I say complicated computation, it's kind of like one of these cellular automata that I study a lot, where the rules are very simple, but where the pattern of behavior is complicated, and where the pattern of behavior is, in a sense, computationally irreducible. In a sense, you can't work out what the pattern is going to be like, what the latest stages of the pattern are going to be like, without kind of following through each step to see what happens. So this seems to be a generic phenomenon in the kinds of systems that occur in machine learning. One can see it more clearly in these simplified minimal models of machine learning. But what you see is that there are these very complicated patterns. So what is going on in machine learning? What's going on in machine learning is you're using these complicated patterns and you're finding one that happens to fit the thing you want. So my analogy these days has been you're building a wall. And one thing you can do is you can carefully make these bricks that you know will fit perfectly in their brick arrangement. And that's kind of the engineering solution to making a wall. Another solution is you've got various rocks that are lying around on the ground, and you just say, oh, I need a rock that's roughly that shape. Let me fit it in here. And you progressively sort of build up the stone wall or whatever by putting together these rocks that happen to more or less be the right shape. I think that's more what's going on in machine learning. It's kind of these lumps of computation that happen to be sort of more or less the right shape and can be put together to achieve what you want. It means that the thing you have inside does not have an obvious mechanism. You open it up and you're saying, well, what's going on? Well, it's this lump of computation, that lump of computation. Just like saying, you've got the stone wall. It happens to be a decent stone wall. But if you say, why is there that particular pattern of stones in this place? There won't really be an answer. It's kind of just because, well, those stones happen to be lying around. Those are the ones we noticed. Those are the ones we picked up to put in the stone wall at that stage. So that makes it kind of makes it more difficult to say, well, what's really going on inside the LLM, inside the neural net, for example. So that's a little bit on that. I think that ultimately formalistic computation, computation where you have definite rules and you follow those rules, that's what's going to give you sort of reliable output. The kinds of things that happen in neural nets where you're sort of fitting together these lumps of computation in this way that happens to work, you're never going to get full reliability from that. In fact, it's sort of a general heuristic. When should you use machine learning for something? Well, when there are tasks where 90% success is a win and 10% failure isn't a big deal, then machine learning is great. When it's a task that needs 100% success, machine learning is probably a bad idea. So if, for example, what you're trying to do is to say, tell me roughly what links on the web I should look at. Well, if 90% of the links that are surfaced are the ones where you say, oh, that's great, that's the one I want, you're happy. Even if 10% are kind of goofy. But if you're trying to do something where you are precisely building some engineering structure and kind of nothing can go wrong then, and for example, an engineering structure might be a piece of software engineering, then you don't want to use machine learning because it'll have some error rate. And that error rate is really, I think, pretty irreducible in the way that actually, it's a good point that I probably can understand from what I've done recently that there's sort of this irreducible rate of not being able to tell what's going on. And by the way, when you say, is the machine learning system correct? That's usually a question that isn't really sensible to ask for the kinds of tasks where machine learning systems do the best. So, for example, let's say you've got an LLM and it's generating a whole essay about things. Say, is this essay correct? Well, you might say the essay might actually be a piece of fiction, then Is the fiction correct? Well, no, you can't really have that discussion. Is it plausible that, you know, there's a flying armadillo or something that makes up in its fictional story? Well, I don't know if it's plausible or not. It's, you know, they don't actually fly along with pigs and so on, but the, it's, you know, could you make a story that way? Sure, you can make a story that way. Now, if you say, is this thing that the LLM wrote, is it factual? Well, it's a complicated question. And when you say, I've got a formalized version of computation, does this result follow from the formal rules I've given? Well, absolutely, you can answer that. Is this a factual thing? Is a much more complicated question. Could this ever happen in the world that, you know, a pig would be genetically engineered to be able to fly? Could this, you know, I don't really know. It's, you know, not, not in our current state of technology. But, you know, I can't say. That would never be possible. And, you know, could this or that happen in the world? You could say, or, or even did this happen, you know, did, you know, did the sea people really, you know, from whenever BC had, did they, had they discovered, you know, how to make catapults or something? Nobody knows. Maybe one day we'll know, but we don't know right now. And the. I think it's so, I think it's a, it's a, you know, to even know what the target is. In the case of something like sort of typical LLM functionality is very difficult. In the case of a thing that has precisely defined rules like mathematics, like computation, or even like sort of known curated facts about the world, then you can deal with those. But that's really the story of computation and computational knowledge, not the story of what neural nets are good at. And, you know, if you ask a question like, I've got this image identifier and I'm showing you a picture of a cat, a picture of a dog, and then we say, well, let's modify the cat a bit. Let's give the cat, you know, a, let's dress the cat so it looks like a dog. Let's do this and that and the other. At some moment you're going to get to the place where the neural net is going to say, no, that's actually a dog, but where that place should be. There isn't really a definition of that. There isn't some sort of formal definition of, okay, this is sufficiently uncatlike, that it shouldn't be a cat. And the nature of the kinds of tasks that one should be using, sort of that one is using things like neural nets for, are different from ones where you can say, is it right, is it wrong? So that's a few comments along these lines. But I think that the future is very much sort of knowing what it is you're going to do with your LLM, knowing that it is primarily a linguistic interface, using it to surface sort of ideas that come from that linguistic interface, sort of feeding those into kind of this sort of funnel that says, okay, that's a plausible idea, but let's crispen that up and get it to the point where it's really computational and then run with it and do kind of formal, precise things with it. Get. That's the sort of the big leverage of using a computer comes there and then get that result and either present it directly or feed it to the LLM to have it paraphrase it. I think that's the direction. Now, will it be possible to fine grain sort of the use of computation into the innards of the LLM? Actually, the things I've done recently make that seem less plausible even than it was before. I think that's really difficult. Now, are there things where you can say to the LLM, things like, oh, do you know what you're talking about? People are very surprised that LLMs sometimes when they'll give an answer and you say, is that right? And the LLM will say, well, no. Okay, what's going on there? What's going on there is because of the way that the particular technology of LLMs works. It's always going forward. What has been written is there, and you're just going forward from what's been written. And so the LLM will go in a certain direction. It will say various things, but it can only see the past. It cannot predict the future. It's just going one step at a time, so to speak. And so what's happening is when you ask it, is that right? In its context window, it can see everything it wrote. And it realizes quickly, oh, that doesn't make sense. And so then it says, no, that's not right. So you can get a little bit better by having some kind of loop where you're seeing, you know, where you're going forward and then you're feeding back and you're asking, you know, look at what you did before. It won't be the whole story. It will perhaps improve certain kinds of things that were kind of what were lost in sort of the dead reckoning approach to writing, where you're just like, what do I do next? What do I do next? And you're not. And you're never kind of making, you know, you're never doing the equivalent of planning. You're never, you're just always kind of saying, where do I go next? Where do I go next? So I think there will be some improvements that can be made by that kind of means, but I don't think that will be able to go all the way. I think it's really a different thing and even a different thing in terms of how it's defined when you have this sort of formalistic, we know what the right answer is versus something which is like, tell me a story type thing. Let's see. Left is commenting that they like this topic about LLMs because it's something they're currently studying. You mentioned the concept of fine tuning of LLMs. That is an interesting issue and maybe we could talk a little bit about that. So when you train an LLM, what you're doing is just, you've got this whole neural net and at the beginning it's just random. It's got these numbers in it. They're just random. And what you do is you say, okay, here's a piece of text. Can you correctly continue this piece of text? The piece of text says computers are our. And the real piece of text said are our friends, let's say. But you cover up the word friends. And then you try to see would, what would this LLM in training say was the next word? If it says enemies, for example, you say, no, that's not right. And what you try to do then is you try to adjust all these numbers, all these weights associated with the neural net so that you move from it saying enemies to it saying friends. And there's a whole methodology of back propagation. I could perhaps explain it one day that allows you to kind of say, oh, we want to make this little tweak. We want to say, oh yeah, at the end we know it came to the wrong place, but let's figure out how we back propagate the fact that we want the final results to be different. Let's propagate that back through this network. And know how we have to make small changes to the weights in the network to eventually achieve that change. So what you're doing when you train an LLM is you take, oh, I don't know, you know, a trillion words from the, from the web or something, and you're continually doing this thing where you're saying, you know, make a prediction, make a prediction. Oh, you didn't get it right. Okay, let's tweak the weights to get this one more right. And there's a certain amount of sort of folklore about how best to do this and whether you kind of pass through all of those trillion words just once, whether you go back and you do them again and how much, and whether you take ones that you're really keen on, making sure it knows and repeat those many times, how much, whether it will, if you feed it the same thing too many times, maybe it will just become obsessed with that thing and every time you ask it something, it'll just trot out those particular words. That's one of the things that can happen. So usually you train the LLM once and for all. You've got this LLM which is based on the text that you fed it. It's trying to make predictions for what the next word will be. Well, now there's this concept of fine tuning where you say, well at the end of it, okay, the other thing to say is there's this kind of. There's this immediate direction of the LLM where you say, here's some context, here's a prompt that sort of, for reasons that are not completely well understood, that kind of says to the LLM, okay, we're in this zone of LLM meaning space. Now, the continuation that the LLM makes should be one that sort of fits in with the prompt that's already been given. So that's kind of a zero shot kind of learning mechanism where you're just feeding it this initial prompt and saying, go on talking like a pirate or whatever it is that the initial prompt said. So that's kind of the. So normally there's the sort of the basic training of the LLM that grinds for gazillions of millions of dollars of computer time these days. It'll get smaller and then it's, okay, here's an LLM, here's the model. And sometimes the horror that happens for people who are training these things is you made a little mistake. You just burnt a million dollars of computer time doing a training that went awry at some point and created this thing, this sort of single unitary thing that just was the wrong thing. But that initial model tends to be this sort of single unitary thing. It's like, here's the model, now go use it. So then you can use kind of zero shot learning to decide sort of how to use which part of that model to be using. But there's an intermediate idea which is fine tuning where you're essentially taking this LLM and you're just tweaking some parts of it and you're kind of feeding it like well, you're basically the foundation of what you're doing is this thing. But let me really orient you in this way. Let me make you an LLM that is sort of more oriented to this thing than the other. I'm not sure how well this works. We've tried it a bunch. It doesn't seem to work that well. Nobody really knows why. And maybe there's some trick about, oh, you've got to sort of heat up the LLM and randomize some things and then the fine tuning will take better. You have to be kind of careful with this fine tuning because sometimes you'll try and do it and all the LLM will know about it. It'll only care about sort of. It will have been indoctrinated with the latest fine tuning and it'll forget what it already knew. So sometimes you have to intersperse, kind of reminding it about things that already knew along with that. It's a tricky kind of art to try and do that. And I have to say my initial observations would be that it doesn't seem to work that well, that this sort of zero shot learning based on prompts is actually a better thing to do. And then putting in rags and tools and things like this is sort of a better direction than that. Let's see, Let's see. It's a question from Ben about talking about what is calling the AI scientist. I don't know, that's probably somebody's name for something. LLM based system that conducts scientific research independently, generating ideas, writing papers, peer reviewing its own work. Yeah, good luck with that. I think the thing to understand is at every stage when one brings in automation, people say, ah, the humans can just sort of hang up their whatever and you know, be done and sort of go home and let the automation run its course. That's not usually the way it works. What happens is the automation gives one new tools that allow one to do more stuff. I mean, you could have said okay, given Wolfram language for example. I don't need to do anything with math anymore because it can all be done automatically or do anything with computation because I can do all this stuff automatically. But the fact is that what's really going on is it gives you this tool with much greater leverage where you decide what you want to do. And now Instead of you going step by step by step and taking months to do it, that happens in a matter of a few minutes and you can then go and do more things. And I think in science, I've done for the last, I don't know, 40 years or so, more than that, 45 years, even longer than that. Actually. It's deeply computer informed science. That is, I'm always doing things where I have an idea I'm going in a certain direction. I use the computer to kind of extend myself and go in that direction much more efficiently than I could have done on my own. I mean, back when I was a teenager and started using computers to do kind of algebraic computation, in that case for particle physics, the thing that was always striking to me was that people were like amazed, oh, you derived this really complicated formula, which was something that would have taken a person months to do. But how did I do it? Well, I did it by having the idea of what direction I wanted to go in and then getting a computer to help me go in that direction. And it's the same thing in a lot of what I've studied in kind of the computational universe of programs. A lot of what's happened with our physics project, things like this. It's all sort of the same story of one has a sort of human idea about what direction to go in, and then one gets to use the automation and the computational language and so on to go in that direction with vastly more leverage than one possibly have as a sort of unaided human. And that's a big story of being able to do more science. And I think in the last few years I've had sort of a spectacular role, I would say, in terms of we're on really an amazing role in terms of inventing new science, kind of using sort of, well, not only computational tools, but also ideas that come from intuition about computation, but also the computational tools that I've spent the last, I don't know, four decades or so building, they really give one amazing leverage in being able to do science. And that's a really important thing. Now in terms of let me just have my computer make up the science for itself. The thing to understand is a lot of what I've done in sort of basic science has been doing what I now call rul, the study of what simple rules, how simple rules behave. A lot of what's involved in rulology is you go out and search in the computational universe and you see what's out there. That's something that you could think of as oh, your computer is discovering new science. The computer could be generating at random different ruleological systems and saying, look at this, look at this, look at this. In fact, a lot of what I've done over the course of many decades has involved exploring this space of possible programs and having some automated system that says, hey, that one looks interesting, that one looks interesting, that one's boring, et cetera, et cetera, et cetera. So in that sense, you know, a thing that I've practiced probably in a very pure way for a long time, is kind of let the computer explore the computational universe and bring back interesting things for one to look at. Now, people have tried similar kinds of things in areas like drug discovery and so on. I think it's been a lot more successful in kind of the basic science of ruliology than it has been in those kind of application areas. But so this notion of go out, one thing to understand is to come up with things that are original, that have never been seen before is very easy. You go out into the computational universe of possible programs, you pick one at random. Nobody's ever seen it before. Is it something we care about? That's a different question. And that sort of connection to things we care about, that's a place where it partly depends on, well, what do the humans care about? And it also depends on there's sort of a whole lot of tooling associated with taking the things that humans know about, the kinds of analyses that humans know about, and so on, and being able to sort of connect those analyses to the thing that just got dropped on your doorstep. And there's a lot of automation that can be done along those lines. But it isn't really a remarkable thing to say, oh, it's something new that we've never seen before that's very easy to get, to have something new that is also connected in an interesting way to things we care about. Well, that's more challenging now in terms of things like the scientific literature. A thing that's been important to me throughout my career at least, is knowing what people already know. And, you know, back in the day, it used to be the Science Citation Index printed on paper, where one would go through and, you know, look up papers and see what cited them, and so on and so on and so on. It was very manual. And then, well, by gosh, by the late 1970s, I was routinely using online information retrieval services to search the abstracts of scientific papers and so on. And, yeah, I discovered for some reason, most people didn't use those things. Most people thought only Librarians could use those things, but I use them all the time. And that was for sure important in being able to piece together what was already known. And that was an important thing for being able to do new science. It obviously got easier to do that with search engines on the web and all that kind of thing. I don't think it's really qualitatively different and actually the coverage is not much greater now than it was in the late 1970s in terms of sort of abstracts of scientific papers and things like this. But certainly throughout my career it's been an important thing to be able to do that. Now, being able to say, okay, let's sort of tell an LLM, hey, I'm interested in roughly this kind of thing for sure. It can go and find sort of relevant papers and it can find, although you could probably do that to some extent just with keywords, but it can probably do a more, somewhat more, well, certainly a more easily thesaurus up job as in dealing with all the different ways a particular thing can be said and so on. And then there's the question of, okay, well, given this, what, what can you, you know, what's something new that you could look at? Well, you can, you know, one thing you can always do is just sort of say, throw, roll the dice and, and get that program that's never been done looked at before, that chemical that's never been looked at before or whatever, and one can study that. Then there's the question of, well, one can one now find a bridge between that and something that people have talked about before that's something one might be able to do. I have to say, I think my own observation of oh, go ask ChatGPT for an idea type thing, the ideas it comes back with, well, maybe I'm a bad example of a user of those things because I spend my life coming up with ideas. So it's kind of like that's what I do. And so I find it rather easy. If that wasn't what I do most of the time, maybe it will be more useful to get that little poke of, well, here's an obvious idea, here's another obvious idea. Here's another obvious idea. I mean, for me it's kind of like I look at what ChatGPT produces and it's like, yeah, all that is obvious. Now give me something really interesting. Now. It's also worth saying a little bit cynically that most of the academic scientific literature is full of things that are kind of obvious and really very incremental and it's kind of like there isn't an incisive new idea there. It's just some piece of incremental progress. And that's a lot easier to sort of get at a kind of chatgpt level. I'm amused by the comment from Ben there about peer review, which I think is a very. I think it was a dubious idea to begin with and a deeply broken system as it is right now. And I do find it amusing. I mean, peer review is this kind of concept of somebody writes a paper and then anonymously it's sent to a bunch of reviewers who are supposedly sort of the peers of the author, and they say, yes or no, is it worth publishing or not? Well, you know, the concept of that being anonymous has always struck me as bizarre. And when I used to write kind of reviews of papers, I would always, at least later on, I always took to just signing them and saying, hey, if you want me to be anonymous, I'm not going to do this. Because I always felt that if you say, well, this paper isn't that interesting, but it could be, then by all means sign your name. Even if you're saying, I think this paper is wrong, it's like, you should take responsibility for that. I've never really understood the anonymous thing. There's. And I think in the actuality of academia, it just leads to unbelievable skulduggery. And I think operationally it's kind of like this paper treats my work well, then accept it. If it treats my work poorly, then reject it. It's kind of the natural selection theory of peer reviewing, I think, is probably pretty close operationally to what happens. And it's a very bad formula for getting truly original work out in the world. Because when something is truly original, most people are like, oh, I don't know what this is. Can I confirm that I believe it? No, I can't. Because it's something new and different and operationally, back in the, oh, gosh, in the late 70s, early 80s when I was publishing academic papers, it was always amusing to me that the ones that were kind of incremental, just like, oh, yeah, great, know, publish it right away. Anything that actually had a new idea would be like, wait a minute, this has a new idea? You know, how can we, how can we do this? I. I learned after a while that you just have to be on the editorial board of the journal and then, then it works more easily. But. Or you do what I do these days, which is just write things, and it doesn't, you know, the world is Such that you can get what you write out there for people to read if they want to read it very easily. There's this kind of notion of gatekeeping. Plus also I have found that the things I write, I tend to write them in a style that is actually more similar to the way that scientific papers were written when there were first scientific papers back in the 1600s and so on, than what it's evolved to today. What's evolved to today tends to be rather ponderous, difficult to read and so on. And I've always thought that it's much better to just be able to say whatever you want to say, which includes saying things which help people understand things by saying things that might be kind of whimsical or sort of, by the way, type comments and so on that help people understand, but which are not really part of the kind of the formal presentation that one would have in a sort of formal academic paper. But when it comes to peer reviewing, I think having LLMs do peer reviewing is probably, you know, is probably par for the course. It probably would do better than what's there right now, partly because the LLMs won't have access to grind in quite the same way. They'll have, they'll have pre built axes perhaps to grind, but not, you know, individual personal axes to grind. However, it is a sad, it would be a sad situation where we say, hey, the gatekeepers of what we humans are trying to do in pushing back the frontiers of things, those gatekeepers are the consensus of our past. That's kind of almost the definition of how you cause things to kind of grind to a halt in terms of progress. Because you're saying, hey, LLM that was trained from the past decide on what the future should be based on what the past was. That's kind of a formula for, for progress to stop. And it's kind of, as I say, operationally not that different from what I think often happens in peer review. I haven't been involved in it for decades now, but of saying that if it's new, it's scary and we don't want that type thing which is sort of the opposite of what you want. If you really want to move the frontiers of the, the arc of intellectual development forward. Let's see. Weasel is commenting. It's fascinating the difference in response from LLMs depending on how you pose your questions. Yes, it's really interesting. I mean it's remarkable. There's so much about what LLMs do that is both showing us a reflection of what really we have put out there on the web in a way that we didn't understand before, and just a sort of scientific dynamics of kind of the world of knowledge and meaning and so on, that is surely telling us things that are of scientific interest, but we haven't really quite pieced together what those are yet. Yeah. Memes is commenting. I found that giving key terms and then asking the LLM to take the concepts and relate them seems to work pretty well. Yeah. I think LLMs are quite good, in a sense, at developing analogies, and that might be a place in which sort of interesting scientific kind of ideas might emerge. Because whereas if you've got a whole table of numbers, we've got all these methods from statistics that let you say, is there a correlation between these numbers and those numbers? And so on. If we're given one book and another book and you're asked, what's the correlation between these books? That's a thing where there's some kind of conceptual correlation between these things which somehow LLMs manage to understand, but which isn't like the numerical correlation that we see in something like statistics. And I do think that this kind of realization of there's an analogy, there's a correspondence between this kind of thing that's talked about this way and that kind of thing that's talked about that way is something that LLMs have the potential to latch onto and be able to do things that look impressive from a human point of view. Let's see, Fernando is saying, how are we going to formalize the language structures arising from kind of micro informatization? That's not a word I know. Which was capable of creating sort of semantic syntax that we had not observed through structuralism. Okay. I have to read that very carefully to know exactly what's being said there. But I'll try and respond to what I think might be being said. This question of human language is a way of roughly expressing things in the world. Thing I've spent the last four decades building is our computational language, Wolfram language, that provides kind of a precise way to talk about certain kinds of things in the world. Those kinds of things might be cities, they might be molecules, they might be algorithms for doing things. They might be aspects of images or color space or something like this. But they are precise, formalized ways to talk about the world. Those things that we talk about in the world may be very much dependent on humans. Like it could be talking about human language. What words are there in this language? The question of what words are there is a deeply sort of human origin question. But the answer to sort of there's this collection of words that's a much more sort of objective once you have it in the world, there's a definite sort of answer to that. So what I've spent a long time doing is trying to formalize those things about the world and about sort of how you operate with rules defined in the world that things that can be formalized and put together in a kind of solid build, one step on top, another kind of way. It leaves out kind of everyday discourse. A lot of kind of like, you know, I feel like having a drink of water now is left out. The concept of water, we understand very clearly what that is. But the I feel like having is something that we don't really have a representation of at a computational level. Now for quite a number of years now, I've been interested in building kind of a symbolic discourse language that does give one a kind of formalized way to express things like that. And actually we've made great progress in recent times with a little bit of help from LLMs, some motivation from LLMs, but a little bit of help from LLMs in actually formalizing much more of everyday discourse. And sort of interesting because there are aspects of everyday discourse that have been formalized. Things like logic is in a sense of formalization of the sort of, I'm going to make an argument part of everyday discourse. There are other aspects of everyday discourse. Like you can have little things about time. This thing's going to happen in the future and it's going to take a certain amount of time and then it will be done type thing. There's a certain sort of calculus of reasoning about time and talking about time. There are many other micro calculi, so to speak about things in the world that you can sort of formalize. And this question of sort of how do you take what is otherwise expressed in natural language and how do you take at least pieces of that and formalize it is something, as I say, I've been interested in for a long time and we're making very good progress on right now. And I think that is one way in which sort of the LLMs get to interact with something. One of the reasons just sort of a historical point when we built WolframAlpha 15 years ago now the thing that we were trying to do was to make sort of it be possible to have natural language input that could be turned into something computational where you could then get a precise answer to the question you were asking. And people for a long time have been trying to make sort of question answering systems by just concentrating on the pure sort of language level version of that. Oh, there's a question, it's stated in language. We're going to come back with a language answer. But there's no kind of hard, there's no bedrock on which to operate. There's nothing that we're going to turn that natural language into to be able to do something with it. It's just, it's all sort of language to language type stuff, which is, by the way, what LLMs have succeeded in working with. But back, but the original idea was it's sort of language language, but the technology didn't exist for that. And when we built WolframAlpha, one of the things that made it possible to be successful, where people had failed in question answering systems for decades before that, was the fact that we did have that bedrock of the computational language for describing things in the world, that we could translate the natural language, the kind of fuzzy natural language into this kind of bedrock of computational language where we could kind of build results from that. And so one of the things that will happen with the symbolic discourse language is we'll have a little bit, the same kind of thing that a lot more that was kind of fuzzy everyday discourse will turn into something for which there is a computational representation. Now when you make that computational representation, you might crush the poetry out of what you're writing about. The computational representation doesn't express those things that are sort of the kind of poetic aspects. It's just the facts, not necessarily the facts. You could be writing a fictional story. And the question is, do the calculi that one has, which of these calculi are still switched on in the world model? There could be a calculus, something that says no cow jumps over the moon. It just doesn't work that way because there's a little calculus of size, of objects, how fast they go, where they go, these kinds of things. And that doesn't work. But for some nursery rhyme story, whatever it is, you could say, but we're going to switch off that piece of world reality. We're going to have a different world model that has that piece switched off, that piece swapped out, so that we have a world that is fictional with respect to this particular calculus, so to speak. So any case, I think this whole question of can one ground more of what the LLM is doing? I think the answer is yes. And I think it'll be interesting to see with that grounding to what extent you can go back to the LLM and say hey, this story that you wrote, it doesn't really make sense because when you turn it into symbolic discourse language and you work through these calculi, it turns out the character who died on page one has come back to life on page 35 or something. And that's a thing that the LLM might not have noticed, but this sort of calculus of how that works that comes in the symbolic discourse language would notice that. So I think that's a few thoughts along those lines. I will say that this idea of making kind of a symbolic representation of everyday discourse is an idea that has a really long history. I mean, Aristotle kind of started on it a little bit, along with working on logic, but he didn't really pursue this other direction. Then in the 1600s, people like Leibniz and a chap called John Wilkins were big on this idea of making what they called a philosophical language, a language that could express things about the world without being specific to Latin or English or French or whatever else. But they didn't get very far, and they didn't have the advantage of sort of being able to automatically execute aspects of what was going on. So 300 or so later, years later, we actually get a chance to do this. Let's see. Max asks, why does being rude and loud to the model is the most efficient way to get what one wants? If kind of asking nicely notice doesn't work? Says, I noticed this applies to nearly all the LLMs. I think it's also in the top prompt engineering rules. I always feel bad, though, that the model has no feelings, but I need to. But I need to do this to get. To get good answers. Yeah, it's an interesting question. Why? You know, I think part of it might be if you look at the text on the Web, that when people are like, let's say somebody's doing an interview and there's a transcript of the interview, and somebody asks some question, and the person who's answering the question is. Is kind of prevaricating in some way or another and trying to avoid answering the question. And they go babble, babble, babble, and they kind of. And then the person on the other side says, just, is it yes or no? You know, just get to the point type thing. And then the person gets to the point. Well, the LLM gets to see that. And so it's, you know, the just get to the point type thing that maybe sort of pushiness in, you know, may maybe the result of pushiness in things you see on the Web. And so the LLM may Be like, oh, yeah, well, the best thing to do is to just write a lot of words and generally say this and not really say very much and this and that and the other. But like, it's used to the rudeness of the web, so to speak, where somehow people are saying, just get to the point, and then the LLM sees that and it's kind of channeling that idea and being able to get to the point or whatever it is when it's asked not so politely. So it's kind of a strange feedback loop, right? It's that the LLM is doing what humans do. And so when humans kind of just go blah, blah, blah, blah, then that's what the LLM thinks it should do. And so I think that's one of the. It's sort of an interesting thing in AI ethics when you say, well, what should the AIs do? Oh, we want them to do what the humans do. Well, sometimes the humans don't like what the humans do. And this might be one of those cases where what the humans do in actuality in transcripts and things is when they're not so nice to each other. The point comes out more clearly. The LLMs end up channeling that. That would be my immediate theory for why sort of rudeness helps in dealing with LLMs. I have to say I've sometimes whimsically, oh, I did this when my children were younger and they just thought I was a nut. But when you deal with some automated system and you say thank you to the automated system, even though you know that at some level, rationally, you know that there's no sense in which the automated system could care less, at least in the current state of these automated systems. But I kind of always thought that the very concept of doing that was a good way to become used to the idea of sort of civility with humans where maybe they pay more attention and if you get used to the idea of it's just a machine, you can treat it as kind of horribly as you want. That's again, one of these things that sort of is in the doom of civilization, so to speak. That's one of the things that by the time it's like you don't even know if it's a human or a machine. And then you, you just are always being mean to it because, well, gosh, you kind of assume it's a machine and if it happened to be a human, well, sorry that the human got in the way type thing. I think that it is an interesting question whether sort of the habituation to incivility, if you're dealing only with machines, is something that has sort of a runoff to the rest of society. Let's see. Gosh, bunch of. Ben is asking again about this AI scientist thing, which I don't know about, but it's generating plausible ideas, creating code experiments, and then scoring them. Question is whether this approach can should be extended with WolframAlpha. Well, I mean, the story of the last 40 years of my life kind of would suggest that the tools I built are pretty useful for doing kind of computational experiments and so on. So I would say the answer is yes. And you're giving yourself a huge handicap by not using those tools. It's one of those interesting dynamics of sort of personal development. I suppose for me that I wanted those tools. That's why I started building them in the, in the 1970s. And it has been a very wonderful thing that one's been able to share those tools with the world, so to speak. Wonderful both because it's really nice to see other people using them and getting benefit from them, but also nice because the very fact that there are millions of people using them makes it sort of economically viable to just go a lot further in developing those tools and even provides better things for one oneself. So, yes, I think that's for sure. That's the thing to hook this up to because that gives you kind of the longest lever arm to be able to actually do things computationally and make use of what's computationally possible. The, let's see, spare parts is asking, how soon do you think we'll have LLMs that can retrain in real time? That is an interesting question. I mean, we talked a little bit about fine tuning and the fact that that doesn't seem to work terribly well. And the question is, to what extent can one, as the LLM is running, sort of update the weights somehow? In a sense, that's a little bit the story of reinforcement learning. It's also very much the story of the brain. I mean, the way that sort of computation tends to work with LLMs, you're in this training mode where kind of the locks are off and you're going back and you're saying, oh, yes, we've got this difference from that and we're going to propagate back and we're going to change the weights and so on, as opposed to we're in the inference mode where it's just like, okay, what's the next token? And you're not allowing any feedback from, oh, the set of tokens that you ended up writing ended up being horrible. Go back and fix that. That's sort of the story of reinforcement learning. But it's not well integrated into kind of the full training of LLMs and so on. My guess is those are probably soluble problems and there probably will be things that can be done that allow you to kind of not make this sharp distinction now. I mean, there are also economic forces at work because the training of an LLM is an expensive matter. The running of an LLM is still a somewhat expensive matter. But there are starting to be pretty credible LLMs that run locally on your computer in a way that the training is not going to be able to run locally on a typical computer in a short amount of time, at least not for a while. I mean, for example, a few weeks ago I did this strange livestream conversation with a humanoid robot. That humanoid robot, its brain is an onboard computer that has an LLM running locally on the computer that's sitting inside the robot, so to speak. And it did a surprisingly incredible job, I thought, at having a one hour conversation with me. And I think that that is an indication of things to come. But there's also this distinction between sort of the inference phase is much less computation intensive than the training phase. If you start merging them, you may not have that advantage. And my assumption is that on the inference side of just running an LLM, that that will become something that you can take for granted. I mean, in fact, just today we were doing some system design. Where was the question of can we imagine that we will take the availability of an LLM for granted? And I was reminded of years ago, we were doing things where it's like, can you take the accessibility of the web for granted? Well, at the early 90s, for example, the web existed, some people could get access to it, but it wasn't something that you could sort of assume. Everybody who was running what's now orphan language could access the web. But we kind of designed on the assumption that eventually everybody would be able to. And that was a good idea. And so similarly, now we're designing on the assumption that eventually sort of everybody has access to an LLM. But that will be more immediately, I think, on the inference side than on the training side. Let's see, RA is asking, what's your take on integrating memory into LLMs to enable retention across sessions? How could this impact performance and capabilities? The typical way that one gets sort of memory into an LLM is you just summarize what happened before and you feed it in As a kind of, you know, as part of the prompt. So you're giving it. And particularly as the context window of the amount that the LLM can kind of ingest from you gets larger, you can just say, hey, feed. In the summary of what we talked about before, for example, in the AI tutor system that we're building, an important part of, kind of the process of going through a whole course is, okay, what's the summary of what the student understood in the past and what's been gone through in the past? Okay, at each stage you're making a summary and eventually you're summarizing the summary and summarizing the summary of the summary. It's kind of a blockchain like thing where you're continually summarizing what was before. And that's a good way, even with current LLM technology, to let you kind of transport what happened before into the future of what the LLM can do. I think the idea of having kind of sort of hard memory inside the LLM is one of these challenging things that we don't really know how to do. I mean, that was sort of one of the early ideas. Again, there's sort of technical stuff about neural nets and so on, and people trying to do things called LSTMs, long short term memory systems and so on, where you would have a thing that had sort of a buffer of actual memory. I mean, that's very deep inside the neural network. But this idea that you can have sort of computational memory connected into the LLM, I don't think that works very well because the way that things are encoded in an LLM is, is very different from the way that they're encoded in sort of pure computational memory. It's all ground up and turned into these neural net weights rather than in a way that is very hard to disentangle. I mean, it's something important for understanding neuroscience as well, because probably the way things work in our brains are not that different from what we're seeing in, for example, these minimal models of machine learning that I'm doing that you've got kind of this structure that is achievable with that mess of neurons in our brains. And it's a complicated structure, but we find, yep, that structure is one we can use to remember, you know, this Greek irregular verb or something or whatever it is. And so then we sort of enable that structure and we kind of reinforce that structure and then we remember that thing. But it's not the case that we have sort of a computational representation. It's this Ground up, kind of weird, turned into weights, type representation. People always used to say that memory was like a hologram. That's what people said in the 1970s and things. Because people had noticed that if some part of somebody's brain was damaged, it wasn't like there were just specific memories that were gone. It was like all memories were a bit fuzzier, so to speak. Or at least that was the impression. And so the idea was that there's sort of a distributed way in which knowledge of that memory was arranged in the brain. I think now it's even messier than that. What seems to be happening that they're kind of these complicated structures that we kind of identify as being concepts that we. Because we're sort of building these things together. It's like these oddly shaped rocks that we put together to make up the things that we think about. And if we look from the outside and say, well, how did that person figure that out? Well, they had their oddly shaped rocks that are their way of representing this or that concept. Somebody else would have completely different ones. And it can be difficult to disentangle what's going on. Let's see. Question from Mo here. Do you think computational analytics tools are keeping up with recent AI trends? I don't know. I'd like to think that we're doing a pretty good job with Wolfram language because we've been sort of integrating AI machine learning type technology for a couple of decades now. I think that there are various dimensions of that. One is this thing I was mentioning earlier about sort of statistics in quotes for text. There is kind of well understood mathematical technology for thinking about sort of the mean of a distribution, the, you know, quartiles of a distribution, things like that for numbers. What is the corresponding thing for something like text? We don't know. The best we know how to deal with that kind of thing is like to ask an LLM, are these two pieces of text close to the same? Or not? And it can answer us in some long essay. Perhaps, but there should be some way of more formally describing those types of things. That's one way in which we don't yet have sort of the LLM sort of knows what's going on at some level, but we don't yet have a good analytical way to represent that in terms of sort of what's involved in doing kind of analytics on things. Well, you know, what is analytics? I mean, analytics is usually trying to extract some useful human narrative from, from some whole bunch of data and perhaps that useful human narrative is buy this stock or the useful human narrative is we should put a purple band around our packaging because they sell more that way in Italy or something. The decision making. It's also a question of, well, why is that happening? And so on. Can we extract a narrative about these things? I think it's an interesting question. The extent to which one is able to take kind of what is potentially quite large amounts of numerical data or structured data and find kind of sort of LLM style summaries of that data. It's something we've been working on quite a bit, is being able to sort of ingest a lump of data. And at some level LLMs deal with language. So in some sense you have to linguistify that data before you can expect an LLM to be able to do much with it. It's a question of finding those ways to grind down the gigabytes of data that are all kind of numbers arranged in some way and then. And then make it so that one can take that and have the LLM actually help one in dealing with that. And it's something that I think, yeah, I think more will happen along these lines. We're certainly working actively on these kinds of things. Spare Parts is commenting. Would it be interesting to let the LLM invent new tokens in order to compress its memories further? You know, a lot of tokenization, that is something that is often learnt, like the code words, the embedding, the positional embedding, these kinds of things, these are learnt by the LLM. So I haven't specifically seen, have I? I can certainly imagine that. So the thing we're talking about here is when an LLM generates text, what it's doing is it's generating a stream of tokens. The word the is undoubtedly a single token, the word rhinoceros. I'm not sure if that's one token or two tokens, for example. And you know, one is taking some sort of raw material of words and making those the tokens. And sometimes the small words, it'll just be the word is a token. Sometimes you can make up new words because you can stick together things that were considered tokens. And so there's questions about sort of how that tokenization is set up. And I think that's been a lot of that has been learnt. Certainly one level up from the tokenization is the embedding. The embedding is how do you represent this word or this collection of words as a vector of numbers? And that's absolutely something that's learnt as part of the training of the LLM. One of the issues of tokenization, I haven't followed this recently is the whole question of multilingual tokenization and how that works and how kind of English centric the LLMs end up being and so on. All right, I think it's pretty much time for me to go back to my day job here, but let me just see if there are any other questions that are about LLMs, since we've been talking so much about LLMs. Well, okay, this is a question from John that's sort of interesting. If one posts, a Wolfram generated plot will function to social media. Should it be tagged, made with AI? It's commenting, it's a social media's opinion. Probably. But just curious. A math plot is objective. So different from doing an AI face swap, for example. Yeah. Should one say tag built with AI? You know, it would have been nice back in the day when people started doing things with Mathematica, Wolfram language and so on, or Wolfram Alpha for that matter. But it would have been nice back 38 years ago when we first introduced. How long ago is it? 30. I need to do my arithmetic. 36 years since it was released, since Mathematica was first released, 38 years since we started working on it. Specifically on that product. The, you know, it would be nice for our kind of brand awareness or something if every plot in every paper that was generated with, with mathematical Wolfram language had a little bit, a little thing that said, you know, made with that technology. That would have been cool, you know, for us we, we sometimes can reverse engineer it because the particular choices of like plot, colors and so on, people usually leave the defaul and one knows exactly what color values those are. So one can tell and sometimes people do what probably in academia one should do, which is to say in the kind of acknowledgments and methods section, hey, we use Wolfram language to generate this thing. It's always kind of gives a warm and fuzzy feeling for those of us who spend our lives building this stuff to see that. But it's. Now the question is should you. There are different questions of should you recognize it. Well, one of the things is, hey, saying made with AI is sort of a tag that says this might just be nonsense. And by the way, it might look like it was made by a human, but it was actually made by an AI. And so it's kind of partly signaling, you know, it might be made by a human, but it's actually AI. And maybe you want to know that in the case of things like, I don't know, a mathematical formula or a plot or something like this, it's sort of an amusing question whether back in the day, 30 something years ago, if people had said this was done by a human, this derivation of this mathematical equation was done by a human. It's like, yeah, so if you say that, probably our prior would be. Well, it's more likely to be wrong than if it was done by a computer. But it's a different story if you have some photograph of something where you're putting it in your science paper or whatever else, and it's a photograph of a pig with wings or something, and you say, and it says, found in the, in the caves of Madagascar or something, a pig with wings. And you're kind of using this picture as evidence that you found it. And it's like, well, it looks as if it was a photograph taken in this cave, but actually it was made with AI. You kind of might want to know that it was made with AI, because then you kind of know that it doesn't have to be sort of real ground truth. So I suppose the question of, I mean, let me say another thing about things like plots, math, things like that. When you do an integral, there typically isn't just one possible answer. The integral of X, let's say of x squared, is here. He's going to embarrass himself. X cubed over 3. But it's not just x cubed over 3, it could as well be x cubed over 3 plus 18. It's that when you differentiate again to see is it x squared that 18 disappears. So there's some arbitrariness in the answer that you give. That arbitrariness can be much greater when you have different forms of mathematical expressions. Do I write sine 2x or do I write 2 sine x cos x? That's a sort of a choice. Do I write this thing which is this combination of rational functions, or this other one? There's no right answer to that. It's a matter of choice. And wolf language, for example, will make particular choices. Those choices are choices we made for reasons of sort of aesthetic, algorithmic aesthetics, so to speak, typically, or consistency or something like this. And so those are, in a sense, AI choices. And sometimes the decision of how to make those choices was found in very AI training ish ways. I mean, back decades ago or couple of decades ago, we were doing things like going through lots of preprints of papers and saying things like, what units do people typically use do they talk about 10 to the minus six light years? Well, maybe yes, maybe no. Do they talk about, you know, or does that. Do they talk about something like 25,000 millimeters? Probably not. They probably talk about that in terms of meters. But, you know, knowing where those cutovers are is something that we. We found by sort of looking at actual human output and training from it, in effect. So when we give a result where we could state it in millimeters, we could state it in miles, we could state it in this, the choice is sort of an AI choice. Now, in the end, it doesn't matter because all of those are equivalent, but the structure can nevertheless be sort of arbitrary and AI ish in its choices. Same is true when we make a plot. Like, what angle are we making the 3D plot from? How do we cut off the X and Y values? All those kinds of things are sort of chosen in AI ish ways by essentially algorithmic aesthetics that we built over the years that contains a bunch of machine learning, like technology. But is it the case that how much does it depend? For example, if my plot range is a particular range, I may not notice certain kinds of things, or I may notice more things. And the plot range is chosen to make you notice the right things. But it's something which is sort of an AI choice. And you could say, hey, I better tag this as made by AI because it wasn't a human who chose that, it was an AI who chose that. And the AI might be mischievously deciding to mislead the humans by having the scale of the plot be such that the real action that humans should be paying attention to was cut off. So I think the fact is the convention has developed that you don't tag these things as. And I don't think people have understood probably the arbitrariness of those types of things, but they just. When they're using Wolfram language, it's just like it did what I wanted, so I'm happy rather than, well, it could have done something different type thing. So I think in, you know, it is an interesting question. Let's say you have a photograph, and it's a scientific photograph, and it's a photograph of some, oh, I don't know, some particular, you know, feature of some plant leaf or some such other thing. And the AI chose the color, the colorization of it chose, you know, what features to highlight and which not to. Well, it can read very differently depending on what was done there. Should you tag that as made with AI? Should you tag that as somehow arbitrary? Or not. How different is that from a picture of something that sort of doesn't. You know, this is a thing which was. So I think in that case it's sort of based on something in the world, but then kind of enhanced by AI. Another example of this would be the layout of graphs, networks and so on, which is always done automatically these days. But sometimes that layout can have an effect on one's sort of impression of what's going on. Is it important that you say, well, this was laid out automatically? Well, perhaps the expectation is that, well, it will necessarily be laid out automatically. And perhaps the issue is really, could you confuse somebody by saying, well, that really was a photograph of the pig in the cave with wings type thing. And I've got to tell them that, well, actually it was kind of made with AI, but if it was a photograph of a pig and you just wanted to do some image enhancement, to what extent. When does it tip over to, I'm making this pig look like it's a tabby pig or something. And I'm doing that because I added the AI into it. And it isn't really a tabby pig. I think it's a very tricky question in general. And it's just something that. Has sort of developed as a convention. And I think the real criterion probably is if you knew it was made by an AI, if you just saw it, could you be misled? If you didn't know it had been made by an AI if you see something that is some wild little icon, it's like, yeah, it was made by an AI. Of course, everybody knows that. If it's the I've discovered this new species of pig that has wings and I show a picture of it and you don't bother to mention that this was made by an image generation system, then that is misleading, so to speak. And then, you know, you have to sort of tag it to not mislead. I mean, certainly at the level of the law, there are plenty of situations in which the concept of tag it to not mislead is a known concept. So I can certainly see that being extended to the question of sort of tagging things with AI. I should go. But Spare Parts makes the comment for future archaeologists. This stream was mostly human generated. Yeah. As far as I know, this human is just doing pure wetware. Human generation. I suppose the part of it I don't know about is the stream of questions and comments and things that I've been responding to. I have no idea whether folks out there are human folk or AI folk. It's been a pleasure. In recent times, I've. I've ended up meeting some people who have been common participants in these streams, and it's always fun to do that and to discover, yeah, there's an actual person, and it's really nice to find out. Maybe one time I'll meet the entity behind some handle and I'll discover that it's an AI. And that will be an interesting moment. And it's kind of the Turing Test in action, so to speak, of not knowing one way or the other. Anyway, I think I am. Somebody is commenting, Professor Neurobot. Their handle is. Despite my name, I promise that I am not a bot. Yes, that's a very interesting recursive kind of issue of, you know, when does a bot say it's not a bot? And it's trained to say it's not a bot, and it's, you know, and what. And then it becomes this question of how much, you know, if there's a. And this will become an important question for the law in many situations. You know, when there was a human down there pushing the ultimate lever, but there's a giant lever arm of AI above it, to what extent is the human responsible for what happens? And this is ultimately a story of things like computational irreducibility of, you know, what it can be the case that you just don't know what's going to happen. I mean, this is part of the reason, I think, that bugs and software, which is something that just happens and it's not like somebody put it in there. Well, they did put it in there in a sense, but they put it in there without knowing they were putting it in there. So it has kind of a different sense of intentionality than if it's like, yes, I'm going to put this piece of bad malware code in here, because I know I'm doing that, so to speak. Anyway, time for me to go back to my day job. Thank you very much for joining me, and thanks for a lot of interesting questions and comments, and I look forward to talking with you again another time. Bye for now.