What is ChatGPT doing...and why does it work?

Okay. Hello, everyone. Well, usually in this time slot each week I do a science and technology Q and A for kids and others, which I've been doing for about three years now, where I try and answer arbitrary questions about science and technology. Today I thought I would do something slightly different. I just wrote a piece about ChatGPT. What's it actually doing? Why does it work? I thought I would talk a bit about that here and then throw this open for questions. And I'm happy to try and talk about all things kind of chatgpt, AI, large language models and so on that I might know about. All right, so bursting onto the scene, what a couple months ago now was our friend chatgpt. I have to say it was a surprise to me that it worked so well. I'd been kind of following the technology of neural nets for. I worked out now 43 years or so, and there have been moments of significant improvement and long periods of time where kind of it was an interesting idea, but it wasn't clear where it was going to go. The fact that ChatGPT can work as well as it does, can produce kind of reasonable human like essays, is quite remarkable, quite unexpected, I think, even unexpected to its creators. And the thing that I want to talk about is, first of all, how does ChatGPT basically work? And second of all, why does it work? Why is it even possible to. To do what has always seemed to be kind of a pinnacle of human kind of intellectual achievement of, you know, write that essay describing something. Why is that possible? I think what ChatGPT is showing us is some things about science and about language and about thinking, things that we kind of might have suspected from long ago but haven't really known. And it's really showing us a piece of sort of scientific evidence for this. Okay, so what is ChatGPT really doing? Basically the kind of the starting point is it is trying to write reasonable. It is trying to take an initial piece of text that you might give and is trying to continue that piece of text in a reasonable human like way that is sort of characteristic of typical human writing. So you give it a prompt, you say something, you ask something, and it's kind of thinking to itself. I've read the whole web, I've read millions of books. How would those typically continue from this prompt that I've been given? What's the sort of the reasonable expected continuation based on kind of some kind of average of a few billion pages from the web, a few million books and so on? So that's what it's Always trying to do. It's always trying to continue from the initial prompt that it's given. It's trying to continue in sort of a statistically sensible way. So let's say, Let me start sharing here. Let's say that you had given it the. You had said initially. The best thing about AI is its ability to. Then ChatGPT has to ask, what's it going to say next? Now, one thing I should explain about ChatGPT that's kind of shocking when you first hear about it is those essays that it's writing. It's writing it one word at a time. As it writes each word, it doesn't have a global plan about what's going to happen. It's simply saying, what's the best word to put down next? Based on what I've already written. And it's remarkable that in the end one can get an essay that sort of feels like it's coherent and has a structure and so on, but really, in a sense it's being written one word at a time. So let's say that the prompt have been the best thing about AI is its ability to. Okay, what's ChatGPT going to do next? Well, what it's going to do is it's going to say, well, what should the next word be? Based on everything I've seen on the web and et cetera, et cetera, et cetera, what's the most likely next word? And it knows certain probabilities. What it figures out are probabilities. It says, learn has probability, 4.5% predict 3.5%, and so on. And so then what it will then do is to put down the next word it thinks it should put down. So one strategy it could adopt is I'll always put down the word that has the highest probability based on what I've seen from the web and so on. Turns out that particular strategy of just saying put down the thing with the highest probability doesn't work very well. Nobody really knows why. One can have some guesses, but it's something where if you do that, you end up getting these kind of very flat, often repetitive, even sometimes word for word repetitive kinds of essays. So it turns out, and this is typical of what one sees in a kind of a large engineering system like this, there's a certain kind of touch of voodoo that's needed to make things work well. And one piece of that is saying, don't always take the highest probability word. Take some with some probability, take a word of lower than highest probability. And there's a whole mechanism. It's usually called its temperature parameter. Temperature, sort of by analogy with statistical physics and so on. You're kind of jiggling things up to a certain extent. And the higher the temperature, the more you're kind of jiggling things up and not just doing the most obvious thing of taking the highest probability word. So it turns out a temperature parameter of 0.8 apparently seems to work best for producing things like assays. So. Okay, well, let's see what it takes. One of the things that's nice to do is to kind of, to get some sort of concrete view of what's going on, we can actually start looking at sort of on our computer. What's it doing? I should say that this, what I'll talk about here is, is based on this piece that I wrote that just came out a couple of days ago. And I should say that every piece of code there is click to copy. So if I click every picture is click to copy. If I click this, I will get a piece of Wolfram language code that will generate that. Let me go down here and start showing you kind of what, How, how this really works. So what chatgpt in the end is. Oops, not seeing screen. Interesting. Oh, well, okay, that's. Oh, there we go. Okay, well, I was, Let me, let me show you again then. What, what I was showing before. This is the piece that I wrote and I just wanted to emphasize that every picture and so on that's in this piece has click to copy code. You just click it, paste it into a Wolfram language notebook on a desktop computer or in the cloud, and you can just run it. Okay, so let's see how, let's see. Let's actually run an approximation at least to ChatGPT. So OpenAI produced a series of models over the last several years, and ChatGPT is based on the GPT 3.5, I think model. These models got progressively bigger, progressively more impossible to run directly on one's local computer. This is a small version of the GPT2 model, which is something you can just run on your computer. And it's part of our Wolfram Neural Net repository and you can just pick it up from there. And this is now the kind of the neural net that's inside a simplified version of ChatGPT. And we'll talk about what all of these innards really are later. But for now we can just do something like say, let's use that model and let's have it tell us the words with the top five Probabilities based on the starting prompt. The best thing about AI is its ability to. So those are the top five words. I probably can ask it 20 words here. So let's say, let's see, these are probably sorted, right? We probably want to sort these in reverse order. And this will now show us the. Oh, I see. This is sorting. Okay, so this is showing us these words with different probabilities here. Actually confused by why this didn't. Oh, I know I didn't. I know I didn't do that. I know why I didn't do that. Let me just make this do what I expect. Okay, here we go. So this is that sequence of words. It's now by the 20th word, we're getting down to keep. I don't know, let's go, just for fun, let's go find out what the 50th word was. Okay, so down here we're seeing words that were thought to be less likely. What does it mean to be less likely? It means that based on essentially ChatGPT's extrapolation from what it has seen on billions of documents on the web, this is the word which. These are the words which have certain probabilities of occurring next in that particular sentence. Okay, so now let's say we want to. We want to go on. We want to say. Let's say we want to say the best thing about AI is ability two. And the next word it might pick might be learn. But then what's the word it's going to pick after that? Well, we could figure that out by just saying here, let's say the next word was learn. Okay, then let's say that what we would get next, we'll fill in the learn there and we just say let's get the next top five probabilities for the next word. Okay, so the next word is from. That's the most probable next word is from. So we could say learn from. And then the next most probable word is experience. All right, so let's write a piece of code that automates that. We're going to nestedly apply this function that is just taking the. The most likely word, so to speak. Let's do that 10 times. And this is now what we get. This is using the GPT2 model. This is asking what the most likely continuation of that piece of text is. Okay, so there it goes. Now this is the case where it's always picking the most probable word. As I said before, it very quickly ends up in this zero temperature case. It very quickly ends up getting Itself kind of tangled in some loop. Let's see if I have the example of what it actually does. In that case, the. Let's see. Yeah, here we go. And. This, this is not a particularly good, impressive essay and it gets itself quite, quite tangled up. If you don't always pick the most probable word, things work much better. So for example, here are some examples of what happens when you use this temperature to kind of jiggle things up a bit and not always pick the word that's estimated is most probable. It's worth realizing that I showed you a few examples of less probable words. There's a huge spectrum of different words that can occur with progressively lower probabilities. It's kind of a typical observation about language which you see here as well, that the nth most common word has probability about 1n and that's what you see for the word that will follow next. And you also see that in general for words and text. Okay, well we could ask what happens in the zero temperature case for A. Let's see, for the actual GPT3 model, this is what it does for zero temperature. Now one feature of this is if you use. Well, for example, this is a, a link to The API for OpenAI that's in our Packlet repository. If you use that link and you simply call GPT3, it will. Because this is always picking the most probable word, it'll be the same every time. So there's no randomness to this. What happens usually when you're picking these words with, when you're picking non zero, when you have non zero temperature and you're picking words that aren't always the most probable word is there's a certain randomness that's being added and that randomness will cause you to get a different essay every time. And that's why if you say regenerate this essay, most likely you will get a different essay every time you press that regenerate button because it's going to pick different random numbers to decide which of the words ranked words it's going to use. So this is a typical example of a temperature 0.8 type essay generated by GPT3. Okay, so the next big question is we've got these probabilities for words and so on. Where do those probabilities come from? So what I was saying is that the probabilities are basically a reflection of what's out there on the web. And those are the things that ChatGPT has learnt from. It's trying to imitate the statistics of what it's seen. All right, so let's take some simpler examples of that. Let's say we're dealing not with. So ChatGPT essentially deals with putting down words at a time. Actually, they're pieces of words, but we can assume for the simpler cases they're just words. But what if. Let's start off to understand this. Let's start off thinking about putting down individual letters at a time. So first question is, if we're going to just put down letters one at a time, what is the. With. With what probability should we put down what letter? How do we work that out? Okay, let's pick some random text. Let's pick the Wikipedia article about cats, and let's just count letters in the Wikipedia article about cats. And we see that E is the winner. A is the runner up. T comes next. So based on the sample of English text from the Wikipedia article about cats, this is what we would think about the statistics of different letters. Let's try the Wikipedia article about dogs. Okay, we have probably slightly different. We have an O shows up more with higher probability, probably because there's an O in the word dog and so on. But if we keep going and we say, well, what about. Really? So that's for these specific samples of English. Let's keep going. Let's make a. Let's see. There we go. Let's use a very large sample of English. Let's say we have a few million books and we use that as our sample of English and ask what are the probabilities for different letters in that very large sample? And we'll see what many people will immediately know that E is the most common letter, followed by T, A, et cetera. Okay, so these are our probabilities. So now let's say that we want to just start generating text according to those probabilities. So this is. Let's see. This is probably just. Yeah, just, just fill those in. Oh, there we go. There are the frequencies. And let's just fill in. Let's just have it start generating letters. This is just generating letters according to the probabilities that we get from. From the occurrence of those letters in English. So that was asking it to generate 500 letters with the correct probabilities to correspond to English text. That's really bad English text there. But that should have. The number of E's should be about 12%. The number of T's should be about 9%, and so on. Okay, we can make it a little bit more like English text by going ahead and let's fill in let's append a certain probability to have a space. And now we can, let's make a bigger version of this. And now this is generating quotes English text with the correct probabilities for letters and spaces and so on. We can make it a little bit more realistic by having it be the case that. The, the word lengths in this case here, we're just chopping it into words by saying there's an 18% chance that a character is a space, which is here what we're doing is we're saying let's insist that words have the correct distribution of lengths. And this is now the text that we get where the words have the correct distribution of length, the letters have the correct probability of occurring, with E being the most common, and so on. Clearly not English. Clearly a lose. If ChatGPT was generating this, it would be a fail. But this is something which at the level of individual letters is statistically correct. If we asked, can you tell that this isn't English by just looking at the chances of different letters? It would say this is English. And different languages, for example, have different characteristic signatures of frequencies. If we were to pick this, or I don't know what, I'm sure if we pick this for English and we were to do the corresponding thing for let's say which we'd pick, let's try Spanish here, for example, and we'll get slightly different frequencies. Okay, those are somewhat similar, but not quite the same. Okay, so that's what happens. If this is sort of generating English text with the correct single letter statistics, we could just plot the, just plot the probabilities for those individual letters. Oh boy. More complicated than it needed to be. Okay, that's just the probability for each letter to occur. So E is the most common, Q is very rare, etc. In this case, what we're assuming is that every letter is sort of picked at random independently. However, in actual English we know that's not the case. For example, if we've had a cue that's been picked, then with overwhelming probability, the next letter that will occur is a U. And similarly, other kinds of combinations of letters, other kinds of 2 grams, other kinds of pairs of letters. So we can, instead of asking for the probability of just an individual letter, we could, for example, say what's the probability for a pair of letters coming together? Let's see, here we go. So this is asking, this is saying, given that the letter B occurred, what's the probability for the next letter to be E? So it's fairly high. The probability for the next letter to be F is very low over here. When there's a Q, the probability for next letters is only substantial when there's a U as the next letter. So that's what it looks like to have. That's what this combination of pairs of letters, the probabilities for combinations of pairs of letters. So now let's say that we try and generate text letter at a time with not just dealing with the individual probabilities of letters, but also the probabilities of pairs of letters. Okay, so now we do that and it's going to start looking a bit more, little bit more like real English text. There's a couple of actual words here like on and the. And, well, Tesla, I guess, is a word of sorts. And this is now sort of getting a bit closer to actual English text because it's capturing more of the statistics of English. We can go on, instead of just dealing with having the correct probabilities for individual letters, pairs of letters, and so on, we can go on and say let's have the correct probabilities for triples of letters, combinations of four letters, and so on. And this is actually, these numbers are probably off by one because those are really letters on their own. These are pairs of letters and so on. So this is six tuples of letters. And we can see that by the time you've got, by the time you're saying, I want to follow the probabilities for, for six tuples of letters, we're getting complete English words like average and so on. And the fact that that's how it finishes, that's why autocomplete, when you type on a phone or something like that can work as well as it does because by the time you have a V E R, there's really only, there's only a limited number of words that can follow that. And so you've pretty much determined it. And that's how the probabilities work when you're dealing with blocks of letters rather than small numbers of letters. Okay, so that's kind of the idea of sort of, you're capturing the statistics of letters, the statistics of sequences of letters, and you're using that to randomly generate kind of text like things. So we can also do that not just with probabilities of individual letters, with probabilities of words. So in English there are maybe 40 or 50,000 sort of fairly commonly used words. And we could simply say, based on some large sample from millions of books or something, what are the probabilities of those different words? And the Probabilities of different words have changed over time and so on. But let's say we. We say, let's say over the course of all books or for the current time, what are the probabilities for all those, let's say 50,000 different words? And now just start generating sentences where we picked those words at random, but with the probabilities that correspond to the frequencies with which they occur in these samples of English text. So there's a sentence we get by by that method, and it's a sentence where, well, these words are occurring with the right probability. This sentence doesn't really mean anything. It's just a collection of random words. Now we can do the same thing we did with letters. Instead of just saying we use a certain probability for an individual word, we say we correctly work out the probabilities for pairs of words based on our sample of English text and so on. We do that. It's actually a computationally already comparatively difficult thing to do this even for pairs of words because we're dealing with sort of 50,000 squared different possibilities, et cetera, et cetera, et cetera. But now let's say we start with a particular word. Let's say we start with the word cat. That's our sort of prompt here. Then these are sentences that are generated with the correct probabilities for pairs of words. So we'll see things like the book and well throughout in is a little bit bizarre, but confirmation procedure, I guess that's a pair of words that occur together a bunch at least in the place where all this text was sampled from. So this is what you get when you're sampling text sort of pairs of words at a time. This is kind of a very pre. Kind of chatgpt. This is a very sort of super minimalist version in which it's just dealing with statistics of pairs of words as opposed to the much more elaborate stuff that it's really doing. Now. You could say, well, how about to do something more like what ChatGPT does. Let's just, instead of picking pairs of words, let's pick combinations of five words or 20 words or 200 words. You know, let's ask it to, given the prompt that we've specified, let's ask it to add in the next 200 words with the probability that at the. With what you would expect based on what's out there on the web. So maybe we just make a table of what's the chance of having this three word combination, four word, five word combination. Okay, here's the problem with that the problem is there just isn't enough English text that's ever been or text of any language that's ever been written to be able to estimate those probabilities in this direct way. Well, in other words, by the time you're at, there may be 40,000 common English words. That means the number of pairs of words that you have to ask the probability of is 1.6 billion. The number of triples is 60 trillion. And you pretty quickly end up with something where you couldn't possibly. There just isn't enough text that's been written in the few billion web pages that exist and so on to be able to sample all of those 60 trillion triples of words and say, what's the probability of each one of these triples? By the time you get to a 20 word essay, you're dealing with the number of possibilities being more than the number of particles in the universe. You wouldn't even be able to record those probabilities even if you had text, you know, written by sort of an infinite collection of monkeys or something imitating humans that was able to do that. So how do we deal with this? How does chatgpt the its. How did. How does it deal with the fact that it can't sample from the web enough text to be able to just make a table of all those probabilities? Well, the key idea, which is a super old idea in the history of science, is to make a model. What is a model? A model is something where you're kind of summarizing data. You're summarizing things in a way where you don't have to have every piece of data. You can just have a model which allows you to predict more data even if you didn't immediately have it. So quintessential example, very early example of modeling was Galileo, late 1500s, trying to figure out things about objects falling under gravity and going up the Tower of Pisa and dropping cannonballs off different levels on the Tower of Pisa and saying, how long does it take for these things to hit the ground? So he could make a plot. Gosh, that's a remarkably complicated way to make this plot. Okay, could make a plot of, I don't know how many floors there actually are in the Tower of Pisa. But imagine there were this number of floors. He could make a plot and he could say, measure in those days by taking his pulse or something. How long did it take for the cannonball to hit the ground? And so this is as a function of what floor it was dropped from, how long it took the cannonball to hit the ground. So there's data about specific times for specific floors. But what if you want to know, how long would it take for the cannonball to hit the ground if you were on the 35th floor, which didn't happen to have been explicitly measured? So this is where kind of the idea of, well, let's make a model comes in and sort of a typical thing you might do is to say, well, let's just assume that it's a straight line. Assume that the time to hit the ground is a function of the floor, and this is the best straight line we can fit through that data. This allows us to predict what the time to hit the ground from a. From a floor that we didn't explicitly visit will be. So essentially, this model is a way of sort of summarizing the data and summarizing what we expect to do when we continue from this data. The reason this is going to be relevant to us is, as I mentioned, there isn't enough data to know these probabilities for different words just from actual text that exists. So you have to have something where you're making a model where you're saying, assume this is sort of how things generally work. This is how we would figure out the answer when we haven't explicitly made a measurement. So we can make different models and we'll get different results. So, for example, we could say, here's another model that we might pick. This is a quadratic curve through these particular data points. Now, it's worth realizing that there's no modelless model. You're always making certain assumptions about how things work. And in the case of these problems in physics, like dropping balls from towers and so on, we have a pretty good expectation that these sort of simple mathematical models, mathematical formulas and so on, are likely to be things that will work. Doesn't always happen that way. You know, this is another mathematical function. This is the best version. It has some parameters in this model. This is the best version of that model for fitting this data. And you can see it's a completely crummy fit to this data. So if we assume that this is sort of, in general the way things work, we won't be able to correctly reproduce what this data is saying. In the case of this model, I think it has three parameters that are trying to fit this data and doesn't do very well. And in the work ChatGPT is doing, it basically has 175 billion parameters that it's trying to fit to make a model of human language. And it's trying to hope that when it has to estimate the probability of something in human language, that it does better than this. That with its 175 billion parameters, that the underlying structure it's using is such that it's going to be able to more correctly than this, for example, estimate the probabilities of things. So let's see. All right, so the next big thing to talk about is doing things like dropping balls from Towers of Pisa and so on. That's something where we've learned over the last 300 years, since Galileo and so on, that there are simple mathematical formulas that govern those kinds of processes, physical processes in nature. But when it comes to a task like what's the most probable next word or some other kind of human like task, we don't have a simple kind of mathematics style model. So, for example, we might say, here's a typical human like task. We're asked to recognize from an array of, from an image, an array of pixels, which digit out of the 10 possibilities is this one? And so we humans do a pretty good job of saying, well, that's a four, that's a two, and so on. But we need to ask sort of, how do we think about this problem? So one thing we could say is let's try and do the thing that we were doing where we say, let's just collect the data and figure out the answer based on collecting data. So we might say, well, let's get ourselves a whole collection of fours. And let's just ask ourselves, when we are presented with a particular array of pixel values, does that array of pixel values match one of the fours that we've got in our sample? The chance of that happening is incredibly small. And it's clear that we humans do something better than that. It doesn't matter where the individual pixels fell here, so long as it roughly is in the shape of the four, we're going to recognize it as a four. So the question then is, how does that work? And what we found is that it's, well, let's say this is using. This is actually using this sort of a standard machine learning problem. This is using a simple neural net to recognize these handwritten digits. And so we see it gets the right answer there. But if we say, well, what's it really doing? Let's say we give it a set of progressively more blurred digits here at the beginning. It gets them right, then it quotes gets them wrong. What does it even mean that it gets them wrong? We know that this was a two, that we put in here and we know we just kept on blurring that 2. And so we can say, well, it got it wrong because we knew it was supposed to be a two. But if we sort of zoom out and ask what's happening at a broader level, we say, well, if we were humans looking at those images, would we conclude that that's a two or not? By the time it gets blurred enough, we humans wouldn't even know it's a two. So to sort of assess whether the machine is doing the right thing, what we're really asking is, does it do something more or less like what we humans do? So that becomes the question, is it not? We don't get to ask for these kind of human like tasks. There's no obvious right answer. It's just, does it do something that follows what us humans do? And that question of what's the right answer? Okay, for humans, we might say, well, up there, most humans would recognize that as a 2. If instead we had visual systems like bees or octopuses or something like this, we might come to completely different conclusions. Once things get sort of blurred out, the question of what we consider to be 2 like might be quite different. It's a very human answer. That, that, to say that that still looks like a 2, for example, depends on our visual system. It's not something where there's sort of a mathematically precise definition of that has to be a two. Okay, so the question is, how do these models, how do these models which we're using for things like image recognition, how do they actually work? The most popular, by far and most successful at present time approach to doing this is to use neural nets. And so, okay, what is a neural net? It's kind of an idealization of what we think is going on in the brain. What's going on in the brain? Well, we all have about 100 billion neurons in our brains, which are nerve cells that have the feature that when they get excited, they produce electrical signals maybe a thousand times a second. And each nerve cell has, it's taking that electrical signal and it has sort of wire like projections from the nerve cell that are connecting to maybe a thousand, maybe ten thousand other nerve cells. And so what happens in a sort of rough approximation is that you'll have electrical activity in one nerve cell and that will kind of get. Communicate itself to other nerve cells. And there's this whole network of nerves that is, has this elaborate pattern of electrical con. Electrical activity. So, and roughly the way it seems to work is that the extent to which one nerve cell will affect others is determined by sort of the weights associated with these different connections. And so one connection might have a very strong positive effect on another nerve cell. If the first nerve cell is fired, then it's like it makes it very likely the next nerve cell will fire. Or that connection might be an inhibitory connection where if one nerve cell fires, it makes it very unlikely for the next nerve cell to fire. There's some whole combination of these weights associated with these different connections between nerve cells. So you know what actually happens when we're trying to recognize a two in an image, for example? Well, the light, the photons from the image fall on the cells at the back of our eye, at our retina. These are photoreceptor cells. They convert that light into electrical signals. The electrical signals end up going through nerves that get to the visual cortex at the back of our head, and there's an array of nerves that correspond to all the different, essentially pixel positions in the image. And then what's happening is that within our brains, there's this sequence of connections. There's sort of layers of neurons that process the electrical signals that are coming in. And eventually we get to the point where we kind of form a thought that that image that we're seeing in front of us is a two. And then we might say it's a two. But that process of sort of forming the thought, that's what we're talking about as kind of this process of recognition. I was talking about it in the actual neural nets that we have in brains. But what is being done in all of these models, including things like ChatGPT, is an idealization of that neural net. Okay, so, for example, in. In the particular neural net we were using for image recognition, this is kind of a Wolfram language representation of that neural net. And we're going to talk about, not in total detail, but we're going to talk about all these pieces in here. It's very kind of engineering biological. There's a lot of different funky little pieces here that go together to actually have the result of recognizing digits and so on. This particular neural net was constructed in 1998, and it really was done as a piece of engineering. So how do we think about the way this neural net works? Essentially, the key idea is the idea of attractors. That's an idea that actually emerged from mathematical physics and so on. But it's a key idea when we're thinking about neural nets and such like. And so what is that idea? The idea is, let's say we've got all these different handwritten digits. They're ones, they're twos, et cetera, et cetera, et cetera. What we want is if we lay all these digits out in some way, what we want is that if we are sort of near the ones, we are kind of attracted to the one spot. If the thing we have is kind of near the twos, we're attracted to the two spot. It's kind of the idea of attractors is imagine that you have some, I don't know, mountainscape or something like this, and you're a drop of water that falls somewhere on the mountain. You are going to sort of roll down the mountain until you get to this minimum that is from your particular part of the mountain. But then there'll be a watershed. And if you're a raindrop that falls somewhere else, you'll roll down to a different minimum, a different lake. And it's the same kind of thing here. When you move far enough away from the thing that looks like a one, you'll roll down into the two's attractor rather than the one's attractor. That's kind of the idea there. Now let's see, let's make a kind of idealized version of this. Let's say we've got a bunch of points on the plane. Let's say those are coffee shops. And you say, I'm always going to go to the closest coffee shop to me. Well, this so called Voronoi diagram shows you this sort of the division, the watersheds between coffee shops. If you're on this side of this watershed, you'll go to this coffee shop. If you're on that side, you'll go to this coffee shop. So that's kind of a minimal version of this idea of attractors. All right, so let's talk about neural nets and their relationship to attractors. So let's take an even simpler version. Let's just take these three attractors. There's the zero attractor, the plus one attractor, the minus one attractor. We're going to say if we fall in this region, we'll have coordinates X and Y coordinates. So if we're in this region here, we're going to, eventually we're going to want to go to say the result is zero. We're in the zero, we're in the basin of the zero attractor and we want to produce a zero. Okay, so we can kind of say, we can say as a function of the position X and Y that we start from. What output do we want to get. Well on this side, we want to get a one, this one we want to get. What is that, a minus one there? We want to get a zero. This is the thing that we are trying to, we're trying to set up is something where we'll have this kind of behavior. Okay, well, now let's pull in a neural net. So this is a typical tiny neural net. Each of these dots represents an artificial neuron. Each of these lines represents a connection between neurons. And the kind of the blue to redness represents the weight associated with that connection, with blue being the most negative, red being the most positive here. And this is showing different, this is showing a neural net with particular choices for these weights by which one neuron affects others. Okay, so how do we use this neural net? Well, we feed in inputs at the top. We say those top two neurons got values 0.5 and minus 0.8, for example, interpreting that in terms of the thing we're trying to work with, that's saying we're at position x equals 0.5, y equals minus 0.8 in that diagram that we had drawn. So now this neural net is basically just computing a certain function of these values X and Y. And at each step, what it's doing is it's taking these weights and it's simply taking. So for this neuron here, what it's doing is it's saying, I want this weight multiplied by this value here, this weight multiplied by this value here. And then what it says is, I'm going to add those two numbers up, the numbers based on the weights multiplied by the original number. Then there's a thing we add, we add a constant offset, different offset, we add this constant offset, and then we say we get some number out. And then the kind of, the weird thing one does, which is sort of inspired by what seems to happen biologically, is we have some kind of thresholding function. We say, for example, this is a very common one to use, relu. If that total number is less than zero, make it be not its actual value, but just zero. If it's greater than zero, make it be its actual value. And there are a variety of different so called activation functions. Activation because they're what determine what the activity of the next neuron sort of down the line will be based on the input to that neuron. So here again, at every step, we're just collecting the values from the neurons at the previous layer, multiplying by weights, adding this offset, applying that activation function relu to get this value minus 3.8 in this case. And what's happening here is we start off with these values, 0.5 minus 0.8. We go through this whole neural net. In this particular case at the end, it comes out with value minus one. Okay, What does that neural net, this neural net here, the one we've just been showing, what does that do as we change those inputs? Well, we can plot it. That's what that neural net actually does. So as a function. So remember, our goal is to. Every time we have a value in this region, we want to give a zero. This region, we want to give a minus one, and so on. This is what that particular neural net succeeds in doing. So it didn't quite make it to give the 0:1 minus 1 values, but it's kind of close. So this is a neural net that's been kind of set up to be as close as it can be for one of that size and shape and so on to giving us the exact function we wanted to compute. Well, how do we think about what this neural net is doing? The neural net is just computing some mathematical function. So for the particular neural net I was showing, if the W's are the weights and the B's are the offsets and so on, the F is the activation function. This is the messy sort of algebraic formula that says what the value of the output is going to be as a function of X and Y, the values of the inputs. So now the question is, well, as we look at simpler neural nets, what kinds of functions can we actually compute? So this is at the sort of minimum level. This is a single. This is a neuron here. It's getting input from two other neurons. What function is it computing? Well, it depends on the weights. These are the functions that get computed for these different choices of weights. Very simple functions, in all cases, just these ramps. So now we can ask, well, okay, let's use a slightly more sophisticated neural net. Here's still a very small neural net. This is the best it can do in reproducing the function. We want to get. Slightly bigger neural net, does slightly better. An even bigger neural net. It pretty much nailed it. Didn't quite nail it right at the boundary. It's a bit confused. Instead of going straight from red to blue, it's got this area where it's giving yellow and so on. But in a first approximation, this little neural net was a pretty good representation of the mathematical function that we wanted to compute. And this is the same story as what we're doing in, in that recognition of digits where again, we've got a neural net, it happens to have, I don't know what it was. I think it's about 40,000 parameters in this particular case that. Specify kind of, that are doing the same kind of thing of working out the function that goes from the array of pixels at the beginning to values zero through nine and so on. Well, again, we can ask the question, you know, is it getting the right answer? Well, again, it's a hard question that's really a human level question because the question of whether it put a one in the wrong place, so to speak, it's a question of how we would define that. Well, we can do similar kinds of things. Let's say we have other kinds of images. We might try and make a neural net that distinguishes cats from dogs. And here we're showing sort of how it distinguishes those things. And mostly the cats are over in this corner, the dogs are over in this corner. But the question of what should it really ultimately do? What should it do if we put a dog in a cat suit, should it say, that's a cat, or should it say, it's a dog? It's going to say some definite thing. The question is, does it sort of agree with what we humans would assess it to be? Well, one question you might ask is, well, what's this neural net doing inside when it works out its sort of catness or its dogness? And let's say we start with let's do this and we can actually do this. Let's say we start with an image. Well, maybe let's say we start with an image of a cat here. Now we can say what's going on inside the neural net when it decides that this is actually an image of a cat. Well, what we can do, normally when we are looking at the insides of a neural net, it's really hard to tell what's happening. In the case where the neural net corresponds to an image, we can at least, at least neural nets tend to be set up so that they preserve the pixel structure of the image. So, for example, here we can go. This is just going. What is this going? This is going 10 layers down. No, this is. No, this is actually. Sorry, this is actually going just one layer down in the neural net. And what happens in this particular neural net is it takes that image of a cat and it breaks it up into a lot of different kind of variants of that image. Now at this level, we can kind of say, well, it's doing things that we can sort of recognize. It's kind of looking at cat outlines without the background. It's trying to pull the cat out of the background. It's doing things that we can sort of imagine, you know, describing in words what's going on. And in fact, many of the things that it's doing are things that we know from studying neurophysiology of brains are what the first levels of visual processing in brains actually do. By the time we're sort of deeper in the neural net, it's much harder to tell what's going on. Let's say we go 10 layers down in the neural net. Then we've got again, sort of, this is in the mind of the neural net, this is what it's thinking about to try and decide, is it a cat or a dog? Things have become much more abstract, much harder to explicitly recognize. But that's kind of what sort of a representation for us of what's happening in the kind of mind of the neural net. And, you know, if we say, well, what's a theory for how cat recognition works? It's not clear. We can have a theory in the sense of sort of a narrative description, a simple way of describing, how does the thing tell that it's a cat? You know, we can't. And if you even ask a sort of human, how do you tell? We say, well, it's got these pointy ears. It's got this and that thing. It's hard, probably, for a human to describe how they do that recognition. And when we look inside the neural net, we don't get to sort of have a. There's no guarantee that there's a sort of simple narrative for what it's doing. And typically there isn't. Okay, So we've talked about how neural nets can successfully go from a cat image to saying, that's a cat versus that's a dog. How do you set the neural net up to do that? So the way we normally write programs is we say, well, I'm thinking about how should this program work, what should it do? Should it first take the image of the cat, Figure out, does it have. What are the shape of its ears, does it have whiskers, all these kinds of things. That's sort of the typical engineering way to make a program. That's what people did back 15 years ago, 20 years ago, in trying to make sort of recognize images of things. That was the typical kind of approach was to try and recognized sort of human explainable features of images and so on as a way to kind of recognize things. The big idea of machine Learning is. You don't have to do that. Instead, what you can do is just give a bunch of examples where you say, this is a cat, this is a dog, and have it be the case that you have a system which can learn from those examples and where you just have to give it enough examples. And then when you show it a new cat image that it's never seen before, it'll correctly say, that's a cat versus that's a dog. Dog. So let's talk about how that's actually done. And what we're interested in is, can we take one of those neural nets? I showed that they're neural nets where they have all these weights, and as you change the weights, you change the function the neural net is computing. Let's say you have a neural net and you want to make it compute a particular function. So let's say, let's take a very simple case. Say we have a neural net and we just want it to compute as a function of X. We want it to compute this particular function here. Okay? So let's pick a neural net. There's a neural net without weights. Let's fill in random weights in that neural net. For every random collection of weights in the neural net, the neural net will compute something. It won't be the function we want, but it'll always compute something. It'll always be the case that when you feed in some value up here, you'll get out some value down here. And these are plots of the function that you get by doing that. Okay? The big idea is that if you do it the right way and you can give enough examples of. What function you are trying to learn, you will be able to progressively tweak the weights in this neural net so that eventually you'll get a neural net that correctly computes this function. So again, what we're doing here is this is. We're just describing if this is X, this is, let's say, you know, G of X down here. This is the value X up here. And this is a G of X for some function G. And that function G that we want is this kind of square wave type thing here. Now, in this particular case, this neural net with these weights is not computing the function we wanted. It's computing this function here. But as we progressively train this neural net, we tweak the weights until eventually we get a neural net that actually computes the function we want. This particular case, it took 10 million examples to get to the point where we have the neural net that we want, okay? So how does this actually work? How is this actually done? How does one. As I said at the beginning, we just had. We started off with neural nets where we had random weights with random weights. This function X to G of X with that particular choice of weights is this thing here, which isn't even close to what we wanted. So even if we have. When we have examples of functions, examples of results, how do we go from those to train the neural nets? Essentially, what we're doing is we say we've got this neural net. We say, let's pick a value of x 0.2, for example. Let's run it through the neural net. Let's see what value we get. Okay, we get this value here. Oh, we say that value is not correct based on what we were trying to. Based on the training data that we have, based on this function that we're trying to train the neural nets to generate that training. It isn't the correct result. It should have been, let's say, a minus one, and it was in fact a 0.7 or something. Okay, so then the idea is that knowing that we got it wrong, we can measure how much we got it wrong. And we can do that for many different samples. We can take, let's say, 1,000 examples of this mapping from value X to function G of X that the neural net computes. And we can say of those thousand examples, how far off were they? And we can compute what's often called the loss, which is take all those values of what we should have got versus what we actually got. And for example, take the sum of the squares of the differences between those values. And that gives us a sense of if all the values were right on, that would be zero. The but in fact, it's not zero because we didn't actually get the right answer with our neural net. And so then what we're trying to do is to progressively reduce that loss. We're trying to progressively tweak the neural net so that we reduce that loss. So, for example, this is what it would typically look like you typically have. This is the loss as a function of the number of examples you've shown. And what you see is that as you show more and more examples, the loss progressively decreases. The reflecting the fact that the function that's being computed by the neural net is getting closer to the function we actually wanted. And eventually the loss is really quite small here. And then the function is really computed by the neural net is really close to the one we wanted. That's kind of the idea of Training a neural net, we're trying to tweak the weights to reduce the loss to get to where we want. Okay, so let's say we've got a neural net, particular form of weights, we compute the loss, the loss is really bad, we're pretty far away. How do we arrange to incrementally get closer to the right answer? Well, we have to tweak the weights, but what direction do we tweak the weights in? Okay, so this is a tricky thing that got figured out well in the 1980s for neural nets. How to do this in a reasonably, it was known how to do this in simple cases. I should say that the idea of neural nets originated in 1943. Warren McCulloch and Walter Pitts were the two guys who wrote this sort of original paper that described these idealized neural nets. And what's inside ChatGPT is basically a big version of what was described in 1943. And there was sort of a long history of people doing things with just one layer of neural nets. And that didn't work very well. And then early 1980s, it started to be some knowledge of how to deal with more layers of neural nets. And then when GPUs started to exist and computers got faster, sort of big breakthrough around 2012, where it became possible to deal with sort of training and using sort of deep neural nets. By the way, for people who are interested, I did a discussion with a friend of mine named Teresinofsky who's been involved with neural nets for about 45 years now and has been quite instrumental in many of the developments that have happened. I did a discussion with him that was live streamed a few days ago, which you can find on the web and so on if you're interested in that history. But back to sort of how these things work, what we want to do is we found the loss is bad, let's reduce the loss. How do we reduce the loss? We need to tweak the weights. What direction do we tweak the weights in in order to reduce the loss? Well, this turns out to be a big application of calculus because basically what's happening is our neural net corresponds to a function. It's a function of the weights, it's a function of when we compute the loss, we are basically working out the value of this neural net function for lots of values of x and Y and so on. And that object, that thing we're computing is a big complicated. We can think of it as an algebraic formula that we can think of as being a function of all those weights. So how do we make the thing better? How do we reduce the overall value? How do we tweak the weights to reduce this overall loss quantity? Well, we can kind of use calculus. We can kind of say, we can think of this as sort of a surface as a function of all of these weights and we can say we want to minimize this function as a function of the weights. So for example, we might have a, in a very simplified case, this is not good. In a very simplified case, we might have. Some as a function of just two weights. So for example, in those neural nets I was just showing they had, I don't know, 15 weights or something. In the real example of an image recognition network, it might be 40,000 weights. In ChatGPT, it's 175 billion weights. But here we're just looking at two weights and we're asking if this was the loss as a function of the value of those weights, how would we find the minimum? How would we find the best values of those weights? Let's see. Oh, there we go. So this is a typical procedure to use so called gradient descent. Basically what you do is you say, I'm at this position on this lost surface, lost surface, where the coordinates of the surface are weights. What I want to do is I want to get to a lower point on this loss surface and I want to do that by changing the weights, always following this gradient vector kind of down the hill, the steepest ascent down the hill. And that's something that you just have to use calculus and you just work out the derivatives at this point as a function of these weights and the direction where you are finding the maximum of these derivatives, you're going down the hill as much as you can. Okay, so that's kind of how you try to minimize the loss is by tweaking the weights so that you follow this gradient descent thing to get to the minimum. Now there's a bit of a bug with this because the surface that corresponds to all the weights it might have, as this picture shows, it might have more than one minimum. And actually these minima might not be all at the same height. So for example, if you're on a mountainscape, there might be a mountain lake, there might be a very high altitude mountain lake, and all of the water that's kind of following Cepas ascent down to get to the minimum only manages to get to that high altitude mountain lake, even though there's a low altitude mountain lake that will be a much lower value of the loss. So to Speak that isn't reached by this gradient descent method. It's never, you, you get stuck in a local minimum, you never reach the more global minimum. And that's kind of what, what potentially happens in, in neural nets is you can be, okay, I'm going to reduce the loss, I'm going to tweak the weights, but whoops, I can't really get very far. I can't reduce the loss enough to be able to successfully reproduce my function with my neural net or whatever. I can't tweak the weights enough because I got stuck in a local minimum. I don't know how to get out of that local minimum. So this was the sort of big breakthrough and surprise of 2012 in the development of neural nets was the following discovery. You might have thought that you'd have the best chance of getting a neural net to work. Well, when it was a simple neural net, you kind of get your arms around it and figure out all these weights and do all these calculations and so on. But actually it turns out things get easier when the neural net and the problem it's trying to solve gets more complicated. And roughly, the intuition seems to be this. Although one didn't expect this, nobody, I think, expected this, I certainly didn't expect this, that it's sort of obvious after the fact. Okay, the issue is, are you going to get stuck as you try and follow this gradient descent? Well, if you're in a kind of low dimensional space, it's quite easy to get stuck. You just get into one of these mountain lakes and you can't go any further. But in a high dimensional space, there are many different directions you could go. And the chances are any local minimum you get to, you'll be able to escape from that local minimum because there'll always be some dimension, some direction you can go that allows you to escape. And that's what seems to be happening. It's not totally obvious it would work that way, but that's what seems to be happening in these neural nets is there's always sort of, when you have a complicated enough neural net, there's always a way to escape, there's always a way to reduce the loss, and so on. Okay, so that's kind of this idea of you tweak the weights to reduce the loss. That's what's going on in all neural nets. And you can, there are different schemes for how you do the gradient descent and how big the steps are. And there are all kinds of different things. There are different ways you can calculate the loss when we're doing it for language, we're calculating probabilities of words based on probabilities of sequences of words based on the model versus based on what we actually see in the data, as opposed to just distances between numbers and so on. But it's the same basic idea. Okay, so when that happens, let's see, we can potentially get, every time we run one of these neural nets, we do all this tweaking of weights and so on, we get something where, yes, we got a neural net that reproduces the thing we want. Okay, so these are the results from four different neural nets that all successfully pretty much reproduce this function. Now, you might ask, well, what happens if I go, let's see, what happens if I. Yeah, what happens if I go outside the range where I had explicitly trained the neural net? I'm telling it, I told it my function X goes to G of X for this range here, the one in white. But then I say, well, I've got the neural net. Now let me try running it for a value of X that I never trained it for. What's it going to give? Well, that will depend on which particular set of choices about which minimum, which weight, tweaking, et cetera, et cetera, et cetera, it went to. And so when the neural net tries to figure out things that it wasn't explicitly trained on, it's going to give completely different answers depending on the details of how the neural net happened to get trained. So it's kind of like it knows the things which it's already seen examples of. It's going to be constrained to basically reproduce those examples. When you're dealing with things that are kind of out of the box, it might think differently out of the box, so to speak, depending on the details of that neural net. All right, so. Let's see. This whole question about training neural nets is it's a giant modern art, so to speak, of how to train a neural net. And over the last, particularly the last decade, there's been sort of increasingly elaborate sort of knowledge of that art of training neural nets. And there's been a certain amount of lore about how these neural nets should get trained. That's. That's developed. So how does that. What's sort of in that law? Well, kind of the first question is what kind of architecture of neural net? How many neurons, how many neurons at each layer? How should they be connected together? What should you use? And there have been a number of kind of observations in sort of the art of neural nets that have emerged. So what Was believed at the beginning was every different task you want a neural net to do, you would need a different architecture to do it. You would somehow optimize the architecture for each task. It's turned out that isn't the case. It's much more that there are generic neural net architectures that seem to go across a lot of different tasks. And you might say, isn't that just like what happens with computers and universal computers? You can run different software on the same computer, same hardware, different software? That was the kind of idea from the 1930s that launched the whole computer revolution, the whole notion of software and so on. Is this a repetition of that? I don't actually think so. I think this is actually something slightly different. I think that the reason that the neural nets, the sort of a small number of architectures, cover a lot of the tasks neural nets can do is because those tasks that neural nets can do are tasks that we humans are also pretty good at doing. And these neural nets are kind of reproducing something about the way we humans do tasks. And so while the tasks you're asking the neural net to do are tasks that are sort of human, like any human, like neural net is going to be able to do those tasks. Now, there are other tasks that are different kinds of computations that neural nets and humans are pretty bad at doing. And those will be sort of out of this zone of it doesn't really matter what architecture you have. Well, okay, so there are all kinds of other things that people sort of wondered about. Like they said, well, let's make. Instead of making these very simple neurons that were just like the ones from 1943, let's make more complicated assemblies of things and let's put more detail into the internal operations of the neural net. Turns out most of that stuff doesn't seem to matter. And I think that's unsurprising from a lot of science that I've done. Not specifically related to neural nets. I think that's something that isn't too surprising. Now, when it comes to neural nets and sort of how they're architected, there are a few features that it is useful to sort of capture a few features. This is not the right thing. That's the right thing. There are a few features of the data that you're looking at with the neural net that it seems useful to capture in the actual architecture of the neural net. It's probably not, in the end, ultimately completely necessary. It's probably the case that you could use a much more generic neural net and with enough training, enough kind of sort of tweaking from the actual data, you'd be able to learn all these things. But for example, if you've got a neural net that's dealing with images, it is useful to initially arrange the neurons in an array that's like the pixels. And so this is sort of representation for the particular network called lunette that we were showing for digit recognition. This is sort of a representation of there's a first layer of neurons here, then it sort of thickens up into multiple different copies of the image which we actually saw looking at those pictures. And then it keeps going and eventually it rearranges. One thing about neural nets to understand is that neural nets take everything they're dealing with and grinds it up into numbers. Computers take everything they're dealing with and eventually grinds it up, grind it up into zeros and ones, into bits. Neural nets right now are grinding things up into arbitrary numbers. 3.72. They're real numbers, not necessarily just zeros and ones. It's not clear how important that is. It is necessary when you're going to incrementally improve weights and kind of use calculus like things to do that. It's necessary to have these continuous numbers to be able to do that. But in any case, whether you're showing the neural net a picture, a piece of text, whatever, in the end it's got to be represented in terms of numbers. And that's sort of. But how those numbers are arranged. Like for example here there's an array of numbers which are sort of arranged in the pixel positions and so on. The whole array is reconstituted and rearranged and flattened and so on. And in the end, you're going to get probabilities for each of the 10 digits that will be just a sequence of numbers here, sort of a rearranged collection of numbers. Okay, so let's see. Right picture. There we go. Okay, so this is. So we're talking about sort of how complicated a neural net do you need to achieve it to perform a particular task? Sometimes it's pretty hard to estimate that because you don't really know how hard the task is. Let's say you want a neural net that plays a game. Well, you can compute the complete game tree for the game. All the possible sequences of games that could occur might be some absolutely huge game tree. But if you want to get human level play for that game, you don't need to reproduce that whole game tree. If you were going to do very systematic computer computation and Just play the game by looking at all the possibilities. You'd need that whole game tree, but. Or you'd need to be able to go through that whole game tree. But in the case of, if you're trying to achieve sort of human like performance, the humans might have found some heuristic that dramatically simplifies it and you might need just some much simpler neural net. So this is an example of, well, if the neural net is way too simple, then it doesn't have the ability to reproduce, in this case the function we wanted. But you'll see that as the neural nets get a bit more complicated, we eventually get to the point where we can indeed reproduce the function we wanted. All right, well, okay, so, and you can ask, you know, are there theorems about what functions you can reproduce with what neural nets? Basically, as soon as you have any neurons in the middle, you can at least in principle reproduce any function, but you might need an extremely large number of neurons to do that. And it's also the case that that neural net might not be trainable. It might not be the case that you can find some, for example, gradient that always makes the loss go down and so on just by tweaking weights. It might be that you couldn't incrementally get to that result. Well, okay, so let's say you've got, you've decided on your architecture of your neural net and now you want to train your neural nets. Okay? So the next big thing is you have to have the data to train your neural net from. And there are two basic categories of training that one does for neural nets. Supervised learning and unsupervised learning. So in supervised learning, you give the neural net a bunch of examples of what you want it to learn. So you might say, here are 10,000 pictures of cats, 10,000 pictures of dogs. The pictures of cats are all tagged as being, this is a picture of a cat, this is a picture of a dog, and you're feeding the neural net these things that are kind of explicit things that you want it to learn. Now that's what one has to do. For many forms of machine learning, it can be non trivial to get the data. Often there are sources of data that, where you're sort of piggybacking on something else. Like you might get images from the web and they might have alt tags that were text describing the image and that's how you might be able to associate the description of the image, the fact this is a cat to the actual image. Or you might have, you know, if you're doing audio kinds of things. You might have something where you, you might say, let's get a bunch of videos which have closed captions. And that will give us the sort of supervised information on. Here's the audio, here's the text that corresponded to that audio. That's what we have to learn. So that's sort of one style of teaching neural nets is supervised learning, where you've got data which explicitly is examples of here's the input that you're going to get, here's the output you're supposed to give. And that's great when you can get it. Sometimes it's very difficult to get the necessary data to be able to train the machine learning system. And when people say, oh, can you use machine learning for this task? Well, if there's no training data, the answer is probably going to be no. Unless that task is something that you can either get a sort of proxy for that task from somewhere else, or you can. Or you just have to blindly hope that something that sort of was transferred from some other domain might work. Just as when you're doing mathematical models, you might sort of say, well, linear models or something worked in these places. Maybe we can blindly hope they'll work here. Doesn't tend to work that well. Okay, the other form of. No, I should explain another thing about neural nets that's kind of important, which is that this is something been very critical over the last decade or so, the notion of transfer learning. So that once you've kind of learnt a certain amount with a neural net, being able to transfer the learning that's happened in one neural net to a new neural net to give it a kind of head start is very important. Now that transfer might be for the first neural net learnt the most important features to pick out an image, let's feed the second neural net those most important features and let it go on from there. Or it might be something where you're using one neural net to provide training data for another neural net or you're making them compete against each other. A variety of other things like that. Those are actually. Those have different names. The transfer learning thing is mostly the first thing I was talking about. Okay, so there are issues about how do you get enough training data. How many times do you show the same example to a neural net? You know, it's probably a little bit like humans, for us, when we memorize things, it's often useful to go back and just rethink about that exact same example that you were trying to memorize before. Again, so it is with neural nets. And there's also questions like, well, you know, you've got the image of a cat that looks like this. Maybe you can get what is the equivalent of another image of a cat just by doing some simple image processing on the first cat. And it turns out that that seems to work. That notion of data augmentation seems to work surprisingly well. Even fairly simple transformations are almost as good as new in terms of providing more data. Well, okay, the sort of a. The other big form of learning methodology that one tends to use is unsupervised learning, where you don't have to explicitly give sort of thing you got as input, example of output. So, for example, in the case of just trying to keep track of. Yeah. In the case of something like ChatGPT, there's a wonderful trick you can use. Let's say ChatGPT's mission is to continue a piece of text. Okay, how do you train it? Well, you've just got a whole bunch of text and you say, okay, you know, ChatGPT Network, here's the text up to this point. Let's mask out the text after that point. Can you predict what's going to come? Can you learn to predict what happens if you take off the mask? And that's the task that you don't have to explicitly give it input output. You're implicitly able to get that just from the original data that you've been provided. So essentially, what's happening when you are training the neural net of ChatGPT is you're saying, here's all this English text. It's from billions of web pages. Now look at the text up to this point and say, can you correctly predict what text will come later? Okay, it gets it wrong. You can say, well, it's getting it wrong. So that provides. That means there's some loss associated with that. Let's see if we can tweak the weights in the neural net to get it closer to correctly predicting what's going to come next. So in any case, the end result of all of this is you make a neural net. I could show you neural net training in. I could show you in morpheme language. It's very easy to train neural nets to. Let's see, maybe I shouldn't do the spell. Let's see. Let's just do one. So here's a collection of handwritten digits. This is. What is this going to be? This maybe 50,000 handwritten digits. Oh, there we go. So this is a supervised training story where here are all the zeros. And they say that's a handwritten zero. And it says it's a zero. Those are the nines. It says it's a nine. Okay, so let's take a random sample of, I don't know, 2,000 of those. And now we're going to use that. Okay, there's our random sample of 2,000 handwritten digit and what it was supposed to be. Okay, so let's take it. Let's get a neural net. Let's say. Let's try taking this Lynette neural net. This is now a. An un. An untrained neural net. And now we can just say, if we wanted to, we should be able to say, just train that neural net with this data. Is that data there? Line 32? Um, let's say. Train this. And so what's going to happen is this is showing us the loss, and this is showing us, as it's being presented with more and more of those examples, and it's being shown the same example many, many times. You'll see the loss is going down and it's gradually learning. Okay, now we have a trained neural net. And now we could go back to our original collection of digits. Let's close that up. Let's go back to our original collection of digits. Let's pick a random digit here. Let's see whether from. Let's just pick. Let's just pick another random sample here. Just pick five examples there from. Oh, I should have not told it to do that. Okay, there we go. So now we can take this trained neural net. Here's our trained neural net. And let's take the trained neural net and let's feed it that particular nine there. Now, remember, we only trained it on 2,000 examples, so it didn't have very much training, but. Oops, I shouldn't have done that. I should have just used that. Okay. Okay. It successfully told us it was a nine. That's kind of what it looks like to train. This is, you know, Wolfram language version of training a neural net. This was a super simple neural net with only 2,000 examples. But that's kind of what it looks like to do that training. Okay, so let's see. The thing with ChatGPT is that you're. Well, let's. Yeah, we can keep going and talk about the training of that. But let's. Before we get to training of ChatGPT, we need to talk about one more thing, which is we need to talk about this question of kind of. Well, let's see. Do we really need to talk about this. Yeah, we probably should talk about this. The question of how you represent kind of things like words with numbers. So let's say we are going to have, we've got all these words and we could just number every word in English. We could say Apple is 75, pair is 43, et cetera, et cetera, et cetera. But there's more useful ways to label words in English by numbers. And the more useful way is to get collections of numbers that have the property that words with nearby meanings have nearby collections of numbers. So it's as if we're placing every word somewhere in some meaning space and we're trying to set it up so that words will have a position in meaning space with the property that if two words are nearby in meaning space, they must mean close to the same thing. So here, for example, are a collection of words laid out in one of these meaning spaces. Sort of actual meaning spaces like the one used by ChatGPT are like, what is that one? It's probably 12,000 dimensional. Maybe this one here is just two dimensional. We're just putting things like dog and cat, alligator, crocodile, and then a bunch of fruits here. And the main thing to notice about this is that things with similar meanings like alligator and crocodile wind up nearby in this meaning space. And, you know, peach and apricot wind up nearby in meaning space. So in other words, we're representing these words by collections of numbers. In this case just pairs of number, just coordinates which have the property that those coordinates are some kind of representation of the meaning of these words. And we can do the same thing when it comes to images. For example, we could ask whether when we looked at, and that's exactly what we had when we were looking at a picture like this, we're sort of laying out different handwritten digits in some kind of meaning of the handwritten digit space, where in that meaning space, the ones that mean one were over here, the ones that mean three were over here, and so on. So a question is, how do you find, how do you actually generate coordinates that represent the so called embeddings of things so that when they're sort of nearby in meaning, they will have nearby coordinates? Okay, so there's a number of neat tricks that are used to do this. So a typical kind of setup is imagine we have, this is just a representation of the neural net that we use to recognize digits. It has these multiple layers, each one is just little Wolfram language representation of that. What actually does this network do? Well, in the end, what it's doing is it's taking that collection of pixels at the beginning. And in the end what it's doing is it's computing what are the probabilities for a particular configuration. It's going to produce a collection of numbers at the end. Because remember neural nets, all they ever deal with are collections of numbers. So what it's going to do is it's going to produce a collection of numbers at the end where each position in this collection of numbers, there'll be 10 numbers here, each position is the probability that that the thing that the neural net was shown corresponded to a 0, a 1, a 2, a 3, a 4. So what you see here is the numbers are absurdly small, except in the case of four. So we can then deduce from this immediately, okay, that image was supposed to be a four. So this is kind of the output of the neural net is this collection of probabilities where in this particular case it was really certain that the thing is a four. So that's what we deduce. Now the thing we can do is we say, well, let's back up one layer in the neural net before we get to that. Let's just say before we had. There's a layer that kind of tries to make the neural net actually make a decision. I think it's a softmax layer that is at the end that's trying to sort of force the decision. It's trying to exponentially pull apart these numbers so that the big number gets bigger and the small numbers get smaller. Okay, but one layer before those numbers are a bit more sober in size, before it's been sort of torn apart to make a decision, those numbers are much more sober in size. And these numbers at this layer give some pretty decent indication of the foreness of what we're seeing. This has more information about what that thing that was shown actually is. And we can think about these numbers as giving some kind of signature, some kind of trace of what kind of a thing we were seeing. This is sort of specifying in some sense features of what we were seeing that later on we'll just decide that's a four. But all these other sort of subsidiary numbers are already useful. We go back so, you know, this is. We can define these feature vectors that represent. This is kind of a feature vector representing that image there. That's the feature representing this image here. And we see that, yeah, the features for different fours, these vectors will be a little bit different, but they're dramatically different between a 4 and an 8. But we can use these vectors to represent kind of the important aspects of. Of this four here, for instance. And if we go back a couple more layers in that neural net, it turns out we can get an array of like 500 numbers that are a pretty good representation, a pretty good sort of feature signature of any of these images. And we do the same thing for pictures of cats and dogs. We can get this kind of signature of what this sort of feature vector associated with what is important about that image. And then we can take those feature vectors and we can say, let's lay things out according to different values in those feature vectors. And then we'll get this kind of embedding in the case of what we can think of as some kind of meaning space in the case of words, if we look at the raw. Yeah. So how do we do that for words? Well, the idea is just like for getting sort of a feature vector associated with, let's say, images, we have a task like we're trying to recognize digits and then we back up from the final answer. We're training a neural net to do that task. But what we end up doing is we back up from that final. We nailed the task and we say, what was the thing that was just before you managed to nail the task? That's our representation of the relevant features of the thing. Or you can do the same thing for words. So, for example, if we say the blank cat and we then ask in our training data, what is that blank likely to be? Is it black, is it white, Whatever else that we could try and make a network that predicts what is that intermediate word likely to be? What are the probabilities for that intermediate word? We can train a network to be good at predicting the probabilities for blackness versus whiteness versus whatever other tabbiness for cats or whatever it is. And once we've got that, we can then back up from the final answer and say, let's look at the innards of the network and let's see what it had done as it got towards coming up with that final result, that thing we get right before it gets to a little bit before it gets to the final result. That will be a good representation of features that were important about those words. And that's how we can then deduce what we can deduce these sort of feature vectors for words. So in the case of GPT2, for example, we can compute those feature vectors. They are extremely uninformative when we look at them in the full feature vectors. What is more informative if we sort of project these feature vectors down to a smaller number of dimensions will discover that the cat one is closer to the dog one probably than it is to the chair one. But that's kind of. So what ChatGPT is doing when it deals with words is it's always representing them using these feature vectors, using this kind of embedding that turns them into these collections of numbers that have the property that nearby words have similar representations. Actually, I'm getting a little bit ahead of myself there because the way ChatGPT works, it uses these kinds of embeddings, but it does so for whole chunks of text rather than for individual words. We'll get there. Okay, so I think we're getting on fairly well here. How about the actuality of ChatGPT? Well, it's big neural net, millions of neurons, 175 billion connections between them. And what is its basic architecture? The. This sort of a big idea that actually came out of language translation networks where the task was start from English, end up with French or whatever else was this idea of what are called transformers. It's an architecture of neural nets. There were more complicated architectures used before. This is actually a simpler one. And the notion is, as I mentioned, when one's dealing with images, it's convenient to have these neurons kind of attached to pixels, at least to sort of laid out in a kind of which pixel is next to which pixel kind of way. Those are so called convolutional neural nets or convnets are the typical things that are used there. In the case of language, what transformers do is they are dealing with the fact that language is in a sequence and with a convnet for an image. One's saying there's this pixel here, what's happening in the neighboring nearby pixels in the image in a transformer. What one's doing is one saying this is. Here's a word, let's look at the preceding words, let's look at the words that came before this word. And in particular let's pay attention differently to different ones of those words. So I mean, this gets quite elaborate and engineering y quite quickly. And you know, it's, it's. It's very typical of a sophisticated engineering system that there's lots of detail here. And I'm not going to go into much of that detail. But, but this is a piece of the, this is sort of the, in a sense the front end of, of. Okay, so Remember, what is ChatGPT ultimately doing? It's a neural net whose goal is to continue a piece of text. So it's going to, it's going to essentially ingest the piece of text so far, reading in each token of the text. The tokens are either words or pieces of words. Like things like the ing at the end of a word might be a separate token. They're sort of convenient pieces of words. They're about 50,000 different possible tokens. It's reading through the text, the prompt that you wrote, the text that it's generated so far, it's reading through all of those things it is then going to generate. Its goal is to then continue that text. In particular, it's going to tell you every time you run through this whole neural net, it's going to give you one new token. It's going to tell you what the next token should be or what the probabilities for different choices of the next token should be. So one piece of this is the embedding part where what's happening is it's reading a token and it is doing. I mean, this is just, you know, it gets into a lot of detail here. So for example, let's say that the sequence we were reading was hello, hello, hello, hello, hello, bye, bye, bye, bye, bye. This is showing the resulting. This is showing the embeddings that you get. Okay, this is showing you're trying to represent. I said before we were talking about embeddings for words. Now we're talking about embeddings for whole chunks of text. And we're asking what is the sequence of numbers that should represent that collection of that piece of text? And the way you set that up, I mean, again, this is getting pretty deep into the entrails of the creature. And well, what you can think of is there are, are different components to this embedding vector. And let's see, what am I doing here? This picture is showing across the page, it's showing the contribution from each word and down the page it's showing the different pieces of the feature vector that are being built up. And the way it works is to. It takes each word and then the position of the word is encoded by a. You could just encode it by saying the position of the word as a binary digit that says is word number seven. It's you know, 00, 0111 or something. But that doesn't work as well as essentially learning this sort of random looking collection of things which are essentially position tags for words. Anyway. The end result is you're going to make this thing that represents where you have both where each level is a different sort of feature associated with each of these words. And that's the thing that's going to be fed into the next level of the neural net. Okay, so the next big piece is so called attention block, in which I don't know how much this is worth explaining. I talk about this a bit more in the thing that I wrote, but essentially what's happening is in the end it's just a great big neural net. But that neural net doesn't have every possible connection in it. It has connections, for example, only connections that look back in that look to places that were earlier in that text. And it is in a sense concentrating differently on different parts of that text. And you can make a picture here of the amount of attention that it is paying. And by attention I mean it's literally the number, the size of effectively the weights that it's using with which it is weighting different parts of the sequence that came in. And the way it works, I think for GPT3, what it does is it. So first of all, it has this embedding vector which for GPT3 is about. Is12288. I don't know why it's that particular. Oh, I do know why it's that number. It's multiples of things long and it's taking. It's trying to put together an embedding vector to represent the text so far in which it has had contributions from words at different positions and it's figured out how much contribution it should get from words at each different position. Well, okay, so it does that. Then it feeds the whole thing to a layer of neural net where sort of it has. It's a. What is it? It's a 12,000 by 12,000 array which specifies where there are 12,000 by 12,000 weights which specify for each incoming neuron, each neuron that's incoming has this weight to this outgoing neuron. And the result is you get this whole assembly of weights which looks like nothing in particular, but these are weights that have been learnt by ChatGPT to be useful for its task of continuing text. And you know, you can play little games, you can try and visualize those weights by kind of making moving averages. And you can kind of see that the weights sort of roughly are kind of like randomly chosen. But this is kind of showing you a little bit of the detail inside that randomness. And in a sense, you can think of this as being sort of a view into the brain of the. Of chatgpt, of showing you at the level of these individual weights that are in this neural net, what its representation of human language is right down the level. It's kind of like you take apart a computer and you look at individual bits inside the cpu. This is kind of the same sort of thing for the representation that ChatGPT has of language. And turns out there isn't just one of these attention layers. Okay. What happens is the different elements of the feature vector for the text get. There are different blocks of that feature vector that get separated out and handled differently. Nobody really knows what the interpretation of those blocks is. It's just been found to be a good thing to do to not treat the whole feature vector the same, but to break it into blocks and treat blocks of pieces in that feature vector differently. Maybe there's an interpretation of one piece of that feature vector that this is, I don't know, words that are about motion or something. It won't be anything like that. It won't be anything as human understandable as that. It's kind of like a human genome or something. All the traits are all mixed up in the specification. It's like what. It's not something where we can easily have a sort of narrative description of what's going on, but what's been found is that you break this kind of feature vector of features of the text up and you have these separate attention heads that have this sort of reweighting process going on for each one. You do that and this is where this is like. It's crazy that things like this work, but you do that, let's see, 96 times for ChatGPT, you're doing the same process 96 times over. And this is for GPT2, the simpler version. This is kind of a representation of the things that come out of these attention layers, attention blocks, what the sort of weights that were used there were. And these may look. There is some regularity. I don't know what it means, but if you look at the size of the weights, they're not perfectly. For some layers they're Gaussian distributed, for some layers they're not. I have no idea what the significance of that is. It's just a feature of what ChatGPT learnt as it was trying to understand human language from the web. So, okay, the. So again, there's, you know, we've talked about kind of what's the. In the end, what's happening is it's just a great big neural net and it's being trained from. We're trying to deduce the weights for the neural net by showing it a whole bunch of text and saying, what weights do you have to have in the neural net. So that the continuation of the text will have the right probabilities for what word comes next? That's its goal. And so I've sort of described the outline of how that's done. In the end, one has to feed it. The reason it's sort of even possible to do this is that there's a lot of training data to feed it. So it's been fed a significant fraction of what's on the web. There are maybe, I don't know, depends how you describe this. But there are maybe 6 billion, maybe 10 billion kind of reasonably human written pages on the web where humans actually typed that stuff. It wasn't mostly machine generated, et cetera, et cetera, et cetera. That's on the publicly visible web. Not having programs go in and not selecting lots of different things and seeing what you get, that's just kind of raw what's on the web page. Maybe there's 10, maybe 100 times as much as that, if you were able to make selections, to drill down to go into internal web pages, things like this. But so you've got something like some number of billions of human written pages. And there's a convenient collection called Common Crawl that's got where one goes, where you start from one web page, you follow all the links, you collect all those pages, you keep going just following links, following links until you've visited all the connected parts of the web. But the result of this is there's a trillion words of text that you can readily get from the web. There are probably 100 million books that have been published. Maybe 100. I think the best estimate is maybe 130 million books that have been published, of which 5 or 10 million exist in digitized form. And you can use those as training data as well. And that's another hundred billion or so words of text. So you've got trillion ish words of text. And there's probably much more than that if you have the transcriptions of videos and things like this. You know, for me personally, I've kind of been, you know, as a kind of a personal estimate of these things, I realized that the things I've written over my lifetime constitute about 3 million words. The emails I've sent over the last 30 years are another 15 million words. And the total number of words I've typed is around 50 million. Interestingly, in the live streams I've done just in the last couple of years, I have spoken another 10 million words. So it gives a sense of what human output is. But the main point is there's a trillion words available that you can use to train a neural net to be able to do this task of continuing from things. It's. Let's see, in. Right. So the actual process of one thing to understand about training a neural net, there's sort of a question. Okay, there's a question. When we looked at those functions before and we said, how many neurons do we have to have to represent this function? Well, how many training examples do we have to give to get the neural net trained to represent that function? In those cases, we didn't need very big neural nets. We needed a lot of training examples. There's all kinds of effort to understand how many training examples do you actually need? How big a neural net do you actually need to do something like do this text translation thing? Well, well, it's not really known, but, you know, with 175 billion weights, the sort of. The surprise is that ChatGPT does pretty well. Now you can ask the question, what, what's the. How much training does it need and how many times does it have to be shown those trillion words? What's the relationship between the trillion words and the number of weights in the network? And it seems to be the case that for text, that the number of weights in the network is sort of comparable to the number of training examples. You sort of show it the training examples about once. If you show it too many times, it actually gets worse in its performance. That's very different from what happens when you're training for mathematical functions and things like this. But one of the things that's an issue is that if you're. Every time. I should say every time. I should explain, by the way, that every time the neural net runs, what's happening is you are giving it, in the case of ChatGPT, you're giving it this collection of numbers that represents the text it's gotten so far. And then that collection of numbers is the input to the neural net. Then you sort of ripple through the neural net layer after layer after layer. It's got about 400 layers, sort of core layers. It ripples through all those layers, and then at the end you get some array of numbers. That array of numbers actually are probabilities for each of the 50,000 possible words in English. And that based on that, it then picks to the next word. So the main operation of ChatGPT is a very. Just straight through. You've got this text so far, given that percolate through this network, say what the next result should be, it just runs through one time. It's actually very different from the way computers tend to work for other purposes. Most non trivial computations, you're taking the same piece of computational material, the same piece of data, and you compute on it over and over and over again in sort of simple models of computation like Turing machines. That's what's happening all the time. That's what's happening. That's what makes computers able to do the non trivial things computers do is that they are taking sort of maybe a small number of pieces of data and they're just reprocessing things over and over again. What's happening in something like ChatGPT is you've got this big network, you just percolate through it once for every token. The only sense in which there's any feedback is that once you get an output, you add that token to the input, you feed it on the next step. So there's kind of an outer loop where you're giving feedback by adding tokens to the text, then that percolates through, then you get another token that percolates through. So it's a very big outer loop is probably the case certainly in computers. In lots of non trivial computations that we do, there are lots of inside loops that are happening. Quite possibly in the brain there are inside loops that are happening as well. But the model that we have in ChatGPT is this kind of just percolate through once kind of model with a very complicated network, but it's just percolating through once. So that's how it works. But one of the things that's tricky is that every time it percolates through, it has to use every single one of those weights. So every token that ChatGPT is producing, it's essentially doing 175 billion mathematical operations to see how to use each of those weights to compute the results. Most likely that's not necessary actually, but we don't know how to do any better than that right now. But that's what it's doing. So every time, if it has, it's percolating through, doing that. Well, when you train CHAT GPT and you are sort of, you're working out, you know, how do you deal with, oh, making the weights change based on the loss, that's another. You're kind of, every time you, you make a training step, you're having to kind of do a reverse version of that forward, so called Inference process. It turns out that reverse process isn't that much more expensive than the forward process, but you have to do it a whole lot of times in the training. So typically, if you have a model of size N roughly for text, it seems like you need about N squared amount of computational effort to do the training. And N is pretty big for the case when you're dealing with sort of language and things of the size of ChatGPT. And so the training process, that just a little bit mathematical square is a really big deal. And it means that to spend hundreds of millions of dollars potentially on doing the training with current GPUs and things like this is what you have to think about doing based on the current model of how neural nets work. Now, I mean, I have to say that there's a lot of aspects of the current model that probably aren't the final model. And we can plainly see that there are big differences between, for example, things the brain manages to do. For example, one big difference is most of the time, when you're training a neural net, most of the memory, and you're doing that by having you have a bunch of things in memory and you have some computation that's going on, but the things that are in memory are mostly idle most of the time. And there's just a little bit of computation that's going on in brains. Every one of our neurons is both a place that stores memory and a place that computes. It's a different kind of setup. And we don't know how to do neural nets training. There are various things that have been looked at from the distant past, actually, about how to do this. Even from the 1940s, people were starting to think about distributed ways to. To do learning in neural nets. But that's not something that's landed yet as a thing we can do. Okay, case of ChatGPT. An important thing was, and this was something, you know, six months ago, a year ago, there were kind of early versions of the GPT family, text completion systems and so on. And they were kind of. The text they produced was only so. So. And then something was done by OpenAI with ChatGPT, which was that there was an additional step, a reinforcement learning training step that was done where essentially what was done was humans told ChatGPT, go and make an essay. Go and be a chatbot. Have a conversation with me. And the humans rated what came out and said, well, that's terrible, that's better, that's terrible. Et cetera, et cetera, et cetera. And the thing that was done was then that little bit of poking turns out to have, seems to have had a very big effect. That little bit of kind of human guidance of, yes, you got the thing from the statistics of the web. Now when you look at what you got, this direction you're going in is a bad direction to go in. It's going to lead to a really boring essay or whatever else. And so that kind of. And by the way, there's a place where a lot of kind of complication about, well, what do the humans really think the network should be, that the system should be producing? If the humans say, we really don't want you to talk about this, we really don't want you to talk about that, that's the place that gets injected is in this reinforcement learning step at the end. But what you do is, for example, is given that sort of the way the humans poked at those essays, you can watch what they did when they poked at those essays and rated what happened and so on. And you can try and machine learn that set of things that the humans did. Then you can use that to provide much more training data to then retrain a piece of this, do retraining of the network based on the tweaking that the humans did. You can do fine tuning of this network based on the particular poking that the humans did, turned into another network that can then be used to do the training, to produce the examples, to do the training of the main network. So that's a thing that seems to have had a big effect on the actual sort of human perception of what happens in ChatGPT. And I think the other thing that is sort of a surprise is that you can give it these long prompts in which you tell it all kinds of things, and it will then sort of make use of that in a rather human kind of way in generating the text that comes later. Okay, big question is how come this works? Why is it that a thing with only, you know, 100 billion or so weights or something can reproduce this sort of amazing thing that seems to require all of the sort of depth of human thinking and brains and things like that, human language, how does that manage to work? And I think the key thing to realize is what it's really telling us is a science fact? It's telling us there's more regularity in human language than we thought there was. It's telling us that this thing that is human language has a lot of structure in it. And what it's done is it's learnt a bunch of that structure and it's learned structure that we never even really noticed was there. And that's what's allowing it to generate these kind of plausible pieces of text that are making use of the structure we know. So we know certain kinds of structure that exists in language. We know the. So for example, here's an example. So one piece of structure that we know share this again, One piece of structure we know is grammatical syntax, the syntactic grammar. We know that sentences aren't random jumbles of words. Sentences are made up with nouns in particular places, verbs in particular places. And we can represent that by a parse tree in which we say, here's the whole sentence. There's a noun phrase, a verb phrase, another noun phrase. These are broken down in certain ways. This is the parse tree. And in order for this to be a grammatically correct sentence, there are only certain possible forms of parse tree that correspond to a grammatically correct sentence. So this is a regularity of language that we've known for a couple of thousand years. It's only really been codified. It was big effort to codify it in 1956, but it was sort of known, this general idea was known for a long time, but then that we can kind of represent the sort of syntactic grammar of language by these kinds of rules that say you can put nouns only together with verbs in this way and that way and to any set of rules. And this has been a big source of controversy in linguistics. To any set of rules you can define, there'll always be some weird exception where people typically say this rather than that. But if you, you know, it's at the. Much like happens in typical machine learning, you know, if you're interested in the 95% result, then there are just rigid rules and there are a few exceptions here and there. Okay, so that's one form of regularity that we know exists in language is, is this syntactic regularity. Now, one thing we can do, we can ask for sort of ChatGPT has effectively implicitly learnt this syntactic grammar. Nobody ever told it verbs and nouns go this way and that way. It implicitly learned it by virtue of seeing a trillion words of text on the web which all have these properties. And when it's saying, well, what are the typical words that follow? Well, it's going to be words that followed in the examples it had, and those will follow mostly correct grammar. Now, we can take a simpler version of this. We can just. To understand what's going on, we can take a Very, very trivial grammar. We can take a grammar that's just a parenthesis, just open and close parentheses, and something is grammatically correct if we open parentheses, and they always eventually close. And this is a parse tree for a parenthesis. Open, open, open, close, open, close, et cetera, et cetera, et cetera. This is the parse tree that sort of shows how you can. It's a representation of the sort of. The parsing of this sequence of open and closed parentheses. Okay, so we might say, well, can we train a neural net to. What would it take to train a neural net to know even this particular kind of syntactic grammar? So we looked at a simple. How big was it? It was pretty small. Okay. We made a transformer net with 8 heads and length 128. So. But our thing was a lot simpler than ChatGPT. But you can use one of these transformers. And if you look at the post I made that there's. The actual transformer is there? And you can play with it in Wolfram language. But in any case, if you give that transformer this sequence here and you say what comes next? It says, okay, well, 54% probability that there's a closed paren there based on. Oh. Its training data was a randomly selected collection of correct open, close, open, close parenthesis sequences. It has a little bit of a goof here because it says with 0.0838% probability this is the end of the sequence. Which would of course be grammatically incorrect because there's no close for the open parentheses here. If we give something which is correctly closing, then it says, okay, Great, there's a 34% probability this is the end of the sequence. There were no further opens here. It has a little bit of a goof here because it says 15% probability there's a closed parenthesis that should occur here, which can't possibly be right because if we put a closed parenthesis here doesn't have a corresponding open parenthesis. It's not grammatically correct. But in any case, this gives a sense of what it takes for one of these transformer nets. We can look inside this transformer net. We can see sort of what it took to learn this very simple grammar. ChatGPT is learning the much more complicated grammar of English. It's actually easier probably to learn the grammar of English because there's so many clues in the actual words that are used to how they're grammatically put together, and there's so many things that we humans wouldn't even notice as wrong in Some sense of wrong, because they're kind of just what we do. But in this more austere case of just this sort of mathematically defined parenthesis language, we do notice. So if we just give it a bunch of open paren, open paren, et cetera, and we ask it, what's the highest probability continuation? You'll see it does pretty good up to this point and then starts losing it. And it's kind of a little bit like what would happen with humans. We can tell at some point here that just by eye that these are correctly closed. It becomes more difficult to tell that when we get out here, and it becomes more difficult for the network to tell that too. And this is a typical feature of these neural nets that with these sort of shallow questions of, oh, you just have, you know, you can just see this block of things and you see another block of things. It does fine when it has to go to much greater depth. It doesn't work so well For a sort of regular computer that can do loops and things inside. It's very easy to figure out what's happening here because you effectively just count up the number of open parens, count down the number of closed parens, and so on. By the way, if you try this in actual chatgpt, it also, it will confidently assert that it's matched parentheses, but it will often be wrong for larger parenthesis sequences. It has the exact same problem. It fails at a slightly larger size, but it's still going to fail. And that's just a feature of this kind of thing. So. Well, okay, so one type of regularity in language that ChatGPT has learned is syntactic grammar. Another type of regularity, there's one more that you can readily identify, and that's logic. And what is logic? Well, originally when logic was invented, you know, by Aristotle, so far as we know, you know, what Aristotle did was effectively a bit like a machine learning system. He looked at lots of examples of rhetoric, lots of example speeches people gave. He said, what are some forms of argument that appear repeatedly? If somebody says something like people might have said, all men are mortal. Socrates is a man, therefore Socrates is mortal. All, all X is a Y, Z is an X, therefore Z is a Y. That logic is taking sort of forms of language and saying, these are patterns that are repeated, possible patterns in these pieces of language that are meaningful sequences. And originally in syllogistic logic, which is what Aristotle originally invented, it really was very language based. And people would memorize. In the Middle Ages, people would memorize these Forms of syllogism, the barbarous syllogism, the syllogism, and so on, which were just these patterns of word usage where you could substitute in a different word for Socrates, but it was still that same pattern, that same structure. So that's kind of another form of regularity. And when ChatGPT says, oh, it's figuring things out. Well, part of what it's figuring out is it knows syllogistic logic because it's seen a zillion examples, just like Aristotle had presumably seen a bunch of examples when he invented logic. It's seen a bunch of examples of this sentence, follows this sentence in this way. And so it's going to do that too. And it says, what's the statistical thing that's going to happen based on the. Based on the web? And so that's. So, by the way, when logic developed by the 1800s, when people like Boole were getting into the picture and making formal logic, it was no longer just these patterns. Boom. It's a pattern that looks like this. It was more this thing. You could build up many, many layers of structure and you could build very complicated logical expressions where the whole thing was deeply nested. And of course, our computers today are based on those deeply nested logical expressions. ChatGPT doesn't stand a chance of decoding what's going on with one of those deeply nested, kind of mathematical, computational style, Boolean expressions, but it does well at this kind of Aristotle level kind of structure of, sort of templated structure of logic. Okay, well, I wanted to talk just for a little bit and then we should wrap up here and I can try and answer some questions about kind of. So what are the regularities that ChatGPT has discovered in this thing that we do, which is language, and all the thinking that goes on around language And I don't know the answer to this. I have some ideas about what's going on. I'll just give a little bit of a tourist. We talked about kind of meaning space, this sort of space of how words arrange in some. How you can arrange words in some kind of meaning space. And we can kind of see how words arrange. These are different parts of speech. For a given word. There may be different places in meaning space where different instances of that word occur. This is the word crane, and this is different sentences. There are two obvious meanings of crane, you know, the bird and the machine. And they sort of break up in meaning space. Where they are. We can look at the sort of structure of meaning space. Another thing we can ask is, is meaning space like physical space? Is it the case that there are parallel lines in meaning space? Are there things where we can go from place A to place B, and we. And then in parallel, we transport to new places? Well, so we can ask, if we have analogies, is it the case that we can go from woman to man, from queen to king, that those are sort of parallel paths in meaning space? The answer is, well, maybe a bit, not very convincingly. That's really the question in space, in physical space. This is the question of whether this is like flat space. It's like if we have things moving in flat space, Newton's first law says if the thing is not acted on by a force, it'll just keep going in a straight line. Well, then we have gravity, and we can represent gravity by talking about the curvature of space here. This question is, when we go from ear to here, eye to C, those are sort of, we're moving in a certain direction in meaning space. And in a sense, the question of whether these things correspond to whether we can do this kind of parallel transport idea is something like, how flat is meaning space? How much effective gravity is there in meaning space? Or something like that? Meaning space is probably not something that's represented in terms of the kinds of things that physical space is represented in terms of. But that's a question. So now when it comes to the operation of ChatGPT, we can think about how is it moving around in meaning space? It's got its prompt. The best thing about AI is its ability to. And that's the prompt, moving around in meaning space, effectively. And then what ChatGPT does is it continues that by continuing to move in meaning space. And so the question is, is there something like a semantic law of motion, an analog of kind of the laws of motion that we have in physical space, in the meaning space of concepts, words, something where we can say, okay, if it's moved around this way, it's like it's got momentum in this direction and meaning space is going to keep going in that meaning space. It's nothing like that simple. But the question is, how do we think about how do we represent kind of the sort of the process of going through meaning space? Well, we can start looking at that. We can say, for example, the different possible continuations that we get. The best thing about AI's ability to. And then what's the next word? Well, we can look at this kind of fan of different directions that it could go in meaning space at that point, and we can Kind of see, there's some direction in meaning space. It tends to go in that direction. It's not going all the way over here, at least not with high probability. Okay, well, if we keep going, we can kind of just see sort of how that fan develops as we go further out, as we continue that sentence. And we can kind of. This is kind of like our motion in meaning space kind of question. And you know, I don't know what this exactly means yet, but this is kind of what it looks like, what the trajectory in meaning space as ChatGPT tries to continue a sentence looks like. The green is the actual thing it chose. I think this is a zero temperature case, and the gray things are the things that were lower probability cases. So that's kind of what, that's a view. If we want to look, we don't want to do Natural Science on ChatGPT and say, what did it discover? What did it discover about how language is put together? One possibility is that there are these sort of semantic laws of motion that describe sort of how meaning, how you move through the space of meanings as you add words into a piece of text. I think a slightly different way to think about this is in terms of what one could call semantic grammar. So syntactic grammar is just about nouns, verbs, things like that, parts of speech, things of that kind. But we can also ask, is there a generalization of that that is sort of more semantic, that doesn't just look at, that has finer gradations than just saying, it's a noun, it's a verb, and says, oh, well, that verb means motion. And when we put this noun together with this noun, that's a thing you can move together with this motion word. It does this. We kind of have buckets of meaning that are finer gradations than just parts of speech, but not necessarily individual words. Is there a kind of a semantic grammar that we can identify that is kind of this construction kit for how we put together not just sentences that are grammatically correct, that are syntactically grammatically correct, but sentences which are somehow semantically correct? Now, I strongly think this is possible, and it's sort of what Aristotle was going for. He even talks about categories of sort of semantic categories and things like this. He talks about a variety of things. He does it in a way that's based on the fact that it was 2,000 years ago and we didn't know about computers and we didn't know about a lot of kinds of formal things that we know about. Now, strangely enough, the amount of work that's been done trying to make kind of a semantic grammar in the last 2000 years has been rather small. There was a bit of an effort in the 1600s with people like Leibniz with his Characteristica universalis and various other people trying to make what they called philosophical languages, sort of language word independent ways of describing meaning. And then they're more recent efforts, but they've tended to be fairly specific, fairly based on linguistics, and fairly based on the details of structure of human language and so on. And I think this idea that you can kind of have a semantic grammar and that that's what's sort of being discovered is that there are these rules that go beyond that are just rules for how you put together a meaningful sentence. Now, you know, you can get a meaningful sentence. Could be something like the elephant flew to the moon. Does that sentence mean something? Sure, it means something. It has a perfectly. We can conjure up an image of what that means. Has it happened in the world? No, it hasn't happened, so far as we know. But could it be in a story? Could it be in a fictional world? Absolutely. So this thing about this sort of semantic grammar will allow you to. Allows you to put together things which are somehow. Which are sort of meaningful things to describe about the world. The question of whether they are realized in the world or have been realized in the world is a separate question. But in any case, the thing that. Is, to me, interesting about this is it's something I've long thought about because I've spent a large part of my life building a computational language, Wolfram language system that is an effort to represent the world computationally, so to speak, to take the things that we know about, about chemicals or lines or images or whatever else, and have a computational representation for all those things and have a computational language which knows how all those things work. It knows how to compute the distance between two cities. It knows all of those kinds of things. And in. And so this is. I've been spending the last four decades or so trying to find a way to represent things in the world in this computational fashion so that you can then compute things about those things in an explicit computational way. It's something where. And we've been very successful at being able to do that. In a sense, the story of modern science is a story of being able to formalize lots of kinds of things in the world. And we're kind of leveraging that in our computational language to be able to formalize things in the world, to compute things about how they'll work. Now, the one feature of that computing about how things work is that inevitably some of those computations are deep computations. They're computations that something like a chatgpt can't possibly do. And in a sense, there's sort of a difference between the things that are the kind of, the sort of shallow computations that you can learn from examples in something like a chatgpt that you can say, this piece of language that I saw on the web here is statistically I can sort of fit that in in this place. Just fitting together these sort of puzzle pieces of language is a very different thing from taking the world and actually representing it in some truly sort of formal way computationally so that you can compute things about how the world works. It's kind of like, well, back before people had kind of thought of this idea of formal formalism maybe 400 years ago or more. Everything that anybody figured out was just, you think about it in terms of language, in terms of words, in terms of sort of immediate human thinking. What then sort of came in with mathematical science at first, and then computation was this idea of formalizing things and getting these much deeper sort of ways to deduce what happens. And thing I've figured out 30, 40 years ago now was this phenomenon of computational irreducibility, this idea that there really are things in the world where to compute what's going to happen. You have no choice but to follow all those computational steps. You can't just jump to the end and say, oh, I know what's going to happen. It's a shallow kind of thing. And so when we look at something like ChatGPT, there are certain kinds of things it can do by sort of matching together, matching these pieces of language. There are other kinds of things it's not going to be able to do. It's not going to be able to do sort of the mathematical computation, the kind of the thing which requires an actual computational representation of the world. For those things like us humans, it's kind of a use tools type, game type situation. And very conveniently, our WolframAlpha system that used in a bunch of intelligent assistants and so on, has this feature that it's using our wolfram language, computational language underneath, but it actually takes natural language input. So it's actually able to take the natural language that is produced by a ChatGPT, for example, take that, and then turn that into computational language, do a computation, work out the results, get the right answer, feed that back to ChatGPT, and then it can Talk sense, so to speak, rather than just following sort of the statistics of words on the web. So it's a way of. By allowing you can get sort of the best of both worlds by having something where you have this sort of flow of language as well as something where you have this sort of depth of computation by having ChatGPT use Wolfram Alpha as a tool. And I wrote a bunch of stuff about that, and all kinds of things are happening with that. But the thing that, you know, talking about, what did chatgpt discover? I think the thing it discovered is there is a semantic grammar to a lot of things. There is a way to represent using sort of computational primitives, lots of things that we talk about in text. And in our computational language, we've got representations of lots of kinds of things, whether it's foods or chemicals or stars or whatever else. But when it comes to something like I'm going to eat a piece of chocolate, we have a great representation of the piece of chocolate. We know all its nutrition properties, we know everything about it, but we don't have a good representation yet of I'm going to eat the I'm going to eat part. What I think ChatGPT has shown us is that it's very plausible to get sort of this semantic grammar of how one has these pieces of representing these sort of lumps of meaning in language. And I think what's going to happen, and I've been interested in doing this for a long time, I think this is now finally the impetus to really roll up one's sleeves and do it. It's a somewhat complicated project for a variety of reasons. Not least, you have to make these kind of. Well, you have to make sort of. This process of designing a language is something I happen to have been doing for 40 years, designing our computational language. This is a language design problem. And those are, to my mind, those are actually the single most concentrated, intellectually difficult thing that I know is this problem of language design. So this is sort of a generalization of that. But I think ChatGPT has kind of shown us what, you know, I didn't know how hard it was going to be. I'm now convinced it's doable, so to speak. So what, what does this, you know, you might ask the question, you know, people might have said, okay, look, you know, we. We've seen neural nets that do speech to text. We've seen neural nets do image identification. Now we've seen neural nets that can write essays. Surely if we have a big enough neural net, it can do everything. Well, not the neural nets of the kind we have so far that have the training structure that they have so far. Not on their own, they will not be able to do these irreducible computations. Now, these irreducible computations are not easy for us humans either, you know, when it comes to doing piece of math or worse, if somebody says, here's a program, run this program in your head, Good luck, you know, very few people can do that. It. It's something where there is a difference between what is sort of immediate and easy for us humans and what is sort of computationally possible. Now, another question is maybe we don't care about the things that aren't easy for humans. It's turned out that we built an awful lot of good technology over the last few centuries based on what amounts to a much deeper level. We haven't really, in our technology, we're not actually going even that far into irreducible computation, but going far enough that it's beyond what we humans can readily do or what we can do with kind of the neural nets that exist today. So I think that's the kind of the thing to understand that there's a certain set of things. What's happening in ChatGPT is it's kind of taking the average of the web plus books and so on, and it's saying, I'm going to fit things together based on that. And that's how it's writing its essays. And when it is deducing things, when it's doing logic, things like that, what it's doing is it's doing logic like the way Aristotle discovered logic. It's figuring out, oh, there's a pattern of words that looks like this, and it tends to follow it like that, because that's what I've seen in a hundred thousand examples on the web. So that's kind of what it's doing, and that gives us some sense of what it's going to be able to do. And I think the most important thing it's able to do is it's a form of user interface. We can get. I might get something where I know, oh, what really matters is three bullet points. But if I'm going to communicate that to somebody else, they're really not going to understand my three bullet points. They need wrapping around that. They need something which is a whole essay describing that's the human interface, so to speak. It's just like you could have the raw bits or something, and that wouldn't be useful to us humans. We have to wrap it in a human like in a sort of human compatible way. And language is sort of our richest human compatible medium. And what ChatGPT is doing is it's able to. I think the way to think about it is it's providing this interface that is. Well, it is just, it's generating pieces of language that are consistent but if you feed it specific things that it will talk about, so to speak, then it's kind of wrapping the specifics with this interface that corresponds to kind of flowing human language. All right, I went on much longer than I intended and I see there are a bunch of questions here and I'm going to go from and to try and address some of these. There's a question from Are constructed languages like Esperanto more amenable to semantic grammar? AI approach. Very good, very interesting question. So I think the one that I was experimenting with was the smallest of the constructed languages. A language called Toki Pona that has only 130 words in is not a language that allows one to express everything one might want to express. But it's a good kind of small talk type language, a small language for doing small talk, so to speak. But it expresses a bunch of decent ideas. And so I was going to look at. Yes, it's a good clue again to semantic grammar that there are these small constructed languages. It also helps, I think. Well, I also think probably the largest of the constructed languages, ITHQL is another interesting source. It's a language which has tried to pull in all of the kind of language structures from all the, all known languages in some first approximation. The. That's. Yeah, that's yes. So I think the answer is that yes, I think they're a good stimulus for thinking about semantic grammar in a sense. When people were trying to do this back in the 1600s, they were very confused about many things but you know, one gives them a lot of. They got a long way given that it was the 1600s. They were confused about things like whether the actual letters that were written as he wrote the language mattered and how that was more so than the structure of things. But there was the beginning of that kind of idea. Okay, I'm going to take these from the end, but I want to go back to some of these others. Okay. Etoria is asking how can one study what's the best way of prompting ChatGPT? Could a semantic law of motion be helpful? Undoubtedly, yes. I don't know the answer to that. I think it's a good question and I don't really know, I think. Yeah, I don't know. Albert is asking, is the 4000 token limits analogous to working memory? Would accessing larger memory be increasing the token limit or increasing search capabilities, Reinforcement learning? Well, I think that the token limits that exist right now are. If you want to have a coherent essay and you want it to know what it was talking about back in that early part of the essay, you better have enough tokens that are being fed into the neural net every time it gets a new token. If it just doesn't know what it was talking about, if it forgot what it was talking about 5,000 tokens ago, it may be saying totally silly things now because it didn't know what was there before. So in some sense, it's like. I don't think it's like our short working memory, but I think it's kind of like you ramble on. I ramble on a lot, talking about things, and I might have forgotten half an hour later that I talked about that already. I might be telling the same story again. I hope I don't do that. I don't think I do that too badly. But, you know, that's a question of what that. That's the kind of thing that happens with this token limit. Let's see. Let me go back to some of the questions that were asked earlier here. Okay. Aaron was asking, talking more about the tension between superintelligence and computational irreducibility. How far can LLM intelligence go? I. I think I talked a little bit about that. I think this question, oh, boy, this is kind of complicated. I mean, so this question about, okay, the universe, the world is full of computational irreducibility. It's full of situations where we know the underlying rules, but we run them as a computation, and you can't shortcut the steps. What we've discovered from our physics project is it looks like the very lowest level of space time works just that way. In fact, just earlier today, saw lovely work about doing practical simulation of space times and things using those ideas and very much supporting, again, this. It's really computationally irreducible at the lowest level. Just like in something like a gas, the molecules are bouncing around in this computationally irreducible way. What we humans do is we sample sort of aspects of the universe that have enough reducibility that we can predict enough that we can kind of go about our lives. Like, we don't pay attention to all those individual gas molecules bouncing around. We only pay attention to the aggregate of the pressure of the gas or whatever else we don't pay attention to all the atoms of space. We only pay attention to the fact that there's this thing that we can think of as more or less continuous space. So our story has been a story of finding slices of reducibility, slices, places where we can predict things about the universe. There's a lot about the universe we cannot predict. We don't know. And if our existence depended on those things, if we had not found kind of these slices of reducibility, we wouldn't be able to have a coherent existence of the kind that we do. So if you ask, sort of, where do you go with that? Well, they're an infinite collection. There's an infinite kind of web of pieces of computational reducibility. There's sort of an infinite set of things to discover about that. We have discovered some of them as we advance in our science and with our technology for things, we get to explore more of that kind of web of reducibility. But that's really the issue now. The problem is that the way that we humans kind of react to that is we have ways to describe what we can describe. We have words that describe things that are common in our world. We have a word for a camera, we have a word for a chair, those kinds of things. We don't have words for things which have not yet been common in our world. And, you know, when we look at the innards of Chatgpt, it's got all kinds of stuff going on in it. Maybe some of those things happen quite, quite often, but we don't have words for those. We don't have a way. We haven't yet found a way to describe them. When we look at the natural world, there are things that we've seen repeatedly in the natural world. We have words to describe them. We've built up this kind of descriptive layer for talking about things. But one of the things that happens is if we kind of jump out to somewhere else in the sort of universe of possible computations, there may be pieces of reducibility there, but we don't have words to describe those things. We only have we know about the things that are near us, so to speak. And so. And gradually, as science advances, we get to expand the domain that we can talk about, so to speak, or everything advances. We get to have more words, we get to be able to talk about more things, but in a sense, to have something which operates. It's this gradual process of us, sort of societally, in a sense, learning more concepts. We kind of can exchange concepts. We can build on those concepts and so on. But if you throw us out into some other place in what I call the ruliad, the space of all possible computational processes, if you throw us out into an arbitrary place, there we will be completely confused because there will be things we can tell. There are actual computations going on here. There are things happening, there's even pieces of reducibility, but we don't relate to those things. So it's kind of like imagine that you're here now and you're chronically frozen for 500 years and you wake up again, and there's all these other things in the world, and it's hard to reorient for all those other things without having seen the intermediate steps. And I think that. That when you talk about kind of where can you go from what we have now? How can you sort of add more? You basically, intelligence is all about these kind of pieces of reducibility, these ways to jump ahead and not just say it's what we. What we think of as sort of human, like intelligence is about those kinds of things. And I think the, you know, so what's the vision of what will happen, you know, when the world is full of AIs? Sort of interesting, because actually we've seen it before. I mean, when the world is full of AIs and they're doing all these things and there's all this computational irreducibility, there are all these pockets of reducibility that we don't have access to because we haven't sort of, you know, incrementally got to that point. What's going to be happening is there's all this stuff happening among the AIs, and it's happening in this layer that we don't understand. It's already happening in plenty of places on the web and bidding for ads or showing you content on the web or whatever. There's a layer of AI that's happening that we don't understand particularly well. We have a very clear model for that, which is nature. Nature is full of things going on that are often computationally irreducible that we don't understand. What we've been able to do is to carve out an existence, so to speak, that is coherent for us, even though there's all this computational irreducibility going on. We've got these little niches with respect to nature which are convenient for us as humans, so to speak. And I think it's sort of the same thing with the AI world as it becomes like the natural world and it becomes sort of not immediately comprehensible to us. We are kind of, our view of it has to be, oh, that's just the operation of nature. That's just something I'm not going to understand. Oh, that's just the operation of the AI is not going to understand that. There's this piece that we've actually managed to humanize that we can understand. So that's a little bit of the thought about how that develops. In other words, you can say, I'm going to throw you out to some random place in the ruliad. There's incredible computations happening. It's like, great, that's nice. I've spent a bunch of my life studying those kinds of things. But pulling them back, reeling them back into something which has sort of direct human understandability is a difficult thing. Aaron is asking more of a business question about Google and the transformer architecture and why. It's been a very interesting thing that the sort of neural nets were this small field, very fragmented for many, many years. And then suddenly things started to work in 2012. And a lot of what worked and what was really worked on was done in a small number of large tech companies and some not so large tech companies. And it's sort of a different picture of where innovation is happening than has existed in other fields. And it's kind of interesting, it's kind of potentially a model for what will happen in other places. But it's always complicated. What causes one group to do this, another group to do that? And there are the entrepreneurial folk who are smaller and more agile and, and there are the folks who have more, the more resources and so on. It's always complicated. Okay. Nicola is asking, do you think that pre training, a large biologically inspired language model might be feasible in the future? I don't know. I think that the figuring out how to train something that is, you know, we don't know what parts of the biology are important. One of the incredibly important things we just learned is that probably there's not much more to brains that really matters for their information processing than the neurons and their connections and so on. It could have been the case that every molecule has some quantum process that's going on and that's where thinking really happens. But it doesn't seem to be the case because this pinnacle of our thinking powers of being able to write long essays and so on, it seems like that can be done with just a bunch of neurons with weights. Now, which other parts of biology are important? Like actually, Terry Sinowski just wrote this paper talking about how there are more backwards going neural connections in brains than forwards going ones. So in that sense it looks like maybe we missed the point with these feedforward networks that something like ChatGPT basically is, and that the feedback is really important, but we haven't yet really got the right idealized model of that. I do think that the what's the next McCulloch Pitts type thing, what's the next sort of simple meta model of this is important. I also think that there's probably a bunch of essential mathematical structure to learn about general mathematical structure to learn. Neural nets. I was interested in neural nets back around 1980 and I kind of was trying to simplify, simplify, simplify models of things and neural nets. I went past them because they weren't simple enough for me. They had all these different weights and all these different network architectures and so on. And I ended up studying cellular automata and generalizations of that where, where you have something where everything is much simpler. There are no real numbers, there are no arbitrary connections, there are no this, that and the other things. But what matters and what doesn't? We just don't know that yet. Paul is asking what about a five senses multimodal model to actually ground the system in the real world with real human like experience? I think that will be important and that will no doubt happen and you'll be more human like look, this chatgpt is pretty human like when it comes to text because by golly, it just read a large fraction of the text that we humans at least publicly wrote. But it didn't know. It hasn't had the experience of walking upstairs and doing this or that thing. And so it's not going to be very human like when it comes to those sorts of things. If it has those experiences, then I think we get to, then that will be interesting. Okay, someone's commenting on the fact that I should do the same kind of description for image generation. Generative AI for images. The thing that I like to think about there is, I think that's one of our first moments of communication with an alien intelligence. In other words, we, in some sense we're talking to the generative AI in English words or whatever and it's going into its alien mind, so to speak, and plucking out the stuff that is these images and so on. It's less so, you know, with ChatGPT, what the output is something that is already intended to be very human. It's human language with an image generation system. It's producing something which has to be somewhat recognizable to us. It's not a random bunch of pixels. It's something that resonates with things we know. But in a sense, it can be more completely creative in what it's showing us. And in a sense, as one tries to sort of, you know, navigate around its space of what it's going to show us, it feels a lot like kind of you're communicating with an alien intelligence and it's kind of, it's kind of showing you things about how it thinks about things by saying, oh, you said those words, I'm going to do this, and so on. I mean, I have to say that if we can't, you know, the other examples of alien intelligences that we have all around our planet are lots of, lots of critters from the cetaceans on, so to speak. That, and I have to believe that if we could correlate kind of the experiences of those critters, cats, dogs, you know, cockatoos, whatever else, and the vocalizations that they have and so on, and we could, you know, that it's talk to the animals time, so to speak. I mean, I think that's a. That feels like that that's, you know, the kinds of things We've learned from ChatGPT about the structure of human language. I am quite certain that if there's any linguistic structure for other animals, it'll be similar because it's one of the lessons of biology is, you know, there are fewer ideas than you think. You know, these things that we have have precursors in biology. Long, long ago, we may have made innovations in language. That's kind of the key innovation of our species. But, but whatever is there had precursors in other organisms. And that's what. And the fact that we now have this much better way of kind of teasing out a model for language in humans means we should be able to do that elsewhere as well. Okay. David is saying ChatGPT's developers seem committed to injecting sort of political curtailments in the code because to avoid it talking about controversial topics. How is that done? It's done through this reinforcement learning stage. I think maybe there's also some actual, you know, if it's starting to use these words, just stop it. Type things. I think maybe that's being done a little bit more with, maybe with Bing than It is with ChatGPT at this point. I think that the, I have to say the one thing that I consider a, you know, so far As I know ChatGPT is a G rated, you know, thing. And that's an achievement in its own right. That it doesn't. Maybe I shouldn't say that because probably, maybe there are horrible counter examples to that. But I think that was a, you know, in terms of one of the things that happens is, well, you have a bunch of humans and they are giving it this training and those humans have opinions and they will have, you know, there'll be this kind of politics or that kind of politics, or they'll believe in this or that or the other. And they are, you know, whether purposefully or not, they're, you know, they're going to impose those opinions because there is no, you know, the opinion. This, what you're doing when you tell ChatGPT that essay is good, that essay isn't good. You know, at some level, that's an opinion. Now, that opinion may or may not be colored into something that is about, you know, politics or something like that, but it's, it's sort of inevitable that you have that. I mean, I have to say, you know, something I've thought about a little bit in connection with general sort of AI injection into sort of the things we see in the world, like social media content and so on, I tend to think that the right way to solve this is to say, okay, let's have multiple, you know, chatbots or whatever, and they are, in effect, trained with different criteria by different groups under different banners, so to speak. And you get to pick the banner of chatbot that you want to be using. And then you're happy because you're not seeing things that horrify you and so on. And you can discuss whether you want to pick the chatbot that accepts the most diverse views or whether you want to. That sort of throws one back into kind of standard issues of political philosophy and things like this. I mean, I think the thing to realize is that there's sort of an ethics. You know, one wants to put ethics somehow into what's going on. But when one says, let's have the AIs, you know, do the ethics, it's like, that's hopeless. Ethics is a. There is no sort of mathematically definable perfect ethics. Ethics is a the way humans want things to be. And then you have to choose, you know, well, is it the average ethics? Is it the ethics which makes only 5% of the people unhappy? Is it this, that and the other? These are old questions of political philosophy that don't really have, so far as we know, good answers. But once thrown into Those questions, there's no, oh, we'll get a machine to do it and it'll be perfect. It won't happen. Because these are questions that aren't solvable for a machine because they're questions that in a sense, come from, right, from us. These are, I mean, the thing to realize about Chat GPT in general. ChatGPT is a mirror on us. It's taken what we wrote on the web, so to speak, in a. In, in aggregate, and it's reflecting that back to us. So insofar as it does goofy things and says goofy things, you know, some. That's really on us. I mean, that's, you know, it's our sort of. It's. It's the average kind of, sort of the average web that we're seeing here. Tenacious is asking about a particular paper which sounds interesting, but I don't know about it. Let's see. Should pop up soon here. Okay. Tragath is wondering how neural net AI compares to other living multicellular intelligence. Plant roots, nerve nets and things like jellyfish and so on. Biofilms. Yeah, well, okay, so one of the big things that's come out of a bunch of science that I've done is this thing I call the principle of computational equivalence, which essentially says that as soon as you have a system that is not computationally trivial, it will ultimately be equivalent in its computational capabilities. And that's an important thing when you talk about computational irreducibility, because computational irreducibility arises because you've got a system doing its computation. There's no system. You can't expect there to be. All other systems will just be equivalent in their computational sophistication. You can't expect a super system that's going to jump ahead and just say, oh, you went through all these computational steps, but I can jump ahead and just get to the answer. Now, a question that is a really good question is when we look at, okay, one of the things that is characteristic of our consciousness, for example, relative to all the computational irreducibility in the universe, is the fact that we have coherent consciousness is a consequence of the fact that we are two things. It seems to me we are computationally bounded. We're not capable of looking at all those molecules bouncing around. We only see various aggregate effects, point one and point two, that we believe that we are persistent in time. We believe we have a persistent thread of existence through time. Turns out, big fact of our last few years for me is that the big facts of physics, general relativity, theory of gravity, Quantum mechanics and statistical mechanics, the second law of thermodynamics, law of entropy, increase. All three of those big theories of physics that arose in the 20th century, all three of those can be derived from knowing that we human observers are noticing those laws and we human observers have those two characteristics I just mentioned. I consider this a very important, beautiful sort of profound result about kind of the fact that we observe the physics we observe because we are observers of the kind that we are now. Interesting question, I suppose, is when we. So we are limited, we are computationally limited things. And the very fact that we observe physics the way we observe physics is a consequence of those computational limitations. So the question is how similar are the computational limitations in these other kinds of systems? In a sense, the fungus as observer, so to speak, how similar is that kind of observer to a human observer? And in terms of sort of what computational capabilities it has and so on, my guess is it's pretty similar. And in fact, one of my next projects is a thing I'm calling observer theory, which is kind of a general theory of kinds of observers that you can have of things. And so maybe we'll learn something from that. But it's very interesting, interesting question. Dugan is commenting. ChatGPT can be improved using an automated fact checking system, like an adversarial network, for instance. Could one, basically, could one train ChatGPT with WolframAlpha and have it get better? The answer is surely up to a point, but then it will lose it just like it does with parentheses. I mean, there's a certain, with a network of that architecture, there's a certain set of things one can learn, but one cannot learn what is computationally irreducible. I mean, in other words, you can learn the common cases, but there'll always be surprises. There'll always be unexpected things that you can only get to by just explicitly doing those computations. Bob is asking, can ChatGPT play a text based adventure game? I bet it can. I don't know, I haven't seen anybody try that, but I bet it can. Okay, there's a question here from software. Aside from being trained on a huge corpus, what is it about GPT3 that makes it so good at language? I think I tried to talk about that a bit, about the fact that there's regularity in language. I think the particulars of the transformer architecture of this kind of looking back on sequences and things, that's been helpful in refining the way that you can train it. And that seems to be important. Let's see. Toria is asking, could feature impact scores help us understand GPT better? Well, so what that's about is when you run a neural net, you can say sort of. How much did some particular feature affect the output that the neural net gave? ChatGPT is just a really pretty complicated thing. I mean, I started digging around, trying to understand sort of what, as a natural scientist, you know, I'm like, I couldn't do sort of neuroscience with actual brains because I'm a hundred times, thousand times too squeamish for that. But, you know, I can dig around inside an artificial brain. And I started trying to do that and it's, it's, it's difficult. I mean, I didn't look at feature impact scores. I think one could. The. Okay, so, but by the way, I mean, I'm, I'm amused by these questions because, because I, I can kind of, you know, I can still tell you guys are not bots, I think. And let's see, Ron is asking about implications like, I have to work late tonight. What does that mean? Yeah, absolutely. ChatGPT is learning stuff like that because it's seen, you know, a bunch of text that says, I have to work, work late tonight, so I can't do this. It's seen examples of that. It's kind of doing the Aristotle again. It's just seeing this, these patterns of language, and that's what it's learning from, so to speak. So, yes, these things, we might say. How do we think about that formally? Oh, it seems kind of complicated to us, but that pattern of language has occurred before. All right, last thing, perhaps. Okay. Albert is asking, do you think humans learn efficiently because they're born with the right networks to learn language more easily, or is there some difference? I think it is important. The architecture of the brain undoubtedly is important. I mean, my impression is that there are, It's a matter for the neuroscientists to go and find out. Now that we know that certain things can be made to work with artificial neural nets, did the actual brain discover those things too? And the answer will be often yes. I mean, just like there are things we probably have learned from the flight of drones or the flight of planes that we can go back and say, oh, did biology actually already have that idea? I think that there are undoubtedly features of human language which depend on aspects of the brain. I mean, like, for example, one talking to Terry Sinofsky. We're talking about the, the loop between the basal ganglia and the cortex and the possibility that the outer loop of chatgpt is a little bit like that loop. And it's kind of like I'm turning things over in my mind. One might say maybe that's actually a loop of data going around, this literal loop from one part of the brain to another. Maybe, maybe not. But sometimes those sayings have a habit of being more true than you think. And maybe the reason that when we think about things, we have these certain time frames when you think about things, the certain times between when words come out and so on, maybe those times are literally associated with the amount of time it takes for signals to propagate through some number of layers in our brains. And I think that in that sense, if that's the case, there will be features of language that. Which are, yes, we've got this brain architecture, we're going to have these features of language. And insofar as language evolves as so far as it's adaptively worthwhile to have a different form of language that is optimized by having some different form of brain structure, that's what will have been driven by natural selection and so on. I mean, I think there are aspects of language like we know if you. We tend to remember five chunks, you know, chunks of five, so to speak, things at a time. And we know that if we try and give a sentence which has more and more and more deeper, deeper, deeper subclauses, we lose it after some point. And that's presumably a hardware limitation of our brains. Okay, Dave is asking, this is a good last question. How difficult will it be for individuals to train something like a personal chatgpt that learns to behave more and more like a clone of the user? I think, I don't know. I'm going to try it. I have a lot of training data, as I mentioned, you know, 50 million typed words type. Yeah, typed words, for example, from me. And my guess is, I mean, I know somebody tried to train an earlier GPT3 on stuff of mine. I didn't think it was terribly good. When I read ones trained for other people, I thought they looked pretty decent when I looked at one trained for myself because I kind of know myself better than I know anybody else. I think it didn't ring true, so to speak, but I do think that that will be a being able to write emails like I write emails. It'll do a decent job of that, I suspect. I would like to believe that as a human one still has an edge because in a sense one knows what the goals are. This system, its goal is to complete English text and the bigger picture of what's going on is not going to be part of what it has except insofar as it learns the aggregate bigger picture from just reading lots of text. But I do think it'll be an interesting I expect that as a person who gets a lot of email, some of which is fairly easy to answer in principle that maybe my bot will be able to answer the easier stuff for me. All right. That's probably a good place to wrap this up. Thanks for joining me. And I would like to say that for those interested in more technical details, some of the folks in our machine learning group are going to be doing some more detailed technical webinars about this material and really going into how you would, you know, how you build these things from scratch and so on and what some of the more detail about what's happening actually is. But I should wrap up here for now and thanks for joining me and bye for now.