Neural Networks Basics: Machine Learning, Training and Embeddings

Sam.

Hi everyone. Doing a quick mic check here. The daily study group is going to get started in about five minutes at the top of the hour and it is our second session so we are going to be looking at the machine learning basics. Another topics from the second part of Stephen Wolfran's book. What is Chat GPT? What is ChatGPT doing and why does it work? Again, just to make sure you have access to that book, I'm going to share a pop up offer on your screen. If you submit any questions in that Q and A pane, we will get ready to review those and then answer them as we go. Again, we'll be getting started in just a few minutes. Thank you. Sa. Sa. Sam, you. Hello everyone. Welcome back and thank you for joining us. I'm Cassidy with the Wolfram youm team and today is our second session in the daily study group. What is Chat GPT doing and why does it work? Our lead instructor for this group is Arbin Calzici. I'm sorry, I can't speak today. Arvin Calzici, also with the Wolfram youm is our instructional Materials manager. He's going to be guiding us section by section through Steven's book through the rest of this week and today pages 15 through 46 of the book will be covered. We also have our machine learning manager Julio Alessandrini here to answer questions in the chat panel. So make sure that you are or the QA panel, make sure you're getting those in throughout the session. And then we also have Mikey with our technology group also available. So we encourage you to get those questions in there as we progress. And then if you do have questions outside of these sessions, most of you know that there is a community thread linked in the chat panel that's going to be the best way to communicate. Arbin is amazing at being engaging and accessible throughout the session in that, in that community thread. So that is our, our goal is really to just be there to best help everyone understand ChatGPT. So throughout the session we're going to use the poll questions. We also have survey questions at the end of every session and then we will also give a quiz out towards the end of the week that you'll be able to take pass with a 60% or higher to get a completion certificate for this series. But for now I think I'm going to go ahead pass things over to Arbin to get started and I'll be back at the end to wrap up. Arvin.

Thanks Cass. Excuse me, let me get my screen shared here. As is the customer. All right, should be shared there. So indeed, today we'll be talking about kind of some of the basics of machine learning. We'll be covering neural networks and training. We'll be talking more about machine learning, tomorrow about embeddings and architecture. But today we're kind of just getting in on the ground floor with some of the basics. So yesterday was pretty high level and today we're going to go a little bit lower. So we'll learn about the kind of fundamental operations of neural networks. We'll see how they are trained and used for machine learning tasks, and explore at the end, just briefly, some of the theoretical shortcomings of neural networks. Now, as a reminder for what we covered last time, remember that what chat, GPT and other LLMs are doing is just adding one word at a time. You can also see that it's not just words, but tokens like periods or pieces of words. But the kind of basic idea is one word at a time. The way it does this is that at each step it sees its input and then it has probabilities for what words should follow next. Because of the ridiculous amount of data that would be needed to perform this in kind of a naive statistical way, ChatGPT and other LLMs need to use a model in order to generate text. It's just a little reminder of the idea of a model if you can't have enough data points. If you draw a model, you can kind of figure out any amount of data points you want, say between here and here and here and here. So you have an unlimited number to choose from if you have a model that way. And then finally, I think we agreed that if you have a model that is intended to perform some sort of human like task, we can say that it does a pretty good job if it does more or less what a human would do. Even if that includes failing in the same way that a human might fail if presented with, say, a blurry or very blurry digit. So let's get into neural nets. And I realized while reviewing this this morning that this is a bit of a block of text, but bear with me here. So this is kind of the background for neural nets. The human brain has around 100 billion neurons. And each of these things can rapidly produce electrical pulses. That's kind of the basic action of the neuron. So these neurons are connected together. It's important to note that not every neuron is connected to every other neuron, but they are connected enough that each neuron can influence maybe thousands of other Neurons. So they're connected to maybe thousands out of the hundred billion total, give or take. The basic idea is that once you have some input or another, once you hear something or look at something, or feel something, that produces some sort of input, which then kind of acts on a neuron, that neuron produces an electrical signal, that pulse is sent forward to other neurons, which then can in turn can, you know, make decisions about whether they ought to produce a signal. And it just kind of keeps cascading through the network of neurons, so to speak. It's important to note that some neuron, neuron connections are stronger and have a larger effect on what happens next, while other connections are weaker. So while neurons are connected on average to say, 1,000 or so other neurons, they're not connected with the same strength to all those other thousand neurons. Neurons. So kind of as an example, I think the example kind of helps to clarify things here. The mechanics of, you know, human task of image recognition. Say you look at something, photons bounce off of that thing and then they bounce into your eye. The photons hit the cells in your eyes, and then the cells in your eyes cause some of your neurons to produce electrical signals. Then those signals spread out through the network of neurons. Again, there's that term in your brain. And that influences each neuron, influences other neurons more or less, based on some kind of pre built connection strength that we don't have to worry about yet. Then once this whole computation of electrical signals has spread throughout the whole neural network again, so to speak, this is what causes you to think, oh, that's a cat, or that's a five. So you have some input, that input causes a response, that response spreads out through some network of little nodes that can talk to each other. And then the final result is that you get, so to speak, an output of, you know, thinking, that's a cat or that's a five, just kind of the implicit recognition of the thing that you're seeing. So the blackbox function that we used at the end of yesterday's talk, the blackbox digit network, as it was very cleverly named, it's a secret function. If I don't suppress the output, we can see that it is a net chain. So indeed this thing is a neural net. It works pretty much like the thing above. There's an input, which in this case is an image, and then there's some weird stuff going on, basically some mathematical steps where the signal from the input is propagated forward through a bunch of computations. And then at the end, what you get out is in this case a class. And the classes here just refer to the 10 digits, 0 through 9. So you get an input, that input causes some response which then kind of cascades through the network. And then at the end you get an output which is just a single digit. And it's worth noting, by the way, that this network is not really theoretically derived. It has some properties that correspond to the fact that it's looking at images. But this network is just kind of something that was found to work in that sense. It's kind of like the human brain. You know, these things evolve under natural selection, but that doesn't really mean that they have a specific like architecture that was designed in mind. Like when the human brain is recognizing images, it's not the case that there's like a specific custom built image classification technique that lives inside the mind. There is just a connection, a series of neurons that are connected together and somehow they have learned to be able to recognize images and, and the neural net functions in much the same way. So we can kind of think of this black box digit knower, even though now we know it's not quite a black box, it's a neural network, a net chain, a chain of the pieces of the neural network. You can kind of think of this as trying to group its inputs into separate boxes. You know, things that look like such and such, they all go in the one box. But things that look like this other type of such and such, these should all go in the seven box and so on. So it's looking at these and it's kind of classifying or boxing them up into different boxes based on something in the input. So here's kind of a simplified example. Maybe instead of maybe taking a whole image and trying to provide a digit like this as output, let's just kind of think of a simpler case where we want a net that takes an XY coordinate pair and then tries to classify it into just three possible boxes labeled zero plus one and minus one. So what we want is, suppose we have the 2D plane. This is representative of the 2D plane. And if I give a point anywhere here, I want it to say zero. And if I give a point anywhere here, I want it to say plus one. And if I give a point anywhere here, I want it to say minus one. So we can kind of represent that like this. We want the neural network, in other words, to compute a function that looks like this. So it's got kind of three different boxes. You can think of these things as. So this is one box up here, this is another box over here, and this is another box over here. So if you exist anywhere in here, we want one to be the output. If you exist anywhere here, we want zero to be the output. And if you exist anywhere here, we want negative one to be the output. So this thing is just taking one XY pair, whereas yesterday where we had the black box digit knower, and that took in a 28 by 28 image, which is 784 total pixels, and it used all of those pixels to place digits or images into one of ten digit boxes. Again, here we're just taking two data points and we're going to try and use those to place our points into one of three height boxes, so to speak. So how do you actually make a neural net learn these things? How do you make it perform this task? So again, a neural net works kind of like a brain. It has neurons and connections of various strengths between the neurons. So each excessive round of operations where one set of neurons talks to another can be thought of as a layer. So you can kind of think of it like this. You've got some input signals here, say X and Y. Those come in and this is the first layer here. And then these two networks send some signal forward to the neurons in the next layer. Those go down, these neurons send some signals to the neurons below, these signals send some neurons or signals to the neurons below, and so on, until you end up with an output which is either 0 plus 1 or minus 1. Here the thickness and color of the lines just represents the different connection strengths. So with the numbers included, we can kind of think of, okay, we come in here, we have some signals. Those signals get propagated throughout the network, and then somehow, some way at the end, we end up with, say, negative one. We could have also ended up with one or with zero, depending on the inputs. But for these inputs we should get negative one. So the kind of idea here with the mathematical operation of the network is that each neuron gets some inputs from the prior layer. So again, if we go here, we can see that this one is getting inputs from all these ones from the prior layer. So what the neuron has to do is it multiplies each input by the connection weight, how strongly it's connected, then it just adds them together, it adds a constant called a bias, and then it applies a thresholding or activation function. So mathematically speaking, this is pretty much just a linear algebra operation. So if you're familiar with linear algebra, you can just think of this as a Matrix multiplied by a vector plus another vector, you have a matrix of weights which tells you how strongly each neuron is connected to each other neuron. You have your input and then you have some constant offset bias vector. It's kind of like your Y intercept in the simple 2D, you know, Y equals MX plus B case. And then once you do the linear algebra part, there's a nonlinear part here called a thresholding or activation function. And you kind of need this for the network to exhibit non trivial behavior. What this does here is this will take your inputs and depending on which activation function you pick, this thing will reweight the outputs. Maybe it just sets. Ok, if you get a negative number, I'm setting you to zero. I'm basically turning off all of the negative inputs. But if you have a positive number, I'll leave you as is. There are different kinds of activation functions as you can see here. And this is just a choice you make when you create your network. Now this is really all just to say I don't want to get too bogged down in the details of that. So it is kind of just a basic linear algebra operation. And at the end of the day, the point point is that any given neural network is just a mathematical function. Typically it is a pretty ugly mathematical function. And the function corresponding to the network above, for example, in terms of like ugly functions, looks like this. But fundamentally the point is that, you know, here we've just got some number and we're going to multiply it by, you know, here we've got another number, just another number. It's just basic multiplication and addition at every step except for the activation function, which in our case is ramp. So it's just setting negative numbers to zero. So basically anytime the input of F is negative, it gets set to zero. That's all it's doing. And indeed, even something AS complex as ChatGPT internally in a sense can be represented by a mathematical function just like this. Except for ChatGPT, it has billions to trillions of these terms here because the weights matrix is so big. So the network we wrote above, just kind of this random network here, the best this network can do at learning our function is this. It's not that good, honestly. It's okay. Kind of learned this one pretty well, and it learned this one pretty well. But this whole area is kind of a mess. It can't really distinguish between these two bins. So we have a nice line there, but not a nice line here. If you give some other networks you can get different results. So you can see here, this network is simpler. It only has a couple of neurons, just a couple of layers. And this one can't really do a good job at all. Just kind of makes a tilted plane and that's it. This one with more connections and more neurons, it's getting closer, but it's still like not quite great. And this one, which has more neurons still and more layers, this one does a really good job of getting that. That shape pretty much how we want it. Again, it's not perfect, but it is quite good. So larger networks, that is, networks that have more neurons and more connections between the neurons, tend to do a better job of approximating our black box closest point detector function.

Okay.

Just two quick poll questions about this stuff. So far we've kind of jumped in comparatively to the deep end compared to yesterday. So just want to kind of make sure folks are keeping up with the ideas. So how is the shape of the network for that black box digit knower derived? Did it just happen to work? Maybe it was derived from first principles. Maybe it has some relevant details that were based on its input, but otherwise it's just kind of thrown together and found to work. Give this a little more time for some responses to come through. Okay. Looks like we have about two thirds of people saying it has some relevant details that are based on its input, but it's otherwise ad hoc. And yeah, that is indeed true. So do I have that defined here? Yes, so. So here, kind of some of the details, just for the sense of it is when you're dealing with images, often you use convolution layers. If you think about an image, for example, an image like this, right? The image has come on paste. There we go. This is an image, it kind of lives on the screen. It's two dimensional. Right. So if you have an image, you expect that there's some 2D structure, right. It's not quite the same as. You can't just take all the pixels and just put them in a line and expect that this represents the same kind of thing. Right. This is not really a meaningful thing here compared to this. So there's a 2D structure to this image that's kind of relevant to, you know, what's contained in the image. Right. In this neural net that we have here, a convolution layer is kind of a way that we can account for the 2D structure here. So we kind of connect the neurons in a way that relates to the fact that the image is two dimensional, but otherwise kind of the details about like, oh, okay, there should be a ramp here and a pooling layer here and a ramp and a pulling layer and so on. This stuff is just kind of found to work and you can kind of back justify it. But the structure like this full structure is not just designed in a kind of, you know, architecturally sound way necessarily. It's just kind of ad hoc. And I have one more question about this. So what type of mathematical operations are being done by a neural network? That is the network itself, not the training process that we'll talk about in a little bit. These kind of the types of operations you'd find in calculus, like, you know, taking integrals or derivatives, a linear algebra, stuff like matrix multiplication. Is it number theory? Are we doing a bunch of modular arithmetic or is it topology? Are we measuring shapes of things in some explicit way?

It.

Okay, we've got a pretty strong response here. And by pretty strong I mean 98% for linear algebra. And indeed that is true. So let me kind of maybe go over this again. I feel like I've done a very good job of explaining some of these examples here. So let me talk about this part here. Let me make this a little bit smaller so maybe I can fit this part on the screen as well. Yeah, there's a trade off between information density and legibility when sharing screens and webinars. But here's the idea. So you have some inputs here, in this case for the kind of point classifier, whether it's going into one of those three different boxes. You've got an input of two numbers, X and Y. Those two inputs get fed to the first layer, which in this case it's just two neurons. Those neurons then send a signal forward to all of these neurons in the next layer. When, say, this neuron sees a signal from this one and this one, it just takes the signal, say 0.5 and negative 0.8, and it weighs them differently. So these different arrows have different strengths. So it's going to say, oh, I think this one's really important. And I think this one's not particularly important. Maybe it does something like that. So maybe it has a big weight for this one and a small weight for this one. So all it's doing is it takes these values and then it multiplies them by how strongly it's connected to each of the neurons producing those values. So it multiplies them by their respective connection weights. Then once it has weighted them, it's given them a weighted or relative importance. It then adds them together. Then there is a step where it Adds a constant. But that part's not particularly interesting or super useful to talk about in this context, so we'll not talk about it. And then after it's added everything together, it applies this thresholding or activation function. So maybe when it adds these two things together, we get some big crazy negative number. And what's been found is that that's not very helpful to the neural network to have that. So what we do is we set that, say to zero. It's if we have a ramp function like this, or we weight it in some way such that, you know, you don't wind up with. With super large negative numbers. In this case, we can keep any positive numbers, but anything that's negative we set to zero. And then once we've figured out those, then this one has a new value. So it has taken, you know, these two N, it's multiplied them by their connections, it's added everything together, and then it does some sort of thresholding operation. And then the same thing is true for this one, which again takes these two things in with two different strengths. And this one takes these two things in again with two different strengths and so on. So they all end up with a number, they all get thresholded and they get sent to the next. They get sent to the next step. So in this fully connected network where each neuron in a layer is connected to each of the neurons in the layer above, what's going on is that, say again, let's look at this one. This guy sees an input from all four of these. He has different weights for the four neurons. Like, it's very strong for this one and fairly strong for this one, comparatively pretty weak for this one. So takes all these inputs, multiplies them by their connection weights, adds them together and thresholds. You keep doing this, you keep doing this, you keep doing this until at the end, you have a neuron which kind of, in this case we just have an output that's one dimensional. It's going to be in one of the three boxes, zero plus one or negative one. And so at the end, you wind up with just one neuron that should have a value of negative 1, 0 or plus 1, ideally is the idea. Hopefully that helps to clarify the operation there. And if you write out that operation in full, that's where the kind of linear algebra part of it becomes clear. You've got elements of your input vector which are X and Y, and they get multiplied by some weight matrix and it just keeps getting propagated forward and forward and forward and forward and in the end, all this leaves you with is a number, ideally plus one, zero or minus one. So you do all this linear algebra and at the end, in this case you get a scalar, oftentimes you get a vector and you can get things that are more complex. But the point is it's always just some numeric operation. You do some linear algebra on some input numbers and out comes either a number or some numbers. And then something like ChatGPT, you get a big list of numbers which correspond to the probabilities for the different words that we talked about yesterday. And again here more complicated networks can generally produce, they can better replicate a function that you'd like them to replicate. Okay, here. Actually there's a question Julia answered from Angelos which asks what determines the values of these weights? And we'll talk about that shortly. But as Julia notes, in the beginning you just set them totally at random. And we're about to see that. So let's kind of return to the task of digit classification. You know, when we're talking about picking the closest point, that is something that is well defined in a mathematical sense. Like there are other non neural nets algorithms which can do that. They do it, you know, more easily. But digit classification is not something that is well defined in the mathematical sense traditionally. So, for example, what is this digit, right? Is this A1? Probably right. According to the official data set I got it from, it's a 1, but it could be a 7 and it could also be a 9 where the loop is kind of really scrunched up, right? And pixelated. Like you could see somebody writing a 9 that looks like that. So it's not really well defined that this, you know, mathematically is a one. It, you know, the person meant to write a one. So we need some sort of model which can emulate human like tasks to recognize that they probably meant to write a A1, even though it could be a 7 or a 9. So if we have a good model for doing human like tasks like say a neural network, we might expect that it might kind of be ambiguous in the same way that a human mind is ambiguous. Right? Like if I look at this and think the data set says it's a one, but I think it could be a seven or maybe even a nine. If we kind of plot this thing according to the neural network, we can see that if we plot a bunch of ones and a bunch of nines and a bunch of sevens together, the thing in question, the neural network actually thinks it looks more 70 than 1e it's more in this kind of sevens group than in the ones group. So I just want to note again something we mentioned yesterday. We can't say exactly what the network is mathematically doing to make the distinctions between ones and nines and sevens. But whatever it is doing, it's pretty close to what a human does. So the network seems to have learned some idea about the fundamentals, say oneness or four ness or sevenness or nine ness, that make these images representative of the digits that they actually represent. And so the outcome is pretty clearly human. Like, even if it's not clear that the internal understanding of the neural net matches up with the way that humans do it, what matters is that in the end it makes similar judgments to what a human would make. And the second thing to note here is that this is true of ChatGPT as well. So we know that ChatGPT has an effective understanding in quotes of language. It can absolutely put together human, like text. But this understanding is not necessarily something that clearly maps onto the human understanding of language. So it's not clear that inside Japan Chat GPT, you'll notice if you say it enough times, just becomes very hard to say. It's not clear that what's going on inside that network maps neatly onto the human understanding of language. But the question is, if these neural nets have this understanding of, say, language or digit recognition or whatever else, how do they come to have that understanding, whether or not it actually maps onto human understanding? Here's where we get into machine learning. So part of the reason that neural networks are so useful, and maybe Julio can tell us a relative percent of maybe why this is why neural nets are so useful. But they can learn via a process called training. So it's not just a set it and forget it algorithm. These neural nets can be trained to improve at whatever task you've set them to. So before a network has been trained, the parameters, that is the weights and biases, like the connection strengths in that network, they don't have any particular value and you just kind of set them at random. What the training process is, is you show the network a bunch of examples and then it will adjust its parameters, its weights and biases gradually until the net does a good job of kind of replicating the things you're showing it. So let's kind of get into an example here. Something that's nice in terms of, say, the blackbox digit knower is that you don't have to provide any details. You don't have to, like, give it A7 and say, okay, network, look here. A7 has a diagonal ish line here at this particular angle. And it's got a horizontal bar at the top. And you'll sometimes see that it has a horizontal bar through the middle. And maybe the relative line lengths are this and that. Instead you just say, hey, look, this thing's a seven. That's a seven. This is also a seven. And guess what? This is also a seven. Learn what it is that makes a seven a seven. Have fun. And by that same process, a neural net can learn, say, how words should fit together or what the differences are between cats and dogs when it sees an image of each. So we'll go to an even simpler example in order to kind of understand the learning process. So let's say we've got this function, very simple function, kind of a simpler analog of the, of the closest point detector function. We've got this. It's just a step function. You know, if you're between negative one and about one third, you should read this. You're between negative one third and one third, you should be one. And if you're greater than one third, it should be zero. So it's a piecewise function. But ideally, the network should be able to figure out how to approximate that. The important thing that we're kind of interested in at the outset is that there's one input which is just represented on this horizontal axis. And that just ranges in this case from minus one to one. And then the output is going to be one of these three values. Negative 1, 1 or 0, 1 input, 1 output. So theoretically, the network then could look something kind of like this. And again, this is just totally random. This is not like some wise thing that if you don't understand the shape of this, you're missing something. This is just a totally random thing. We've got an input and we got an output and everything in between. We're just saying, put some stuff here and connect it. Who knows? I just want to really push that. It's a totally random network. The random number of neurons, random connection structure. In this case, it's fully connected. But a random number of layers, random number of neurons per layer, it's all just, you know, randomly picked here. The only thing that actually matters with respect to our problem is that we have one input and one output. So whatever it is, it's a thing and it's got one input and one output. And what we want to do is we want to check whether this network can be trained to reproduce the graph above. And so if we just Totally randomly pick connection weights for the connections here. These are kind of the best we can do at approximating the function. So these are all really bad, obviously. Right. But you shouldn't be surprised by that. If you just randomly pick a bunch of weights, you shouldn't expect that your network is going to magically just reproduce the correct function automatically. This is where the idea of training comes in. You take the network and you feed it a bunch of input output examples to help better learn how to approximate the function. And again, this is a bit human. Like, if you were given a bunch of X, F of X pairs, a couple of parameters to play with, you could figure out a lot of functions too. So, for example, you know, maybe I just, I point at you and I say, I want you to figure out this function. What function is generating these numbers? Got 1, 2, we've got 2, 4, we've got 3, 9, we've got 4, 16, and so on. And so if you looked at this and if I gave you enough examples in this case, you know, humans are pretty good at pattern recognition. So it's pretty. I changed my mind about which function I wanted to use early on. Humans are pretty good at pattern recognition. So you can kind of quickly see, oh, okay, this is representing the function F of X is equal to X squared. But the point is, if you look at enough examples, you can learn something about how the inputs relate to the outputs. And the network is going to do the same thing. We're going to give it a bunch of these input output examples, and then through those examples, we hope that it will learn how to replicate the function that is generating those input and output pairs. So if we Give this network 10,000 examples, this is the best we can do. If we give it 100,000 examples, this is the best we can do. And so on. As we give it more and more examples, it's able to learn better and better what the shape of the function is. And you can see it does require a good number of examples here. So the way that the network learns by looking at examples is that. And we'll go into a little bit more detail about this, but we give a network these examples and it kind of consider these examples to be the facts on the ground. And so it will keep adjusting its parameters so that if it puts in an input, it gets an output closer to what's expected. So let's say it puts in an input and its parameters are totally busted and random and wrong and they don't make any sense, and it gets a crazy output what it's going to do is say, oh, okay, I see that that output is really far from what I should get. So what I'm going to do is I'm going to adjust my parameters a little so that the difference between what I'm getting and what I should get is smaller. I'm going to minimize the difference between what I'm getting and what I should get. And so as you give it more and more and more examples, the error that it experiences between what it has and what it wants gets smaller and smaller and smaller, and it tends to follow this sort of curve. So each step of training, the network computes some outputs, and then it calculates a loss by comparing them to the ground truth that we have provided. And then it updates the network's parameters to reduce that loss. The question I think then naturally is, well, how does it know to change the parameters to make the loss smaller? Isn't it possible that you could change the parameters and the loss would get bigger? Like, you know, if you just kind of nudge the parameters, can't you possibly, you know, make the situation worse? The answer is, yes, you could. But fortunately, there is kind of a little calculus or numerical analysis at hand here. Let's simplify this a little further. Let's assume that we only have two weights in our entire neural network, just two. We're not going to have all these crazy connections between all these networks. We just got two weights. That's it. A simple, beautiful world. What we can do then is we could compute a function of the two weights that tells us how far away we are from the answer that we want to get. And so here, redder numbers are higher, we're further away from where we want to be, blue numbers are lower, we're closer to where we want to be. You kind of see that in the 3D plot here. And so what we can do is just say, well, I want to adjust my parameters, my two weights, such that I get the smallest possible difference between my. My desired outputs and the outputs my network is calculating. So if you have this kind of picture, and if you've had calculus, you might know that you can quickly find a way that points to the bottom. You can follow the path of steepest descent. So basically, the network has all these different weights. It knows that by changing the weights, it's going to change how far away it is from the ideal output. And what it can do is then say, oh, okay, well, if I know how changing my weights changes my error, what I can do here is I can Just follow the path of steepest descent and I can get to this place where I can pick values for the weights, where I have the lowest possible, where I have the lowest possible error, I can get as close as I can to the desired output of my network. And this is kind of another fact from calculus. This process will always result in hitting a local minimum, if not a global one. You could imagine that over here, maybe if you take another path around outside this curve, if you take another path around, maybe there's a really steep hole over here that you don't reach by this method, but you will at least reach a local minimum. And something that's pretty cool is that even if instead of just having two dimensions, two weights that you're trying to minimize the loss over, if you have, say, billions or trillions, like in ChatGPT, this process still works. And in a kind of surprising turn of events, if you have more weights, in some sense, it's actually easier to find the global minimum and totally minimize the loss. As in two dimensions, you can see, you can kind of get stuck following one path and then you kind of wind up at your local minimum and that's it. If you have a bunch of dimensions, you can imagine there are more paths to get to your global minimum. And you can imagine here that I'm kind of waving my hands around in a really ridiculous way indicating the multidimensional paths that you can take through the very high dimensional space to the global minimum of the loss. So because a neural net is just a big mathematical function, tools from calculus and numerical analysis let you very quickly, in a sense, minimize the error that your net is experiencing. And one thing to note, I'll scroll back up a serial scroller is that it's not the case that there's like one specific value of all the weights. Maybe you have a billion weights. There's not just one length, one billion vector that specifically points to the minimum. Like you can see here, you can have a loss function where you could be here or here, and you have the same total loss, but you have different values of the two weights. So when you minimize the loss in your network, the weights are not going to some like, preordained truth, they're just kind of following along, following along. And eventually you end up at a place where they're hopefully minimized and they are locally minimized. So here are some examples from the earlier net where different weights are used, but they all produce the same output more or less, and over the range of interest, they all produce pretty much the same output, but those weights can result in some pretty wildly different outputs beyond the range of interest. Now we might look and say, oh, okay, well, this one and this one are pretty good, right? That looks very reasonable. But the network doesn't know anything about what came before or what comes after the training data. So we don't have, you know, we don't have anything to say about the truth of what happens before or after the training data. So the network is not wrong to say that, you know, maybe this one's flat and maybe this one goes up, or maybe they're both pretty flat, or maybe this one goes up really steeply and this one kind of goes up a little bit. You know, there's nothing that's true or false or correct about any of these. It's just a matter of, you know, we haven't provided it the information, so it's going to just make some guesses and that's fine. We've only trained it to be good in this region and all of these are pretty good in that region. Questions here. Which of the following is an advantage of the machine learning process? A, you do not have to specify the important features for the network to learn. B, training can use a basic concept from calculus to minimize the error or the loss, or C, the number of parameters means that there are multiple places in parameter space, so to speak, that could solve your problem. Or D, all of the above and once again, water.

It.

Okay, in the meantime, we've had a Strong showing with 95% for all of the above. You know, I'm cognizant of the fact that when you see all of the above, these questions tend to be a little easier. But given that the poll questions are not a quiz that you're taking for some credit or something, they're just meant to kind of reinforce what you've seen in the lecture so far. I think that's okay. So indeed, all of the above are true. So let me just kind of indicate these things point by point really quickly so you don't have to specify the important features for a network to learn. So for example, when the network above was trained to learn this shape, we never said, hey, by the way, network, remember, it's really important that it's a step function, okay? And remember that it only has like two full steps and then that results in three plateaus. We never specified any of this. We just said, here are some points that exist on this weird curve, quote, unquote. It's not really a curve, it's discontinuous or it's not smooth. Rather we just said, you know, hey, here are some points. Figure it out. So that's a really nice thing about neural networks. And the way it does figure it out does use this basic concept from calculus where we could just follow the path of steepest descent and eventually we wind up in some sort of minimum for our error or our loss. So we get closer and closer to the truth by perturbing our parameters a little bit. And it's also nice that you have all these different parameters available and there are many different places in parameter space that you could get to that could minimize your error or at least solve your problem. So these might all have slightly different total errors, but they all more or less solve the problem. Like I expect that this one has a little bit higher of an error, but they all more or less solve the problem. So these are all positive things about the machine learning process. And then there's one quick question about training process. Will it always hit a global minimum in the laws? Is that true or false? Okay, let's see these results. I was checking out the questions. Okay. Indeed, as is often the case, again with question design, if you have an always in your statement, typically the answer is false. So you can metagame it a bit. Although of course I know you can metagame it. So I will sometimes meta metagame it and give you something that is always true. That said, this one is indeed false. And we can see that up here. So again, you always follow the path of steepest descent, but the path of steepest descent is only ever guaranteed to take you to a local minimum. It could be that over here again, there's a spot that we could have gotten to by a more gradual path. So it would be like a slow, nice set of steps over here. But then over here, maybe there's a giant hole that has an even lower error. That's possible. And again, in higher dimensions, it's actually easier to find those global minima, but you always get at least a local minimum. Okay, let us move on to the last part of today's lecture. So we've kind of seen a neural network do things like guess this weird step function or figure out that three dimensional digit, not digit, but point closeness classification task. We've seen it. We've seen a network learn how to recognize digits. And we know from ChatGPT that a network can even. And we'll talk about how it does this tomorrow. But we know that the network can even write human like text. So you might think, well, writing human like Text. Now, that's a complicated thing, and it might take billions or trillions of parameters, but that just kind of tells us that if we add more and more and more and more parameters, we could make a network that could do anything. This is not actually true. It's an understandable thing to think, but it's not actually true. You can keep adding more and more and more weights, but that doesn't mean that a network can learn to do anything. The way Stephen puts it, which I think is a nice way to think about this, is this concept of computational irreducibility. So let me explain what's meant by that, using an example that I like. So when you throw a ball, right, you got a baseball, you got a. Whatever kind of ball you throw it. You can use Newton's laws of motion to predict the motion of the ball in the future, right? You don't have to wait for the ball to move through its arc and see what will happen. You don't have to look at the ball moving through space step by step and see what goes on. You can just say, well, I know that given the initial angle, if I have the initial velocity here, you know, I can figure out exactly where the ball is going to land. So if I have, you know, a different angle or if I have a different speed, I can always figure out where the ball will be at any given time in the future. And that's that. I don't need to, you know, I don't need to sit here and wait for the ball to land. I can just say, okay, I know where it's going to end up. I can just kind of take a shortcut from, say, this time to this time, and I just know the result. So we're able to reduce the computation. Instead of having to go step by step through time here, you know, every little micro instant of time calculating the arc of the ball, we can just go straight from here to here using Newton's laws, and we're good to go. So that's kind of something that we can learn. We've picked up a way of computing this sort of thing using just a little bit of information. We have, say, the inputs of the initial speed, the initial angle, and the time. And then what that lets us do is we can get an output of whatever, wherever the ball is going to be at that time, right? So we can kind of take a shortcut. However, not all processes can be reduced like this. People who have been, you know, paying attention to Wolfram for any amount of time will probably have run into cellular automata. So here's an example where, you know, it kind of looks like there's an interesting structure here. But the point is, even though there are simple rules to this thing, which I have not enumerated here, I enumerate in a later talk, there are very simple rules that govern the behavior of this thing, but they don't let you skip forward in time. We can't say, oh, okay, we're starting from one blue pixel in the middle, and we're going to run for however many steps. And then that's what that row is going to be like. There's no way to do it. We have to go from this step to the next step, to the next step, to the next step to the next step, the next step. We have to follow every single step along the way. So, in a sense, there's nothing to learn here. So you could keep adding more and more and more parameters to a neural network, but there's no way for the neural network to just learn what any given output at the end state will be. There's no shortcut from beginning to end like there is with basic physics here. So you can keep adding more and more stuff to your network. And what it will do instead of learning is it will memorize the result. If you give a network enough space, it will just memorize the input output pairs, and it will not actually have learned anything. So it won't have learned the rules or the structure. It will just know, okay, if you give me this input, I'm going to give you that output and that's it. And that's kind of a fundamentally different thing from this sort of learning about a shortcut from here to here. So again, the kind of point is that there exist processes which cannot be reduced to simple rules. And these things are things that even a very big network can't really learn to do. There's something else I wanted to note about this, Right? Let me give a quick example of overfitting. So suppose I have some data points. Let's do this. And we'll have x go from 0 to PI 0.1. So as a human, we can look at this, right? And we can say, oh, okay, this kind of looks like a sine curve, and then it's got a little bit of noise on top. No problem. Right. But the issue is what you don't want to happen is you don't want. Let me check. I always mix up two functions here. Yeah, What you don't want is to try and learn the shape of this thing. Which should be. We can show our list plot, and then we can show a plot of sine of x. X goes from 0 to 2 PI. We'll do. So here we can see we've got our experimental data that's a little bit too thin. Let me get rid of that. And we've got our. You know, what we can kind of recognize as the ground truth. If you give a network too much leeway, what can happen is that it can do something that looks like this. Let me break these up so you can see a little bit better. Make it. Well, we wouldn't want to do green for red, green, colorblind reasons. Let's do purple. If you give the network too many parameters and you just keep let it training forever, what it will do, and there are ways to prevent this, but what it will do kind of by default is it will just learn the input output pair so that really properly learning the underlying shape. So what you would want, given this input data, ideally, is that you learn that it's always just a sine curve. That's no problem, right? But if you give a network enough parameters and enough examples, and maybe it's something that it can't figure out, maybe like it can't figure out again, the cellular automaton, what it will do is it will just say, oh, okay, no problem. I know exactly how to match the input and output pairs for you. No problem. That's great. And clearly this is bad and not what you want. So in the case of these computationally irreducible things like this, the network can't learn how to go from an input to an output. But if you give it enough parameters, it could learn to memorize the inputs to the outputs. So you have to be a little bit careful about that. But the point is really just that fundamentally there are some things that networks are not really well suited for, even if you give them a lot of training and a lot of parameters. So even a really big network can't do everything. There's a quick question about that and then we'll be done for the day. So which of the following are things that a neural net could be good at learning to do? And you should be able to select multiple options here. Computing where a ball will land if you throw it a certain way. Computing, say, the kinetic energy of a block that it would have at the bottom of a fall from a certain height. Maybe computing the N step of a cellular automaton, Maybe factoring a large prime number, Distinguishing between two types of bacteria based on topographical images of bacteria. Couple of examples here. Okay, people are thinking about their responses here. Okay, they're coming up. Things are filling out. All right. We have results that are ones that I like, which are primarily A, B and E. So these are all things that have kind of implicit rules where you can again, kind of shortcut your way from beginning to end. Well, A and B are at least, these are kind of physics things. So you can imagine you could provide a network with, you know, an initial speed and initial angle or equivalently an initial velocity at a time. And those would be the inputs, and then the output would be, you know, where it lands or where it is at that given time. And just like it learned the step function, a neural net probably would do a pretty good job of being able to learn that function because it's a basic kinematics function. Similarly, computing the kinetic energy of a block at the bottom of a fall, well, if you give the inputs as the mass and the height, and then you give the output as the kinetic energy, it should be able to figure out that, you know, it's square root of 2gh is the velocity and you square it, you multiply by 1/2 m and you get the thing. So we'll be able to figure out that function too. However, computing the nth step of a cellular automaton is not so easy because it has to go through each step individually. There's no nice way to kind of learn that process. And so this is the sort of thing that if it does it at all, it would do it via memorization. The same thing goes for factoring a large prime number. To my knowledge, this is a thing that neural nets are not good at. They can't really follow these algorithmic steps. They can learn functions, but they can't really follow more general algorithmic computational steps super accurately. This is also why, you know, sometimes, you know, you ask ChatGPT what you might think would be a simple question. You know, you sometimes see screenshots of this. How many A's are in, you know, some word with four A's, and it's like, there are six A's. And you say, no, there are two ways. And it's like, oh, I'm sorry, yeah, there are two A's. It's. It can be not so good at these sorts of algorithmic tasks where it actually has to kind of follow some steps of that fashion. And, you know, to be fair, that's true of people too, right? People are not good at just computing the next steps of the cellular automaton. Without going through each step individually. And people make these sorts of counting mistakes too. And the last example is a little self indulgent thing I did in grad school, which was about distinguishing between two different inoculated bacteria types on an agar plate. You could train a neural network to very quickly distinguish between is this bacteria A or bacteria B based on how bumpy the surface was. At least that's my suspicion about what it did, because you can't really know the heart of a neural network. So again, A, B and E are the answers there. And then just a quick summary at the end. So for all we talked about, you know, the structure and, you know, connections and all the, all that jazz. Neural nets are essentially big mathematical functions and they have a lot of parameters. Just like yesterday, we had this kind of, you know, F of x is equal to ax squared plus bx plus C. This is a model F which has an input X and parameters A, B and C. Fundamentally, a neural net is no different. You've got a net model, it's got its inputs, and then it has, you know, some horrible number of parameters and also the input, and then that results in some output. The way you teach a network about how to figure out how to take an input and some parameters and get an output is that you show it a bunch of input and output examples and it will continuously adjust the parameters in order to best recreate the input output pairs that you feed does that by gradient descent. So nets with more parameters are capable of learning to perform more complex tasks in general. So you can imagine if you have to learn a really complicated function, but you only have, you know, two numbers to describe it, that might not be enough. Like if you had to learn that step function we had from before, just giving you two numbers to play with, that's not enough to learn the step function in terms of just like transforming the input into the output. So more parameters means that you can do more. However, while it does mean that you can do more and more complex human like tasks, it does not mean that you can do anything specifically. There's a weakness when it comes to general computation. And again, this is kind of represented in, thanks to the zoom in this badly scaled thing here. So you have a neural network which just learns to replicate functions. If you have more parameters, you can better replicate a function. Neural networks learn by training on input output pair examples. And the more examples you show them, the better they get at reproducing the input output pairs. They learn by gradient descent. They reduce their loss by updating the parameters repeatedly each step. So each time they run through the process of training, they're stepping, stepping, stepping, stepping all their parameters to lower the loss and get a better approximation of the function you want. And then the final thing was just the difference between reducible kind of learnable computations and more general, possibly irreducible computations. And this is something that we'll talk about again over the next two days and then we'll talk about in a lot of detail on the last day when alan from the WolframAlpha team is here because he'll be talking about integrating Wolfram Language into chatgpt, which, what do you know, Wolfram Language is perfectly suited for doing these sorts of more general computations. So just like people, if you have access to a tool that can do these more general computations, ChatGPT can kind of COVID up a big hole in its, in its knowledge and its capabilities. All right, that's it for us today. Tomorrow we will be talking about embeddings and the internal architecture of CHAT gbt. So that'll be a very fun one. We'll get a little bit more into the weeds in some places and just cover some nice high level ideas and others. So I'll pass it back to Cassidy to close us out. I do want to thank Mike and Julio for answering the very numerous questions we got today in the Q and A pane.

Yes, thank you Julio and Mike for that allowing Arbyn to really be able to just focus on presenting. That's much appreciated and thank you again everyone for being here. As always, we've got that survey, I kind of changed it up. Instead of asking for what you want to see in review time, kind of what more examples and like what functionality examples you would like to see in these sessions. And as always that community thread is there for you to use in between the sessions. Arvin is one of our best presenters in terms of reviewing and giving thoughtful and resourceful responses. So thank you again everyone. We will be back here tomorrow, have a.