Neural Network Representations (C1W3L02)

See me draw a few pictures of neural networks. In this video, we'll talk about exactly what those pictures means, in other words, exactly what those little neural networks we've been drawing represent. And we'll start with focusing on the case of neural networks with what's called a single hidden layer. Here's a picture of a neural net. Let's give different parts of these pictures some names. We have the input features x1, x2, x3 stacked up vertically, and this is is called the input layer of the neural network. So maybe not surprisingly, this contains the inputs to the neural network. Then there's another layer of circles, and this is called a hidden layer of the neural network. I'll come back in a second to say what the word hidden means, but the final layer here is formed by, in this case, just one node. And this single node layer is called the output layer and is responsible for generating the predicted value Y hat in a neural network that you train with supervised learning, the training set contains values of the inputs X as well as the target outputs y. So the term hidden layer refers to the fact that in a training set, the true values for these nodes in the middle are not observed. That is, you don't see what they should be in the training set. You see what the inputs are, you see what the output should be, but the things in the hidden layer are not seen in the training set. So that kind of explains the name hidden layer, just means you don't see it in the training set. Let's introduce a bit more notation. Whereas previously we were using the vector X to denote the input features, an alternative notation for the values of the input features will be a superscript square bracket zero. And the term A also stands for activations and it refers to the values that different layers of the neural network are passing on to the subsequent layers. So the input layer passes on the value X to the hidden layer. So we're going to call that, call the activations of the input layer a superscript 0. The next layer, the hidden layer, will in turn generate some set of activations which I'm going to write as a superscript square bracket one. So in particular, this first unit or this first node will generate a value, a superscript square bracket 1 subscript 1. This, this second node will generate a value now with a subscript 2 and so on. And so a superscript square bracket one. This is a four dimensional vector, or if you want in Python, I guess this be a four by one matrix or four column vector which looks like this and it's four dimensional because in this case we have four nodes or four units or four hidden units in this hidden layer. Then finally the output layer would generate some value A2, which is just a real number. And so, so Y hat is going to take on the value of a two. So this is analogous to how in logistic regression we had Y hat equals a. And in logistic regression we didn't, we only had that one output layer, so we didn't use the superscript square brackets. But with a neural network, we're now going to use this superscript square bracket to explicitly indicate which layer it came from. One funny thing about notational conventions in neural networks is that this network that you're seeing here is called a two layer neural network. And the reason is that when we count layers in neural networks, we don't count the input layer. So the hidden layer is layer one and the output layer is layer two. In our notational convention we're calling the input layer layer zero. So technically, maybe there are three layers in this neural network because there's the input layer, the hidden layer and the output layer. But in conventional usage, if you read research papers and elsewhere in the course, you see people refer to this particular neural network as a two layer neural network because we don't count the input layer as an official layer. Finally, something that we'll get to later is that the hidden layer and the output layers will have parameters associated with it. So the hidden layer will have associated with it parameters W and B. And going to Write superscript square bracket 1 to indicate that these are parameters associated with layer 1. With a hidden layer, we'll see later that w will be a 4 by 33 matrix and b will be a 4 by 1 vector. In this example, where the first coordinate 4 comes from the fact that we have four nodes or four hidden units in the layer, and 3 comes from the fact that we have three input features. We'll talk later about the dimensions of these matrices and it might make more sense at that time. But then similarly, the output layer has associated with it also parameters W, superscript square bracket 2 and B. And it turns out the dimensions of these are 1x4 and 1x1. And these 1 by 4 is because the hidden layer has four hidden units, the output layer has just one unit. But again, we'll go over the dimensions of these matrices and vectors in a later video. So you've just seen what a two layer neural network looks like. That is a neural network with one hidden layer. In the next video, let's go deeper into exactly what this neural network is computing. That is how this neural network inputs X and goes all the way to computing its output. Y hat.