Andrew Ng explains how to build deep neural networks with forward and back propagation. The lecture covers vectorization and the notation you need to implement these networks from scratch.
Welcome to the fourth week of this class. By now you've seen forward propagation and back propagation in the context of a neural network with a single hidden layer, as well as logistic regression. And you've learned about vectorization and when it's important, initialize the weights randomly. If you've done the past couple weeks homeworks, you've also implemented and seen some of these ideas work for yourself. So by now you've actually seen most of the ideas you need to implement a deep neural network. What we're going to do in this week is take those ideas and put them together so that you'll be able to implement your own deep neural network. Because this week's programming exercise is longer and just has a bit more work going to keep the videos for this week shorter so you can get through the videos a little bit more quickly and then have more time to do a significant programming exercise at the end, which I hope will leave you having built a deep neural network that you feel proud of. So what is a deep neural network? You've seen this picture for logistic regression and you've also seen neural networks with a single hidden layer. So here's an example of a neural network with two hidden layers and a neural network with five hidden layers. We say that logistic regression is a very shallow model, whereas this model here is, is a much deeper model and shallow versus depth is a matter of degree. So neural network with a single hidden layer, this would be a two layer neural network. Remember, when we count layers in neural network, we don't count the input layer, we just count the hidden layers as well as the output layer. So this would be a two layer neural network is still quite shallow, but not as shallow as logistic regression. Um, technically logistic regression is a, you know, one layer neural network. But over the last several years, the AI and the machine learning community has realized that there are functions that very deep neural networks can learn that shallower models are often unable to. Although for any given problem, it might be hard to predict in advance exactly how deep a neural network you would want. So it would be reasonable to try logistic regression. Try one and then two hidden layers and view the number of hidden layers as another hyperparameter that you could try a variety of values of and evaluate on holdout cross validation data or on your development set. Say more about that later as well. Let's now go through the notation we'll use to describe deep neural networks. Here is a 1, 2, 3, 4 layer neural network with three hidden layers and the number of Units in these hidden layers are, I guess 5, 5, 3, and then this one output unit. So the notation we're gonna use is gonna use capital L to denote the number of layers in the network. So in this case L is equal to 4. And so that's the number of layers. And we're gonna use N superscript L to denote the number of nodes or the number of units in layer lowercase l. So if we index this, the input as layer 0, this is layer 1, this is layer 2, this is layer 3, and this is layer 4. Then we have that, for example, n 1, that would be this. The first hidden layer would be equal to 5 because we have five hidden units there. For this one we have that n 1, 2. The number of units in the second hidden layer is also equal to 5, n 3 is equal to 3, and n 4, which is n capital L is the number of units. This number of output units is equal to 1, because here capital L is equal to 4. And we're also going to have here that for the input layer, n0 is just equal to nx is equal to 3. Okay, so that's the notation we'll use to describe the number of nodes we have in different layers for each layer L. We're also also going to use al to denote the activations in layer L. So we'll see later that in forward propagation, you end up computing alpha as the activation G applied to zl, and perhaps the activation is indexed by the layer L as well. And then we'll use wl to denote, you know, the weights for computing the value zl in layer L. And similarly, BL is used to compute zl. Finally, just to wrap up on the notation, the input features are called x, but X is also the activations of layer zero. So a zero is equal to X and the activation of the final layer, a capital L is equal to Y hat. So a superscript square bracket capital L is equal to the predicted output to prediction Y hat of the neural network. So you now know what a deep neural network looks like as well as the notation we use to describe and to compute with deep networks. I know we've introduced a lot of notation in this video, but if you ever forget what some symbol means, we've also posted on the course website a notation sheet or notation guide that you can use to look up what these different symbols means. Next, I'd like to describe what forward propagation in this type of network looks like. Let's go on to the next video.
