Learn how to compute forward propagation through an L-layer neural network. For each layer, calculate z = Wa + b, then a = g(z), and repeat through all L layers to get your prediction. The vectorized version stacks training examples as matrix columns so you compute predictions for all examples at once.
In the last video, we described what is a deep L layer neural network, and also talked about the notation we use to describe such networks. In this video, you see how you can perform forward propagation in a deep network. As usual, let's first go over what forward propagation will look like for a single training example X. And then later on we'll talk about the vectorized version where you want to carry out forward propagation on the entire training set at the same time. But given a single training example X. Here's how you compute deactivations of the first layer. So for this first layer, you would compute z1 equals w1 times x plus b1. So w1 and b1 are the parameters that affect the activations in layer one, right? This is layer one of the neural network. Um, and then you compute the activations for that layer to be equal to G of Z1. Um, and if the activation function G depends on what layer you're at, then maybe we'll index that as the activation function for layer one. So if you do that, you've now computed the activations for layer one. How about layer two, say that layer? Well, you would then compute z2 equals w2 a1 plus v2. And then, so the activation of layer two is the weight matrix times the output of layer one. So it's that value plus the bias Vector for layer two. And then A2 equals the activation function applied to Z2. Okay, so that's it for layer two, and so on and so forth until you get to the output layer. That's layer four, where you would have that Z4 is equal to the parameters for that layer times deactivations from the previous layer plus that bias vector. And then similarly, a 4 equals G of Z4. And so that's how you compute your estimated output Y hat. So just one thing to notice, X here is also equal to a zero, because the input feature vector X is also the activations of layer zero. So if we scratch out X and when I cross out X and put a zero here, then, you know, all of these equations basically look the same, right? The general rule is that ZL is equal to WL times A of L minus 1 plus BL one there. And then the activations for that layer is the activation function applied to the values z. So that's the general forward propagation equation. So we've done all this for a single training example. How about for doing it in a vectorized way for the whole training set at the same time, the equations look quite similar as before. For the first layer you would have capital z1 equals w1 times capital x plus b1, and then a1 equals g of z1, right? And bearing in mind that X is equal to a zero, these are just, you know, the training examples stacked in different columns. You could take this, Let me scratch out X, you can put a zero there. And then for the next layer, it looks similar. Z2 equals W2, A1 plus B2 and A2 equals G of Z2. We're just taking these vectors Z or A and so on and stacking them up. This is Z vector for the first training example, Z vector for the second training example, and so on down to the nth training example and stacking these in columns and calling this capital Z, right. And similarly for capital A, just as capital X, all the training examples are column vectors, stack left to right. And then at the end of this process, you end up with Y hat, which is equal to j g of z4. This is also equal to a4, and that's the predictions on all of your training examples stacked horizontally. So just to summarize our notation, I'm going to modify this up here. Our notation allows us to replace lowercase z and a with the uppercase counterparts, because that already looks like a capital Z and, and that gives you the vectorized version of forward propagation that you carry out on the entire training set at a time where a zero is x. Now, if you look at this implementation of vectorization, it looks like that there is going to be a for loop here, right? So it's left For l equals 1 to 4 or for l equals 1 through capital l that you have to compute the activations for layer one, then for layer two, then for layer three, and then then for layer four. So it seems that there is a for lo here. And I know that when implementing neural networks, we usually want to get rid of explicit for loops. But this is one place where I don't think there's any way to implement this without an explicit for loop. So when implementing forward propagation, it is perfectly okay to have a for loop that computes the activations for layer one, then layer two, then layer three, then layer four. No one knows. And I don't think there exists any way to do this without a for loop that goes from 1 to capital L from 1 through the total number of layers in the neural network. So in this place, it's perfectly okay to have an explicit for loop. So that's it for the notation for deep neural networks, as well as how to do forward propagation in these networks. If the pieces we've seen so far looks a little bit familiar to you, that's because what we've seen is taking a piece very similar to what you've seen in the neural network with a single hidden layer, and just repeating that more times. Now, it turns out that when you implement a deep neural network, one of the ways to increase your odds of having a bug free implementation is to think very systematic and carefully about the matrix dimensions you're working with. So when I'm trying to debug my own code, I'll often pull a piece of paper and just think carefully through some of the dimensions of the matrix I'm working with. Let's see how you could do that in the next video.
