Learn how to stack forward and backward propagation across layers to build a deep neural network. The lecture covers the core equations Zl = Wl·A{l-1} + bl and Al = g(Zl), why you cache values during forward pass, and how backward functions use those cached values to compute gradients. Then it shows how forward pass, backward pass, and gradient descent fit together in a training loop.
In the earlier videos from this week, as well as from the videos from the past several weeks, you've already seen the basic building blocks of forward propagation and back propagation, the key components you need to implement a deep neural network. Let's see how you can put these components together to build your deep net. Here's a network with a few layers. Let's pick one layer and look at the computations, focusing on just that layer for now. So, for layer L, you have some parameters, WL and BL. And for the forward prop, you will input deactivations AL -1 from the previous layer and output al. So the way we did this previously was you compute Z L Bengals wl times al minus 1 plus bl, and then al equals g of zl, right? So that's how you go from the input al -1 to the output al. And it turns out that for later use, it'll be useful to also cache the value zl. So let me include this cache as well, because storing the value zl will be useful for backward, for the back propagation step later and then for the backward step for the back propagation step. Again, focusing on the computation for this layer L, you're going to implement a function that inputs DA of L and outputs DAL minus one. And just to flesh out the details, the input is actually DAFL as well as the cache. So you have available to you the value of ZL that you computed. And then in addition to outputting DAL minus one, you will output, you know, the gradients you want in order to implement gradient descent for learning. Okay, so this is the basic structure of how you implement this forward step. We're going to call the forward function as well as this backward step, which I'm going to call a backward function. So just to summarize, in layer L, you're going to have, you know, the forward step or the forward prop of the forward function, input Al minus 1 and output Al. And in order to make this computation, you need to use WL and Bl, um, and it'll also output a cache which contains zl. And then the backward function used in the backprop step will be another function that now inputs DA of L and outputs DAL minus one. So it tells you, given the derivatives respect to these activations, that's DA of L, how, what are the derivatives? How much do I wish, you know, al minus one changes? So compute the derivatives with respect to deactivations from the previous layer, um, within this box where you need to use WL and Bl. And it turns out along the way, you end up computing dzl. And then this box, this backward function, can also output DWL and dbl. And I was sometimes using red arrows to denote the backward iteration. So if you prefer, we could draw these arrows in red. So if you can implement these two functions, then the basic computation of the neural network will be as follows. You're going to take the input features A zero, feed that in, and that will compute the activations of the first layer. Let's call that A1. And to do that you need W1 and B1, and then we'll also, you know, cache away Z1. Now, having done that, you feed that to the second layer and then using W2 and B2, you're gonna compute the activations of the next layer, A2 and so on, until eventually you end up outputting a capital L which is equal to Y hat. And along the way we cached all of these values Z. So that's the forward propagation step. Now for the back propagation step. What we're going to do will be a backward sequence of iterations in which you are going backwards and computing gradients like so. So we're actually going to feed in here da of l, and then this box will give us da of l minus 1 and so on until we get da 2, da 1. You could actually get one more output to compute da 0, but this is derivative with respect to your input features, which is not useful, at least for training the weights of these supervised neural networks. So you could just stop it there. Along the way, backprop also ends up outputting DWL, DBL, right? Just use the parameters WL and BL. This would output DW3, DB3 and so on. So you end up computing all the derivatives you need. Um, and so just to maybe fill in the structure of this a little bit more, right, these boxes will use those parameters as well. Um, wl, bl. And it turns out that we'll see later that that inside these boxes will end up computing DZs as well. So one iteration of training for a neural network involves starting with a zero, which is X, and going through forward prop, as follows, computing Y hat, and then using that to compute this and then backprop, right, doing that. And now you have all these derivative terms. And so you know, W will get updated as W minus the learning rate times dw, right, for each of the layers and similarly for B, right? Now that you've computed back prop and have all these derivatives. So that's one iteration of gradient descent for your neural network. Now, before moving on, just one more implementational detail. Conceptually, it'll be useful to think of the cache here as storing the value of Z for the backward functions. But when you implement this, and you see this in the programming exercise, when you implement this, you find that the cache may be a convenient way to get this value of the parameters of W1, B1 into the backward function as well. So in the programming exercise, you actually store and you cache Z as well as W and B. Right? So to store Z2, W2B2. But from an implementational standpoint, I just find this a convenient way to just, you know, get the parameters copied to where you need to need to use them later when you're computing backpropagation. So that's just an implementational detail that you see when you do the programming exercise. So you've now seen what are the basic building blocks for implementing a deep neural network. In each layer, there's a forward propagation step, and there's a corresponding backward propagation step, and there's a cache to pass information from one to the other. In the next video, we'll talk about how you can actually implement these building blocks. Let's go on to the next video.
