A neural network stacks logistic regression units across layers. The lecture introduces notation for layers and training examples, then previews forward and backward passes that compute activations and gradients like dA, dZ, dW, and dB.
Welcome back. In this week, you learned to implement a neural network. Before diving into the technical details, I wanted in this video to give you a quick overview of what you'll be seeing in this week's videos. So if you don't follow all the details in this video, don't worry about it. We'll delve into technical details in the next few videos. But for now, let's give a quick overview of how you implement in your network. Last week we had talked about logistic regression and and we saw how this model corresponds to the following computation graph where you didn't put the features X in parameters W and b. That allows you to compute Z, which is then used to compute A. And we were using A interchangeably with this output Y hat. And then you can compute the loss function L. A neural network looks like this. And as I'd already previously alluded to, you can form a neural network by stacking together a lot of little sigmoid units. Whereas previously, this node corresponds to two steps of calculations. The first is compute the z value. Second is to compute this A value in this neural network. This stack of nodes will correspond to a Z like calculation like this, as well as an A like calculation like that. And then that node will correspond to another Z and another A like calculation. So the notation which we introduced later will look like this. First we'll input the features X together with some parameters W and b. And this will allow you to compute z1. So new notation that we introduce is that we'll use superscript square bracket 1 to refer to quantities associated with this stack of nodes. It's called a layer. And then later we'll use superscript square bracket 2 to refer to quantities associated with that node. Really, that's called another layer of the neural network. And the superscript square brackets like we have here are not to be confused with the superscript round brackets which we use to refer to individual training examples. So whereas x superscript round bracket I Refer to the ITH training example, superscript square bracket 1 and 2 refer to these different layers, layer 1 and layer 2 in this neural network. But so going on, after computing Z1, similar to logistic regression, there'll be a computation to compute a1, and that's just sigmoid of z1. And then you compute z2 using another sort of linear equation and then compute a2 and a2 is the final output of the neural network and will also be used interchangeably with Y hat. So I know that was a lot of details, but the key intuition to take away is that whereas for logistic regression we had this Z followed by a calculation, in this neural network, here we just do it multiple times. There's a Z followed by an A calculation and a Z followed by an A calculation. And then you finally compute the loss at the end. And you remember that for logistic regression we had this backward calculation in order to compute derivatives, right? So computing your derivative DA DZ and so on. So in the same way a neural network will end up doing a backward calculation that looks like this, in which you end up computing DA2DZ2 that allows you to compute DW2 DB2 and so on in this sort of right to left backward calculation that is denoting with the red arrows. So that gives you a quick overview of what a neural network looks like. It's basically taking logistic regression and repeating it twice. I know there was a lot of new notation, a lot of new details. Don't worry about it if you didn't follow everything. We'll go into the details more slowly in the next few videos. So let's go on to the next video. We'll start to talk about the neural network representation.
