Learn how to compute logistic regression predictions for all training examples at once using matrix operations. Instead of looping through each example, you represent inputs as X (n x m), compute Z = w^T X + b with broadcasting, then apply sigmoid to get activations A.
We've talked about how vectorization lets you speed up your code significantly. In this video, we'll talk about how you can vectorize your implementation of logistic regression so that you can process an entire training set, that is, implement a single iteration of gradient descent with respect to an entire training set without using even a single explicit for loop. Um, I'm super excited about this technique and when we talk about neural networks later, we. Without using even a single explicit for loop. Let's get started. Let's first examine the forward propagation step of logistic regression. So if you have m training examples, then to make a prediction on the first example you need to compute that, right? Compute Z using this familiar formula, then compute deactivation or to compute Y hat in the first example. Then to make a prediction on the second training example, you need to compute that. Then to make a prediction on the third example, you need to compute that, and so on. And you might need to do this M times if you have m training examples. So it turns out that in order to carry out the forward propagation step, that is to compute these predictions on all M training examples, there is a way to do so without needing an explicit for loop. See how you can do it. First, remember that we defined the matrix capital X to be your training inputs stacked together in different columns like this. So this is a matrix that is a nx by M matrix. So I'm writing this as a Python numpy shape, but this just means that X is a an X by M dimensional matrix. Now the first thing I want to do is show how you can compute z1, z2, z3 and so on, all in one step, in fact with one line of code. So I'm going to construct a one by M matrix. This is really a row vector where I'm going to compute Z1, Z2 and so on down to ZM all at the same time. It turns out that this can be expressed as W transpose the capital matrix X plus and then this vector B, B and so on. B, where this thing, this B B, B B thing, is a one by M vector or one by M matrix, or that is a M dimensional row vector. So depending on how familiar you are with matrix multiplication, you might see that w transpose x1, x2 and so on through xm that w transpose is going to be a row vector, right? So this W transpose will be a row vector like that. And so this first term will evaluate to w transpose x1, w transpose x2 and so on. W transpose xm and then when you add this second term, BBBB and so on, you end up adding B to each element, so you end up with another one by n vector, where that's the first element, that's the second element, and so on, and that's the nth element. And if you refer to the definitions above, this first element is exactly the definition of z1. The second element is exactly the definition of z2, and so on. So just as X was what you obtained when you took your training examples and stacked them next to each other, stacked them horizontally, I'm going to define capital Z to be this, where you take the lowercase z's and stack them horizontally, okay? So when you stack the lowercase x's corresponding to different training examples horizontally, you get this variable capital x. In the same way, when you take these lowercase z variables and stack them horizontally, you get this variable we should want to know by capital z. And it turns out that in order to implement this, the numpy command is capital z equals np.w.t as w transpose x and then plus b. Now, there is a subtlety in Python, which is that here B is a row number, or if you want to say, you know, one by one matrix, it's just a normal real number. But when you add this vector to this real number, Python automatically takes this real number B and expands it out to this one by m row vector. So in case this operation seems a little bit mysterious, this is called broadcasting in Python and you don't worry about it for now. We'll talk about it some more in the next video. But the takeaway is that with just one line of code, with this line of code, you can calculate capital Z. And capital Z is going to be a 1 by M matrix that contains all of the lowercase Zs lowercase Z1 through lowercase Zm. So that was Z. How about these values? You know, A, right? What we like to do next is find a way to compute A1, A2 and so on through AM all at the same time. And just as stacking lowercase x's resulted in capital X, and stacking means stacking horizontally, lowercase Z is resulting capital Z. Stacking lowercase a's is going to result in a new variable which we define as capital A. And in the program assignment, you see how to implement a vector valued sigmoid function so that the sigmoid function inputs this capital Z as a variable and very efficiently outputs capital A. So you see the details of that in the programming assignment so just to recap what we've seen on this slide is that instead of needing to loop over m training examples to compute lowercase z and lowercase a one at a time, you can implement this one line of code to compute all the z's at the same time, and then this one line of code with appropriate implementation of lowercase sigma, to compute all the lowercase a's all at the same time. So this is how you implement a vectorized implementation of the forward propagation for all m training examples at the same time. So to summarize, you've just seen how you can use vectorization to very efficiently compute all the activations, all the lowercase a's, sort of at the same time. Next, it turns out you can also use vectorization to very efficiently compute the backward propagation to compute the gradients. Let's see how you can do that in the next video.
