An instructor walks through NumPy vectorization techniques with concrete examples: matrix-vector multiplication, elementwise operations like np.exp and np.maximum, and how removing loops from logistic regression gradient descent makes code faster.
In a previous video, you saw a few examples of how vectorization by using built in functions and by avoiding explicit for loops allows you to speed up your code significantly. Let's take a look at a few more examples. The rule of thumb to keep in mind is when you're programming your neural networks or when you're programming logistic regression, whenever possible, avoid explicit for loops. And it's not always possible to never use a for loop. But when you can use a built in function or find some other way to compute whatever you need, it will often go faster than if you had an explicit for loop. Let's look at another example. If ever you want to compute a vector U as the product of a matrix A and another vector V, then the definition of a matrix multiply is that UI is equal to sum over J AIJ bj, right? That's how you define ui. And so the non vectorized implementation of this would be, you know, to set u equals np zeros would be n by 1 and then you know, for I, and so on for j, and so on, right? And then UI plus equals, you know, aij times bj. So now this has two for loops looping over both I and J. So that's a non vectorized version. The vectorized implementation would be to say u equals np.a, v and the implementation on the right, the vectorized version now eliminates two different for loops and it's going to be way faster. Let's go through one more example. This. Let's say you already have a vector V in memory and you want to apply the exponential operation on every element of, say this vector V. So you compute U equals, you know, a vector that's e to the V1, e to the V2, and so on down to e to the VN. So this would be a non vectorized implementation, right? Which is that first you initialize U to a vector of zeros and then you have a for loop that computes U the elements one at a time. But it turns out that Python and Numpy have many built in functions that allow you to compute these vectors with just a single call to a single function. So what I would do to implement this is import numpy as np and then what you just call U equals NP dot e to the V. And so notice that whereas previously you had an explicit for loop with just one line of code here, just V as an input vector, U as an output vector, you've gotten rid of the explicit for loop and the implementation on the right will be much faster than the one needing an explicit for loop. In fact, the numpy library has many other vector value functions. So NP.log of V will compute the element wise log. NP ABS computes the absolute value. NP maximum computes the element wise maximums. You can take the max of every element of V with zero v star star two. This takes the element y square of each element of V1, one over V takes the element wise inverse, and so on. So whenever you're tempted to write a for loop, take a look and see if there's a way to call a numpy built in function to do it without that for loop. So let's take all of these learnings and apply it to our logistic regression gradient descent implementation and see if we can at least get rid of one of the two for loops we had. So here's our code for computing the derivatives for for logistic regression. And we had two for loops. One was this one up here, and the second one was this one, right? So in our example we had N or NX equals 2. But if you had more features than just two features, then you need to have a for loop over DW1, DW2, DW3 and so on. So it's as if there's actually a for j equals 1 to nx, you know, dwj dwj gets updated. So we like to eliminate this second for loop. That's what we'll do on this slide. So the way we'll do so is that instead of explicitly initializing DW1, DW2 and so on to zeros, we're going to get rid of this and instead make DW a vector. So we're going to set dw w equals NP zeros and let's make this a nx by one dimensional vector. Then here, instead of this, you know, for loop over the individual components, we just use this vector value operation, dw equals xi times dzi. And then finally instead of this, we will just have DW device equals m. So now we've gone from having two for loops to just one for loop. We still have this one for loop that loops over the individual training examples. So I hope this video gave you a sense of vectorization and by getting rid of one for loop, your code will already run faster. But it turns out we could do even better. So the next video we'll talk about how to vectorize logistic regression even further. And you see a pretty surprising result that without using any for loops, without needing a for loop over the training examples, you could write code to process the entire training set. So pretty much all at the same time. So let's see that in the next video.
