Learn vectorization in NumPy: how to compute linear model predictions with a dot product instead of loops. Vectorized code is shorter, clearer, and runs much faster because NumPy uses parallel hardware under the hood.
In this video, you see a very useful idea called vectorization. When you're implementing a learning algorithm, using vectorization will both make your code shorter and also make it run much more efficiently. Learning how to write vectorized code will allow you to also take advantage of modern numerics in your computer, but turns out, can be used when you write vectorized code to also help you execute your code much more quickly. Let's look at a concrete example of what vectorization means. Here's an example with parameters W and B, where W is a vector with three numbers, and you also have a vector of features x with also 3 numbers. So here n is equal to 3. So notice that in linear algebra, the index or the Counting starts from 1, and so the first value is subscripted W1 and X1. In Python code, you can define these variables W, B and x using arrays like this. Here I'm actually using a numerical linear algebra library in Python called NumPy, which is by far the most widely used numerical linear algebra library in Python and in machine learning. Because in Python, the indexing of arrays or counting in arrays starts from 0, you would access the first value of w using w squared bracket 0, the second value using w square bracket 1, and the third using w square bracket 2. So the indexing here goes from 01 to 2 rather than 12 to 3. Similarly, to access individual features of X, you would use x0, x1 and x2. Many programming languages, including Python, start counting from zero rather than one. Now let's look at an implementation without vectorization for computing the model's prediction. In code, it would look like this. You take each parameter w and multiply it by its associated feature. Now, you could write your code like this, but what If n isn't 3, but instead n is 100 or 100,000? It's both inefficient for you to code and inefficient for your computer to compute. So here's another way, still without using vectorization, but using A for loop. In math, you can use a summation operator to add all the products of WJ and XJ. For J equals 1 through N. Then outside the summation, you add B at the end. So the summation goes from j1 up to and including n. For n3, j therefore goes from 12 to 3. In code, you can initialize f to 0. Then for j in range from 0 to n, this actually makes j go from 0 to n1. So from 01 to 2, you can then add to F the product of wj times xj. Finally, outside the for loop, you add B. Notice that in Python, the range 0 to n means that j goes from 0 all the way to n 1 and does not include n itself. And more commonly, this is written range n in Python. But in this video I added a zero here just to emphasize that it starts from zero. While this implementation is a bit better than the first one, it still doesn't use vectorization and isn't that efficient. Now let's look at how you can do this using vectorization. This is the math expression of the function f, which is the dot product of W and x plus b. And now you can implement this with a single line of code by computing FP equals np. I said because the first dot is the period and the second dot is the function or the method called dot, but is FP equals NP w x. And this implements the mathematical dot product between the vectors W and x. And then finally you can add B to it at the end. This numpy dot function is a vectorized implementation of the dot product operation between two vectors. And especially when n is large, this will run much faster than the two previous code examples. I want to emphasize that vectorization actually has two distinct benefits. First, it makes the code shorter. It's now just one line of code. Isn't that cool? And second, it also results in your code running much faster than either of the two previous implementations that did not use vectorization. And the reason that the vectorized implementation is much faster is behind the scenes. The numpy dot function is able to use parallel hardware in your computer. And this is true whether you're running this on a normal computer, that is on a normal computer cpu, or if you are using a gpu, a graphics processor unit that's often used to accelerate machine learning jobs. And the ability of the numpy dot function to use parallel hardware makes it much more efficient than the for loop or the sequential calculation that we saw previously. Now, this version is much more practical when N is large because you are not typing w 0 times x 0 plus w 1 times x 1 plus lots of additional terms like you would have had for the previous version. But while this saves a lot on the typing, it's still not that computationally efficient because it still doesn't use vectorization. So to recap, vectorization makes your code shorter, so hopefully easier to write and easier for you or others to read. And it also makes it run much faster. But what is this magic behind vectorization that makes this run so much faster? Let's take a look at what your computer is actually doing behind the scenes to make vectorized code run so much faster.
