#5 Stop Writing Slow Code: Learn NumPy Broadcasting & Vectorization

So hello everyone, welcome back to another video on NumPy. Yes, this is going to be a one-shot video. After this video, you are not required to watch any other video because in this one I am going to tackle each and everything from NumPy, what exactly you require for this entire machine learning journey, data analyst, data science, or whatever field you are choosing wherever the NumPy is required. So in this particular video, going to decode everything. Yes, and this video is practical one, right? So we are not going to jump on to any part of theory because we have already discussed in the part 1. So go check this video out and then come back to this video for the practical exposure and hands-on experience on NumPy. So let us begin. So if you guys are excited, do let me know in the comments. Just write down, I am excited, and let us begin with this video. So yeah, let us begin with our coding session. Now I have opened a Google Colab. You can open Visual Studio Code, Anaconda, wherever you wanted to write down the code. Just go grab it and let us begin with the code. I'm going to add a code cell. Very first step in any Python coding environment is whatever you need, you have to write it. So I'm writing import numpy as np. np is what? np is a nickname given to that. Now this will print— this will basically get that NumPy library in your notebook. And next step is to print the version. I want to check exactly which version am I using. So you have to write down underscore underscore version after np. This will show 2.0.2 for me. But for you, it is going to— might be it shows 1.8, 1.9, or maybe 2.2 or something, right? So this is a version. It means this is the level of NumPy which you are using, not the latest one, or maybe not even the very old one, right? So np.version is nothing but updated version of NumPy. And then I am going to create something. So now, as we all know, in NumPy we create arrays, right? So arrays means you can try to store anything in it. You can try to store any null values, you can try to store 1, 2, 3, 4, any numeric value, anything basically, right? So I'm going to create a variable c in which I am going to create a vec— a zeros array, right? A vector which is going to have 10 values and that has to be all zeros. So that is why I am using a function called zeros. Zeros will automatically add 0, 0, 0 in that data, but you have to define the size, that what should be the size. Now this is how you create any array, create any vector in NumPy. Now I wanted to check what is the memory size of any array that I have declared. So let's say c is— let's say z, I'm creating one more. So z is equal to np.zeros again, but this time I'm going to use two values as in the format of tuple, 10, 10. And then I will print the size of this array. So whenever we calculate the size, first we take the container size and then we multiply it with the item size, right? So I will just simply write this command, which is %d bytes %zsize multiplied by zitemsize, and then I will close the last bracket. It is showing 800 bytes. It means this is the memory size of my array, which I have created with 10 comma 10. You can simply do it like this. Otherwise, you can also use one function which is called— if I'm going to write down the same stuff, let's say I'm going to copy this, okay, and I will paste it up over here. Now look at it carefully. What I have done is I have written again %d. Why %d are we using? We are using %d because we are going to work with the digits. So when we work with digits, we have to write down %d. And obviously I want the digit value. I want that what is an integer value for the bytes. And then you have to come out of the bracket to mention %z.nbytes. What's the function? z.nbytes. This will also give you the same result. This is the second method of checking what is the memory size of any array. Got it? Now let us say I wanted to basically change some value. Let's say I already have them, this zeros. If you remember the last video, we discussed about how to exactly, you know, do indexing, how to get on to one value and change that value. So same application is over here. It means the same phenomena was work— will work over here. So I have the z variable, so I will write z Now within z, I want to change which element? So let's say I wanted to work on c, let's say, for which we are able to see the output. Let's say I wanted to change 5th item, so I will write down 4, obviously, because the indexing starts from 0. And now what is the value that I wanted to replace it with? So let's say I wanted to replace it with 85. So I can simply write it like this and then print c and check if 85 is there at the 5th position or not. So that's how you can change the values in the arrays that you have created. Now if I take you to this one, which is z, where we have written 2 values in the zeros function. Now let me tell you why 2 values. When we were writing single value, it was just the number of items required in the, uh, vector that we are creating. Now I will show you, if I'm going to print z, what will happen. So this is z. It means now I have rows and columns format. It means in one row there will be 10 values and there will be in total 10 rows. So let me explain it to you again. So again, what I am telling to try— telling to emphasize over here is when I am writing 0, 0, 0, I want zeros. I want a vector of zeros, but those vectors of zeros should have 10 rows and 10 columns. So you can create a kind of tuple and mention that, okay, 0 comma 0, and then it will initialize a vector with 0, 0. Next is, let's say I have a variable x and I'm trying to write down the function called arrange. You can arrange it as well. So let's say I have written 10 comma 50, and then if I will print x, it will create array like this. It means starting value and last value. Obviously, the last value every time in Python, in programming languages, goes one greater than what you are expecting. I repeat, end value has to be one greater than what you are expecting, right? So np.arange— arrange— it is not arrange, it is a range. It means a particular range that we are providing. So that is 10, 50, and this is the value of range that I want my arrays to have. Simple. This is a vector which we have created. This is a vector, correct? Now, is— let's say I wanted to just give one value, I can also generate the same thing. So let's say I'm writing v is equals to np.arrange, and I'm just writing 50. What will happen in this case? Will it work? Yes, it will, but it will be taking very minimum value, that is 0, and from 0 to that number which you have mentioned, it will create a vector. Done. If you wanted to reverse it, same string concepts will be applicable. So let's say I'm writing print v colon colon minus 1. If you have not watched this logic, go watch out because I have already explained it in detail what exactly this is. This is basically starting to end but from the last. Starting to end, everything I want to reverse but from the last. Start from the last, it means start printing from the last. So this is how it reverse the vector. You can see the same vector has been reversed. So that's how you reverse it. Now this is a vector. Vector is a kind of list. Vector is a kind of list. Vector is a kind of normal one-dimensional stuff, right? Now I am going to show you how you can create a matrix, which is two-dimensional. Matrix are two-dimensional. So how do we do it? We basically have to mention which type of matrix do we want. So I will use the same thing. So this time again, I will use v np.arange, and let's say the range is 9, only 9 values I want, but I want to reshape it. What do I want? I want to reshape it. So we will write down in the same line .reshape. What should be this shape? I want 3 and 3, 9 values, right? In total, it means 0 to 8 value it will generate. In total, we are going to get 9 values, and those 9 values I want to reshape into 3 comma 3. That's how I want to make it, right? Okay, I've written the wrong spelling, so let me just correct it. I was just wondering why it is showing wrong. Okay, now I will just print v and I will just run it. You can see now I have the same num— set of values. It means the starting— not just random values, but the range which I have provided, 0 to 8 values, but in rows and columns, rows and columns. So this is matrix. Earlier what we were doing is we were working on vectors. Now this is matrix. Now let us jump on to the next one. So the next what we have is we have to work on, uh, creating a non-zero element. So when we are to add a non-zero element, then how can we add it? So we again have to create one variable. So any variable, whatever you want to write, you can write it. Let's say vx is my variable. np. Np.nonzero. What do we want? We want to work with nonzero. What should be the nonzero value? So you can write down any value that you are trying to store, and then simply just go print vx. What will it do? You look at it carefully. So we created an array, right? So it says Yeah, so now you look at it carefully. So what we have is 1, 2, 3, 2, 5, 0, 1, 0. So basically I have tried to create an array, right? So this is what— this is which type of array? This is basically to find the indexes of non-zero elements. So if you wanted to check the indexes, this is how it goes, right? Now let's say I want to create 3 into 3 identity matrix. Now what is identity matrix? Those who have learned about mathematics and matrix in maths, then they already know what is matrix, right? So let's say I am creating I3 and then print ZX. Now this is called identity matrix in which we have the diagonals as 1, 1, 1. So this is how you create a identity matrix. Now let's say I wanted to work on 3 into 3 into 3 array with some random values. Now with some random values. So there are many, uh, places where you will be required to work on random values. Why do we need random values? So let's say there is a column in which you have the company data, obviously, for sure. Now in this company data you have some null values. Which values? You have some null values. Now either I can generate something randomly from the existing data, that's the first options. Second, I can take an average of the value if that is feasible for me. Third, I can take mean. Fourth, I can take mode. So basically, I might have to do something, but the very first step that I might get from my boss or manager is, okay, randomly generate something from our data and then put it there. So for that reason, we have to work with num.random. Okay, so you do not need to randomly go— do not need to basically specifically go and mention that, okay, I need a random library, because in NumPy also we have random function. So random function is random.random, and then you mention what exactly do you want to generate. I want to generate 10, 10. I want to generate 4, 5. I want to— whatever you want to generate, just mention it there. Correct. And then simply go fx and then run it. So you can see this is 3-dimensional kind of, right? So we have rows, column, and within rows also we have some values. So it means you can create an array which is 3 into 3 into 3 with the random values like this. You can create any array basically. If I'm going to take the value and I wanted to just enter two values, 10, 10, then you can get the array of 10, 10 as well. It means 10, 10 values you can get. Now let's say I want to get the minimum value and the maximum value which is generated by the random library or by the numpy random function. So I— then I will create two variables. So let's say I have a variable, I will just make sure that I will use a relatable one. Fx, right? So I will use fx— not fx— fx_min, fx_max. It means two values I'm taking. If you want to store, you can store it like this also: fx_min, enter, fx_max is equal to fx_max, and then you print fx_min, fx_max. Simple. Now you can see this is the minimum value which is there in our data, and this is the maximum value which we have in the data. You can even write it within the same line. So you will generally find it in the same line. That's how developers do. Let me just take it in the single line. Yeah, so this is fx_min, fx_max. The first variable will take the first value after equals 2, and the second value after equals to will take the second value. It means if you wanted to flip them, you wanted to store max value in the min variable, then you just change this over here. This is generally asked in interviews, that how you can swap the values within the variable without using any third variable. So you can do it like swapping like this. So when I'm writing like this, first value will take the after equals to first value, second value will take the after equals to second value. But when you wanted to swap, you just interchange these. That's it. That's how you can calculate the minimum and the maximum values, and you can print them even. Now if you wanted to generate the mean value, then let's say I want— just like I created zeros, right? Same way I can also create ones. It means a vector which is full of ones, a 2D array with full of ones.. If it is 2D array, then write 2 values. If it is 1-dimensional array or 1-dimensional values like vectors or something, you can write it, write a single value, right? When I'm writing np.ones, it means it all— like all the values in that particular NumPy vector or the array, which is 2-dimensional array, will be filled with just 1, 1, 1, 1, 1, 1. That's how it works. Now if you wanted to, let's say, calculate the mean— so dc.mean is a function that we have. And you can simply get the mean. Okay, so I will store it in the variable, let's say j is equals to this, and then I will say print j. I have not executed that. Let me get— yeah, now you can see 1 is the mean of this, right? So because this is a vector of 1, because this is the, uh, matrix of 1 only, array of 1 only. So instead what I will do is instead of dx I will write fx. Fx, dc, I will write fx, and then I will just run this. Now this comes with a meaningful value which is 0.49 something. So it means now I'm generating the two-dimensional array, but at the same time I'm also calculating the mean. You can even create a random vector. Random vector, let's say k is a variable, np.random.random, and then mention what is the value. Then you can simply print k and then run it. You can see this is how it works. And if you want to create a two-dimensional array with 1 on the borders and zeros inside, so it's like 1 1 1 1 1 1 1 1 1 like this, and in mid you want zeros to be filled, then how can you create this? So you can create it with the help of ones. Let me show you. So let's say my variable is r. r is equals to np.ones because on borders I want ones. Okay, you can do it with the zeros function also, so that is fine. So I want rows and columns both. So let's say 10 comma 10. Now where exactly do you want the ones? So let's say 10, right? I have written, so it means there has to be 10 rows. 1, 2, 3, 4, 5, 6. Okay, so I will take 6 for now. Then 1, 2, 3, 4, 5, 6. And then this has to be like this, right? Now notice one thing carefully. Whatever is required, you have to first understand the logic behind it. Now see, first one, if I talk about it starting from 0, it means this is 0, this is 0. It means if I talk about indexes row index 0, column index 0, right? This also is the last one. Where do I want to place my zeros? First you have to decide that. So these are my zeros. I have created zeros function, so it means initially everything will be 0. Initially everything will be— sorry, sorry, sorry— np.ones. So everything will be 1s. Now I have to decide where exactly do I need 0. So I need 0 on row— sorry, column 1 and row 1. Column 1 and row 1, right? And if I talk about the last, it's also the second last. So it means I have to mention the indexes. So I will write it like r square bracket, start from 1 till the -1, then comma, 1 colon -1. And what do I want to fill it with? I want to fill it with 0. And then if I print r, will it work? Yes, it will, because I have mentioned the correct indexes starting from 1 till the -1, starting from 1 till the -1. So from row side and column side, I have mentioned both. This is row, this is column, and I want to fill it with 0. If you want to fill it with 5, so you can write it 5. So it depends upon what type of data you are trying to generate, what type of values do you want to generate. Let's say there is an array and you wanted to add a border onto that array. So any array we can pick up, um, we will just pick up the meaningful one. Okay, so let's say this one, f is the array in which I wanted to work on, on which I basically wanted to work on. So how to add it is like f is equals to np.pad, means padding we are going to do Where do you want your padding to be added? I want the padding to be added on F. What should be the padding width? So write down the width. I want the width has to be 1. Now what should be the mode? Mode has to be whatever mode, like the constant, constant mode I want. So mode is equals to constant. What should be the constant value? So constant value is, let's say, 5. And then I will print f, and we'll run it. Now you can see, initially it was this. Now I added a border, this is how it became. Now if I say width 2, how will it work? This is how width 2 will look like. 0, 1, 2— sorry, 0, 1, 2, yeah, correct. And then the constant value I want, so I have written the constant value. That's how it works. This is how you can add the padding. Okay, so you do not have to use, uh, anything extra. You just have to mention the pad as a function and you can directly add the padding to the function. Now if I am going to— let's say I want to create a matrix with some values just below the diagonal, right below the diagonal I want to work. So if you know the matrix, you might be getting this more efficiently. Okay, so let's say I'm writing np.diag np.arange— let's say I want to add 1, 2, 3, 4, comma, k is equals to 1. Now let us store it in some variable. Let's say w is equals to this. Print w. Now you see 1, 2, 3, 4. It means just above the diagonal we got the values. If I will write minus, so you can see the values are like not increasing, not decreasing, but the values are not that consistent. So diag is basically to mention or give the reference of diagonal. Now I have the 4 values for the diagonal. This is how you can create an np, matrix of 5 into 5, and you can fill in the values like this. Got it? Now next is np. Let's say I want to create a matrix and fill it with the checkerboard pattern. What is the checkerboard pattern? Let me show you how the output looks like. So z— not z, let's say i is equals to np. Zeros, comma, dtype— this data type is integer— and checkerboard I want, right? So I will write it like 1 colon colon 2 comma colon colon 2, and then we'll run it. Hmm, over here nothing I want to enter, and then I want i and this also i and this also 1 comma. Okay, I want it in the round brackets, not square bracket, and then again square bracket. E— I do not have to add. Let's run it. Yeah, so this is the checkerboard pattern where we have 1, 0, 1, 0, 1, 0, 1, 0, 1, 0. And that's how you can create it. You have to excel in indexing. You have to excel in the primary way of extracting the values, slicing the datasets, and so on. So the basic concepts, that's why I say, are very important, because without them you cannot survive in machine learning, in artificial intelligence, and you cannot do anything basically within the maths and, you know, AI and ML. Now let's say I want to check what is the index of any value in any shape. Okay, let's say there is one data which is of shape, let's say, 6 rows, 7 columns, and 8— or let's— anything, right? For now, just let's assume this. Okay, this is the shape of my data, and I want to check the index of any, any value. Okay, so you have to use a function called— and print first, write down print np.unravel, unreveal basically, index. Which index do you want to know? I want to know, let's say this selected one, I'm just taking 100th. Wait, why are we adding 2? 1, 2, 3 brackets I have closed. 2 of them, one is this, one more I want. Reverse, it says unravel, reversing ownership. Oh, okay, wait, now it works. Yeah, so this is going to give you the index of any element that you wanted to check, and that's how you get it. So I hope you got it. If you want to create the custom data type, uh, for let's say you wanted to describe anything specifically, right? So let me just show you. So you create a variable called color, let's say np.dtype bracket square bracket round bracket, or r is for, let's say, I want u_bytes. Comma, enter. Gcream for np.ubyte again. Ubyte, yeah. This is how you can give the custom data type that describes as a color. For any unassigned, uh, bytes, like RGB I'm taking as a colors. And you can get it— dtype spelling is wrong, let me correct it— yb. So you got it how it works. So this is how you can give the colors. Now the next is we have to work with the multiplication. Now what is for multiplication? Let's say I want to multiply two different matrix. So if you wanted to multiply any matrix, so let's say I have np.ones as one matrix, and let's say the values are 4, 5, and then comma np.ones. Again, I'm creating one more. So let's say this time it's just 3, 2. So what I'm trying to mention is if you wanted to multiply two of them, then you have to use a function called matmul. Matmul means matrix multiplication. Matmul means matrix multiplication. And then you go at the end and then close the bracket. And the values which we are mentioning has to be in the tuple, right? So let us just enclose it in tuple. This also has to be in tuples. Yes. So 2 values over here and 2 values over here. And let us store it in some variable. Let's say h is the variable. And then one mismatch. Why is it mismatch? matmul np.ones then we have the values, then comma, MatMul 5, 3. Yeah, now it is perfect. So basically it was a matrix, uh, uh, details like real matrix has to be with the perfect details. Like you cannot make, uh, irregular one, right? So you cannot do that. So now I'm going to print edge. It will do the multiplication and give you the matrix in this format. So that's how you do matrix multiplication. Now let's say there is one, uh, one-dimensional array which are given to you. Now you have to negate all the elements which are between 3 and 8, and you have to make the changes in actual, right? So you go create one one-dimensional array. So let's say I'm creating one one-dimensional array with the some values. Let's say 11 is my value. Now what I will do— now I have 3, because the range which is given to us is 3 to 8. Anything between this range, within the value, you have to make the changes. So for that changes, I'm writing this logic. What is the logic? Logic is saying if the f is greater than 3 and f is lesser than equal to 8, then replace it with -1. So everything with the value— within the value will be replaced with the minus symbol. Negate the value means every value has to be in the negative format, and that's how you generate the negative format. You got it? That's how you generate the negative format. So I hope you have understood NumPy in depth, in detail, and you have also practiced alongside while watching this video. And if you have in case any doubt, do let me know in the comments. I will surely reply back to you and answer your queries, resolve your questions. And if you are practicing or participating in any hackathon, do let me know in the comments. I will surely help.