Numpy for Data Analysis

Hello. Yeah, yeah, how are you guys? Okay, so in the last session we discussed about libraries and we discussed about modules in Python. Now specifically, we are going to cover the NumPy library as you all are focusing on data analysis. So in this video, in this session itself, I'm going to teach you about NumPy library and how you have to use NumPy library for creating arrays, for calculating any statistical stuff like creating mean, median, moders, anything related to this. Okay, so let's get started. So NumPy is basically numerical Python library. Num— Num stands for numerical and Py stands for Python, and this is used to work with an array. So basically, using NumPy library of Python, you can create n-dimensional array. When I say n-dimensional array means you can create any dimensional array using NumPy. This is not possible simply with Python list, but this is possible with NumPy. So to get started with NumPy, you have to install NumPy in your Python. So simply you have to use pip command, just like we have discussed the same in the last session as well. pip install numpy will install that particular library. For you. Once you have installed that library, second step is to import that library. To import that library, we simply have a command import numpy as— as we are writing just to give it a nickname. When we provide a nickname to this, then it becomes easier for us to call this library whenever we want that. So I'm giving it a nickname that is np. So import numpy as np. It means NumPy— to NumPy library, I have given np as a nickname. Okay, let me just increase the size. Okay. Now next is that you have to— thing that comes up in NumPy is you have to learn how to create arrays in NumPy. So simply, just like in other libraries, we have built-in functions, built-in methods. We have to use the same in the NumPy library as well. To create NumPy array, there are multiple ways to do it. So first way of creating a NumPy array is, let's say you have array function itself. So you can simply say Let's say f is equal to np.array bracket. This is going to create an array for you. So now you can mention the values that you wanted to add up in the array, and once you will do print, it will be generated like this. Now let's suppose you wanted to create an array, but the— you want that everywhere it should be 0. So for an example, let's say I wanted to create a table, okay, a table where there should be, let's say, 3 rows and let's say 4 columns. 1, 2, 3, and 4. I want 3 rows and 4 columns to be within the NumPy, and I want all of these columns to be filled with 0. It means I'm creating array of zeros. When you want to create this, you have to use different function which is np. Let's say f1 is equal— we are already using f, so I'm using p. p is equal to np.zeros and then you have to mention the shape. So when I'm opening the bracket, it means this is a function, this is the method which requires parameters. When it comes to the point of parameters, it means now you have to mention that how many rows or how many columns you want in the result. So I want 3, 4— 3 comma 4. But this is the shape, so you have to type it within another bracket. And I'm going to say print p. So now you can see you have created 3 rows and 4 columns, and all the columns and rows, the cells, are filled with 0. So this is what we call as array of zeros. When you wanted to create array of ones, there are some scenarios in which you will be required to create arrays of one. So at that time you have to use different function, which is— I'm going to take the same copy of this, but this time I'm going to make it p1, and instead of zeros I'm going to mention 1. Wait, p1 it should be, and then So now this is the area of zeros and this is the area of 1. Anyone, any question in this? NumPy is simply for arrays. However you wanted to create arrays, you have to use the built-in methods of array. So these are the 3 methods for creating basic type of array. Now let's say you wanted to create an array with the value, let's say 0 to 9. So in this case, you have to create array with the range— array with a range. So p2 is equal to np.arange, and then you have to mention the values. As I told you that the values starts with 0 automatically, like this is the fixed thing in Python, so indexing begins with 0. So now I am writing np.arange. So now this has created an array of this particular range. So this is how you create it within a range. If you know the range, then you can go with this method. Now let's say you wanted to create an array with evenly spaced values. When it comes to evenly spaced values, right? So let me give you an example. So p3 is equal to np. For this you have to use the function called linspace, and then here you have to mention 3 values: start, stop, and the evenly spaced one. Okay, so suppose I'm writing 0, 10, let's say 5, and I am going to print p3. So this is generating 5 values for us because we have written 5 over here, and all the values, if you can see, are from this particular range 0 to 10. If you wanted to increase the range, let's say I want 10, 0 to 20, and I want 5 evenly distributed value. Evenly distributed means you can see there is, there are, there like between the numbers, if you are going to observe, you will have the same gap. 0 to 2.5, 2.5 to 5, then 5 to 7.5, 7.5 to 10. All are evenly spaced. Now when I am saying 5 values from 0 to 20, then 0, 5, 10, 15, and 20 are the values arranged in the format of array. Any question in this? Yeah, anyone, any question? So the arrays that we are trying to create is 1D array. I hope you know about 1D array, 2D array, and 3D array. Do anyone know about 1D, 2D, 3D? Yeah. What is 2D array? Uh, means two-dimensional array, like rows and columns. Correct. So when you want it just like in list, when we are creating something with list, we have to use the concept of nested list. With the nested list, you can create 2D array, 3D array kind of stuff as well. But within NumPy also, you have to use the same formatting if you wanted to go for 2D and 3D array. Okay, any dimension array you can create, and you can also check which dimension array you have created. So let's say if I'm writing print p3.n-dimensional— oh wait, I shouldn't write this. Yeah, so now you can see it shows 1. It means this is 1-dimensional array. So this method, which is ndim, which is going to show you the number of dimensions that you have created. If you have created 2D array, let's take an example of 2D array, uh, let's say p4 is equal to np.array bracket 1 comma 2 comma 3 comma square bracket again 4 comma 5 comma 6. Print p4. Uh, which I have to add it in one more bracket. Yeah, so now I have created this. Now I'm going to check the dimensions. So p4.ndim is going to give us the number of dimension, which is 2. If I'm going to keep on increasing this, so will it change the number of dimension? Which type of array is this? Or if, let's say, I wanted to create 3-dimensional array, then how do I have to do it? What do I need to change in this? Yeah, anyone, any idea? I want to create 3D array using this np.array. Then what do I have to change is that in this particular statement on my screen. Simply, when you wanted to move to the 3-dimensional array or more than this, then you have to create— use, uh, the different, different brackets to add it. For an example, let's say I'm going to add one more bracket and then opening a bracket, adding two of them in the square bracket again. Okay, I think we need to add one more. Yeah. So now this is 3-dimensional array. So you can create higher dimensional arrays as well. An array can have any number of dimensions. It is totally up to you how you want it to create. Now let's say I want to create an array of, uh, 5 dimensions. So you need not to, you go through with these brackets because Python is for making our task easy. So when it comes to making our task easy, then you can simply follow the method. Let's say p5 is equal to np.array bracket 2, 3, 4, 5 comma ndim is equal to, let's say, 5. Print p5. 1, 2, 3, 4, 5. Save it. And I'm mentioning— Now this is the 5-dimensional array. You can see these brackets are automatically added. Okay, so you have to use this argument which is ndim, okay, which is the short form of the number of dimensions. Any question in this? Now there are more built-in function. For example, let's say I have grades as a variable, and within this grades variable I have some values. Now I wanted to create mean of this, then I am going to use mean, me, as a variable. np.mean is a built-in function, then you have to pass the variable in this, and then I'm going to print the mean. Value, which is 42.2. So that's how simply you can create mean. Now when you wanted to create median, then we have, let's say, y is equal to np.median, then grade, and then going to print the value of y, which is 30.5, which is the median value. Similarly, when you want to create mode value, then you have to use, let's say, x is equal to np.mode and then grade, printing the value of x. So basically, now in the updated version of NumPy, we don't have mode as a direct function. Okay, so mode has been added to the next library, which is scipy library. Okay, which is basically all for statistical stuff. So when you are going with the mean, median, mode— mean and median you can directly perform with NumPy, but for mode you have to use scipy library. So you can simply import that scipy library and then you can use it. So to import it, same way, import scipy, and then instead of using this np, use the scipy.mode to get that. Wait, the— within scipy library, we have a specific stats also. So for making it more specific, because I have to update the versions of these libraries, so I'm going to import it with a different way. From scipy import, let's say, stats. This way we will try once. Wait, what? Yeah, so now we have stats. So this is the alternate way of importing a library along with a specific thing. So libraries are the package, a package of multiple modules. When we are loading any multiple modules on our system, then it may take some time to load. First thing. Second thing to notice, whenever we load a library, it is bit heavier than our memory, computer's memory. So you have to be specific, you have to just import what you need. So when it comes to you wanted to just import what you need, then you have to use this method. From this library, just import stats. SciPy library can do a lot more thing, but I don't want the entire SciPy library to work on this file.ipynb. I just want the stats, functionality to be there in this file because I just wanted to use that. So I am not going to install everything or import everything. So I'm just importing that stats from that particular library. So stats is like, you can say, if a library contains 10 files, then stats is one of them. And we are not importing all 10 files, we are just importing one single file which is stats. Is this clear to everyone? Yeah, okay. So you can also generate some errors, you can also create some errors using random things. Okay, so let's say I wanted to create random array with the NumPy library. So for that, as we have discussed, the random library, same functions are applicable in the NumPy as well. So if I'm going to say np.random.rand, it is going to ask me for the shape, that how many rows and how many columns you want to add. Okay, so again, as I told you in the last video when we were discussing random.random, I told you the values that it is going to generate will be from 0 to 1 only. You can see this is the last class execution, random.random is going to generate the numbers from 0 to 1. The range will be 0 to 1. Okay, so same will be happening in the NumPy library as well. So np.random— let me just use p6 is equal to np.random .rand, then you have to mention the size. So let's say 3 comma 3, print p6. Now you can see all the values are from the— are from like the range which is 0 to 1. Okay, same way if you wanted to create array but not with the value 0 to 1, you wanted to provide a range. Along with the size, then in that case you can use different method which is random— sorry, np.random. np.random because we are randomizing the values, okay? And then you have to mention which type of values do you want. Random integers I want. And then you have to mention the value range. So I'm saying, well, 1 comma 10. Then you have to mention the size. What should be the size of your array that you are trying to generate? Over here you have to mention the number of rows and number of columns that you want. So let's say I'm writing this 2 comma 3, printing p7. Now you can see all the values which we have is 2 rows and 3 columns I am generating, and with the values 1 2, 10. All the values are between this particular range only. If I want 10 to 100, then the values will look something like this. So it totally depends upon how you want it to generate it. You can also perform some mathematical operation on the arrays when maybe you wanted to add the arrays, you wanted to subtract, you wanted to do scalar application— operation. Scalar operation means you wanted to do multiplications, single value you wanted to multiply. For example, we can say that— you will say I'm going to create two different array, 1D array. Okay, so I'm going to say d1 is equal to np.array 1 comma 2 comma 3 within the square brackets. Now I'm going to create one more. d2 is equal to np.array bracket 4 comma 5 comma 6. Now I wanted to perform addition. So just like if you wanted to go with the element-wise— element-wise means you want all the values to be added. Then you simply can apply, let's say, s is equal to d1 plus d2, then print s. So this is the final result. Now let's say in the same value, uh, I want each and every value to be multiplied by, let's say, 2. So I want these 3 values to be multiplied by 2. So I can simply do the scalar operation. Scalar operation means multiplying a single value. So if I'm going to say d1 multiply by 2, and I'm going to store it in the variable, let's say a1 is equal to this, print a1. So now you can see each and every value has been multiplied with the 10— 2. So if I want, let's say I wanted to multiply by 100 to make it more visible. So now you can see 1 multiplied by 100, 2 multiplied by 100, 3 multiplied by 100. Any number if you wanted to multiply by, then you can do this. Same methods just like we have discussed in the concept of string manipulation when we have discussed Same methods and same thing that we have discussed in string manipulation, and we have discussed in slicing, then we have discussed in list. So those concept of indexing and slicing are applicable over here in the arrays as well. So whenever we are creating arrays and we wanted to get 2 values from the array, 3 values from array, or single value from array, then you have to use the method of indexing. So indexing means it is going to start from 0 and moving onwards to the number of digits, it's going to increase. If you want slicing, then you have to use the slicing method, same as it is from the strings concept, or you can say from the list concept. When you are dealing with multi-dimensional array, then also you have to use the same method. Now let's say I'm going to just take an example of this D1 to just explain you. So let's say A2 is equal to— I want to print something, so let's directly print it. So D1, from D1, I want value which is at the first index. It's going to print that value for me. So now you can see the first index, we were having 2 as a value. 2 was the value of the first index. Now I'm just going to randomly update it with some more values. Okay, so just going to add it up over here. Okay, so now we are getting 2 as a value. Now if you want more values, like starting from 1 you want up till 5, then you will get something like this: 2, 3, 98, and 63 are 4 values which you are getting because I am using the concept of slicing. Now let's take an example. If you have multi-dimensional array like 2D array, let's take an example of 2D array. So I— we have already created, so I'm just going to take the same one. I think we updated it with some values. S1 is equal to np.array, okay, comma. Now when I have written s1, 1 comma 2, 1 comma 2 means I want the first row and the second value. Second index. First row, second index. So this is the zeroth one, this is the first one. From first one, I want the second index. Second index means 0, 1, 2. So we are getting 963 printed in the output. If you want to display this 85— 850, sorry— so then you need not to mention this because this we are targeting as 0. And then you will get 0 comma 3 means the first— from the first kind of a list, you can say From first array or first list, I want the third index. So 0, 1, 2, and this is 3, which is 850 in the results. Okay, now let's say you want to, uh, you can also use advanced indexing. Advanced indexing means you need not to mention all of these. Okay, so if you want the values from, uh, let's say simply I'm going to print s1, and from the s1 variable itself I want the values from 3 onwards. So now you can see which— what you are getting. As I said, 3, and these are the values you are getting from this one and the other values from this one. It means we are skipping value. When you wanted to do fancy indexing to extract specific element, or you can say sub-array, then you can use Any question, anyone, in this? I mean, zero, uh, print s1. Uh, if you, if you were not write zero, by default it can be This one you're saying? Yes, ma'am. Sometimes it gives you an error because of the syntax error, because you— in this method we are using this comma, okay? When it comes to the point comma, either we are trying to make it a list. If you are not trying to make it a list, then you have to mention the value. When— if you are using the slicing one, then you can skip it. Okay, if you are going to skip it, you will get results something like this. But what we are expecting, we are expecting single value. When you're extracting single value from multi-dimensional array, then you have to use this method of comma. Okay, you have to use comma instead of that colon. Okay, okay, anyone else, any other question? Now let's say you wanted to reshape it. When it comes to reshape any existing array, then we have the built-in function for this also, which is the reshape function. So N1, uh, like NumPy library which you have, within that only you will use this. So let's say we have already created an array. So I'm not going to use this s1 because this is 2D array. First, let's do with the 1D array, which is d1. So now I wanted to use d1 and I wanted to reshape it. So d1.reshape. So let's say d_up is equal to d1. Reshape, and then you have to mention the shape in which you wanted to reshape it. Now let's say the same shape I'm going to use, which is 0 comma— oh sorry, 2 comma 3, which is 2 rows— sorry, yeah, 2 rows and 3 columns. And now I'm going to print it. d_up is the output. Cannot reshape an array of size 8 into 6. Yeah. When we are using this one, okay? So in total we have 8 values in this array, and when we have 8 values, then you have to make sure you are adding, uh, we are providing the values which is the multiplication of both. For an example, I'm going to just remove 2 of them, okay? Now we have in total 6 values. Now I'm going to run this. And now we will try this. Now you can see the array has been reshaped, reason being you have to make sure that you are providing the enough number of values to the reshape function. Value 1 shows the number of rows, value 2 shows the number of columns. So it is basically going to create a table format for you where you will have 2 rows and you will have 3 columns. 2 rows, 3 columns means you want in total how many values? 6 values, which is the multiplication of 2 multiplied by 3, which is going to give you 6. It means in total, if you have an array of 6 values, then only you can create a— you can reshape it to 2 comma 2 multiplied by 3. If you have any less value than that, then you will encounter the array that we were just getting. Did you get it now? Yeah. So simply, when you're reshaping, make sure you are going to pass the same number of columns. Now let's say you wanted to get minimum value, like which one is the minimum value from this array, or maximum values from this array. Then you simply have the inbuilt functions for the same, which is np.min and then you will pass, let's say, D1. So this is the smallest value. And then if you want the maximum one, then we have D2, which is going to give you again the maximum value. Okay, I'm using the wrong one. That is why— yeah, 98 is the highest value from that particular array. So we use arrays. It, it looks similar to the Python list, but this is not similar because NumPy is much faster than we use the list, okay?. So if you want to do vectorization, if you wanted to perform something on it, it eliminates also the need of explicit loops to— like, if you wanted to create nested 3D loop, 3D list, or 2D list kind of nested list, then you have to use loops also. But in this case of NumPy, you don't have to do it. You can do computation more faster than the regular one. You can use it for statistical analysis, for just getting the information about the data that you are going to work on. Okay, we have many more functions in this. Like suppose you want— if you have an array of string So arrays are not just supposed to be of, uh, the numbers. Array can be of strings as well, means a text as well. So then at that time, if you wanted to sort your array in the alphabetical order, then you have np.sort to sort everything. Okay, you just have to pass the variable in which you have stored the values. You can sort any type of arrays. You can do filter on the array based on the conditions as well. Okay, so anyone has any question in this? No. Okay, so your today's task is that you have to practice this NumPy library, and then tomorrow we are going to have the last session of Python, which we are going to discuss about pandas. Okay, so we will have the session at the same time, 8 PM. Until then, you need to just practice NumPy. Okay, thank you, ma'am.