Win Kaggle's Flood Prediction Challenge (Complete Solution)

Ladies and gentlemen, hello everyone, welcome back to my YouTube channel. In today's video, we are going to see how we can solve a problem which is hosted as a competition on Kaggle. So on Kaggle website, there are multiple competitions through which you can enhance your skills and also showcase it in your resume. It will be a project which you are going to work on. So let's see on the screen what is the project and how we are going to build it. So in Kaggle, I found one on today, like it was regression project with the flood prediction dataset. So what we have to do in this is it has already started, and I think the end date is somehow, uh, May 31st. So you can submit your outcomes in this, and I recommend you to not to copy everything that is in this video. You can modify it a bit. Although I'm not going to make a submission on this, I'm just sharing my suggestion that how you can create it. So let's first see which type of a data we do have in this. So in this data, we have 3 files. One is the train.csv, another is test.csv, and there they are expecting us to give them a sample_submission.csv. This should be in the correct format. So if I see this sample Submission.csv, this should have 2 columns. One is ID, another one is the flood prediction. So on the basis of the ID, it has to predict the floods. And for this, for training, you have the test_train.csv, and then we have the test.csv. I have downloaded all 3 data and I have created a file on this Yeah, so here is my folder with the name Kaggle, and I have added all 3 datasets in the data column— data folder, basically. And then I have created one file which is start.ipynb. So now I'm going to show you my suggestion that how you can build it. It's totally up to you what you wanted to modify, what you wanted to add on in this. So first, what I have done is I need pandas for this purpose. Obviously, in the starting, we are going to work on the data. So for working on the data, I have started with the pandas import. So I've imported the pandas library, then I have done the read_csv to read the file which I have. So I have done copy and copy_path method to add the path of this, and then I have— I'm checking the shape of the data, that how many rows or how many columns do we have. Then I am doing data.describe, it means I'm getting the info of about data, that what does the data includes, what is the minimum value, and everything about the data, basically a summary, small summary. Then I am using info to check if there is any missing value or not, and I am using the original data from Kaggle website. So I figured it out that there is no missing value in this. If you can see in the info one, there was no missing value. All the columns have the same number of data. And another thing to notice over here is that the data type is integer data type, and we do not have to do one-hot encoding or any other part to just, you know, transform the data. So So data manipulation is not much required, nor even the data cleaning or the missing value handling. So you just directly have to, you know, switch to the next one. I am checking still with the isnull that is there any or not. So I found out, like, I figured it out that there's no missing value. I did with the .value_count also to check it, cross-verify basically, to check if there is any. Still I was not satisfied that how come the Kaggle can host something which is related to— this is not by the Kaggle. Anyone can host a competition on the Kaggle website. I just took it for just to make you guys understand that how to work on any datasets. So then I have done the .sum. This will give us the entire values that how, like, in which column do we have, and if we have any missing value, it will count it and will say— suppose if this column would have any missing value, then it, it would be like 5, 6, or any n number. Okay, then I am trying to build the linear regression model on this. So I have imported the train_test_split and the linear regression in this along with pandas, which I have already imported, but I am still doing it again. You can skip this part. Then what I am doing is I am splitting my data. So I have to get two variables. One should be x, another should be y. So, so to separate the features x and the target variable y, what I have done is in the column— in the variable x, I have dropped the flood probability and kept the X is equal to 1. So what will happen in this case? It will collect all the data except the flood probab— another is I have the Y column in which I have only the target column which is flood probab. Now I'm using the train test split and I am dividing my entire data into 48— 80/20 ratio. 80/20 ratio I'm using. You can use the 70/30 also, it depends upon you, but make sure that it is greater than like 70, like 72 or 18 you can use. Any value above 70 will work. So I have done the random state 48. Then moving forward, I'm creating the linear regression mode. So I have created a variable, then I created a object, linear object, linear regression. Then I'm fitting it with the data, training data, which is X_train and Y_train. Then I'm doing the predict one, which is X_value. Now I'm trying to get the mean squared error. I'm checking it whether there are like more errors or it's okay to go with it. So when I figured it out, I found that there are no such errors. It is mean squared error, so it means there is no such errors. We are going good with the project. So now what I have to do is I have created the model, I have trained my model on the train.csv. Now it's time to test it on the test data. They have already provided us the test data, test.csv. So I'm going to check it on the test.csv data. So this is test.csv data which I have imported just now. Now we need not to drop it flood. In case suppose the train.csv consists of the target column, so in that case you have to drop that. But in our case, that particular column was already removed. It means already, uh, you know, structured data they have provided to us. So you can directly move to the next step, which is to make the prediction. So I am making the prediction on the test data, which I have imported in the test data variable. Then I have to create a file. Whatever prediction my model is going to make on the test data, I want it in the column in the two columns, ID and flood probability, in the new file which is submission.csv. So what I have to do is I have to create a DataFrame ID with the test ID and the flood probability with the predicted outcome. Now what I have done is I just want— after points I want only one digit, so that is why I am rounding off my number to one digit only. And then I am saving my file locally to the submission.csv. Index is equal to false. And then I am rechecking all the errors that we were supposed to check it, and we have to check it to check whether our model is giving us the good accuracy or not. So here I have tested the mean square— mean squared error, which we tested above also, which was good enough to go ahead. And then when I'm checking the R-squared error, we are getting 0.84. Then mean absolute error, which is also good to go. So that's how we completed this project, and you can also do the same. If I will show you the submission.csv, here you can see that we have the ID and the fraud probability, and we have the values over here. They have provided a sample CSV also that this is how your output should look like, and we are getting the same output from here. So if you want the dataset or the, you know, all the files and all, you can check out the GitHub link which I have added below in the description section. And if you like this video, this was totally for beginners, so if you like this video Don't forget to comment down your views, and also if you want more videos like, like this, then drop your comments.