I got Detectron2 working! | Airbnb Machine Learning Project Part 4

Oh, hey, wait, whoa, that's pretty cool. Welcome back to Part 4 of replicating Airbnb's amenity detection. In this video, we're going to try and build a small model using Detectron 2, am I right? Let's see how we go. Let the record show I'm stuck. Oh, I was beating my head up against a wall yesterday for about 3 hours. Finished the day feeling defeated. It is day 12 now. My goal is to build a small model by the end of the week. Oh, at least that's what the whiteboard said for week 2. And it's getting close to the end of week 2 and I haven't built a small model yet. My data manipulation code, in my head it should be working, but as you can see here, I'm getting a runtime error. CUDA runtime error 59. So this is the way I troubleshoot errors. I take that and I go here and I copy and paste that. But as you will see, I've been to each and every one of these links trying to fix my own problem. I mean, I could go a little bit deeper. There's one there. I'll probably go into that after we finish this little clip. But I've tried a few things on my own. I've left some notes in Notion of what I've tried. I've, I've tried adjusting the size of my bounding boxes. I've tried different— a different GPU in, in Colab because there might be some sort of issue with, with CUDA. CUDA is, in a nutshell, a really basic nutshell, is code on a GPU that helps your Python code run faster. So I write this code in Python, and then when it goes to the GPU, CUDA is what executes a bunch of different calculations on the GPU a lot faster. Again, really basic understanding of that. But what I'm going to do next is post a little issue on— where is it— on the Detectron 2 GitHub of just exactly what, what I'm facing, what I'm going through. I'm going to post that issue while I try and troubleshoot it further myself, but we still have the goal of building a small model. Oh well, in the meantime, before I post that issue, how about a little story time? You know, I had a job interview at Airbnb once. I mean, this is quite fitting because this is an Airbnb project. It wasn't for a technical role, but it was for, uh, like employee— oh, actually it was, yeah, but I didn't really see it as like a technical role. It wasn't coding, it was technical support for Airbnb employees. So I'd been working at Apple at the time at the Genius Bar and saw this role come up in Airbnb Ireland, and I was I was like, you know what, I'm going to apply for it. And so I did apply for it, but then I realized, hey, a lot of people are going to be probably wanting to work for Airbnb through the, through the job portal. So what I did was I went on LinkedIn and I searched for people who worked for Airbnb in the particular zone, the employee tech support zone that I wanted to work in. And I messaged a few people and one of them said, hey, I can't really help you. Try and message someone else. And then I messaged that someone else and I said, Hey, I'm applying for this role, or actually I think my words were something like, before I apply for this role, do you mind giving a 1-minute once-over of my resume and cover letter just to make sure I'm on the right path? And turns out they did, and instead of me applying through the, through the job portal, they offered to give me a phone call the next day. And so I had that phone call. Long story short, didn't get the job, but that's just one example of how if you're looking for a job at said company, rather than just shooting your resume through some job portal where it's likely to get filtered, do your research and reach out directly. You're smart enough to figure out who you should talk to. But that's enough for that story. Another little update is that I'm using Colab at the moment to try and build a small model. Thank you, someone pointed this out on Twitter, was that, thank you very much by the way, I don't necessarily have to use a deep learning VM. You saw me in a previous video create a deep learning VM on Google. That costs money to run. Colab and Jupyter Notebooks running locally are free. So all of my data manipulation code I've done locally and for free. But when I do need to do more heavy compute, compute, or use a more powerful computer than what I have available here, I'm going to be using Google Cloud's Deep Learning VM. But that's enough talking. I've got an error to fix. I'm going to post a GitHub issue and then probably bang my head up against the wall for another hour or so. And then if I still haven't made any progress, I'm going to call it a day and then come back once I've fixed it, because I'm going to fix this error. I believe we've made a break. Through. Hold on, it's a little bit dark here. So it's still the same day, still week 2. Let me show you what I'm up to. Have a look at this. So we've got here the train cell. Again, I've adapted this from the original Detectron2 tutorial. This is my version. This one in Colab is an even more adapted version with a whole bunch of preprocessing functions with a lot to do. And I will show you a little tidbit. I put to-do wherever I need to do something. So basically you can see there's like 16 of these. So when I need to come back and update something, which I do need to do a lot because this notebook is pretty messy at the moment, I can come back, fix it. And basically what I wanted to do in these first 2 weeks is just get something running. And then I can go back and make sure the code is pretty after I've got something running. But let's go down here. I made a little breakthrough. So you can see I was in the, in the middle of posting a GitHub issue. I was going to post my entire Colab notebook into the GitHub issue saying, hey, this is the code, you can run it yourself, what is going on? But then have a look at this. My Notion document is now 7:13, but 4:46 PM. A bunch of troubleshooting hasn't prevailed, cry face. Time to ask for some help. And that's what I was going to do, ask an issue on GitHub, but a wild breakthrough appeared. That's what we got there, 5:11 PM. I think I found my issue. Since I'm only focused on the coffee maker class, it has to be the only class in my training data. But when I run my preprocessing function, I find that images with Coffee Maker have other classes in them. So the mistake was the list of classes I was passing to my model. It was getting dimension errors. And so instead of just using the Coffee Maker class— and by the way, that's what I'm doing in the beginning, is I'm building a small model on one single class. So although Airbnb want to use 30 classes, In the beginning, when you're starting out, reduce your problem space. So only work on small subset of the problem, which is what I'm doing. I'm working on one class. I wanted to get a model working for coffee makers, mostly because I like coffee as well. So this is probably giving my model grief. Time to fix it. And that's what I did. I went to this link. As you can see, I've got a whole bunch of links. Even I've got— look at this— 3 windows full of links. I'm going to struggle to find what I'm supposed to run my, my code in a minute, but I read through a whole bunch of links on the fast.ai forums. Shout out to fast.ai. Figured out it's a dimensionality issue, which is always the problem. Dimensionality. When you pass the wrong shape tensor to your machine learning model, the wrong shape tensor input expecting a different shape output. I will say it until I go hoarse. Your inputs must line up with the machine learning model that you're trying to use and your outputs. And then I figured out a little bit more using Google Colab's runtime logs. I didn't know you could do this. So this is a beautiful thing about exploring. Look, where's my Colab notebook? Let's bring it back. You can go into runtime and then go view runtime logs and it's going to come out with a terminal output of what your notebook is outputting. So if you get an error and the error message in the actual notebook isn't very helpful, check the logs. You might get something that's a little bit more helpful. That's how I figured out my problem. So with that being said, without any further ado, let's come down to here because this little cell should get us a jumping and a jiving. Fingers crossed. Remember what the goal of the week was? To train a model, to get a small model trained. We come back to Notion one last time. This is this week: convert all images and labels to Detectron2 style labeling. We haven't done that. Build a small working model on a custom dataset based on Detectron2. We come into here. I've gone to the Detectron2 model zoo, right? So a model zoo is just— if you come up here Detectron2 Model Zoo. This is on GitHub, basically a whole bunch of Detectron2 pre-trained models that you can adapt to your own dataset, which is what I'm trying to do. These were pre-trained on the COCO object detection dataset, but I want to, remember, train it on the Open Images dataset, which is what Airbnb used, or partial part of their dataset that they used. So we come back to our notebook. I chose this model here, Faster R-CNN R50 FPM-1X. It's instantiated. If I run this cell here, Detectron 2 should kick off. Let's have a look. Oh, here we go. Starting training from iteration 0. Is that training? Here we go, iteration 19. This is where I was getting the CUDA runtime error before. There we go, iterations are going up, ETA. Oh, it's only going to take 1 minute because we've got a fast GPU up here. And you know, I'm going to keep to my promise. You know what I said? You know what I said? If I get Detectron 2 running, what am I doing? If I get Detectron 2 running, what am I doing? I will dance though once Detectron 2 starts to run. Hold me to that. I'll tell you what I'm doing. I'm dancing. I'm having too much fun to stop. Alright. We better pause that, probably gonna get copyrighted, but it's worth it. Look at this, total training time. Woo! That's what you need to do when you're sitting down coding for too long. You need to break it up with a good little dance. Training time, 1 minute 17, so that was pretty quick. We're only using about 300 or so images. So let's come down here, see if we can make some inference. We can. Let's have a look at what's been predicted. So this is gonna— this should draw a bounding box in our image. Oh, it didn't draw anyone on that. See, num_instances is 0. That means it didn't find a coffee maker in that picture, so it failed on that one. This should work. It's picking a random image out of the validation set. Oh, Zero. The moment of truth, and we're failing. Please find one coffee maker. Come on. Yes! It's found a coffee maker. Oh, 79%. Beautiful. Look at that. Let's look at another one for good measure. Nope, didn't find— well, think about that. Would you look at that as a coffee maker? I mean, yeah, fair enough, it's a coffee maker, but If you're looking at a house photo, maybe not. Oh, same random image. Come on, one more for the road. This video is getting too long. Yes! Ah! Oh! There we go! Another coffee maker. There we go, we found it. Alright, so this is a great start. We've got a small working model on one of our classes. That means, after a little bit of tidying up, of our preprocessing function of this notebook, right? Small model building. I'm probably going to turn this into number 5 and go big model building and use the whole dataset because I'm starting to get my code to work with Detectron too. All right, let's, uh, I believe this will also give us some actual metrics rather than just visualizations, you know. Oh my goodness, look at that. I'm not sure. Alexa, it's right. Even Alexa's excited. Excited. AEP, I believe that is average precision. 55.341. Now if we remember, let's come back over, let's finalize this video. Come back. What was Airbnb's MVP? 50% mean average precision. Let's come back. Look at that. Obviously this is not mean average precision yet because we're only using one class, but so far on one class we are 55.341 average precision. And of course we may have got a lucky test split, but this is a beautiful way to end week 2. Here we go. We haven't, we haven't actually completed that. Um, we'll fix that. That's the next stage. Is public data enough? We don't know yet, so we're gonna We're going to fix that up and build a small working model. We can actually— let's tick this off. Let's tick this off. Boom. Boom. What a way to finish week 2. Look at that, 12 out of 42. I'm going to have a little bit of rest this weekend. So that's going to be day 13 and 14. And then we'll be back for week 3 where— oh! Start scaling our model with data sources. That's where we're up to. We've got a small working model. Next is to introduce all of the other classes that we have to use. So I'll see you in week 3. Yes, I'm going to come.