Learn how to define inputs and targets for ML projects and why inconsistent labeling kills model performance. The instructor shows real examples: iguana bounding boxes and phone defects labeled differently by different annotators, confusing the algorithm.
You're now in the third and final week of this course. Just one more week and then you'll be done with this first course of the specialization. In this week, we'll dive into data. How do you get data that sets up your training, your modeling for success? But first, why is defining what data to use even hard? Let's look at an example. I'm going to use the example of detecting iguanas. One of my friends, Kian Cutenfourouche, really likes iguanas. So I have a bunch of iguana pictures floating around. Let's say that you've gone into the forest and collected hundreds of pictures like these, and you send these pictures to laborers with the instructions. Please use bounding boxes to indicate the position of iguanas. One laborer may label it like this and say, one iguana, two iguanas. This labeler did a good job. A second laborer that is equally hard working, equally diligent, may say, look, the iguana on the left has a tail that goes all the way to the right of this image. So the second laborer may say, one iguana, two iguanas. Good job, labor. Hard to fault this labor either. A third label may say, well, I'm going to look through all hundreds of images and label them all. And I'm going to use bounding boxes. And so let me indicate the position iguanas and draw a bounding box like that. Three diligent, hard working labelers can come up with these three very different ways of labeling iguanas. And maybe any of these is actually fine. I, I would prefer the top two rather than the third one. But any of these labeling conventions could result in your learning algorithm learning a pretty good iguana detector. But what is not fine is if one third of your labelers use the first and one third the second and one third the third labeling convention, because then your labels are inconsistent and, and this is confusing to the learning algorithm. While the iguana example was a fun one, you see this type of effect in many practical computer vision problems as well. Let's use the phone defect detection example. If you ask the laborer to use bounding boxes to indicate significant defects, say maybe one labeler will look at it and go, oh well, clearly the scratch is the most significant defect. Let me draw a bounding box on that. A second labeler may look at this phone and say, there are actually two significant defects. There's a big scratch and then there's that small mark there. It's called a pit mark, kind of like if someone poked a phone with a sharp screwdriver. I think the second label probably did a better job. But then a third labeler may look at this and say, well, here's a bounding box that shows you where the defects are between these three labels. Probably the one in the middle would work the best. But this is a very typical example of inconsistent labeling that you will get back from a labeling process with even slightly ambiguous labeling instructions. And if you can consistently label the data with one convention, maybe the one in the middle, your learning algorithm will do better. What we will do in this week is dive into best practices for the data stage of the full cycle of a machine learning project. Specifically, we'll talk about how to define what is the data, what should be X and what should be Y, and establish a baseline. And doing that well will set you up to label and organize the data well, which should give you a good data set for when you move into the modeling phase, which you already saw. Last week, many machine learning researchers and many machine learning engineers had started off doing downloading data off the Internet to experiment with models. So using data prepared by someone else, nothing at all wrong with that. And for many practical applications, the way you prepare your data sets will have a huge impact on the success of your machine learning project. In the next video, we'll take a look at some more examples of how data can be ambiguous so that this will set us up later this week for some techniques for improving the quality of your data. Let's go on to the next video.
