5:48
Learn how sliding-window detection works: train a binary classifier on cropped image patches, then scan test images with windows at different sizes. The catch is computational cost, which a convolutional implementation can fix.
Learn how sliding-window detection works: train a binary classifier on cropped image patches, then scan test images with windows at different sizes. The catch is computational cost, which a convolutional implementation can fix.
You've learned about object localization as well as landmark detection. Now let's build up toward an object detection algorithm. In this video, you learn how to use a convnet to perform object detection using something called the sliding windows detection algorithm. Let's say you want to build a car detection algorithm. Here's what you can do. You can first create a label training set so x and y with closely cropped examples of cars. So this is a image. X, there's a positive example, is a car. There's a car is a car, and then there's not a car. There's not a car. And for our purposes, in this training set, you can start off with what I'm going to call closely cropped images, meaning that X is, you know, pretty much only the car. So you can take a picture and crop out and just cut out anything else that's not part of a car. So you end up with a car centered and in pretty much the entire image. Given this label training set, Given this label training set, you can then train a convnet that inputs an image like one of these closely cropped images, and then the job of the convnet is to output y01. Is this a car or not? Once you've trained up this confnet, you can then use it in sliding windows detection. So the way you do that is if you have a test image like this, what you do is you start by picking a certain window size that I'm shown down there, and then you would input into this convnet a small rectangular region. So take just this little red square, input that into the convnet, and have a convnet make a prediction. And presumably for that little region in the red square, it'll say, no, that little red square does not contain a car in the sliding windows detection algorithm. What you do is you then pass as input a second image now bounded by this red square, shift it a little bit over, and feed that to the convnet. So it's feeding just the region of the image in the red square to the convnet, and run the convnet again. And then you do that with a third image and so on, and you keep going until you've, you know, slid the window across every position in the image. And I'm using a pretty large stride in this example just to make the animation go faster. But the idea is you basically go through every region of this size and pass lots of little cropped images into the convnet and have it classified 0 or 1 for each position at some stride. Now, having done this once with running this, what's called a sliding window through the image, you then repeat it, but now use a larger window. So now you take a slightly larger region and run that region. So resize this region into whatever input size the ConvNet is expecting and feed that to the confnet and have it output, and then slide the window over again using some stride and so on, and you run that throughout your entire image until you get to the end, and then you might do it a third time using even larger windows and so on. And the hope is that if you do this, then so long as there's a car somewhere in the image, that there will be a window where, for example, if you are passing in this window into the ConvNet, hopefully the ConvNet will have output one for that input region. So then you detect that there is a car there. So this algorithm is called sliding windows detection, because you take these windows, these square boxes, and slide them across the entire image and classify every square region with some stride as containing a car or not. Now, there's a huge disadvantage of sliding windows detection, which is the computational cost, because you're cropping out so many different square regions in the image and running each of them independently through a convnet. And if you use a very coarse stride, a very big stride, or very big step size, then that will reduce the number of windows you need to pass through the convnet. But that coarser granularity may hurt performance. Whereas if you use a very fine granularity or a very small stride, then the huge number of all these little regions you're passing through the ConvNet means that there's a very high computational cost. So before the rise of neural networks, people used to use much simpler classifiers, like a simple linear classifier over hand engineered features in order to perform object detection. And in that error, because each classifier was relatively cheap to compute, it was just a linear function, sliding windows detection ran okay. It was not a bad method. But with confnet now, running a single classification task is much more expensive. And sliding windows this way is infeasibly slow. And unless you use a very fine granularity or a very small stride, you end up not able to localize the objects that accurately within the image as well. Fortunately, however, this problem of computational cost has a pretty good solution. In particular, the sliding windows object detector can be implemented convolutionally or much more efficiently. Let's see in the next video how you can do that.
