Andrew Ng explains region proposal methods for object detection, covering R-CNN and its faster variants against sliding-window approaches. He contrasts these with single-stage detectors like YOLO and argues for the efficiency of single-shot methods.
If you look at the object detection literature, there's a set of ideas called region proposals that's been very influential in computer vision as well. I wanted to make this video optional because I tend to use the region proposal set of algorithms a bit less often. But nonetheless, it has been an influential body of work and an idea that you might come across in your own work. Let's take a look. So if you recall the sliding windows idea, you would take a train classifier and run it across all of these different windows and run a detector to see if there's a car, pedestrian, maybe a motorcycle. Now, you could run the algorithm convolutionally, but one downside of the algorithm is it just classifies a lot of regions where there's clearly no object, right? So this rectangle down here is pretty much blank. It's to going clearly nothing interesting there to classify. And maybe it was also running it on this rectangle, which, you know, looks like there's nothing that interesting there. So what Ross Gertrude, Jeff Donahue, Trevor Darrell and Jitendra Malik proposed in a paper cited at the bottom of the slide is an algorithm called rcnn, which stands for regions with convolutional networks or regions with CNNs. And what that does is it tries to pick just a few regions that make sense to to run your confnet classifier on. So rather than running your sliding windows on every single window, you instead select just a few windows and run your confnet classifier on just a few windows. The way that they perform the region proposals is to run an algorithm called a segmentation algorithm that results in this output on the right in order to figure out what could be objects. So, for example, the segmentation algorithm finds a blob over here. And so you might pick that bounding box and say, let's run a classifier on that blob. Looks like it's this little green thing. It finds a blob there. So you might also run a classifier on that rectangle to see if there's anything interesting there. And in this case, this blue blob, if it runs a classifier on that, hopefully you'll find the pedestrian. And if you run it on this light cyan blob, maybe you'll find a car, maybe not. Not sure. So the details of this, this is called a segmentation algorithm. And what you do is you find maybe 2,000 blobs and place bounding boxes around about 2,000 blobs and run your classifier on just those 2,000 blobs, which and this can be a much smaller number of positions on which to run your confident classifier than if you had to run it at every single position throughout the image. And this is especially the case if you are running your confnet not just on square shaped regions, but running them on tall skinny regions to try to find pedestrians, but running them on your wide fat regions to try to find cars, and running them at multiple scales as well. So that's the RCNN or the region with CNN or region with CNN features idea. Now, it turns out the RCNN algorithm is still quite slow, so there's been a line of work to explore how to speed up this algorithm. So the basic RCNN algorithm would propose regions using some algorithm and then classify the proposed regions one at a time. And for each of the regions, it will output the label. So is there a car? Is there a pedestrian, Is there a motorcycle there? And it also outputs a bounding box, so you can get an accurate bounding box if indeed there is a object in that region. So just to be clear, the RCNN algorithm doesn't just trust the bounding box it was given, it also outputs a bounding box BXBY bhbw in order to get a more accurate bounding box than whatever happened to surround the blob that the image segmentation algorithm gave it. So you can get pretty accurate bounding boxes. Now, one downside of the RCNN algorithm was that it's actually quite slow. So, so over the years there have been a few improvements to the RCNN algorithm. Ross Gershwig proposed the fast RCNN algorithm, and it's basically the RCNN algorithm, but with a convolutional implementation of sliding windows. So the original implementation would actually classify the regions one at a time. So fast R CNN used a convolutional implementation of sliding windows. And this is roughly similar to the idea you saw in the fourth video of this week. And that speeds up RCNN quite a bit. It turns out that one of the problems of fast RCNN algorithm is that the clustering step to propose the regions is still quite slow. And so a different group, Shaoxing Ren Kai, Minh Roskozwig and Jian sun, proposed the Foster RCNN algorithm, which uses a convolutional neural network instead of one of the more traditional segmentation algorithms to propose the blobs or propose the regions. And that wound up running quite a bit faster than the fast RCNN algorithm, although I think the faster RCNN algorithm most implementations are usually still quite a bit slower than the YOLO algorithm. So the idea of region proposals has been quite, quite influential in computer vision. And I wanted you to know about these ideas because you see, others still use these ideas. Um, for myself, and this is my personal opinion, not the opinion of the computer vision research community as a whole, I think that region proposals is an interesting idea, but that not having two steps first proposed regions and then classify it, um, being able to do, you know, everything more all at the same time, similar to the YOLO or the you only look one's algorithm, that seems to me like a more promising direction for the long term. But that's my personal opinion and not necessarily the opinion of the whole computer vision research community. So feel free to take that with a grain of salt. Um, but I think that the RCNN idea you might come up, you might come across others using it, so it's well worth knowing as well, so you can understand others algorithms better. So with that, uh, we've now finished up our material for this week on object detection. I hope you enjoy working on this week's programming exercise and I look forward to seeing you next week.
