Andrew Ng explains non-max suppression for object detection. On a 19x19 grid, many cells predict overlapping boxes for the same object. NMS keeps the highest-scoring box and removes others with high overlap, repeating until done. Apply NMS separately for each class in multi-class detection.
19 by 19 (361 grid cells).
PC ≤ 0.6 (threshold example: 0.6).
Example box scores mentioned: 0.9 (highest), 0.8, 0.7, 0.6.
One of the problems of object detection as you've learned about it so far, is that your algorithm may find multiple detections of the same object. So rather than detecting an object just once, it might detect it multiple times. Non max suppression is a way for you to make sure that your algorithm detects each object only once. Let's go through an example. Let's say you want to detect pedestrians, cars and motorcycles in this image. You might place a grid over this, and this is a 19 by 19 grid. Now, while technically this car has just one midpoint, so it should be assigned just to one grid cell. And the car on the left also has just one midpoint. So technically only one of those grid cells should predict that there is a car. In practice, you're running an object classification and localization algorithm for every one of these grid cells. So it's quite possible that this grid cell might think that the center of a car is in it. And so might this. And so might this. And for the car on the left as well, maybe not only this box, if this is a test image you've seen before, not only that box might decide that things has found a car. Maybe this box and this box, and maybe others as well will also think that the they found a car. So let's step through an example of how non max suppression will work. So because you're running the image classification and localization algorithm on every grid cell, on 361 grid cells, it's possible that many of them will raise their hand and say my PC, my chance of thinking I have an object in it is large, rather than just having two of the grid cells out of the 19 squared or 361 think they've detected an object. So when you run your algorithm, you might end up with multiple detections of each object. So what non max suppression does is it cleans up these detections so that you end up with just one detection per car, rather than multiple detections per car. So concretely, what it does is it first looks at the probabilities associated with each of these detections. Kind of the PCs, although there are some details you learn about in this week's programming exercise, is actually PC times C1 or C2 or C3. But for now let's just say is PC with the probability of a detection, and it first takes the largest one, which in this case is 0.9 and says that's my most confident detection. So let's highlight that. I'm just saying I found a car there. Having done that, the non Max suppression part then looks at all of the remaining rectangles and all the ones with a high overlap, with a high IOU with this one that you've just output will get suppressed. So those two rectangles with the 0.6 and the 0.7, both of those overlap a lot with the light blue rectangle. So those you're going to suppress going to darken them to show that they're being suppressed. Next, you then go through the remaining rectangles and find the one with the highest probability, the highest PC, which in this case is this one with 0.8. So let's commit to that and just say, oh, I've detected a car there. And then the non max suppression part is to then get rid of any other ones with a high iou. So now every rectangle has been either highlighted or darkened and, and if you just get rid of the darkened rectangles, you're left with just the highlighted ones. And these are your two final predictions. So this is non max suppression and non max means that you're going to output the your maximal probabilities classifications, but suppress the close by ones that are non maximal. So that's hence the name non max suppression. So let's go through the details of the algorithm. First, on this 19 by 19 grid you're going to run, you're going to get a 19 by 19 by 8 output volume. Although for this example I'm going to simplify it to say that you're only doing car detection. So let me get rid of the C1, C2, C3 and pretend for this slide that each output for each of the 19 by 19. So for each of the 361, which is 19 squared for each of the 361 positions you get an output prediction of the following, which, which is the chance that there's an object and then the bounding box. And if you have only one object, there's no, you know, C1, C2, C3 prediction. Um, the details of what happens, you have multiple objects I'll leave to the programming exercise which you work on at the, towards the end of this week. Now, to implement non max suppression, the first thing you can do is discard all the boxes or discard all the predictions, all the bounding boxes with PC less than or equal to some threshold, let's say 0.6. So we're going to say that unless you think there's at least a 0.6 chance that there's an object there, let's just get rid of it. So this has got all the low probability output boxes. So the way to think about this is for each of the 361 positions, you output a bounding box together with a probability of that bounding box being a good one. So we're just gonna discard all the bounding boxes that were assigned a low probability. Next, while there are any remaining bounding boxes that you've not yet discarded or processed, you're going to repeatedly pick the box at the highest probability with the highest PC and and then output that as a prediction. So this is our process on the previous slide of taking one of the bounding boxes and making it lighter in color. So you commit to outputting that as a prediction for that there's a car there. Next, you then discard any remaining box. So any box that you have not output as a prediction and that was not previously discarded. So discard any remaining box with a high overlap with a high IOU with the box that you just output in the previous step. So this second step in the while loop was when on the previous slide, you would darken any remaining bounding box that had a high overlap with a bounding box that we just made lighter, that we just highlighted. And so you keep doing this while there's still any remaining boxes that you've not yet processed, until you've taken each of the boxes and either output it as a prediction or or discarded it as having too high an overlap or too high an IOU with one of the boxes that you have just output as a predicted position for one of the detected objects. I've described the algorithm using just a single object on this slide. If you actually try to detect three objects, say pedestrians, cause and motorcycles, then the output vector will have three additional components. And it turns out the right thing to do is to independently carry out non max suppression three times, one on each of the output classes. But the details of that I'll leave to this week's programming exercise, where you get to implement that yourself, where you get to implement non max suppression yourself on multiple object classes. So that's it for non max suppression. And if you implement the object detection algorithm we've described, you actually get pretty decent results. But before wrapping up our discussion of the YOLO algorithm, there's just one last idea I want to share with you which makes the algorithm work much better, which is the idea of using anchor boxes. Let's go on to the next video.
