Researchers at University of Tübingen built bagnets, models that classify image patches and combine results, and found they match state-of-the-art ResNets on some tasks. The finding raises questions about what makes neural networks actually better and whether recent claims about their superiority hold up.
Dear fellow scholars, this is two minute papers with. In the previous episode we talked about image classification, which means that we have an image as an input and we ask a computer to figure out what is seen in this image. Learning algorithms, such as convolutional neural networks are amazing at it. However, we just found out that even though their results are excellent, it is still quite hard to find out how they get to make a decision that an image depicts a dog or a cat. This is in contrast to an old and simple technique that goes by the name bag of Words. It works a bit like looking for keywords in a document and by using those trying to find out what the writing is about. Kind of like the shortcut students like to take for mandatory readings. We have all done it. Now imagine the same for images where we slice up the image into small pieces and keep a score on what is seen in these snippets. Floppy ears, black snout, fur. Ok, we're good. We can conclude that we have a dog over here. But wait, I hear what you are saying, Caroj. Why do we need to digress from AI to bag of words? Why talk about this old method? Well, let's look at the advantages and disadvantages and you will see in a moment. The advantage of bag of features is that it is quite easy to interpret because it is an open book. It gives us the scores for all of these small snippets. We know exactly how a decision is being made. A disadvantage, one would say, is that because it works per snippet, it ignores the bigger spatial relationships in an image and therefore overall it must be vastly inferior to a neural network. Right. Well, let's set up an experiment and see. This is a paper from the same group as the previous episode at the University of Tubingen. The experiment works the following way. Let's try to combine bag of words with neural networks by slicing up the image into the same patches and then feed them into a neural network and ask it to classify them. In this case, the neural network will do many small classification tasks on image snippets instead of one big decision for the full image. The paper discusses that the final classification also involves evaluating heat maps and more. This way, we are hoping that we get a technique where a neural network would explain its decisions, much like how bag of words works. For now, let's call these networks bagnets. And now hold on to your papers because the results are really surprising. As expected, it is true that looking at small snippets of the image can lead to misunderstandings. For instance, this image contains a soccer ball, but when zooming into small patches it might seem like this is a cowboy hat on top of the head of this child. However, what is unexpected is that even with this, bagnet produces surprisingly similar results to a state of the art neural network by the name resnet. This is. Wow. This has several corollaries. Let's start with the cool one. This means that neural networks are great at identifying objects in scrambled images, but humans are not. The reason for that is that the order of the tiles don't really matter. We now have a better reason why this is the case. Doing all this classification for many small tiles independently has superpowers when it comes to processing scrambled images. The other more controversial corollary is that this inevitably means that some results that show the superiority of deep neural networks over the good old bag of features can come not from using a superior method, but from careful fine tuning. Not all results, Some results As always, a good piece of research challenges our underlying assumptions and sometimes in this case even our sanity. There's a lot to say about this topic and we have only scratched the surface. So take this as a thought provoking idea that is worthy of further discussion. Really cool work. I love it. This video has been supported by Audible. By using Audible you get two excellent audiobooks free of charge. I recommend that you click the link in the video description, sign up for free, and check out the book Superintelligence by Nick Bostrom. Some more AI for you whenever you are stuck in traffic or have to clean the house. I talked about this book earlier and I see that many of you fellow scholars have been enjoying it. If you haven't read it, make sure to sign up now because this book discusses how it could be possible to build a super intelligent AI and what such an all knowing being would be like. You get this book free of charge and you can cancel at any time. You can't go wrong with this. Head on to the video description and sign up under the appropriate links. Thanks for watching and for your generous support and I'll see you next time.
