Károly Zsolnai-Fehér asks whether we should accept deep learning results in vision conferences if they work but we can't understand how. He argues for extracting compact, human-readable rules from neural networks, using DeepMind's Atari agent and neural programmer interpreters as examples of models that reveal what they actually learned.
Dear fellow scholars, this is two Minute papers with I have recently been witnessing a few heated conversations regarding the submission of deep learning papers to computer vision conferences. The forums are up in arms about the fact that despite some of these papers showcase remarkably good results, they were rejected on the basis of, from what I have heard, not adding too much to the tree of knowledge. They argue that we don't understand what is going on in these neural networks and cannot really learn anything new from them. I'll try to understand and rephrase their argument differently. We know exactly how to train a neural network. It's just that as an output of this process, we get a model of something that resembles a brain as a collection of neurons and circumstances under which these neurons are activated. We store these in a file that can take up to several gigabytes, and the best solutions are often not intuitively understandable for us. For instance, in this video we are training a neural network to classify these points correctly. But what exactly can we learn if we look into these neurons? Now imagine that in practice we don't have a handful of these boxes, but millions of them and more complex than the ones you see here. Let's start with a simple example that hopefully helps getting a better grip of this argument. Now I'll be damned if this video won't be more than a couple minutes, so this is going to be one of those slightly extended two minute papers episodes. I I hope you don't mind the grammatical rules of my native language. A lot of them are contained in enormous tomes that everyone has to go through during their school years. Rules are important. They give the scaffoldings for constructing sentences that are grammatically correct. Can we explain or even enumerate these rules? Well, unless you are a linguist, the answer is no. Almost no one really remembers more than a few rules. But every native speaker knows how their language should be spoken, and it is because we have heard a lot of sentences that are correct and learned it by heart. What makes a proper sentence and what is gibberish? This is exactly what neural networks do. They are trained in a very similar way. In fact, they are so effective at it that if we try to forcefully insert some of our knowledge in there, the solutions are going to get worse. It is therefore an appropriate time to ask questions like what merits a paper? And what do we define as scientific progress? What if we have extremely accurate algorithms where we don't know what is going on under the hood? Or simpler, more intuitive algorithms that may be subpar in accuracy? If we have a top Tier scientific conference where only a very limited number of papers get accepted. Which ones shall we accept? I hope that this question will spark a productive discussion and hopefully scientific research venues will be more vigilant about this question in the future. Okay, so the question is crystal clear. Knowledge or efficiency? How about possible solutions? Can we extract scientific insights out of these neural networks? Model compression is a way to essentially compress the information in this brainish thing, this collection of neurons we described earlier. To demonstrate why this is such a cool idea, let's quickly jump to this program by DeepMind that plays Atari games at an amazingly high level in breakout. The solution program that you see here is essentially an enormous table that describes what the program should do when it sees different inputs. It is so enormous that it has many millions of records in there. A manual of many thousand pages, if you will. It is easy to execute for a computer, but completely impossible for us to understand why and how it works. However, if we intuitively think about the game itself, we could actually write a super simple program in one line of code that would almost be as good as this. All we need to do is try to follow the ball with the paddle. One line of code and pretty decent results. Not optimal, but quite decent. From such a program we can actually learn something about the game. Essentially what we could do with these enormous tables is compressing them into much, much smaller ones. Ones that are so tiny that we can actually build an intuition. This way. The output of a machine learning technique wouldn't only be an extremely efficient program, but finally the output of the procedure would be knowledge insight. If you think about it, such an algorithm would essentially do research by itself. At first it would randomly try experimenting and after a large amount of observations are collected, these observations would be explained by a small number of rules. That is exactly the definition of research. And perhaps this is one of the more interesting future frontiers of machine learning research. And by the way, earlier we have talked about a fantastic paper on neural programmer interpreters that also aim to output complete algorithms that can be directly used and understood. The link is available in the description box. Thanks for watching and for your generous support. Thank you and I'll see you next time.
