Building Blocks of AI Interpretability | Two Minute Papers #234

Dear fellow scholars, this is two Minute papers with hold on to your papers because this is an exclusive look at a new neural network visualization paper that came from a collaboration between Google and the Carnegie Mellon University. The paper is as fresh as it gets because this is the first time I have been given an exclusive look before the paper came out. And this means that this video and the paper itself will be published at the same time. This is really cool and it's quite an honor. Thank you very much. Neural networks are powerful learning based tools that are super useful for tasks that are difficult to explain, but easy to demonstrate. For instance, it is hard to mathematically define what a traffic sign is, but we have plenty of photographs of them. So the idea is simple. We label a bunch of photographs with additional data that says this is a traffic sign and and this one isn't, and feed this to a learning algorithm. As a result, neural networks have been able to perform traffic sign detection at a superhuman level for many years now. Scientists at Google DeepMind have also shown us that if we combine a neural network with reinforcement learning, we can get it to look at a screen and play computer games on a very high level. It is incredible to see problems that seemed impossible for many decades and crumble one by one in quick succession over the last few years. However, we have a problem, and that problem is interpretability. There is no doubt that these neural networks are efficient. However, they cannot explain their decisions to us, at least not in a way that we can interpret. To alleviate this, earlier works tried to visualize these networks on the level of neurons, particularly what kinds of inputs make these individual neurons extremely excited. This paper is about combining previously known techniques to unlock more powerful ways to visualize these networks. For instance, we can combine the individual neuron visualization with class attributions. This offers a better way of understanding how a neural network decides whether a photo depicts a Labrador or a tiger cat. Here we can see which part of the image activates a given neuron and what the neuron is looking for. Below we see the final decision as to which class this image should belong to. The next visualization technique shows us which set of detectors contributed to the final decision and how much they contributed. Exactly. Another way towards better interpretability is to decrease the overwhelming number of neurons into smaller groups with more semantic meaning. This process is referred to as factorization or neuron grouping in the paper. If we do this, we can obtain highly descriptive labels that we can endow with intuitive meanings. For instance, here we see that in order for the network to classify the image as a Labrador. It needs to see a combination of floppy ears, doggy forehead, doggy mouth, and a bunch of fur. And we can also construct a nice activation map to show which part of the image makes our groups excited. Please note that we have only scratched the surface. This is a beautiful paper and it has tons of more results available exactly from this moment, with plenty of interactive examples you can play with. Not only that, but the code is open sourced, so you are also able to reproduce these visualizations with little to no setup. Make sure to have a look at it in the video description. Thanks for watching and for your generous support and I'll see you next time. Ra.