Jason Yosinski from Uber AI Labs demonstrates how to visualize what's happening inside convolutional neural networks. He shows the DeepViz Toolbox, walks through how networks detect edges in early layers, and runs live demos with a webcam to show how the network classifies images like school buses and zebras.
Chapters
Extremely simple. Like, what have we done? We've just, like, detected, like, oh, this edge is kind of like an edge. But that's all this is. That's all the first layer is, is detecting this edge, that edge, and 94 other edges. And all neural networks are is just that same process of, like, finding patterns and firing, just stacked on top of each other, on top of each other, on top of each other, until at the end, one of those patterns should be gorilla or school bus. I'm going to show you some cool stuff about how we can start to understand this. So the very first way we can try to understand what happens in the middle of the network is just to plot it. So I'm going to show you this little toolbox we've written called the DeepViz Toolbox. You can actually download it yourself and try it yourself if you'd like. I will show you a quick demo now. Okay, cool. So here we have— on the left side, we have this school bus. So this is an image. We're giving this image to the network. We put it in the first layer. Here are the responses of the first layer. This is a little— A little tricky. Basically, on the first layer, there's a very simple, like, pattern detector. It's called a convolutional neuron that looks at each patch of this input, and it, like, scans for different patterns. And we scan not for one pattern, but we scan for actually 96 patterns. So there's 96 little boxes here showing, like, the 96 patterns that were detected or not detected in this image. This is the first layer. Then we have more layers. So this is pooling. We reduce the spatial resolution, contrast normalization, more convolution, more convolution, and so on. Up to here, we get to the first fully connected layer. This is— it's simply a different type of neural layer. Now every single dimension is just a single dot rather than like having some spatial awareness. Now this layer doesn't really know where things are in the image. It just fires or not fires, for example, if it sees part of a school bus or a wheel or something. Next layer, next layer. This is the second— this is basically the last layer. Well, here's the second last layer and the last layer. So the very last layer, we trained it to do something. We trained it to fire this one neuron, which actually corresponds to school bus. So you see all these other neurons are not firing. School bus is firing. I'll quickly change the school bus to something else. One second. Here's a zebra. Okay, so now we put a zebra in, and it goes through the whole network and it fires a different neuron here. So that's 1.0 zebra. The other one was school bus. We can try some other inputs. Here's a shark. This is kind of cool, actually. So sometimes the network is unsure. So I showed you the zebra. It said 1.0, you know, 100% zebra. School bus, it said 100% school bus. Here I'm showing a shark, and it's actually not quite sure if this is a tiger shark or a great white shark. So it gives them each a probability here, 0.55 probability tiger shark, 0.44 great white shark. Does anyone know what that actually is? Great white shark? I don't know either, but I actually most— when I give this talk, most people say great white shark, so probably it's that. I don't really know. Sure. In any case, the point is the network is able to represent its uncertainty in things, and it fires those sharks. The network also knows a lot of stuff about crazy little categories, like this dog here. So this is a Yorkshire Terrier. Does anyone know a lot about dogs? Yorkshire Terrier. This is a Norfolk Terrier. And you see the neuron that's firing is changing each time. A Maltese Terrier and a Norwich Terrier. So I remember the first time I trained, actually, the same network. It was running on a GPU, running on my computer under my desk in grad school for like a couple days. And like, on the one hand, it took a long time to like run this program for days to train this network. On the other hand, I basically like clicked go on some Python script, went and got a coffee, went home, made dinner. I came back a couple days later, and there was this network, and I was proud of it. Like, I showed it a school bus, it could say it was a school bus. I showed it a zebra, it could say it was a zebra. And I was, I was happy. I was, I was proud of my network. It learned as much about school buses as I know, right? Yeah. And then I showed it like all these dogs, which are different categories than ImageNet. And I realized like the network actually knew more about dogs than I do, right? I actually didn't know what a Norwich Terrier or a Norfolk Terrier was before this. And like just by showing it all these pictures of dogs, it learned like really subtle, cool things about dogs that I never knew. Like certain little tufts of hair on these ears are actually very indicative of what breed of dog it is. So in terms of things I learned in grad school that I didn't think I would, Lots of breeds of dogs would be one thing I learned. Cool. So we can do some other stuff here. We can actually also use as input a webcam. So here I have a webcam on top, and you can see it's, like, responding. It's saying different things. I can't even see this, actually, sadly. I'm just going to pause it. So there's me. I can point it at you guys in a second. I want to go through these, like, these basic patterns it's detecting a little bit more. So these are the kind of neuron firings. This is how much it finds each pattern. And here's a view of like what patterns it's looking for. So I'll show you those again. So there are 96 patterns. There are 96 little patterns I'm showing you here. Each neuron is like looking for one of these. And the amount it finds that pattern is shown here. So for example, let's just go down to— This guy here. See these 2 neurons here, 1 and 2? So this first neuron here is detecting, like, simple patterns of, like, light to dark from left to right. And the neighboring pattern is detecting dark to light patterns. So if we go back to the activation view, probably what I can do is show you my phone. Yeah, okay. So my shirt is white, as you can see, and my phone is kind of dark. So I've highlighted this neuron here that's looking for light to dark edges, and where should that fire? Which side of my phone should that fire on? Left side. Probably the left side. So you see that? See how it's like firing on the left side of my phone there, but not the right side, because the right side is the opposite pattern. Now, on the other hand, I'm going to move over to the next neuron over, so now I'm on the other one, which is the dark to light neuron. And now you can see it's firing on the kind of the right side of my phone, but not the left side, right? Kind of cool. On the other hand, on the other hand, extremely simple. Like, what have we done? We've just like detected like, oh, this edge is kind of like an edge. But that's all this is. That's all the first layer is, is detecting this edge, that edge, and 94 other edges. And all neural networks are It's just that same process of finding patterns and firing, just stacked on top of each other, on top of each other, on top of each other, until at the end, one of those patterns should be gorilla or school bus or whatever. So on the one hand, simple. On the other hand, you add it all together and the effect is very complicated. So that's it.
