Jason Yosinski explains how neural networks learn by adjusting roughly 60 million parameters on labeled images. He uses AlexNet and a gorilla photo as an example, then argues that while we can build AI systems that work spectacularly well, we barely understand how they operate. He calls for "AI neuroscience" to probe these black boxes before they become more powerful.
Chapters
The training process, basically we show it this gorilla and we tell it, actually, that's not a school bus, it's a gorilla. Please, network, next time you see the picture, you should say gorilla a little bit more. As it trains, it kind of moves slowly around in different ways till at the end it works. And the question is, okay, now we had 60 million things that changed and now it works. What did we just build? So far we saw some robots, some language models. I'm actually going to talk about vision models for a while because I think they're really cool and the pictures are pretty and everything. But cool. So we have these models, we have these datasets, larger and larger, computers are faster and faster, models are getting better and better. They can do cool stuff like I just showed you. There's another thing we could consider, another kind of aspect of human progress we can consider, which is our ability to understand what in the world we've built. So we can plot on here another line which I call like our scientific understanding of what's going on. Now, I've plotted it here with a different slope because I don't think it's driven by the same factors. Just because computers get twice as fast doesn't mean we understand twice as much about, about the models we're building. So we can identify a couple of regions in this plot. First, here on the left side, we have this bubble back here. This is like way in the past. This is when people, you know, were really smart. They knew what they wanted to do, but they just couldn't do it. They were sitting waiting around for double Es electrical engineers to invent faster computers, and for someone to invent the internet and then upload a bunch of cat videos so we could train models on it. That's the past. Now we're here. We're here in this, like, future region. So what is this region? In this region, we have AI that works very well, and we can't even quite understand it. So this is like a new era of human history. I think it represents also a new field. We could maybe think of it like AI neuroscience. So what is AI neuroscience? It's like probing AIs the same way neuroscientists probe squishy brains. So I think this represents a lot of ways the world will be changing. So first, we will start to use these models as tools. We'll use them to build companies, enable new technologies, hopefully improvements to life. Some people unfortunately will also use them for nefarious purposes. But to have any handle on any of that, we're going to need to understand what's going on. So I'm going to tell you about some work I've done and some collaborators have done to try to understand more deeply what happens inside of, like, big neural networks. So to start with, we're going to start with this neural network here. This is called AlexNet. This is the network— cartoon of a network, rather— that processes images and gives classifications for them. So for example, we can take a gorilla picture here, and we feed it in the left side of the network, all the pixels there. We do forward prop. And we eventually train the network so that it fires a single neuron here, one neuron corresponding to gorilla. So if I wanted to tell you kind of like how this network works, I could describe to you that it has like 5 convolutional layers and 3 fully connected layers. I could show you the shapes of all these things. In fact, using modern frameworks, modern machine learning frameworks, I could show you the code for this network and it would fit on about a half page. It's not very complicated at all to like assemble the basic building blocks to create this network. However, the complicated part here is actually what happens in the middle. So what happens in the middle is there's thousands and thousands of neurons that are firing in different ways, and the way these neurons fire isn't fundamentally just, you know, 5 convolutional layers, 3 fully connected layers. It has to do with all the parameters that are learned. So who knows how we, how we train neural networks? Anyone want to give a quick summary? Build an architecture for the network. Build an architecture for the network, and then what? And then feed it through the data. And then we feed the data, yeah. So, yeah, and then you train it. So, so first, I'll just summarize it differently, I guess. So we have the architecture for the network. We assemble it like this. It has a bunch of weights, a bunch of connections between neurons, and if those connections are in just the right configuration, the network will work. If they're not in the right configuration, it won't work. How do we know the right configuration? Well, actually we don't. So what we do is we first build the network, we populate it with random connections, so we just randomly connect things to other things with different strengths. Then we give it some data. We show it this gorilla picture. The neurons go through the network, fire, fire, fire, fire, fire, and out this side comes what? Gorilla. Initially, it comes the wrong answer. It doesn't say it's a gorilla. It says it's like some random other thing, like a school bus or a zebra or things I'll show you later. So initially, the network is broken. The training process, basically we show it this gorilla and we tell it, actually, that's not a school bus. It's a gorilla. Please, network, next time you see the picture, you should say gorilla a little bit more. And then we don't dwell on gorilla forever. We show it a new picture of a lion or something, and we say, this is a lion, this is a school bus, this is a gorilla. Every time we say that, these connections in the middle, they change position just very slightly. So you can think of these connections as like a bunch of numbers. There's actually 60 million numbers in this network. Think of how many numbers 60 million is, right? If you plotted one number per pixel, so like every pixel on your screen, make it like some brightness corresponding to this number, You would need like 30 screens worth of pixels just to like see every connection in this network. So initially, those, those 60 million numbers are in some configuration, and the network's broken. As it trains, they kind of move slowly around in different ways till at the end it works. And the question is, okay, now we had 60 million things that changed, and now it works. What did we just build? How does it work, right? It sounds maybe trivial because this network does perform well, but like understanding what happened with 60 million things is, is difficult. It's, it's a huge gap between what we'd like to know and what we do know.
