DeepMind trained a neural network to play grandmaster-level chess by learning from 15 billion Stockfish evaluations instead of self-play. The surprisingly small model, just 270 million parameters, learns to predict winning moves from single board positions and appears to approximate algorithms, with potential applications in program synthesis and autonomous driving.
Questions this video answers
15 billion board states
Stockfish
270 million parameters
A single board state (not a sequence of positions)
About 20 moves per second on a PC with a ~$200 GPU; about 2 moves per second on a CPU
Chapters
Scientists at Google DeepMind have already created an AI system that plays chess on the level of a grandmaster. Actually, these are so good, no human has a reasonable chance to beat them. So why write a new paper on it, especially one that does not perform as well? How does this make any sense? Well, to accomplish this earlier, they created an incredible AI system. It starts from zero and could learn from each match it plays. It had two key ideas. One of the key ideas was to make a copy of this AI and have it play against itself billions of times and learn from that. The other was search. This means that it would think several steps ahead. It would often even simulate the game right until the very last move. And all this before deciding on a move. And every single variant of this AI involved sophisticated techniques for search, search, search, search, search everywhere. And now here is their new paper. And this AI did absolutely none of these things. No self play, no search, none. But why? Why would they do that? And why does all this matter? Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Yes, that's right. They created a grandmaster level AI technique without search and self play. Sort of. Let's focus on the no search part. How well it learned from Stockfish, a powerful handcrafted chess engine, one of the best ones around. It looked at 15 billion board states and the moves that this strong chess engine would make on these boards. Just moves, not full matches. So what is the result? Now hold on to your papers, fellow scholars, because the result is that it can play on the level of a human grandmaster. If it were a human, it would be one of the best players in the world. That is fantastic. And all this without ever having played a single game from start to end. That is an incredible result. And oh my, look at that. 270 million parameters for their bigger model. It might sound like a large number, but a seasoned fellow scholar knows that this neural network is tiny. That is about 3,000 times smaller than GPT4. So it is tiny and it gets better. They also have a version that is half of that in size and performs nearly as well. So it is tiny, but is it fast? Well, it can give you 20 moves per second on a personal computer with a graphics card that costs $200. And if you don't have a graphics card still two moves per second on your processor, it might even run on the phone in your pocket. Yet it is able to play extremely well. It is much, much better than the 3,000 times bigger GPT4 in chess. Wow. Now wait, there are two surprises here. Two key assumptions, none of which will seem to make sense at first. One, in this work, only one board gets in. Not a sequence of board positions, not the whole match, just the current state of the match. Two, it sees only one move ahead and makes sure that the move has a high probability to win the game. So why do we have these assumptions? The these assumptions do not necessarily lead to the most usable chess engine. You could do way better just by throwing these out. So why. And here comes the key part that I think is easy to misunderstand here. The goal here is not necessarily to create a strong chess engine. No sir. But wait, I hear you asking Caroj, this is a paper on chess and you are saying that chess is not important here? Yes, kind of. You see, the goal here is to demonstrate that a transformer type neural network can learn the expertise of a master just by watching the master at work. And of course this is not just a copying machine. It really learned how to make good moves and generalized well to novel situations. And the key to understand why this matters and why this is an incredible achievement, we have to move back in time to more than 8002 minute papers episodes ago. Oh yes, a really old paper by two scientists in London. I love the smell of an amazing dusty old paper. The neural programmer interpreter. The goal of this work was to create an AI that doesn't just give us answers that but one that can give us full algorithms. It could write readable programs to rotate these cars or to sort numbers. It was simple, it was crude and not very useful. But more importantly, it was a glimpse of the future. And that future is now. You see, this new chess technique learned not just how to make moves, it learned to approximate algorithms. So if we were able to look into these neural networks, we might just be able to pull out not just the moves, but the chess playing algorithm itself. And once again, this is not just about chess. This is a concept that could be used to create self driving cars, amazing new ray tracing algorithms and so much more. And good news fellow scholars. The luckily scientists at Anthropic are already working on looking into these neural networks and finally they have made meaningful progress on that. So with these works, we might be one step closer to an amazing AI technique that can look at the world around us and create useful algorithms that we can use and understand Algorithms, not just a pile of numbers. What a time to be alive. And if you wish to run your own experiments in the cloud, make sure to check out Microsoft Azure AI. Azure AI is a powerful cloud platform that offers you the best tools for your AI projects with responsible AI built in. And here comes the best part. You can even try it out for free through the link in the video description.
