DeepMind's AlphaZero learned chess from scratch in four hours and beat Stockfish without losing a single game. It evaluated far fewer positions per second than Stockfish yet won 28 games and drew 72. The same algorithm works on Shogi and other games.
Dear fellow scholars, this is two minute papers with after defeating pretty much every highly ranked professional player in the game Of Go, Google DeepMind now ventured into the realm of chess. They recently challenged not the best humans. No no no, that was long ago. They challenged Stockfish, the best computer chess engine in existence, in quite possibly the most exciting chess related event since Kasparov's matches against Deep Blue. I will note that I was told by DeepMind that this is the preliminary version of the paper, so now we shall have an initial look and perhaps make a part two video with the newer results when the final paper drops. AlphaZero is based on a neural network and reinforcement learning and is trained entirely through self play. After being given the rules of the game, it is not to be confused with AlphaGo Zero that played Go. It is also noted that this is not simply AlphaGo Zero applied to chess. This is a new variant of the algorithm. The differences 1. The rules of chess are asymmetric. For instance, pawns only move forward, castling is different on kingside and queenside and this means that neural network based techniques are less effective at it. 2. The algorithm not only has to predict a binary win or loss probability when given a move, but draws are also a possibility and that is to be taken into consideration. Sometimes a draw is the best we can do. Actually, there are many more changes to the previous incarnation of the algorithm. Please make sure to have a look at the paper for details before we start with the results and more details. A word on ELO ratings for perspective. The ELO rating is a number that measures the relative skill level of a player. Currently, the human player with the highest ELO rating, Magnus Carson, is hovering around 2800. This man played chess blindfolded against 10 opponents simultaneously in Vienna a couple years ago and won most of these games. That's how good he is. And Stockfish is one of the best current chess engines. With an elo rating over 3300. The difference of 500 Elo points means that if it were to play against Magnus Carson, it would be expected to win at least 95 games out of 100, though it is noted that there is a rule suggesting a hard cutoff at around the 400 point difference. The two algorithms then played each other AlphaZero versus Stockfish. They were both given 60 seconds of thinking time per move, which is considered to be plenty given that both of the algorithms take around 10 seconds at most per move. And here are the results. AlphaZero was able to outperform Stockfish in about four hours of learning from scratch. They played 100 games. AlphaZero won 28 times, drew 72 times, and never lost to Stockfish. Holy mother of papers. Do you hear that? Stockfish is already unfathomably powerful compared to even the best human prodigies. And AlphaZero basically crushed it after four hours of self play. And it was run with a similar hardware as AlphaGo Zero one machine with four tensor processing units. This is hardly commodity hardware, but given the trajectory of the improvements we've seen lately, it might very well be in a couple of years. Note that Stockfish does not use machine learning and is a handcrafted algorithm. People like to refer to computer opponents in computer games as AI, but it is not doing any sort of learning. So you know what the best part is? AlphaZero is a much more general algorithm that can also play Shogi on an extremely high level, which is also referred to as Japanese Chess. And this is one of the Most interesting points. AlphaZero would be highly useful even if it were slightly weaker than Stockfish because it is built on more general learning algorithms that can be reused for other tasks without investing significant human effort. But in fact it is more general and it also crushes Stockfish. With every paper from DeepMind, the algorithm becomes better and more and more general. I can tell you this is very, very rarely the case. Total Insanity Two more interesting tidbits about the paper. 1. All the domain knowledge the algorithm is given is stated precisely for clarity. 2. One might think that as computers and processing power increases over time, all we have to do is add more brute force to the algorithm and just evaluate more positions. If you think this is the case, have a look at this. It is noted that AlphaZero was able to reliably defeat Stockfish while evaluating 10 times fewer positions per second. Maybe we could call this the AI equivalent of intuition. In other words, being able to identify a small number of promising moves and focusing on them. Chills run down my spine as I read this paper. Being a researcher is the best job in the world and we are even being paid for this. Unreal. This is a hot paper. There's a lot of discussions out there on this. Lots of chess experts analyze and try to make sense of the games. I had a ton of fun reading and watching through some of these. As always, 2 Minute Papers encourages you to explore and read more and the video description is ample. In useful materials you will find videos with some really cool analysis from Grandmaster Daniel Kingdom, International Chess Master Daniel rensc and the YouTube channel Chess Network. All quality materials. And if you have enjoyed this episode and you think that eight of these videos a month is worth a few dollars, please throw a coin our way on Patreon. Or if you favor cryptocurrencies instead, you can throw Bitcoin or Ethereum our way. Your support has been amazing as always. Thanks so much for keeping with us through thick and thin, even in times when weird Patreon decisions happen. Luckily this last one has been reverted. I'm honored to have supporters like you, fellow scholars. Thanks for watching and for your generous support, and I'll see you next time.
