An AI learns to play racing games by dreaming up imaginary tracks to practice on. It compresses what it sees into a compact memory, then runs internal simulations to improve. On 100 random tracks, this dream-trained model scored nearly three times higher than the standard baseline.
Dear fellow scholars, this is two minute papers with. Today we are going to talk about an AI that not only plays video games really well, but can also dream up new unseen scenarios and more. This is an interesting new framework that contains a vision model that compresses what it has seen in the game into an internal code. As you see here, these latent variables are responsible to capture different level designs. And this variable simulates time and shows how the fireballs move towards us over time. This is a highly compressed internal representation that captures the most important aspects of the game. We also have a memory unit that not only stores previous experiences, but similarly to how an earlier work predicted the next pen strokes of a drawing. This can also dream up new gameplay. Finally, it is also endowed with a controller unit that is responsible for making decisions as to how to play the game. Here you see the algorithm in action. On the left there is the actual gameplay, and on the right you see its compressed internal representation. This is how the AI thinks about the game. The point is that it is lossy, therefore some information is lost, but the essence of the game is retained. So this sounds great, the novelty is clear, but how well does it play the game? Well, in this racing game, on a selection of a hundred random tracks, its average score is almost three times that of DeepMind's groundbreaking deep cue learning algorithm. This was the AI that took the world by storm when DeepMind demonstrated how it learned to play Atari Breakout and many other games on a superhuman level. This is almost three times better than that on the racetrack game, though it is to be noted that DeepMind has also made great strides since their original DQN work. And now comes the even more exciting part, because it can create an internal dream representation of the game. And this representation really captures the essence of the game. And then it means that it is also able to play and train within these dreams. Essentially, it makes up dream scenarios and learns how to deal with them without playing the actual game. It is a bit like how we prepare for a first date, imagining what to say and how to say it. Or imagining how we would incapacitate an attacker with our karate chops if someone were to attack us. And the cool thing is that with this AI, this dream training actually works. Which means that the newly learned dream strategies translate really well to the real game. We really have only scratched the surface, so make sure to read the paper in the description. This is a really new and fresh idea and I think it will give birth to a number of follow up papers. Cannot wait to report on these. Back to you. So stay tuned and make sure to subscribe and hit the bell icon to never miss an episode. Thanks for watching and for your generous support, and I'll see you next time.
