A Two Minute Papers YouTube episode (Two Minute Papers / Dr. Neural Network, uploaded 2020-06-20) discusses NVIDIA’s GameGAN work, where a neural network learns to recreate a playable Pac-Man-like game purely from gameplay video. The narrator situates this result alongside an earlier 2015 DeepMind "neuroprogrammer interpreter" that trained networks to output programs, highlights GameGAN’s learned capabilities (game rules, memory, foreground/background separation, neural rendering), notes the system trained on roughly 120 hours of footage, and reflects on broader implications for AI-generated code and simulation. The video includes a sponsor segment for Weights & Biases and a subscribe/CTA closing.
Questions this video answers
2020-06-20
Approximately 120 hours
DeepMind's 2015 "neuroprogrammer interpreter" paper
Weights & Biases (W&B)
Chapters
Dear fellow scholars, this is two minute papers with Dr. Neural Network based learning methods are capable of wondrous things these days. They can do classification, which means that they can look at an image and the output is a decision whether we see a dog or a cat or a sentence that describes an image. In the case of DeepMind's AI playing Atari games, the input is the video footage of the game and the output is an action that decides what we do with our character next. In OpenAI's amazing jukebox paper, the input was a style of someone, a music genre and lyrics, and the output was a waveform, or in other words, a song we can listen to. But a few hundred episodes ago, we covered a paper from 2015 where scientists at DeepMind asked the question, why? What if we would get these neural networks to output? Not sentences, decisions, waveforms or any of that sort. What if the output would be a computer program? Can we teach a neural network programming? I was convinced that the answer is no until I saw these results. So what is happening here? The input is a scratch pad where we are performing multi digit addition in front of the curious eyes of the neural network. And if it has looked for long enough, it was indeed able to reproduce a computer program that could eventually perform addition. It could also perform sorting and would even be able to rotate the images of these cars into a target position. It was called a neuroprogrammer interpreter, and of course it was slow and a bit inefficient, but no matter, because it could finally make something previously impossible but possible. That is an amazing leap. So why are we talking about this work from 2015? Well, apart from the fact that there are many amazing works that are timeless and this is one of them in this series, I always say two more papers down the line and it will be improved significantly. So here is the two minute paper's moment of truth. How has this area improved with this follow up work? Let's have a look at this paper from scientists at Nvidia that implements a similar concept for computer games. So how is that even possible? Normally, if we wish to write a computer game, we first envision the game in our mind, then we sit down and do the programming. But this new paper does this completely differently. Now hold on to your papers. Because this is a neural network based method that first looks at someone playing the game and then it is able to implement the game so that it not only looks like it, but it also behaves the same way to our key presses. You see it at work here? Yes, this Means that we can even play with it. And it learns the internal rules of the game and the graphics just by looking at some gameplay. Note that the key part here is that we are not doing any programming by hand. The entirety of the program is written by the AI. We don't need access to the source code or the internal workings of the game. As long as we can just look at it, it can learn the rules, everything truly behaves as expected. We can even pick up the capsule and eat the ghosts as well. This already sounds like science fiction and we are not nearly done yet. There are additional goodies. It has memory and uses it consistently. In other words, things don't just happen arbitrarily. If we return to a state of the game that we visited before, it will remember to present us with very similar information. It also has an understanding of foreground and background, dynamic and static objects as well. So we can experiment with replacing these parts, thereby reskinning our games. It still needs quite a bit of data to perform all this. And as it has looked at approximately 120 hours of footage of the game being played, however, now something is possible that was previously impossible. And of course two more papers down the line this will be improved significantly, I am sure. I think this work is going to be one of those important milestones that remind us that many of the things that we had handcrafted methods for will over time be replaced with these learning algorithms. They already know the physics of fluids, or in other words, they are already capable of looking at videos of these simulations and learn the underlying physical laws. And they can demonstrate having learned general knowledge of the rules by being able to continue these simulations, even if we change the scene around quite a bit. In light transport research, we also have decades of progress in simulating how rays of light interact with scenes and and we can create these beautiful images. Parts of these algorithms, for instance noise filtering, are already taken over by AI based techniques and I can't help but feel that a bigger tidal wave is coming. This tidal wave will be an entirely AI driven technique that will write the code for the entirety of the system. Sure, the first ones will be limited. For instance, this is a neural renderer from one of our papers that is limited to this scene and lighting setup. But. But you know the saying, two more papers down the line and it will be an order of magnitude better. I can't wait to tell all about it to you with a video when this happens. Make sure to subscribe and hit the bell icon to not miss any follow up works. Goodness, I love my job. What a time to be alive. This episode has been supported by Weights and Biases. In this post they show you how to visualize molecular structures using their system. Weights and Biases provides tools to track your experiments in your deep learning projects. Their system is designed to save you a ton of time and money and it is actively used in projects at prestigious labs such as OpenAI, Toyota Research, GitHub and more. And the best part is that if you have an open source, academic or personal project, you can use their tools for free. It really is as good as it gets. Make sure to Visit them through wnb.com papers or just click the link in the video description to start tracking your experiments in five minutes. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
