DeepMind's DVD-GAN generates realistic videos across many topics by using two discriminators to check spatial and temporal quality. It creates 2-second videos at once rather than frame-by-frame, and learns to separate foreground from background without being told how.
Questions this video answers
256×256
48 frames
About 2 seconds
A spatial discriminator and a temporal discriminator
No — it generates the entire video in one go
Chapters
Dear fellow scholars, this is two minute papers with in the last few years, the pace of progress in machine learning research has been staggering. Neural network based learning algorithms are now able to look at an image and describe what's seen in this image, or even better the other way around, generating images from a written description. You see here a set of results from Biggan, a state of the art image generation technique, and marvel at the fact that all of these images are indeed synthetic. The GAN part of this technique abbreviates the term generative adversarial network. This means a pair of neural networks that battle each other over time to master a task. For instance, to generate realistic looking images. When given a theme, these detailed images are great. But what about generating video with the dual video discriminator gan dvdgan? In short, DeepMind's naming game is still as strong as ever. It is now possible to create longer and higher resolution videos than was previously possible. The Exact numbers are 256 x 256 in terms of resolution and 48 frames, which is about 2 seconds. It also learned the concept of changes in the camera view zooming in on an object, and understands that if someone draws something with a pen, the ink has to remain on the paper unchanged. The dual discriminator part of the name reveals one of the key ideas of the paper. In a classical gan, we have a discriminator network that looks at the images of the generator network and critiques them. As a result, the discriminator learns to tell fake and real images apart better, but at the same time provides ample feedback for the generator neural network so it can come up with better images. In this work we have not one but two discriminators. One is called a spatial discriminator that looks at just one image and assesses how good it is structurally, while the second temporal discriminator critiques the quality of movement in these videos. This additional information provides better teaching for the generator, which will in return be able to generate better videos for us. The paper contains all the details that you could possibly want to learn about this algorithm. In fact, let me give you two that I found to be particularly interesting. One, it does not get any additional information about where the foreground and the background is, and is able to leverage the learning capacity of these neural networks to learn these concepts by itself. And two, it does not generate the video frame by frame sequentially, but it creates the entire video in one go. That's wild. Now, 256x256 is not a particularly high video resolution, but if you have been watching this series for a while, you are probably already saying that. Two more papers down the line and we may be watching HD videos that are also longer than we have the patience to watch. And all this through the power of machine learning research. For now, let's applaud DeepMind for this amazing paper and I can't wait to have a look at more results and see some follow up works on it. What a time to be alive. Thanks for watching and for your generous support and I'll see you next time.
