Google DeepMind released Veo2, an AI video generator that produces 4K output with strong visual quality and prompt following. It outperforms earlier systems like Video Poet, though it still struggles with fast motion and sometimes loses track of objects or renders faces at low resolution.
Okay, due to popular request from you fellow scholars, let's talk about Google DeepMind's new AI video generator called VO2. Now look at this. Is this VO2? It is not. So what is it then? This is called Video Poet, which was one of the state of the art AI video generators back in the day. Now what do I mean by saying back in the day like five years ago? No, absolutely not. This was a state of the art video AI less than a year ago. And today we can do this. Whoa. Just look at the difference. Holy mother of papers. The new VO2 can create videos up to 4K resolution. That is stunning. And I gotta say, I was very, very surprised by the results that I am going to show you now. Okay, but we are fellow scholars here, so we have questions. What can it? What can't it do? How does it work? And of course, how does it compare to its competitors like OpenAI's Sora? Can this compete? You will get answers to all of these questions. Dear fellow scholars, this is two minute papers with Dr. Zsolnai. Now first, if you are one of our OG fellow scholars, I hope this says something to you. Oh yes, I am a computer graphics researcher by trade and we have tons and tons of simulation programs like this where we have to write the laws of physics into a computer program and this comes out. Yes, these are all computer simulations. However, today we don't even need to do that. An AI video generator can create all of these pixels by itself and we only need to add a text prompt. Just look at that quality. Incredible, honey. Much better than the company with the same name. Also, when creating humans, this fellow scholar is super lifelike and I see barely any or no flickering at all. Super coherent video. So cool. The usual suspects like a drifting car or animated movies are working superbly. And of course you can also use this AI to let your imagination run wild and dream up worlds that don't even exist. Or maybe can't even exist. So good. I am almost ready to believe that this can do absolutely anything. So can it? Well, no, it is not perfect. It has its own limitations. For instance, when we have this high frequency motion with skateboarding, we have lots and lots of temporal coherence issues, which I will explain in a moment. But there are cases with other high frequency movement with tons and tons of beats here. And this seems really well done to me. Really interesting. Also, sometimes humans come out with a really coarse low resolution and it gets worse. Look at that. The face turning away from us is more like morphing into the Back of the head. And if we look behind the shoulder of this woman. Oh yes, the two minute papers special, that is Object permanence issues. Something appears and when we see it again, it becomes something else. Okay, but we have two more questions to go. How does all this wizardry work? Well, the architecture they use for this is called a diffusion transformer model. Okay, but what does that mean? Well, when you do text to image, the model starts out from a bunch of noise and over time reorganizes this noise to make it resemble your text prompt. However, here we are talking about video, not just one image. So easy just do it many times after each other, right? No, no, no, not at all. You see, if you do that, you get something like this. You get this flickering effect because the neural network does not remember well images it made previously and the differences show up as flickering. You can't do that. What you need to do is create not just one bunch of noise, but many, many bunches and refine them at the same time while taking into consideration not just their neighbors, but every single image. This is how you get long term temporal coherence. And this is why we have these consistency problems we just talked about. It is getting better and better over time, but it still there now it's good to look at these results, but how does it compare to its competitors? And now hold on to your papers fellow scholars. And oh my goodness, look at that. I would say it is heavily favored against its competitors, especially against the amazing sora. That is a stunning result. Wow. But it gets better. This was about overall quality, but in terms of prompt adherence. Wow. It is also doing extraordinarily well. This is where scientists test how closely the videos are following your text prompts. Super important because something can look amazing. Might not be what you asked for at all. This is not the case here. It is also interesting how the results correlate with the quality as well. Now note that this is not from a peer reviewed study. So of course every wise fellow scholar like you applies a small grain of salt here. I wanted to try it out too and ah yes, maybe you fellow scholars will have better luck with it. There is a link in the description, so please let me know in the comments below. And to think that this was possible less than a year ago and now this. It is unbelievable how far we have come. There is so much to be excited about in the near future. What a time to be alive. What do you fellow scholars think? Let me know in the comments below. To run your own experiments on an Nvidia gpu. Check out Lambda. I use it myself regularly for these videos. Hmm, look at that. You can generate high quality images in less than a second per image. I did a ton more of them and paid less than a dollar for all this crazy. Seriously. Try it out now@lambdalabs.com papers or click the link in the description.
