YouTube video (Dr. Zsolnai/LARRUZ) commenting on rapid advances in text-to-video AI. The speaker highlights new systems claiming near-real-time generation (e.g., "one minute video clips within one minute" but clarified as approaching one second per second), describes a technique that generates an image first then animates it to speed up video creation (sparsification step), notes models that can run on a single consumer GPU, and surveys related systems (Sora, DeepMind real-time demos, Phantom, Step Video, Deepseek). The speaker praises open-source/open-science progress while noting dataset biases (human-centric/cinematic) and trade-offs between speed and visual quality.
Chapters
What a day. Free Text to video AIs are popping up like crazy where you write a short text prompt and you get a video of it. And now you are seeing a brand new system here and I am delighted by how incredible it is. You see the title of the paper says it can generate one minute video clips within one minute. Is that true? Well, kind of. Let me explain. This might give you the impression that we are going to see one minute long generated videos, which we are not. Instead, they probably meant to imply video generation times that are approaching real time speed. That means generating one second of video footage within one second of time in real life. Which is kind of mind blowing. I mean less than a year ago I was able to try sora at the OpenAI lab and unfortunately I couldn't show it to you. Everything was super secret. And. And now less than a year later you are telling me that there are dozens of these text to video AI systems with matching quality, perhaps even better. I mean look at the kind of variety it is capable of. And all this in real time. Wow, what a time to be alive. Just think about it. Here is a huge pack of already existing techniques, many of which are less than a year old. And now bam. This one is 12 times faster. That sounds insane. I mean how do you even do that? How is that even possible? Dear fellow scholars, this is two minute papers with Dr. Zsolnai. Well here is a crazy idea. It does something that we fellow scholars are already doing. You see if text to video takes too long, we don't want to try the same prompt a hundred times to get something good. Instead we have tons of free text to image AIs out there. So we write a short text prompt, create a still image quickly and then if we like the image, give that to a text to video AI. And we only have to generate one video, not a hundred. And given that we like the image, we will probably like the video too. And that is what this technique does. First generate an image from text and then make that image move. Now they also use a sparsification step, cutting a few corners if you will. And this leads to nearly real time text to video. My understanding is that this can run on one consumer graphics card. Not a cheap one mind you, but it can be done. And that really makes me think, I mean Real Time DeepMind's AI is able to take just one image and create these video games out of it. They are simple games mind you, but games nonetheless. That was less than a year ago and now you are saying that we have the capability to generate games out of one photo in real time at home. I have been doing this channel for almost 10 years and nearly a thousand videos. But I still have a hard time thinking that this is real. Unbelievable. Now, not even this technique is perfect. It was trained on an unbalanced data set that is very dense in human centric and cinematic stuff. So this is a limitation of this particular trained model, but not the concept itself. That means an even better model can be trained. Whoa. So good. But it doesn't end. There is another system that just came out called Phantom that does something incredible. Not just text to video, but subject to video. You can have a person, a place or an object and it will create a video about them. But here comes the twist in a manner that preserves their identities. This was and kind of still is a problem even with image generator systems. Imagine trying to create a comic with a central character. You need many images, but the character keeps changing every time you generate them. This system is a bit lower in terms of visual quality. However, you can recall the same identities as many times as you want. The full research paper is available for this one too. And we are still not done yet. Now that we can create videos efficiently, the content often comes out as we wanted because we write the prompts for them. But sometimes the presentation is not exactly as we envisioned. But this new tool does this really amazing relighting for an input video without changing it much. If you want to look a bit more dramatic, not a problem. Put your cat in a cyberpunk world. Easy. Now hold on to your papers, fellow scholars, for a final paper for Step Video. If you are okay to wait a bit longer for video creation. But in return they promise higher visual quality. And I got to say, I am not sure if this is strictly better than the first one, which was 12 times faster. But I left it here to show you that things are progressing so quickly we can barely keep up with all the things that are offered for us for free. So many papers to read and so many models to try. This is open science and open source at its best. That is what I want. And just imagine that all of these works came out just a few days apart. Loving it. What a time to be alive. So what do you think? What would you fellow scholars use this for? Let me know in the comments below. Would you like to run your own copy of Deepseek in the cloud cheaply without using the official app? Yes. Then try Lambda GPU Cloud. They have so many powerful Nvidia GPUs with tons of memory. I use them regularly too. Seriously, Try it out now@lambdalabs.com papers or click the link in the description.
