AI Like OpenAI’s Sora...But Free To Try!

As we are all itching to try OpenAI's video generator, AI Sora. And after looking through a lot of research papers on this topic, finally, here is something called the Dream Machine that can do this, this and this. So, is it as good as Sora? And why does it matter? Well, let's have a look. And I'll tell you right now that as of the making of this video, you fellow scholars can all try for free. And yes, I've done what must be done for science too. Dear fellow scholars, this is two minute papers with Dr. Zsolnai. Well, we have some good news and some bad news. First, I would say it is not as good as Sora. Nothing that I've seen yet is. And this new one has lots of limitations. Let's start with those and we shall see how it gets a lot better in a moment. First, things can morph from one thing to another. Second, movement may not always be amazing or exist at all. And it is also not great at the Achilles heel of almost all AI Text to image and video systems. And that is generating text. I will commend Luma Labs for candidly coming forward and discussing these failure cases. Love it. And we are not done with the limitations. Here is a potentially big one with Sora. We saw up to 60 second clips, but this one can do only five seconds. So that's a bit of a problem, right? Well, surprisingly not so much. Watch this. Now, we did not publish a video on this immediately. I wanted to wait a little to see how you fellow scholars are using it in practice and to be able to create a better video for you. Hope you appreciate it. And wait a second. This video has been going on for a while. Certainly much longer than five seconds. How is that possible? Well, it is possible through two steps. One, it has an image to video mode. Which means. Are you thinking what I am thinking? Oh yes. It can bring these legendary memes to life and we can finally find out what what really happened here. And yes, it has some issues as the background can suddenly become the foreground, but still an amazing piece of technology. Or we can do this too. Hope he finds what he's looking for eventually. Plus, even though it is not great at creating text from scratch, but given a good starting point, it can make a pretty cool animated intro with it. And now, one of my favorites. This is fine. We can also find out what could have happened here according to the AI or even tell it how it should be continuing it. But I will note that this is just one panel of the original comic strip. I've put a link to it in the video description. I will warn you that it is not pretty. Or you can even use your favorite text to image AI and use that as a starting point. A full workflow that needs nothing else, only your imagination. Loving it. So wait a second, how does this image to video thing help us create longer videos? Well, hold on to your papers fellow scholars, because you create your first five second snippet. Then you take the last image and ask the AI to create a video starting with this image. You can do this as many times as you wish and bam. You get an arbitrarily long, almost completely seamless video. How cool is that? And with this you can really let your imagination run wild. You can make up new kinds of animals, even solve the trolley problem. Although, oof, this might not be the solution you are looking for. And now of course we go like this. Oh yes, videos of scholars holding on tightly to their papers must be done. And what is this? This is an excerpt from a feature length AI generated movie. In this case you see that the cuts take place Approximately at the 5 second mark. Previously, to make a movie that looks anything like this, it you needed a professional crew, lots of funding, and with tools like this, today everyone can become a film director, even for free or for very little. What a time to be alive. And worry not, according to OpenAI, Sora will also be available sometime this year. And just imagine what we will be capable of just two more papers down the line. You will surely be able to do it on the phone locally, right in your pocket. And that's just the start. My goodness. And a quick note that we don't have any kind of relationship with Luma Labs, the company behind this technique. So what did you fellow scholars make? Let me know in the comments below. If you're looking for an Nvidia GPU cloud service for large scale AI projects, check out One Click Clusters by Lambda, featuring up to 512 Nvidia H100 GPUs with InfiniBand networking and hold on to your papers because with one click clusters you can access all these GPUs on demand with reservations starting at just two weeks. Try it out@lambdalabs.com paper or click the link in the description.