Dr. Zsolnai breaks down OpenAI's unreleased Sora model, praising its physics simulation but noting text rendering fails and occasional glitches like extra limbs. Dream Machine and Runway Gen 3 are already good enough that you might not want to wait for Sora.
This is OpenAI's text to video AI called Sora. As it is still not out publicly yet and as everyone is hungering for it, I will tell you about two alternatives that you can use right now. These paid AIs are excellent and more importantly available right now. And I was really surprised because one of them is I think close to as good as Sora. In the meantime, OpenAI also gave Sora two amazing creative people to see what it is capable of. And surprisingly very few people saw them. And I found quite a few interesting things in them. Let's have a look. Dear fellow scholars, this is two minute papers with Dr. Zsolnai. The goal with these text to video AIs is is that we can finally imagine new creative projects that would be too expensive to try in real life or things that are flat out impossible. Alright, so let's have a look at what these amazing artists were able to do with Sora. Oh yes, it is fantastic at physics based simulations. This almost looks like reality. Wow. It is also great at object permanence. If something gets occluded, it reappears to be very similar or often the same. Now it is of course not perfect. I think the leg movement here is a little excessive also, as you smart fellow scholars have guessed. Oh yes, text. It is still not great. Modern text to image solutions such as the Amazing and Free Flux is doing better on these also. Is he sitting or is he walking? Neither, because he had grown an additional leg. I told you this can do things that are otherwise impossible. Also look at our little old friend the hands every time. Now two alternative systems that are available right now. One is the Dream Machine for from Luma Labs. This is the cheaper alternative and this you can try for free. It is really good. However, Runway's pricier Gen 3 is perhaps even better. Just look at this. This is not just text to video, but a set of amazing creative tools around it. Note that this video is not sponsored by any of these companies. While we look at some more Results With Runway Gen 3, I really wanted to make sure that you fellow scholars know about these alternatives. Why? Because they are so good. You might find that you don't want to wait for Sora instead. And yes, today we can all become creative movie directors. And you know the drill. Just two more papers down the line and I am sure we will be able to do this for super cheap. Perhaps even right on your phone in your pocket. What a time to be alive. So what do you think? What would you fellow scholars use this for? Let me know in the comments below. We need new tools for the era of LLMs and weights and biases. Now has Weave a lightweight toolkit to confidently iterate on LLM applications, use traces to debug how data flows through each step of your app, and use evaluations to measure your progress. It is the best. Try it out now at WNB Me papersllm or click the link in the description below.
