YouTube video commentary (Two Minute Papers) by Dr. Zsolnai reviewing OpenAI's Sora—a text-to-video AI. He praises Sora's quality (especially humans and creative scenes), demonstrates features (Remix, Recut, Blend, styles/looping), raises concerns about competition (Luma Labs, Runway) and release timing, notes pricing and usage limits (part of ChatGPT Plus/Pro), and predicts rapid local-model parity within a year.
Yes. OpenAI. Sora is finally here. This is an amazing AI that takes a small piece of text from you and yes, creates an incredible video. The first results were showcased earlier this year, but I am a little worried. I am worried because since then a lot of time has passed and we saw many competitors actually release their models in the meantime. And I am not sure if Sora would still be good by the time it is released. You see, Luma Labs has such an AI, Cheap but OK quality, not super detailed, Runway a bit more expensive, much higher quality cling and so on. The competition is strong and you can use them right now. So by the time Sora is released, can it really compete? Well, these results are already an indication, but let's have a look together. Dear fellow scholars, this is two minute papers with Dr. Zsolnai. Well, my first favorite is of course the Best of the Good Boys Super Pup. In the world of runaway inflation, the world needs you, Super Pup. And from now on, including this one, every single video that you will see here will be AI generated. Sora can also create crazy things like a dragon made of shampoo bubbles. And as expected, it is extraordinarily good at creating and animating humans. I had the honor of trying Sora a bit more than half a year ago and I was stunned by the results I saw. And I am so happy that I can finally share this beautiful moment with you fellow scholars. You can now make a family of extraordinarily civilized bears eating sushi. And interestingly, look at this carefully. Have you noticed we talked a great deal about how good it is at object permanence? Things that disappear and appear again are typically the same. However, here some golden retrievers fail some of the object permanence tests. I will now slow this down. Have you found any appearing or disappearing dogs? Let me know in the comments below, but I'll give you a real good one. Look at this good boy. And then. Oof. It becomes the rear end of a different good boy. Or maybe the same, I don't know, crazy. But it does not end there. In fact, this is just a start. Now comes the best part where we look at four amazing features and one not so amazing feature that it also has one. Now hold on to your papers, fellow scholars. Four Remix. If you are happy with some of the aspects of your result, you want to keep it, but only replace the doors. It has to open to the same scene. Not a problem. Or if you wish to remix the interior instead, that is also possible. 2. Recut. You don't have to Accept the whole video as a result. Just choose the parts that you like for put them on a timeline and feel what happens before the good part and maybe after too. 3. Blend. Throw two videos together and you'll see that snowflakes and flower petals go really well together. Here is the original video of a nature documentary, also AI made, but if you would like to recreate it in a different style, I love the Papercraft version. The archival version has a nice analog warmth to it and so on. If something caught your eye, you can save the style to reuse later, thereby enabling you to create a movie that is not just a bunch of footage cobbled together. No, no. But something that follows a nice overarching design. Fantastic. And one Is this amazing or not so amazing looping videos? I would say this is not so amazing as they have these sudden jumps within the video and if there is such a thing, you might as well just copy paste the video. Many get a similar jumpy result. This is a bit better, but as a research scientist I can't resist mentioning this research work that was done almost 25 years ago and could make non looping videos loop. Stunning. Overall, I love the fact that the SORA team did super good on the research side, but at the same time did not neglect the engineering aspect of the product. Very few companies can do the two at the same time. That is excellent. So where can you use it and can it compete? Well, it is now part of the ChatGPT plus and Pro subscriptions. It is pricey. There are heavy limits on usage. I think they even had to disable signups for a while because so many of you fellow scholars were interested. If you ever listen to an entrepreneur talk about product market fit, this is it. They literally can't get enough capacity to keep up with the demand. So yes, SuperPAP can compete and yes it is pricey. But I think less than a year from now you are going to get this kind of quality for free on your own machine with local models, possibly even better and possibly even earlier. What a time to be alive. Not so long ago we were only able to create images using AI and not even very good ones. And now we got video that is so much harder. Every frame has to relate to the previous one and just one small mistake and the illusion is gone. And this is just Sora version 1. This is the worst version of it. If you will imagine what the third version will look like, we will all be able to become film directors. My goodness. So what do you think? What would you fellow scholars use this for? Let me know in the comments below. To run your own experiments on an Nvidia gpu, check out Lambda. I use it myself regularly for these videos. Hmm, look at that. You can generate high quality images in less than a second per image. I did a ton more of them and paid less than a dollar for all this. Crazy. Seriously. Try it out now@lambdalabs.com papers or click the link in the description.
