Ross Zeiger (speaker) reviews OpenAI's new text-to-video demo Sora (March 2024 context), shows example outputs (a woman in neon-lit Tokyo, flocking paper-airplanes), and argues the technology is a step-change for generative video despite current artifacts. He connects Sora to Descript’s existing text-to-image (DALL·E) features, notes Descript upgraded from DALL·E2 to DALL·E3 in January 2024, and predicts Descript will integrate Sora or similar text-to-video features soon (beta within months of public Sora release) for b-roll, avatars, and in-scene object additions — ultimately speculating on on-demand custom movies. Tone: speculative, optimistic about Descript’s positioning.
A few weeks ago, OpenAI, the company behind ChatGPT, rolled out some demos from their new tool called Sora. And along with it, we all got a glimpse into the near future of video. I think this is going to be massively disruptive to Hollywood, to Netflix, and yes, you and I as video editors and creators, I'm going to make some predictions about what this means for descript and video editing in general. And spoiler, I think descript is going to benefit more than most other softwares.
Sora is a text to video tool.
Meaning that you prompt it in plain English and it generates a video based off of your prompt. Now this tech has existed for several years, but it's never been really good until now. So let's check it out. This is the Sora landing page on OpenAI.com Sora and if we scroll down a little bit, they got a bunch of examples of different things that have been created. And this one, the first one, is probably the most famous video. You've probably already seen it, it's been all over the Internet. And it's this woman walking down the street in Tokyo. And right below it is the prompt.
That they used to create it.
So they wrote, a stylish woman walks down a Tokyo street filled with the warm glowing neon and animated city signage.
She wears a black leather jacket, a.
Long red dress and black boots. And I think it continues, but that's, oh yeah, more. And carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights many pedestrians walk about. So it's not perfect, but check out.
The lights, the shadows, the other people.
Here in the background. Um, it's, it's not perfect if you pick it apart. If you zoomed in, you'll find imperfections. There's weird things in the physics that the feet don't always turn in the right way. And you know, not in this one, but in other ones there's things where, like the wind doesn't create flames to blow in the right direction or trees to blow in the right direction. But how impressive is this? And there's tons of other examples here. I especially love the hero section of this site, this one right here with the birds. Or it's paper airplanes. The prompt was something like create paper airplanes flocking through a forest like birds. So Sora and future tools like it will allow you to create things from your imagination that can't exist in real life. Paper airplanes flying like birds and without any knowledge of cgi or animation whatsoever. I could talk for hours about what this means for the industry in the medium and long term.
But I teach Descript here and I.
Have some bold predictions for what this means for Descript in the short term. This year, 2024, as I record this.
Descript has had Dall?
E built into it for a long time, for well over a year at this point, and in January of this year. So just a couple months ago, they upgraded the backend from Dall? E2 to Dall E3, which was a significant upgrade. It went from an almost unusable feature.
To a really pretty good feature.
And Dall? E, if you don't know, is a text image feature from OpenAI, the same company that created Sora and ChatGPT and happens to be an investor in.
That's right, descript. So to use Dall?
E in your project, inside of Descript, you can open your media bin, you can come over here to Images, this images link, and then right here you got photos, illustrations, unsplash, and text to Image. Text to Image will be your Dall? E. And you can prompt it again in plain English, whatever you want it to. So let me type something in here. Um, let's do a sunset over the Pacific Ocean. I don't know, something random. And let me spell Pacific correctly and hit Enter. And it needs a moment to think about it and check it out.
That's pretty cool.
Pretty, pretty stunning images. And so I could use any of these in my videos just by clicking on it. I could regenerate it by clicking on that, generate variations, and I could, I could re prompt it. I could say make it photorealistic because it made these look kind of like paintings or something.
But these are pretty cool.
I like these a lot. And so this could be the background of a video. And so that's Dall? E. That's the, that's the technology we currently have for creating images. So it's not a big leap to predict that here in the videos section, there might be another tab right here.
That says text to video.
And you'll be able to prompt it in plain English to generate original B.
Roll for your videos.
But when will we have this again? I'm completely speculating here. And Sora isn't even available to the.
Public as I record this.
And I don't know when it will be, but my uninformed guess is that Sora will be available in the next three months and descript will have it integrated, at least in a beta form of some type, within three months. Of that within three months of Sora being released to the public. But that's only one very narrow use case, which is using it for B roll. Because I want you to try to imagine where else this could be headed. Remember, this is version one. This is like the beta version of Sora. This is the worst that this technology will ever be. It's entirely possible that the way we have AI speaker right now to recreate your voice, there might be an AI.
Avatar feature where you give Descript a photo or a short video of you.
And it allows you to generate an entire video of you from a text prompt. You could say, I don't know, create a video of me lecturing a stadium full or a college room full of people about, I don't know, the Roman Empire, some history lesson, and it'll generate you at a podium, in a stadium, in a room, in a church, cathedral, whatever you want it to be. Anything of your choice, speaking in front of a room full of people. Your imagination is the only limit with.
That type of capability.
And the next idea I have with where this might be heading is adding things to your video. So right now, if I mention a ticking clock, I can add a little GIF right here and a sound effect, but with AI, maybe I could add a realistic one that blends in with the lighting of this room and it could be sitting on that stack of books right there. And if we take this to its ultimate conclusion, its logical conclusion, progression, there may be a future where you sit down in front of your TV or put on your VR headset your Apple Vision Pro, right, whatever it might be in a few years, and you prompt Netflix or Sora or the service of the day. Create me a brand new movie similar to the other movies I've enjoyed in the past, starring my favorite actor or the actress I find most attractive. And I only have an hour, so make it an hour long and it'll generate a custom movie for you on the fly, an original movie that no one else has ever seen or even thought of before. So these are exciting times. No one knows how these technologies will look or even exactly how they'll come about, but I still think it's well worth learning. Description Because I think it's well positioned to stay on the bleeding edge of these AI tools.
