I Was Wrong About AI Video...

The world of AI generative video just keeps getting crazier and crazier with new announcements coming out all the time, really cool research, and more and more flexibility and controllability over the videos we create. It's crazy because some of the predictions that I made in this 2024 predictions video have already come true. And there's one prediction I made that I couldn't have been further off on. I don't think in 2024 I could be wrong, but I don't think in 2024 we're going to have that. I type in a prompt and it's going to spit out a 30 minute episode of south park for me. I don't think we're gonna see that in 2024, but I do think we're going to see some much higher quality AI video along with a lot longer generations in that higher quality. I posted that video on January 2, 2024, and on January 23, just three weeks later, I completely changed my stance and course, corrected and said I was wrong. In my 2024 predictions video, I made a comment that I don't think AI is going to be creating full length shows and and films this year. 30 to 60 minute stories that were coherent and enjoyable won't be available this year. I was wrong. The next two months will be wild. That was January 23rd. I'm recording this on February 28th, one month later. And here we are. People's minds are getting blown by AI video. When I made this tweet, I wasn't referring to any one specific tool. It was just a handful of things I had gotten sneak peeks of. And now most of them, except for I think one of them has been revealed. So there's still one tool yet to come in the AI video space that I've seen behind the scenes, but haven't been able to talk about yet and still can't. But I think everything else that we're seeing right now is crazy impressive. So this is going to be a pretty quick video. I just want to do a rundown of what we've seen in the AI video space and what we can now or almost do that we weren't able to do two months ago when I made my predictions video, starting with Sora. Sora is the obvious one, right? It came out recently and and just blew everybody's minds with the realism and the ability to do longer generation videos and how it seemed to properly follow the laws of physics, which a lot of the other AI video generators kind of didn't very well, also this week, Pika Labs rolled out a new feature for their AI video generation tool called Lip Sync, which allows you to actually make your AI generated videos talk. You can actually upload audio files and it will move the mouse to what the character is saying while still continuing to do an animation that you had it generate. And they look pretty good. And if you have a pro account at Pika, you can head over to Pika Art and when you go to generate a video, you'll see a new option that says lip sync on your video. I can take this video here that I made of Tom Hanks playing basketball, click edit on it, click the lip sync button here, and then I can either input text or even upload an audio file that I recorded. Let's make the voice, Dave, and say I love basketball. And click generate voice. I love basketball. Doesn't really sound like Tom Hanks, but let's roll with it and attach and continue. And we get a video that looks like this. I love basketball. Not the best lip sync I've ever seen, but we've come a long way and this is the worst it's ever going to get. But the same week we got this, we also got some research out of Alibaba Group called Emo Emote Portrait Alive Generating expressive portrait videos with audio to video diffusion model under weak conditions. This really blew my mind. So check this out. They gave this input image. They gave a song with this input image and this is what came out of it.

I never knew you were the someone waiting for me.

I don't want to play too much of these songs because I'm sure it's copyrighted music and this video will get demonetized, but it is really, really impressive. Here's another example of an input image that was given. A vocal sample from Mara Moratti here was also inputted. And this is the output result.

Yeah, I think this is right now an inflection point where we're sort of, you know, redefining how we interact with, with digital information.

And what's really impressive about this is, yes, the lip sync is really, really good, but the emotions on their face and the movement of their head as they're talking is the most realistic we've seen so far. Here's another example. Watch how much movement there is in the heads and the blinking and emotions really match her singing. So check this out.

Talk to myself for hours.

Say things.

You don't understand.

Again, don't want to play too long of a clip I'm already probably treading on thin ice with what I've shown already. This was the one that I think was the most impressive to me because as this image of a girl was input and she sings, the face is so expressive.

Notice how as she's singing these higher, longer notes, she's closing her eyes as she sings. This was all just AI generated, just an image and the song and it picked up on the emotions that should be conveyed while singing this song. And I'll link up to this page in the description because it is really cool and there are a lot of examples here. I'm just sort of avoiding playing too many of them because a lot of it I think is using copyrighted music. So if you're watching this video down the line and the videos I just played are muted for whatever reason, that's because YouTube decided I can't play those songs unfortunately. But again, I will link this up in the description so you can go check out these videos directly from the source. This again is called Emo. It is really impressive. Like this is one of the things that came out recently that's really blown me away. News came out this week also that Stability partnered with More Studios to make this AI filmmaking platform here. This one's interesting because More Studio has its own text to video model and Stability AI also has their own text to video model in stable video diffusion. But what really makes this one stand out is this sort of storyboard visual drag and drop builder that they have where you can have a whole bunch of video shots created and and then you kind of string them together and then export them into a longer video. You can see here they took a three second clip, a three second clip, strung them together into a six second clip. Morph calls the process of generating, editing and cross cutting a workflow. Users can share their unique production workflow in Morph's creator community where others can replicate and modify the templates simply by changing the AI prompts. So it seems like a very similar sort of thing to like ComfyUI looks quite a bit less complex than ComfyUI but a similar sort of visual workflow. We also got the announcement of LTX Studio this week. Now this is a tool made by Lightrix, the company Behind Motion Leap VideoLeap PhotoLeap, really really popular mobile video and image editing software. This is sort of an all in one video creation platform. Using AI, you give it a prompt like a futuristic space drama or NYC courtroom drama and it will create an entire storyboard of generated videos with for an entire film. And below each scene in the video you can see the prompt that was used to generate that scene. And you can get in and tweak every single shot, every single scene in the video, one by one if you'd like, or let it generate all the scenes for you. This is really capable of generating entire movies from a prompt. And it's not using stock footage. It's generating every scene with the AI. It's also capable of creating the music, the dialog, the sound effects. You can make any sort of editing edits you want, like can we make the car red? And then it makes the same shot as the original, but now the car is red. And when the storyboard's all complete, you can just drag and drop scenes around to put your video together. LTX Studio also prides itself on character consistency. This is one of the big differentiators and selling points of this platform, is that once you get a character dialed in, you obviously want it to look like the same character in all the future scenes. Well, this platform's designed to do that as well. You can customize the lighting and pretty much do everything from A to Z. When you have an idea for a video, you rearrange it, you dial it in the way you want. You change characters, you change lighting, you change scenes. You can make a whole movie with this single platform. I'm not going to go too much deeper into this one because LTX Studio is a company that I am going to be working with in the future. So you will see me talk about it in more videos. They actually didn't pay me to sponsor this video or anything like that, but they did pay to sponsor future videos. So you will see a lot more about LTX Studio on future videos. There's also a group of AI animators that put together a Terminator 2 remake using entirely AI generated video clips. You can see some shots here on their Twitter of what this is going to look like. And you can see it's all AI generated scenes trying to remake Terminator 2. And they're going all out with this one. They're actually premiering it in Hollywood on March 6, doing a live stream with the cast and crew on March 9th. And they also put together some behind the scenes content. So if you want to know how it's made, they're going to be showing you how it was made as well. Pretty cool to see a full length movie come together. Now, this wasn't a quickly thrown together AI generated video. You can tell they spent a lot of time getting the exact right scenes for it and dialing it in the way you want. This doesn't look like it was an easy AI video to make, but it should be interesting to see how it's received. After seeing what tools like Sora and that emo research and what LTX Studio are capable of, this style of AI animation is almost already starting to feel a little bit archaic, if I'm honest. But as we always say, this is as bad as it's ever going to get. And somebody's got to make this first AI generated movie or remake, and this is the first one. But I only imagine future iterations of movies like this are going to be better and better and better with the technology that we have rolling out. And one last cool thing that I wanted to show off before I wrap this one up is that this company, Anamorph, created this generative technology that can reorder scenes to create unlimited versions of one film. The guys over at Anamorph put together a movie all about the visual artist Brian eno. They had 500 hours of restored archival footage and interviews, as well as animated visuals and music. So Animorph is able to take all of that footage and generate a new version of the movie. Every single time somebody watches that movie. It says Animorph system is able to generate billions of potential sequences, resulting in a unique viewing experience for each audience. You can still have an engaging narrative arc in a film, sort of what we expect when we see a normal documentary. Even if the scenes, footage, music, and the sequences change, we can still get an engaging, cohesive story. It also helps that the first and last scenes of Eno are always the same. Plus there are certain scenes pinned to the same time slot in each version, including the scene where Eno discusses generative art. So with their video, they made an intro, an ending, and then had certain scenes that they wanted to make sure absolutely showed up in every variation of the film. And then the whole rest of the film was just pulling from the footage that they had available and generating a new film for every single audience that watched it. Every audience got a completely different experience. I'm sure at some point this will be put online so others can watch it. And every person viewing it, every new experience is going to be a slightly different experience than anybody else who watched that same film. Got pretty interesting, pretty different take on film and what generative AI can do for the film industry. Who knows, maybe in the future we're going to have this movie going experience where we go to see a new movie and everybody sees a slightly different variation of the movie that could also really up the rewatch value because if you watch a movie and when you watch it again the second time, a movie could be completely different than the first time. Well, you have incentive to watch that movie again. That'd be really cool on YouTube videos actually. Anyway, I feel like this is exciting, innovative, sort of novel use of this technology and I'm excited to see how it plays out. All that to say I was totally wrong. We will see more and more crazy amazing videos coming out generated by AI in 2024, including longer and longer form videos that are actually videos people want to watch. I'm absolutely convinced of that now and based on some of the research and technology we're seeing roll out right now, maybe I convince some of you as well. I think it's going to be a really exciting year regardless and and I'm totally here for it. You want to follow along to what's going on in the world of AI? AI news, AI tech gadgets, AI and film chatbots. All the good stuff. Make sure you're subscribed to this channel and you like this video and I'll make sure more videos like this show up in your feed. Also, if you haven't Already, check out FutureTools IE. This is where I curate all of the coolest tools in the AI world as they come out, as well as keep you up to date on all of the latest AI news. There's a free newsletter here. Sign up for the free newsletter and I'll hook you up with the awesome database of interesting ways to make money with AI. And you'll also be subscribed to the Future Tools newsletter where I will keep you in the loop with just the most interesting tools, the most interesting news, just the cream of the crop, the best of the best for each week that you're going to want to know about. So check that all out at FutureTools IO and I really appreciate you tuning in and hanging out and nerding out about cool AI stuff with me. I really appreciate you. I'll see you in the next video. Bye Bye.