Luma Labs Stunning "DREAM MACHINE" Is Bigger Than You Think!

So this past week there was something fascinating released. Finally, a Sora based competitor that we could actually use. This video is going to give you guys an in depth guide on this actual software and a variety of different comparisons because you'll be quite surprised at what the competition is up to. So essentially, what do we have here, ladies and gentlemen? Introducing Dream Machine. So Dream Machine is an AI model that makes high quality, realistic videos for fast from text and images. It is highly scalable and efficient transformer model that trained directly on videos, making it capable of generating physically accurate, consistent and eventful shots. Dream Machine is Luma's first step towards building a universal imagination engine. And luckily for us, it's actually available to everyone now. So I know that this video has been a little late, but I still wanted to give my opinions on them because I do believe that this announcement is very worthwhile. This is by far one of the most impressive text to video AI softwares that we can actively get our hands on compared to a lot of the other AI systems that we've currently seen. In the recent weeks we saw around three to four different demos, most of those being from China. The only problem was that we could never use them. Now this one right here, Luma Labs. Their new Dream Machine system is something that is very, very effective because it has a multitude of effects that are really, really impressive. One of the first things that it's really, really consistent in certain areas. And in this video I'll talk about the actual strengths of this and of course the limitations because, because some of the experiments that I've seen people do over the weekend this week have been truly, truly fascinating because it shows us that we are truly about to enter a new era where there's going to be a lot more creative people who are allowed to express their creativity through these tools. And I do believe that this is a very useful tool. Whilst yes, of course there is the risk there, I think this is something that is very, very useful for those who are creatively inclined. So one of the first things I wanted to actually show you guys is the impressive consistency. One of the things that can happen with Luma Labs text to video is that the temporal consistency in the videos shows a very, very good understanding of the movement dynamics that occur when you're actually using video. So this right here is one of the best examples that we can see this because it shows that, you know, someone going into this environment, not only does the camera angle look relatively, you know, stable and relatively well as it bounces, you can also see interestingly Enough, and this is something that most people didn't really pay attention to, is that the light kind of goes on as you are going inside. Now right here, it does get a little bit darker, but along the edge right here, what you can see is you can see a little bit that outline of the torch. And I think those subtle things that, you know, this can pick up on. It's really important to kind of, you know, pay attention to that because this is something that it kind of slips by a lot of people's, I guess you could say attention, like most people wouldn't realize that, but you can see as you walk more inside, you can see that the torches are more pronounced and the shadows continue to conform. So this is clearly just a model that is rather effective at what's going on. We know now that when we look at these demos by Lumalabs that in comparison to other AI systems, and I will get into that moment in a moment, but in comparison with others, I think what Luma Labs actually showcases is what we actually have here is an AI system that actually genuinely understands what's going on. In the image, for example, right here, you can see that this is a man walking. So we can see that the man is, you know, walking up and down, which shows us that this actually means that, you know, there's an understanding of exactly what's going on here. And I think this difference is important because certain other, you know, video, you know, makers, and I'm going to show you guys that too. The difference is, is that what we have here is we have, you know, something that understands how to move this, you know, how to move this character effectively, rather than just doing a still that kind of rotates around the character. We also have this one right here. We can see literally a car. And then of course, it does a 360 around the car, which shows us that it kind of understands the motion of this and it understands exactly what is going on there. So it's truly, truly effective to make sure that we can understand exactly, you know, how the motion is in these clips and of course, get a visual look at the understanding between these. Because if we look at prior models and how they would have, you know, done this clip, it would have been like this slow moving thing. But here there's this dynamic understanding which shows us. And of course, this isn't crazy by any means. Some people have clearly stated that, you know, maybe they just need to throw more compute at this. But I do think that this one is really effective. Now there are also some you know, you know, longer versions here. I do think that this was made firstly with Mid Journey, with the character consistency there or another AI system, because you can see that of course, whilst this does look pretty good, the character consistency between this little blue creature and then of course, this little blue creature right here, there is a little bit of difference. But overall what we can see here is a situation on our hands where we have a tool that is a lot more effective than the previous ones, because Luma Labs AI is a tool that truly understands what's going on in the clip. And then of course, maybe it was just training on, I don't know how it did that. Maybe it was trained on more data. Maybe they took a few things from Video Poet, you know, Google's new VO software. They looked at all of those things there, incorporated those into this and then they've actually finally released it. Now I think this is absolutely incredible because this is the first kind of thing that we got that is actually, you know, finally released. And it is something that showcases what is done when, you know, these other larger companies are put under the spotlight because they haven't released these tools yet. And, and of course Luma Labs have managed to get to market first, even if their tool isn't as good as some of these other ones. Now, I'm not going to say that everything about Luma Labs is best because of course there are some limitations. We can see here that there is of course the morphing, which is, which is what happens when you have a model and then as you're rotating, essentially what can happen is the translation. Not the translation, but like how the model changes through that. So sometimes it gets lost as you kind of walk around. So there is that problem and that, that does kind of occur. Of course, we do have here as well. Sometimes in some of the movement clips, what we will have is a scenario where the movement just won't happen. Like the legs won't touch the ground, they won't move, but the background will as well. So this is something that, you know, does tend to happen and we do see this from time to time in many different AI softwares. Movement is of course something that is pretty hard to solve. Then of course we have text. Text is one that is notoriously hard for generative AI to solve. Not sure what methods OpenAI are using. They pretty much have solved it at this point. But it is something that, because of how generative AI is, it doesn't really understand text as well. So after coherency, I do think that text will likely be solved last. That's kind of what we saw in things like mid journey. Usually they focus on making the quality great first and then of course you think about the text lastly. So that is something that I think will get solved, but it's not something that most people will care about. Like if you had a really nice AI text generator, but the text wasn't good, but the video was good, I'm sure that you would like that. And of course this is where we have Janus, which is where we have a reference to two faced things. And this is the kind of issue that you get with morphing as well, where you have creatures that can have all sorts of, sorts of different weird faces and they can immediately switch. So we got the polar bear going to one way like this and then we got the polar bear, you know, switching just like that, which is pretty strange. But it is pretty, pretty cool. There are so many different things. Now before I go on to some of the current comparisons with some of these state of the art models, I want to show you guys the pricing because I think this probably shows why this is something that still hasn't been released yet. Because for free we get 30 generations per month, which is pretty nice. But you can see for Standard we get 120, which is four times here, and then for Pro we get 400 and then for Premiere we get, you know, 500amonth. Now I think at this level this is something that is pretty expensive and I'm going to tell you guys why. That doesn't mean you shouldn't buy it, that doesn't mean you shouldn't try. I'm just saying that this is going to show you guys where we're at in the industry. Basically what we have is a situation on our hands where we have the pricing for this being much more expensive than it is to effectively, I guess you could say not go out and shoot a clip, but maybe you could spend your money more effectively. Because what I've heard is that with 30 generations per month, if you do a lot of generations, sometimes the generations don't come out well, so you have to regenerate them. So let's say you're going through five to six generations to get maybe one usable clip. And, and if we take a look at the standard generations and if we can say 120 divided by five or six, that's about 20 clips for around $30. That's not that bad. But considering the fact that you need character, consistency and all of that good stuff, some people are stating that, you know, they need immediately like 400 generations if they want a certain product around 2000 generations per month. There was a really in depth tweet that broke it down with that to why this still needs to be pretty cheaper before we do get like mass scale know, video stuff. But the point here is that, you know, filmmaking is a very extensive process and of course I think this is going to be fun for the meantime for like indie projects and stuff like that. And I think with Sora, you know, if we take a look at Sora and those kind of things there, I think because you have to do like, you really do have to run multiple generations a lot of the times sometimes this can become tedious because it's a generative AI system. It might not immediately understand what you're doing. And, and of course with that you might be running through your generations per month, especially during, you know, certain periods. Like even right now, if you go onto Luma Labs website, it will say that things are pretty, pretty busy at the moment. So this is going to be something that I think is a fun tool right now, but doesn't seem to be a game changing film tool that many people will have, you know, used. Now one of the things that I wanted to show you guys was the comparisons. I'm going to leave a link to all of these tweets in the description below. But you can see here that the comparison that the comparisons between Luma Picker and Runway are pretty, pretty stark. We can see here that there's a deep understanding of Luma being able to, you know, pan the camera down and being able to move it nicely. P Collabs right here you can see it's basically just a still image with a few things moving around that it looks okay. Like I'm not going to say this looks completely awful. You can see Runway as well. There's no actual steps, no actual movement there. But of course with Luma Labs, you can see that the character on this walking shot right here. So you can see this walking shot right here. It understands that it's a legs and these legs need to be moved forward and backwards in order to make the character move forward. So it's truly, truly different here. I do think that like in the future, maybe in order to get consistency, you might have to run five generations for every one image. And of course that's going to take a lot longer, going to take a lot more compute, going to be an issue. But I think just the comparisons between, you know, Runway, Pika and Luma I think that we can see here that it's pretty surprising that Luma and Pika. No, not, not Luma and Pika, that Luma managed to take, you know, out Pika and Runway as the, you know, premier state of the art video model that we can access for free. So what this should show you is that AI, you know, is ramping up because a lot of other companies, like I said before, and now starting to realize that, wait, this is going to be a trillion, multi trillion dollar industry where billions and trillions of dollars are going to be flowing through. They also want their piece of the pie. And I think this is why, you know, like they said, maybe not exponential growth, but, you know, the dynamics in capitalism where you have companies that are always fighting to get to things first and to be the best for their customers. I think we're going to see a situation on our hands where we get a lot more tools than we even know what to do with and a lot quicker than we know what to do with. Now the. There was this awesome, you know, example that someone did on Twitter. I'll leave a link to this. But they basically showcased the direct examples between Sora and Luma AI. We can see here that Sora AI versus Luma AI. The main, main differentiating factor here that we can have is of course, the character quality. One of the issues that we have here that you can all see is that Luma AI just doesn't have extremely high resolution clips. I'm not sure what causes this, but I do know that with many videos, video generators, they do struggle to, you know, have clips that are up to 1080p. Of course, it's probably something to do with the architecture, but you can see that Sora's quality, like on direct one, on one comparison, is definitely a huge leap ahead. Okay, definitely like a huge leap ahead in terms of what, you know, kind of quality you'd expect. So with Sora, it's definitely pretty crazy, like looking back at these, because we look at these clips right here and of course this does look pretty cool. But we, when we look down below, we're like, whoa, okay, Sora, the kind of dynamics that we have there, you know, the clips of the cat, you know, in a bed with someone, I mean, it's pretty, pretty crazy how far ahead open AI are. And of course, it's not even their main thing. I do want to say that of course, you know, Luma Labs AI is a pretty going to. Going to be a pretty fun tool. The fact that it can do all of this stuff. But when we do look at Sora and how crazy this technology is, like, it's truly still even hard to to believe that this is AI generated. It's something that I think, you know. You know, maybe with another iteration cycle, we're going to see these other companies catch up. Because once again, now, whatever Picker Labs or Runway we're working on, they have to make sure that they now beat Luma Labs, because if they don't, then nobody's going to care. And that's just the half truth of this race. But it seems like OpenAI, for now, are still in the lead in terms of their video generation model. So this was an interesting update. I think this is something that's very effective. Let me know what you all thought about this, and I'll see you guys in the next one.