Open AI SUED, Midjourney DRAMA, Stunning New TEXT TO Video and More (AI NEWS #27)

So this was actually quite an interesting week in AI. So let's not waste any time to figure out some of the key news stories you might have missed in artificial intelligence. So one of the first things that actually was released was a text video model for zero shot video generation by Google. So essentially this means unlike other video models where you need to input an image to then get out a cool video, this was something that they finally released. Now I, I think this one was something that did go a little bit under the radar. I'm not sure if I've released a video on it just yet, but there was a deep dive and if it is released, I'll leave a link to that in the description below. But essentially the reason this was so cool is because it had many new features that other video models haven't simply done yet. So one of the cool features about this was that it had video to audio where essentially you have the video being generated by the model. And then of course it not only does that, it actually manages to get the correct audio for that as well. So for example, in this clip right here, you can see it's a cat playing a piano. Then if I were to unmute this, you can hear that of course there is a piano sound playing as the cat, you know, hits the keystrokes. So there's multiple examples of this. This is just one of the things, video to audio. There was also this thing by Google Research where they did like a little story that, that was based on this raccoon, which is kind of cute, and I will show you guys that in a moment. But essentially this was something that many people didn't realize because everyone right now is focused on Gemini. So when Google releases other things, it's not going to be that big as a focus. But one thing that I would say is the main takeaway about Google's Video Poet is the fact that the accuracy of this model is really, really surprising. Like sometimes we try to do other things in other models it's not that accurate. But in these two examples here, we can see that, you know, for example, two raccoons on motorbikes, A meteor shower falls behind the raccoons and the meteor explodes right behind them. You can see that that looks pretty accurate. And then this one also, lightning flashes in the background. Purple smoke emits figure of water. That is also really accurate. The lightning flashes and the purple smoke does come out after. Now there are also some interesting other things where we had longer video generation that was really good. Now what's also interesting was that Literally just after this was released, Pika released something where they also do have this as well, but that doesn't really matter. I'll be showing you guys Pika in a second. But they also have controllable video editing, where essentially you can follow different motions and different dance styles. Also interactive video editing, where essentially extending input videos just from a list of examples. And you can change that via text conditioning. And you can see that the text conditioning makes the videos so much better. And of course, we do have a really decent image to video generation, where I. I even tried this image in other AI video generators and it actually didn't go as well as this one. So clearly Google might have something secret here. Like, I'm not sure what framework they're using. I have looked at the previous Transformers that they did use for this, but I still think that even though they've shown what they're using, they might have some secret sauce in this. And then of course we have zero sort of stylization, which is like runways one style to another style. But I would say that this one is a little bit better. You're going to see that later on in the video. There is also another text to video model that that was released. And it's really surprising because when I look to the community, the community was insanely small on Twitter. Like the post about the stuff only had around 10 to 20 likes and the the tool is incredible. So I can't wait to show you guys that in a moment. But you can see applying a visual style and effects here on this part is really, really nice. So overall, there's going to be many different things maybe in the future. I'm not sure if other companies are going to be using this or if Google is ever going to release this, but it was something that Google did release that I do want you to know about. Next. We had Pika Labs actually finally release their model to pretty much everyone. So if you want to sign up for Pika Labs, you can, and you're going to get immediate access. And what's good about Pika Labs is unlike Midjourney, that uses Discord. And this is no hate to Midjourney whatsoever. I just feel like it's really cool that they do have a web app you can see here that you can also explore everyone else's generations. So you can see just how good PCollab's quality is. I mean, comparing this to some of the other stuff, and it definitely does just take the cake. And one of the things I really like about nation styles, like this raccoon. So I think those two ones, I'm not sure what Mid Journey did, maybe they just kept putting Disney movies into it, but those ones do look really, really effective. And like I said, what you can do here as well is you can add something to this. So you can click edit. And then I think it's here. Yeah, you can click add 4 seconds. So if you want, if you've got a clip, you can just keep adding four seconds to it. And this is where I did try the same generation from Google and Pika Labs, but it didn't work. So it goes to show that Pika Labs and Google are using different techniques to generate their videos, from text to video. So this is Domo. So it's video to video. And I think the quality of this is crazy because although we did see this with stable diffusion and with Runway, whatever they're using is absolutely incredible because I've seen some examples on midjourney and I played around with this myself and. And it looks absolutely outstanding. So it's called Domo AI. And I, like I said, I don't know what architecture they're using. Maybe they're using stable diffusion, I'm not entirely sure. But it looks so consistent and some of the styles that they've chosen to use are very, very effective. So I think this AI tool, like I said when I was looking at it on Twitter, you can see here that it's only got 50 retweets and 100 likes, but it's really, really effective at doing the stuff. Now, I'm going to show you guys a quick Twitter thread that showcases some of the cool stuff. And here's this quick Twitter thread that shows you exactly. This is. So we have the source video. And you can see here that this one is. It's just crazy. Like, it's very surprising in terms of how good it is. But you can see that the consistency is completely there. And I'm not sure if this is going to be used to make animated movies. I'm sure that there are different ways that yours make this stuff. But like I said, I think that this kind of technology goes to show us how crazy this technology is, because usually when you do see stuff like this, you usually. You usually do see stuff that usually looks completely AI generated. But if I did see this, I would initially think that someone did animate this, could say that much coverage out there. But it is something that is really good. So I'm not sure if other companies are going to get in on this as well, but you can see that the consistency of this is pretty incredible. And then this is one more example that I did want to show you guys that was on Twitter. And you can see, like, if I saw this, I would definitely not say that that's AI generated. This is completely different. So we'll be interesting to see how this progresses in the near future. Then we had something that was rather frustrating. Slash. I don't even know what to say. The. The real take on this is. But Mid Journey has been involved in a little bit of drama, and it's not the good kind of drama because everybody knows that Mid Journey is pretty much the top tier when it comes to AI image generation. But essentially Mid Journey has been in some Twitter drama recently for copyright infringement, you know, alleged copyright infringement. Because essentially what people are claiming is that Mid Journey has essentially, you know, ripped off these, you know, movies. And essentially how people are trying to prove that is they're prompting it and they're saying, get a screenshot from a movie. So the prompt that someone put into the model was thanos Infinity War 2018. Screenshot from a movie. Movie scene 4K. And then Midjourney output this image on the right hand side, which is eerily similar to the image on the left hand side. And these images from the left hand side are images that are directly from a movie. So people are saying that, you know, this, what we have right here is essentially something that's stealing stuff from these movies. Now, Midjourney hasn't been sued by any of these companies, but I do think it is interesting because. Because we have a situation where Midjourney hasn't really responded in the best ways. And the problem here is that apparently they banned this guy just for making this Twitter thread. So I don't think that's a good look on Midjourney. And Gary Marcus here, you know, a leading expert on AI that actually spoke at the US Senate AI hearing, actually basically was talking this and was saying, dear Mid Journey, reinstate this guy or drop your new terms of service. But people have spoken and it's not even close. So basically they're saying that they should reinstate this guy. And I'm not sure if they banned him or if they just were working on the systems. Either way, it would be nice to get some kind of statement from Midjourney. But I do know that Midjourney is in a pretty precarious situation because if they explicitly state that, look, this is kind of a mistake, then they open themselves up to some kind of litigation because if they admit that this is from Infinity War, and they admit that they're actually using other things to train their models, then like I said, they're admitting potentially to copyright there. So it is a tricky situation. But I do hope that this guy's account does get reinstated. But the only thing that I can say is that it is Midjourney's versions. Also, don't think that this is a major surprise. I'm pretty sure that everybody knows that all these AI companies just steal the data like you're about to see later on in the video. So I'm not sure why this is such a big issue. 2 seconds of video and it looks pretty good. I'm not gonna lie, it looks pretty good. It might not be better than PCoLLabs or whatever, but like I said, text to video is improving. And so far we're getting the increased high quality, we're getting increased frames per second, increased stability, consistency, and this is the exact direction that I do want to see us going in. So I won't be surprised if eventually Stability does get another model. But of course, right now, two seconds isn't great, but it's better than nothing. So Stability did release this, and you can of course use this if you want, but it's another development that you probably did miss. Well, they also did release, and this was something that I was supposed to talk about as well, was of course stable012,3 quality 3D object generation from single images. And essentially you just use Images to generate 3D models. And I've seen this technology get better and better and better, and I honestly can't imagine where we're going to get just three years from now just based on an image. Like, will people even need to model images anymore? So it will be interesting to see where this does go. But like I said, I think Stability is pushing out all of this stuff because they recently did actually have some problems in terms of their finances and they were actually, you know, trying to. I think they were trying to raise more money and they were essentially in a pretty precarious situation because they're not making as much money as these other AI companies. So I'm sure that if they manage to put out more stuff, they're going to be able to eventually survive. But of course, as you do know, AI is a growing field and I'm pretty sure that the demand is there and they're going to be able to work around this stuff. Then, of course, we had ChatGPT's Project Sunshine. Now, if you haven't been paying attention, I made A video on this around three or four days ago, and it basically does talk about GPT's advanced features. So this is a kind of big story that most people did miss. And if you think that this is some crazy leak, you can actually go on OpenAI's official webpage and then you can check the code and see that it actually does exist there on the JavaScript or whatever, the open source code, you can go ahead and check it's not open source. You can just go ahead and check the inspect element. Like when you click inspect element, you can click Control F. Then you can actually find these things on the web page. So essentially these are going to be the features that potentially OpenAI might have. This is what they currently have on the web page, but of course it's not confirmed because it's not released yet. So they could be just, you know, alpha testing it with a set amount of uses. They could be alpha testing it on their end. But it is something that we do know is there and is probably likely going to be the future of GPTs. So it says here that your GPTs can now learn from your chats. And I'm going to include a clip from the other video because there's no point of me going over this again, but I'm going to include one or two minutes from that video where I just pretty much explain exactly what's going on here. But the look, but testing it with a set amount of uses, they could be alpha testing it on their end, but it is something that we do know is there and is probably likely going to be the future of GPTs. So it says here that your GPTs can now learn from your chats. And I'm going to include a clip from the other video because there's no point of me going over this again, but I'm gonna include one or two minutes from that video where I just pretty much explain exactly what's going on here. But the look, but the TLDR is basically that GPTs are going to be really incredible. I'll leave a link to the full video because it's a full deep dive. Like there's a lot that you might miss, but it is something that you do want to see. We also did have this tweet from someone that works at OpenAI, Stephen Heidel, who fine tunes LLMs and essentially he talked about, brace yourselves, AGI is coming. And this was in a response to OpenAI's preparedness framework. And once again, I did a deep dive on this and this is really, really Important, because there are these measures that essentially show us what kind of models we're going to be getting. And with this kind of framework that they have, we basically know that really dangerous models are never going to be deployed. So that's not something we should worry about. And kind of dangerous models may be developed by OpenAI, but those ones are never going to be deployed. So if a model is Even remotely dangerous, OpenAI is never going to deploy that to the public. But of course, as you know, there are ways for people to have prompt injection attacks and for ways for people to essentially jailbreak these Systems. But to OpenAI's knowledge, they're never going to release a model that is higher than stage. And essentially, you can see here on this scale, what they had was they had something from low to critical. And if there is any model that is high or critical, they will never release that. And if a model is low to medium risk in terms of the capabilities, then they will release that system. So it goes to show that the future models aren't just going to be some AI death murder bots, which is of course, something that we didn't think would happen. But at the same time, it goes to show that they're taking AI safety really, really seriously, even though a lot of people don't think that they are. But I do think I'm really glad that they did release this, because although OpenAI was actually criticized for steaming, like, going full steam ahead with all of their projects, it's good to see that they still maintaining a very good stance on AI safety, despite the previous concerns. Something that was rather, I guess you could say, frustrating. Interesting, fascinating, slash, innovative was this. And people did actually talk about this romantic ar, romantic companion. But the problem was that they've been AR romantic companions for quite some time. And it just really did stir up a huge debate around how the future is going to be regarding these kind of models. Now, the app itself isn't that good. You can go ahead and check the reviews. And this isn't me dogging on the app. It's not paid. It's not like hate or anything like that. It's just essentially all one star reviews. But the point is that I don't think we're going to have a future where people are just going to be obsessed with AI models. But at the end of the day, we do know that large language models are increasing their capabilities in terms of persuasion, in terms of how they sound, in terms of their realness. And I do think that with an ostracized part of the population, there's definitely going to be some of them who are going to use these services whether or not we like it. So we'll be interesting to see how this kind of does play out. But it is one of those really controversial topics that does have a lot of information on both sides that's really. Now we had something that was really, really cool here. We had something from Sudo AI So essentially they released this, you know, text to music generator. And I'm going to show you guys the clips from that trailer because it's really, really cool. So take a head, take a look at that. So I think that is really fascinating stuff because I mean, music generation is something that is kind of weird in the sense that, you know, it's. It's kind of something that you do really like to be done by one person. I mean, the whole point of music is that you're listening to someone. So I'm not sure if people are going to take this on board and are going to start listening to AI generated songs. But what I do think that this video did showcase was the fact that we might be in a future where we have AI generated entertainment because the guy's walking home, these girls are sitting in a car and rather, and I know this is just of course a demonstration, but rather than, you know, picking their favorite song, they just simply put in a text prompt and listen to it. Now, of course, you know, in the future, I think that an AI system that has all of your, you know, favorite music tracks, all of your interests, all of your hobbies, your, and knows absolutely everything about you could possibly generate some of the best songs and you may never listen to another artist again. But at the end of the day, it's something that is fascinating because it brings up the discussion of how much AI will actually impact us and these other industries. So I do think this tool will just help musicians make stuff faster rather than replace them. But nonetheless, it is something that you should probably play around with. And I think it does solve the copyright issue where some people, some creators, even people such as myself, do struggle to find soundtracks that are copyright free. You could go ahead and generate one with AI and then of course just use that. So like I said, this is going to be something that's pretty fascinating and I'd love to know your opinions on this specific one. Then of course, we had Leonardo AI Motion. So essentially what this is is it's basically just image to video. And essentially you can just use your image and you put that into the system and then it turns it into a video. So you can see here it's image to motion, you can see the motion strength. And then you generate that and then in a couple of seconds, you know, using your credits or whatever, you then get output a video. And I think it's really decent based on the first, you know, because this is their first video model that we're seeing. And I'm going to show you guys another thread as well. But it goes to show that competition is good. And Pika Labs does have some very good competition in terms of the quality of this model. And there's also a thread here and essentially this guy was doing a sketch to video project, as you can see here, for example, like this donut that looks pretty good. Then he has this robot that was upscaled by Magnific. And if you haven't seen that one, it's a really insane AI tool. You can see this one right here, you can see that like that how it is. And I wonder the day that we're going to get, you know, really high quality videos that are high quality enough like on the same level of mid journey, that would be absolutely incredible. Also got like an apple here, you know, the first one was really, really cool. And of course we've also got this like Viking dude in a forest and then we've got this drawing of a car. So it's actually a car, but you know, if you look at the sketch that's drawing what it's taking the sketch as and then it's made that into that. So I think this is really interesting because it actually does help the creative process out so much. And I mean like, imagine you're like a kid or something, you're like 4 or 5 years old and you want to make some cool stuff. And your drawings, you know, your drawings are like that. You could essentially make videos like this. I know that's not really like a crazy application, but it's still something that you know, you can creatively enjoy, something that you can probably use. And of course we had OpenAI absolutely getting sued by the New York Times. And essentially why they're getting sued is because apparently OpenAI trained on proprietary data from the New York Times. And essentially when you prompted certain things on GPT4, it just output the exact same text from the New York Times. So it's really interesting because like I said before, we had midjourney facing this issue. And then you can see here that the output from GPT4 is the exact same text from New York Times. So it's Fascinating, because this is something that we didn't see before. And the copyright law is so weird because the problem is these AI companies didn't ask for permission. They've already trained on the data. So it's like we're like, you know, going back in time to kind of sue these guys because it's like suing them for something that they already did steal. I mean, it's kind of like a weird thing. I'm not even sure if they're going to win this case. I mean, if they do, what does that mean for OpenAI? Does it mean that any company ever that's trained on data that it didn't ask for, if someone does a prompt and then they find the exact test, are they going to be able to then sue them as well? Because that would open up the floodgates and then would that slow down AI LLM production? And then with that, increased synthetic data production? I mean, all of these things are things that you definitely do have to think about because, I mean, it could change the landscape. Who knows? So the ruling of this case will be one to definitely watch because the problem is, is that with generative AI and with all these new technologies is that there are problems that you haven't seen before, which means that there are laws that don't take into account certain things, which means these cases are going to be fundamental to shaping the future of how future cases are tried and, you know, ruled. So this is why I'm really going to be paying attention to this. And essentially the New York Times are suing them for billions of dollars and saying that, you know, you've cost us billions because people are using your software rather than us. So I mean, will be interesting to see how this is ruled because this will set the future for future. So then we had a viral video, okay, where someone said that Ayo Rob Chat GPT just recognized himself in the mirror. So essentially what this person did was they put ChatGPT into this small robot that was based on the Raspberry PI platform. And that's just a platform that people can make really cool stuff with. And essentially just watch the video first and then I'm going to tell you guys why this thing there. But there is some kind of music and I'm not sure if I'm going to be able to play it. So. But the point is, okay, is that in this video they say, hey Rob, let's test your vision. And then the robot essentially says, wow, this is crazy. I can't believe I'm seeing myself for the first time, this is like a sense of not consciousness, but this is like a sense of realism, yada, yada, yada. And at first people were freaking out about this clip because they were like, yo, this, this robot recognized itself. And there were a bunch of comments on TikTok, on YouTube, on Twitter, saying that, well, this is the moment that people have finally been waiting for where, you know, these systems that we don't think are conscious are actually really conscious. But however, there is a caveat to this response where you do have to kind of be honest about how they actually achieved this result. So you can see right here that this guy actually said yes for this test. I overly described the mirror thing in the prompt. So when he wasn't in front of a mirror, he mentioned that he didn't see anything that resembled himself and then proceeded to describe what else he saw. Either I simplify the prompt not mentioning anything about a mirror or I'll add in statement to tell it, if he doesn't see himself, he describes what he sees. And basically this guy actually did unfortunately prompt the large language model inside ChatGPT to basically tell him what was going on. So it wasn't just him turning on this robot and turning on Vision and then the robot realizing that, oh my God, I'm awake. This is crazy, because if it was, I'm pretty sure it would have been a more viral moment. But that's why this clip did go viral. So that is why a lot of people were like, okay, this is absolutely insane, but this is the honesty. But of course there will be more tests and there will be, you know, further times where this is something that does happen. Because this isn't going to be the first time that LLMs are going to be a robots combined with vision, because adoc, the guy that is the founder of Figure Robot, which is a robotics company, so this company right here, which is figure AI, essentially he talks about 2024 being the embodiment of AI and robotics. And it definitely is because there are so many companies, I've seen so many different press releases and things that don't even have that much attention. And I will do a full video on robotics because there's so much news that people are just missing out on. But basically he said that robotics is going to be that crazy, crazy next step. And that's exactly what his company is focusing on. And he wasn't the only one that said this. Dr. Jim Fan, the lead of AI agents at Nvidia and this seeder and the senior Nvidia researcher basically said that 2024 is going to be absolutely incredible. So 2024 is going to be the next step in robotics and it's going to be the thing other than LLMs that takes away the game. He said we are three years away from the ChatGPT moment for physical AI agents. We've been cursed by the motherex paradox for too long, which is the counter intuitive phenomenon that tasks human find easy are extremely hard for AI and tasks that humans find hard are extremely easy for AI. So he basically talks about the future foundation models and platforms for robots. So the multimodal LLMs with robot arms and physical I O devices such as Vimar, RT2, RT1, Palm E, Google Robocat, DeepMind and then there's, and then there's so many different algorithms that bridge the gap and of course they always talk about the data and of course Nvidia's Isaac Sim can actually simulate reality at a thousand times faster. So he's basically saying that 2024 is going to be incredible for robotics. And I honestly can't wait. The last thing we have here is that basically Sam Altman's working with someone who used to work at iPhone to build one of the first AI devices. And of course there already is the AI device, the Humane Labs device. But this is going to be something next level. So it will be interesting to see how this platform does develop because app and although it isn't actually Apple, it's iPhone's design chief. But it will be interesting to see what kind of AI project they're going to be developing, what kind of, you know, I guess you could say physical products is going to be, if it actually does go to mass market, if people are, you know, going to be using this, if people are rejecting it. Because making something that everybody uses like a mobile phone or like a bracelet or something that really, really takes off is really, really hard to do because people are so ingrained with their current devices. So if there is any wearable tech that does take off, it will start an entire and it will be interesting to see if AI actually is able to do that.