AI News WAVE Continues! AI Video, LLMs, & World Models!

Welcome back everybody to another AI news roundup. Plenty to talk about today. It looks like these last few weeks of December are going to be an insane time in the AI space. First of all, we have OpenAI doing their 12 days of ship miss. We saw them start out by releasing full 01 and chat GPT Pro and the day after that move on to fine tuning for devs. So as this week continues on, we'll likely see some pretty awesome releases for OpenAI. Not to mention everything I already have to talk about in today's video. So let's dive right into it. To kick things off, Llama 3.3 was released in a 70B model size open source as usual from Meta here, but this 70B model is a chart topper in its weight class. It delivers the performance apparently of their 405B model, but obviously this model is a lot cheaper to run, it's more accessible to those with consumer grade hardware, and it's just insane that they were able to extract so much and are just releasing the model open source. Props to Meta for that. I don't think they've missed once with the Llama series of models here. And this is just another great addition. You can see this thing is pretty much tied with GPT4 omni. MMLU we have an 86, GPT4O is an 87, so is Gemini Pro 1.5 like right in line here, state of the art with the rest of the LLMs with instruction following we have a 92 versus an 84.6 for GPT4 and even in human eval here we got an 88.4 versus GPT4's 86. So it's crazy. I mean GPT4 is a great model but llama 3.70B is catching right up basically on par and it's fully open source. Guess what's not GPT4O. The pricing is even more crazy since this is open source. The pricing for 1 million input tokens is $0.10, while the pricing for 1 million input tokens on GPT4 omni since it's API based through OpenAI is $2.50 and the output tokens are even crazier. For llama 3.370 B it is $0.40 per 1 million output tokens and $10 for GPT4O. So not only is it basically as good as GPT4 omni, but it undercuts the price by so much and also in comparison to the competitors by the way, it is by far the cheapest and same quality. Meta is kicking butt in doing good work for the open source world. You have to commend them for that as usual. Also going to be available either on Meta's website or we have the hugging face links. So I'm going to put everything down below for you folks who want to try to get Llama 3.370 B up and running if you have the kind of hardware and the means. Otherwise the API is extremely cheap. So next up guys, we're going to talk about our sponsor, but don't click off yet. This fits right into our news roundup. Today's video is sponsored by none other than Leonardo AI. Leonardo AI has a brand new feature called Flow State. It's a new feature designed to keep you in ideation mode, helping overcome creative roadblocks and exploring new areas of the AI model's latent space. I'm a big fan of this and it reminds me of the early days of Dall E2, before it was even public. Let me explain. Free users have a soft limit on the number of image generations that can be made with Flow State, but premium users have virtually unlimited generations. Anyways, here is how it works. I'm going to type in a prompt at the top. Anthropomorphic lemon character vibing on the beach wearing sunglasses. I'll click Generate and then an endless stream of images is going to pour out of Leonardo in all kinds of different styles with all kinds of different creative nuances. And the more I scroll, the more images get generated. They just keep coming, which is really cool. And you can already see a ton of unique ideas here. This one's got like a nice purple hue. This one is where it's raining. This seems to be origami or paper mache. You know what, I actually really like the rain. So I'm going to click the more like this button. And as you can see, it has these styles. Rainy and burge blue. When I click on it, it's going to add on the timeline up here another spot and start generating in that specific style we selected. So we've got the rain, we've got the blue, and I can refine from there. So I like this one for example, it's a lot cleaner. I'm actually going to go ahead and upscale this image because it's exactly what I was looking looking for. But if I want to go back to those original generations, all I have to do is go to the timeline and click on one and it will bring me back. And then I can go and explore other ideas as well, like black and White, for example. And as I said, the more I keep scrolling, the more images keep generating. Back in the very early days of image generation with Dall E2, for example, the beta was like this, where you could just keep scrolling and it would just keep generating. And it allows you to really see what the model is capable of, all the different kinds of generations that can come from the latent space. And it really helps with creativity and just finding really cool generations overall. Like this one. I never would have found this without flow state. What a crazy image. If you want to learn more about the brand new flow state, there is a link down in the description below. Huge thank you to Leonardo AI for sponsoring this video. And of course, thank you guys so much for watching my sponsored videos. Sponsors literally keep this channel alive. Let's get into the next piece of news. All right, let's move on to our third topic. Microsoft's Copilot AI now has live vision. Basically, you can show it your screen live and it can update and talk to you in real time, which is pretty cool. If you guys know Microsoft has very close ties and connections to OpenAI, so this makes sense. But still, in ChatGPT, we don't have advanced voice with vision. I hope we get it this week. Probably going to come as one of those OpenAI updates for the 12 days of ship. Ms. Copilot's got it first. As Mustafa CEO at Microsoft AI, points out, you can now show not tell AI what you want help with in real time as you scroll, shop or even guess locations in geoguessr, which is what they did for this demo. Hey, Anthony.

Hey.

Going hey.

I'm just playing this new game called GeoGuessr.

First of all, GeoGuessr is not a new game. Don't know why they said that.

What is GeoGuessr? I cannot believe you guys haven't heard of this.

Yeah, I'm with her.

Is this like a new game or.

No, no, no. Yeah. Okay, good. Oh, no.

Okay, let's just try it and we can use Copilot Vision. It'll teach you as we go.

Okay.

Hey, Anthony. I'm here.

Hey, copilot. I just need some help playing GeoGuessr.

Ooh, sounds fun.

Apparently I'm supposed to guess where this image is.

Okay, let's look closely at the architecture, vehicle size, and they're cheating. Let's focus on any languages, symbols, or distinctive features in the image.

What about the language here?

The big red characters are Chinese and the smaller letters in English give a clue.

Yeah, where is that? Where is Passe City?

It's located in the Philippines.

Oh, we're going to try to guess the Philippines.

Let's see.

Yeah, let's try it out.

Great idea.

Do you know if Passe is in the north or south part of Philippines?

It's in the northern part of the Philippines.

Great.

That's good enough.

Let's do it.

Oh, nice teamwork.

Thanks Copilot. I think your strategies helped out a lot.

You're welcome.

Use the AI to cheat @GeoGuessr immediately. No, I'm with that. I think that's a pretty decent ad. I'm glad that they had that lady there to to clear the water. Geoguessr is not a new game. It's been around. This is pretty funny though. This sort of feels like one of those ads for like a cheap free to play mobile game loaded with transactions. You know the kind of ads I'm talking about. You probably get them all the time anyways. Yeah, that's actually a pretty darn cool feature. Copilot can now see your screen in real time. Don't know why the Chat GPT app doesn't have this yet. Like I said, I hope it's coming this upcoming week with the whole OpenAI thing. But Copilot's got it now if you guys want to learn a little bit more. There's actually an entire blog post about this. I can tell you just dealing with Windows in general from Microsoft. I've needed Copilot many a time. You long term Windows users know exactly what I'm talking about. Driver issues, registry issues, etc. Things like that can happen on a Windows machine so this is very necessary. Next up I have to show you guys this. This is so cool. Our demonstration here is brought to us by Dreaming Toloopa on Twitter. Highly recommend following this account by the way. This is really really awesome. This is essentially motion control, very fine tuned motion control for AI video generation and there is a lot you can do with it. Right now it GitHub project with no code released yet but it looks super promising. Hopefully this is fully open source when it is actually released. As you can see this is basically called motion prompting and there is many different ways you can set it up. In this example here we have a static image of a cloud and then you can actually dynamically drag your mouse around and move it and it will generate that video with the motion of your mouse flinging the smoke around in a realistic physics esque way. Really, really cool. This is stuff that you used to have to do, you know, complex simulations for, but now The AI can do it. We've also got, you know, a static image of some branches and we want them to sway like they're swaying in the wind. And if you just shake them a little bit, that's exactly what it will generate. Again, really, really cool stuff. It's like these dynamic interactive images. They're not exactly like a playing video, much more like a video game where it's interactive. But again, it's not a video game because it's just AI generating the images based on expected outcome. Super crazy. We've also got this sand demo with multiple patches that you can pick up and like move around and you can see the sand falls in like a very realistic way. Again, super interactive, really cool to see. We've got the pearl earring girl, whom in a recent video I called the Mona Lisa. My apologies, folks, I was very tired. Anyways, this pearl earring girl can be moved dynamically. You can position her head and you can actually see like her body slightly move as well. Her earrings react dynamically, the clothes react dynamically, and you can sort of pull and move her face. But it really does look like you're moving this woman around and you're actually dragging a real object. It's really quite crazy to look at. Again, we've got another example of photo like a selfie of a girl and you can drag the hair around in a realistic way. She even like blinks and moves her eyes and mouth slightly. And same thing for the, the pearl earring girl. We've got the same thing going on for an owl. You can, you know, drag his movements around to be exactly how you would expect them. Moving a big piece of neck fat on this cow down here and then finally moving the chest piece, which is really, really impressive. Again, how far away are we from dynamic video games that have extremely realistic graphics and physics? That's all being done with AI? That's what this mostly reminds me of. But it's also very good photograph and video generation control as well. You can also straight up control the camera as you can see from these examp. Pretty stellar as well. You know, increasing the FOV as we move in closer to the subject. Or you could combine the motion and camera movement together to do again, very, very specific, precise movements. This is the most precise video generation control I think I've ever seen come out of the AI world thus far. And yeah, this is. This is pretty magical. This feels like the next step in regards to controlling motion. And you can even do motion transfer so you can upload a video of Yourself moving your head, for example, and it will transpose that onto whatever we're looking at. Just like Runways Act 1. And again, guys, no code for this just yet, but we do have a nice paper, and there is way, way more information here on the full page, along with more examples. I mean, this is just absolutely crazy. Controlling the individual eyeball on the frog. So, yeah, this. This one's cool. Keep an eye out for this. I'll link the full project page down in the description. Next up, folks, we're talking about Runway act one, which just got an update before. You would upload a video of yourself acting out whatever you want to see happen in the performance. And then you'd upload an image, and it would transpose your acting onto that image. But now, instead of an image, you can do a video. So things can be going on in the video, and then your acting can be transposed right on top of whatever is going on. So Runway has provided a little demo for us to take a quick peek at. Original video, obviously, is a slow motion shot, a real shot of a man, and then the car moves in the background, and then he does acting, which could be anywhere. He could be sitting at his house doing the acting, or it could be someone else. And then we put that acting on top of that video.

I have been out here waiting for the bus all day. It was light outside when I got here, and now it's dark and I'm cold and want to go home. My Uber driver has canceled on me twice now and then restarted the ride. What is. What is that? Does he actually want to take you where I meet to go? I need to get out of here. As I was leaving the house, my mom literally said, bring a jacket. It's gonna get cold. And I said, mom, you don't know what you're talking about. And guess what? Mom was right. It's cold.

Pretty good. It's spooky. Like, you can barely tell that the original video had no mouth movements at all. I mean, it's. It's. It's very, very good. Except for the car moving really slow in the background. That's kind of a giveaway. It's. It's crazy. I can't wait to give this one a shot. I might do a deeper dive video on this because I had so much fun with Runway Act 1 when it first dropped, and this just supercharged it. We've got a few more examples.

Okay, so you'll let me know when I'm supposed to hear something? Yeah. Okay. I still don't I still don't hear anything. Maybe something really light. I think I hear something really light. Is it supposed to be loud? Am I supposed to hear something loud? I think. Oh, the volume wasn't up. Now I hear it. Now I hear it.

Oh, pretty good demo, man. It's crazy. You can see her lift up her hand. It goes with, you know, the acting, obviously, the performance and the voice and everything really is gonna have to come through. Right? So at the end of the day, I feel like this tool is right on that cusp of being fully usable for creating more professional productions. It's just, you got to get that acting right, you got to get the mood right. That's what it's going to come down to.

I don't want to move on but I move on, move on I don't want to move on but I move on.

Pretty crazy as it zooms in and then pans out. Oh, finally we've got some animation style stuff, which I'm always a fan of. So I think it was back in 1992 when it all started. And, yeah, that's pretty much the story. We good here? It's a God again, this one, like, you can barely tell that this is AI Generated.

Yeah. Okay. James. James.

Yeah?

We're not moving. We're not moving. I'm not gonna tell you again, okay? We've talked about this. I'm asking you to not move so I can paint.

Yeah, that's nuts. Like, what an update, man. Look at this, for example. That's just crazy. Changing the facial performance of an existing video. Copyrighted music, for sure. So I don't want to. I don't want to get screwed over, but that's crazy, dude. The original side by side, Mr. John Finger also got right to work here demoing this. Obviously, this started as an AI Generated video, but came out pretty darn good.

All right, so this is pretty exciting. It seems like you can now do facial performance on top of video with Runway Act 1, which is much more useful than, you know, just putting on an image. So let's see how it looks. And hopefully this will be, you know, very convincing that I'm that character and that all the driving that I'm doing with my face translates. We'll see.

It's pretty good. It did the face more so than that original head movement, but still so darn cool, man. I love it. It's crazy how far we've come in regards to AI Video in such a short amount of time. All right, folks, moving on. This one's A little bit old from December 3rd, but 11 labs released their own conversational AI agents and specifically a way to build them very easily. This is also more designed for businesses. It should only take, you know, a few minutes to end up making the conversational agent and you can have it for these very specific use cases. I might have to do a deeper dive video on this one as well. Let me know if that's something you'd be interested in. But you've got tons of models to pick from and basically this is like a ready to go, easy to build solution for pretty much any conversational AI agent use case. Good work 11 labs. Can you. Pretty darn impressive.

Building AI agents that can speak is now easier than ever.

Hi, what would you like to talk about today?

Introducing conversational AI with ElevenLabs build, test and deploy all in one platform for the most natural way to communicate with technology.

Hey, can I check the status of my order?

Yes, of course. Let me check for you now.

Your order is due to arrive at.

Around 2pm Building conversational AI agents used to be immensely complex and demand huge resources. But not anymore. Start building your agent by creating a voice or exploring our vast high quality library to discover one that's perfect for your application.

Hello and welcome to DHK Online. How can I help you today?

With a wide range of the best LLMs to choose from, you can then upload your knowledge base and define the goal and personality of your your agent. Or alternatively, you can integrate your own server to take full control.

Hi, this is Eric from Bound Events. How can I help?

Hey, I accidentally bought a ticket twice. Could I get a refund please?

Sure. Please read out your reference number and I can begin to process that right now.

Our system will analyze and evaluate your transcripts, providing you with valuable insights, conversation playback and the ability to test and define success criteria. And it's all possible in 32 different languages.

Welcome to GC Medical. How can I help?

Hello. Hablazo espanol?

Si por suppuesto que hablo espanol en que puedo ayo garle.

And best yet, effortlessly deploy your agent into websites with a single copy and paste. Or leverage our SDK for creating apps, services or even video games. So whatever you're creating, get to production in days, not months. Months and all with enterprise grade security to keep your users data safe and secure. It's time to explore the huge potential of conversational AI with ElevenLabs.

ElevenLabs is good man. They constantly ship very consistent quality apps that are genuinely useful for a wide variety of voice related or audio related scenarios. Their sound effect generator is really good as well. They have a music generator on the way. I haven't really seen 11 labs miss, so this is probably pretty great. And again, I might have to give this a little bit more attention. I love this comment here by Jason Scott. You're saying I can make an agent with my own voice clone and powered by any LLM and talk to myself? Yeah, absolutely. It's actually a pretty good idea for a video to test this out. Next up we have another Perplexity competitor. This one is called Mindsearch and it's open source. They're calling this an AI search engine framework and allows you to hook this thing up to either closed source or open source LLMs and allow them to search the web. This one also comes with a demo on Hugging Face that should be free if you want to test it out and make sure it's good for your use case. But yeah, this one would be pretty cool. For devs who want to implement some sort of search functionality and really don't want to rely on the Perplexity API for example, this definitely could end up being a lot cheaper, especially with those open source LLM capabilities. Think about what we just talked about in this video. Llama 3.370 B GPT4O performance with really good search performance and at fractions of the price. Finally, we're ending things off here with Google's Genie 2. This is another AI video generation model that's basically working in real time to allow you to play AI generated video games, which is crazy. They're literally generated on the fly and take the input commands from your keyboard to, you know, walk around or move around your environment, adjust the camera, etc. This is a really, really impressive model and obviously this is the best in its class that I've ever seen to date. For something that works in real time and produces kind of like realistic video game gameplay. I mean it's not perfect. Obviously you're not going to sit down and spend hours into playing one of these games. But as just a very low level concept demo, this is totally mind blowing. Like generate a playable world on a spaceship, it makes the initial image and then you can, you know, start playing with W, S, A and D and then looking around the robot and the futuristic city. I really like the sailboat demo because it shows you how good the graphics can be on these AI generated video games. Obviously it's not perfect yet, but think about what this will look like in just a few years you can have flawless, and I mean flawless graphics pretty seamlessly. And again, they have a larger blog post on this that I'm gonna link down below. But you can see all the different actions here. W to move forward, A to move left, S to move backward, D to move right and space to jump. Now they aren't consistent amongst each other and they're not all perfect, but it is at least listening to the action. I like the humanoid robot in ancient Egypt. This one's pretty darn cool. First person view of a robot on a purple planet like that looks pretty high quality. I mean it can do virtually anything. It does make some screwed up weird looking UI as well. And obviously that's still going to be an issue. It's a, it's something that we saw with that Minecraft demo, but this is a lot more diverse than that Minecraft demo and I think it's a little bit more consistent and smooth. They also say that Genie 2 is capable of remembering specific parts of the world and that are no longer in view and then re rendering them accurately when they become observable again. And that is absolutely true. This is something again, that was a huge issue that we had seen with the Minecraft AI video game demo, but Genie 2 seems to have fixed that at least somewhat short term. Apparently it can maintain a consistent world right now for up to a minute. Again, you're not gonna sit down here and play full video games but, but it's a start, it's a peer into what is going to be possible in the future. Fully AI generated video games is such a wild concept. It's so far removed from everything I understand, but apparently it's, it's going to be possible in the future. Like this is, this is the first baby steps towards that man. Unlike the Minecraft as well, there also seems to be a higher level of ability to interact with certain objects in the video game worlds. Like, you know, this is smashing balloons up here or blowing up this oil thing. Apparently it can even generate some NPCs as well and do some interactions. And of course, you know, physics is going to be another big bump with this and we already saw that earlier with the motion AI video generation. That kind of feels similar to this. I think this does have some real world use case application though, as they talk about in this blog post. You can do rapid prototyping. So let's say you were going to actually build, you know, one of these video games from scratch, but you wanted to demo maybe what something might feel or look like. This would be a great place to start, at least for brainstorming ideas and exploring new concepts. They're calling it overall a diffusion world model. Obviously it's trained on a video game data set. It passes through an encoder. Latent frames from the video are then passed to a large transformer dynamics model trained with causal masks similar to that used by large language models. And at inference time JNI2 can be sampled in an auto aggressive fashion, taking individual actions and in past latent frames to obviously predict what should be happening next. And obviously they're using some guidance to improve the controllability. Overall it's pretty dope. I think we're still a long way from like a fully AI generated video game that's just playable. Really cool model coming from DeepMind obviously. Alright, that's all I have for you guys today in regards to AI news. Like I said, I think this upcoming week is going to be pretty huge so. So I've got my work cut out for me but I'm going to be doing my best to deliver you the latest and greatest. Thanks so much for watching folks. I'll see you in the next video and goodbye.