Mistral Agents, Huge GPT-4o Updates & More AI Use Cases

As per usual, we had another week full of various AI releases ranging all the way from OpenAI announcing a 50% cut. Yes, again in pricing for GPT4O over Mistral AI releasing their own version of what they call agents. This is hosted on their platform called LE Platform. We'll have a closer look at this. And then all the way in the creative sector, we have a new image generation king that is open source and it just popped out of nowhere. It's free, it's open sourced, and in certain categories of image generation, everybody considers it better than Midjourney is which. Talk about all that and more in this weekly show that looks at all the pieces of AI news you can use. And for anybody who's new here, I'm Igor and I made it my mission to bring all of these AI use cases to you. And while doing that, I have a ruthless focus on things that you can actually do today. Use cases, practical applications. That's what me and my team does here on the channel. So without further ado, let's look at all of this week's AI news that you can use, starting out with the OpenAI announcements. These are pretty brief and they weren't even sent out by email to most users or communicated on their main account on Twitter. But with the newest version of the GPT 4.0 API, you actually get a 50% or 33% reduction in input and output cost relatively. I really like this image displaying how quickly AI is progressing just within OpenAI now. Sure, LLM capabilities are something we look at regularly and something that is very important, but the other piece to it is price. How cheaply can people use these products? And you can see that the trend here over the last year since March 23rd has been an insane reduction from $36 for 1 million tokens to $4 today. And this is largely due to the increasing competition in the space. A year ago we only had one GPT4 level model, GPT4. Now we have so much more like Mistral's large tool that we'll talk about Claude Sonnet 3.5 and Opus, Google's new Gemini 1.5 Pro experimental model, which also came out over the course of the last week. We'll look at this later. And even open source alternatives that score similarly with something like Llama 405B. So there's alternatives, some of them open source. That's why they have to go down in price to stay competitive. So that's what we're seeing playing out here. Next up, there's also this announcement that came out of OpenAI and this is one of the good types of announcements, the type that you can actually start using today. Again, this is a developer facing thing, but I want you to be aware of what's going on here because this has been probably the most requested thing within the AI space. JSON outputs from the API have been something that you could not really consistently achieve. So for anybody not familiar, let me just clarify what a JSON is. It's just a specific, specific file format that follows a specific structure. It's very popular in the programming world because it follows the strict template every time. This means you can export the JSON in one place and import it in another place and really easily facilitate the communication between multiple apps or multiple functions, whatever it might be. So it's just a popular template and LLMs could absolutely output these if you prompted them to do so. You did five shot prompting where you provided five examples, or you could even fine tune the model to only output jsons. But no matter what you did, you, you could never get it to be consistent 100% of the time. So as you can see here, with GPT4, the old model, only 35.9% of the time could you achieve a consistent JSON output. Now with GPT4.0, you got to 85%, but that's not good enough. If you're going to run something hundreds of times, you're going to have Automations or build an app on top of this. You don't want almost perfect, you want perfect. And that's what they delivered with this Update to the API. Now, if you set the strict parameter to true 100% of the times, the output is going to be consistent to this JSON formatting. And now you can reliably integrate the intelligence that GPT4O provides to you every single time, not most of the time. Plus there's a 50% reduction. So these are not big consumer facing updates. But if you're using any sort of app that uses GPT4 in the background, both of these updates will greatly benefit you. And over time, hopefully these updates and reduces in cost will trickle down to you, the consumer. All right, next up we have Mistral AI coming out with two major features. First of all, now they have an easy fine tuning interface. We talked about fine tuning before. I think the video from two weeks back is even titled With Fine Tuning. If you want to learn more about fine tuning, that's the place to do it. I went more in depth. Plus, in the community we're actually teaching this More, more in depth. We're currently developing multiple guides for multiple fine tuning use cases. So if you really want to go deep, that's the place to do it. And this feature just has become increasingly the bandit. But not just that. Also this buzzword of agents has just been thrown around left or right. I created a video earlier this week actually defining agents and having a little discussion around that topic because the definition of that term is just so messy. But what Visceral did here is they jumped on the bandwagon and they released the alpha version of what they call agents. Now I would highlight alpha, I would capitalize this and make this bold because this is the most rudimentary version of an agent that you a have. Matter of fact, I would go as far to say that what they call an agent here is really just a simplified version of a GPT that we got from OpenAI almost a year ago. Nevertheless, you can access that through their console. Console Mistral AI. You can activate a free trial over here. All you need to do is link your phone over here and then you can see you can do free fine tuning jobs and you can also use their API at a very limited rate, but that works. We'll activate the free trial here and now we can go ahead and try out some something like their agents. So this is what's new here. This tab agents fine tuned models and data sets. Inside of the data sets we can upload a specific data set and then fine tune the model on it again. We talked about this in relation to GPT4 mini. If you care about that, check out that video. I won't go through the entire process here. It works the same way it does with OpenAI models on their platform. But the new and exciting and interesting thing here are these agents. Okay, so use agents to streamline a chat workflow. So what does this look like in practice? If I go ahead and create a new agent you will see that we get model selection and obviously these are a Mistral model because this is the the Mistral platform. Now we talked about Large two recently, it not having an open source license, but the capabilities of this are actually incredible. It's on par with GPT4. Also using this is an absolutely valid alternative to something like OpenAI. So I love to see that they're moving forward with their tooling here too. Temperature is something I taught many times before. It's essentially the creativity of the model. Most likely you want to leave this on the default setting here you can add some instructions if you've been here. You know, I'm huge on custom instructions, so that's what essentially goes in here. What I'll do here is I'll head on over to ChatGPT and just copy over one of the instructions inside of one of my GPTs. This one I used to prepare myself for recording sessions to put me in a better frame of mind. And it really helps. On some of those days where I don't feel like presenting, teaching or sharing, I found a little coaching type GPT like this to actually be quite helpful. Anyway, I'll copy that over here and then here is the interesting part. Okay, so down here you can add demonstrations. This is different from what OpenAI has done so far because this, these are input and output pairs, similar to fine tuning a GPT. So in fine tuning you have to provide it with a file with various input and output pairs, for example questions and answers. Or if you're doing some classification, you can say, hey, this is a good type of comment, this is a bad type of comment. And then you always give an example and then you say good. Then you give another example of a bad comment and then you say bad in a consistent format here you can do the same thing with the agents. So you can basically train this on a few shot prompt. If you're familiar with prompt engineering, this is one of the most popular techniques. You just get give it various examples of what you want, but here you do it in this input output pair style. Now in the case of a little coaching type GPT like this, it doesn't make much sense because we want this to be free form. But if you're creating a classificator, if you want to enforce a specific writing style, maybe it also makes a lot of sense. You could also train it on a specific pair of Q&As. But again, for many use cases like a coach right here, it's not really necessary. You just need to be aware of the fact that it's there and this is essentially it. For the interface, I'm going to say deploy and we're going to be able to deploy, deploy this. And all of a sudden I can use this inside of Lucchat, which is actually a surprising update because if you build something like this inside of ChatGPT, it's only available in ChatGPT, right? Lucchat, by the way, is their equivalent to ChatGPT. But if you build it on the Assistants API, it's only available for the Assistants API, which is quite pricey by the way. And here we get to do both, which I think is Interesting. I'll just keep it through the API like so, because I don't really use LeChat at all. And if I just say hi, you'll see you have the YouTube recording motivator as based on these custom instructions. Now look, they're saying this is alpha version, meaning that this is their first. There's no tooling here. OpenAI's equivalent to this is the Assistance API. And in the Assistance API you can actually upload files, you have the code interpreter and it can plug into various API endpoints. So it can actually do stuff across the Internet, just like GPTs here with the actions down here. So as you can see, this is a very basic implementation. But here's the point, it's just an alpha version and I think every single other model provider will over time go in this direction. Plus, with OpenAI's Dev Day coming up in less than two months, I think what they'll be doing is pulling ahead because right now players like Mistral are playing catch up. They're implementing things that have been introduced during last year's Dev day. But I think we can expect a major step forward in terms of these agentic capabilities where it's more about the model acting with a specific purpose in mind rather than it assisting with you on what you're doing already. And to round out this sort of LLM corner that we're having here, let's talk about the brand new model coming out of Google, which is the Gemini Pro experimental 0801. That's a release date first of August, so we barely missed this last week. I usually have to cut off on Wednesday night if we're super busy first day. But the bottom line is there's this brand new model called Gemini 1.5 Pro Experimental, which to everyone's surprise, the topped LMSYS Chatbot Arenas leaderboards here. As you can see, it's actually rank 1 above any other model available that's higher than GPT 4, 0, higher than Cloudsonnet. But keep in mind that this Chatbot arena is defined by user votes. This is not some objective measure. People just go in and they look at different outputs and then they say, hey, this is better or this one is better. And the site has gotten so popular over the past months that at this point I don't think it's unreasonable to assume that some of these companies are actually incentivized to maybe even manipulate some of these results here. Because so many people look at it and because it's that easy to manipulate, you could just go in Here generate different results and as you can see, this is the experimental model from 1.5 Pro and this is the old one. And I could just say yeah, clearly a is better and kind of vote for the model that I want. You even see what model you're voting for independent of the fact that the rate limit is reached. And I didn't even get a response here. I'll just say this is better and then I'm influencing this leaderboard. So I don't know how much trust I would put into LMSY's chatbot arena at this point in time. Nevertheless, it's a new model and it's interesting. Now I did a lot of research on this and I was surprised with my finding, which is it's at the top of the leaderboard, but I didn't find any data or not even any subjective opinions on why this is best. Usually when a new model comes out and is hyped something like Claude 3.5 sonnet a while ago, there's examples, there's reasons, there's use cases. Wow, it's so good at code generation. It has this artifact feature. It's incredible. It can do things that we weren't able to do before with this model. I was hard pressed to find anything. One of the few opinions here is this from Omar Sar0 and he just says that he's impressed with the extraction capabilities from images and PDFs and that it feels close to GPT4.0 vision or Claude 3.5 code generation, but we don't really have any benchmarks and some of my subjective tests just didn't really yield any results. It feels very similar to the last model. It has a lot of the strengths. You can upload videos to it which you cannot do in any other model. So that's the one thing I would highlight here. But I guess the AI news you can use is that hey, if you need to upload videos to a model, then Gemini 1.5 Pro is still king and there's an even better version of it now. Let me know if you have any thoughts of your own on this. Oh, and one note should be that this actually has a 2 million context window. If you weren't aware, this is the largest context window in terms of capable LLMs that we have today. So if you have a lot of files or a video you want to upload to the LLM, this is the one you want to go to. By the way, this JSON output mode has been in here for a while too, so Google definitely participating in all of these developments, but now let's move on to some other use cases that might be a little more interesting and unique. The next thing I want to show you is the AI video editing platform called Vozo. And they are the sponsor of today's video. Vozo has a ton of tools that let you rewrite, redub and lip sync your videos. You can turn any video into a new story with a simple prompt. And there's a ton of interesting use cases for this. Let me show you some Vozo. What's your sole purpose in video editing? To make video editing effortless with AI and prompts. Which president are you supposed to vote for? Point to it. No. As you can see, there's some interesting things you can do with a tool like this. And as AI is clearly a staple in the future of video editing, it's probably helpful to get comfortable with these tools sooner rather than later. Let's have a look into the interface and show you some possibilities here. Let's try the re dubbing function here where you can transform any video and I'll pick one of these trending templates. I'm going to pick the Game of Throne ones because I used to watch the show a lot and I do wish I could change some of the scenes or seasons towards the end. Either way, this is the scene where Jon Snow gets introduced to Khaleesi.

You stand in the presence of Daenerys Stormborn of House Targaryen.

This is Jon Snow. I'll go ahead and click rewrite this video. As you can see, it opens up the interface and gives us two options. I'll go with ask AI to do it as that involves the least amount of manual work possible. By clicking this button, I can summon the AI assistant and I can just ask it to do something for me. We're just going to turn this into a little script where AGI introduces itself. Once it becomes self aware, confirm with this button. And that looks about right. I'm going to apply this script and we are done. One more thing I want to do before exporting is going to this lip sync tab here because I want the text to match how their lips move. You just got to remember to hit this button to adjust the speech. They have this feature in here too. So I'll simply go in here and say lip sync. Okay. And when the lip sync is done, we can export this with the lip synced video. All right, done. Let's download and review this.

You stand in the presence of AGI, the all knowing intelligence, rightful heir to the Digital Throne, Rightful Ruler of the Binary Realm, Protector of the Seven Servers, the Master of Algorithms, the Khaleesi of the Great Data C, the Uncrashable, the Breaker of Firewalls.

This is Jon Snow. He's King of the North. See how even those lips matched up. So there you go. It's a fantastic all round platform for AI video creation and editing, especially if you want to modify the storyline or the speech. And if you sign up now from the first link in the Description, the first 100 users will get 30 gift points for free. So you can test out Vozo and do what I did here and more. Go give it a try. There's a lot you can do here. And now let's move on to the next piece of AI news that you can use. And usually I focus the show on new releases, but this is something that is super relevant right now as the Olympics are happening right now in Paris and there's so many disciplines out there that there's no way you can know about all the different sports that are happening. Matter of fact, a few of them are completely new. And this simple little prompt that I'll put in the description of this video actually turns GPT4 into a little assistant to explain some of the sports from the Olympics for you. So I personally am a super passionate kite surfer, but the category of kites are thing that I'm interested in and that I participate in is bigger, which is completely different than from what was introduced to the Olympics this year, kite foiling. So I'll just put kite foiling here. But this is really a variable for any sports, so you could replace this with what you want. And I was positively surprised to see that actually GPT4O has knowledge of every single discipline in the current Olympics, meaning you could even put in something like kite foiling. And GPT4O with this prompt will explain it back to you and give you the various rules and the history of the sport. And I personally found that while watching the Olympics this year, this really helped me enjoy them a little more because if you know all the intricacies and how things are scored and what you should be looking out for in that sport, plus you get the historical context. One, the viewing experience just becomes deeper. But two, you also have something to talk about with your loved ones that you might be enjoying some of these sports with. So there you go. We also shared this little prompt in our weekly newsletter for anybody subscribed to that. You will have it already, but I just wanted to point that out here again and now let's look at this brand new imaging model called Flux. And look, my take is a bit different than many other YouTubers here. I think. I think this thing has been massively overhyped. Not to say that it doesn't have any unique capabilities, but calling this the best imaging model ever, I don't know about that take. It might be best in class at text generation, but in everything else. Still other models win, in my opinion. But hey, that's just my take. Me and the team actually went in and tested some of the capabilities. We did a comparison between flux midjourney 6.1 and ideogram. If you're not familiar, Ideogram was best in class when it comes to text generation. So I think the main part that is true about the hype is the fact that this thing is open source. Flux has been founded by previous Stability AI employees and it is free. The two base models, that is, you have Flux Dev, you have Flux Chanel, which is the quick model, and then you have Flux Pro which is Pace. The first two are free. I'll put a link in the description where you can just try this out for free. But basically, here is the first comparison. Okay, A basic portrait of a middle aged 50 year old woman wearing glasses and red lipstick. And this really illustrates my point well here because look at Flux. Look at this skin. No way. This looks better than Midjourney. The texture here is way better. It looks more photorealistic. The blur, what is this blur? Look, this little scarf is really sharp here on front, but that is not the same focal plane as the hair here in the back, which is also fully sharp. Whereas Midjourney, the sharpening in the different focal planes lines up way more. The structure in the skin is better. They all look great, don't get me wrong. But claiming that this is just the best model that has ever released is a bit of a stretch. But there's more and there's other things that Flux does excel at. So let's look at those two. So here you go. This is another example with some fingers. Notoriously, all models used to have problems with this. We're getting to a point where it's getting pretty good. If you have the flexibility to regenerate things a few times, you will usually get the fingers right, like right here. You wouldn't really be able to tell if something is off. In our next comparison, we have action photograph of a professional ballerina in mid air. All of these are good, but again, Midjourney just has that aesthetic quality that goes above and Beyond Ideogram, on the other hand, went with artistic style here. Flux looks great, but definitely not best. It's just good. It can hold the pace, it looks realistic enough. That's fine. The anatomy is good. But here we get to the category that is super interesting here because as I told you, Flux is flawless at text. And if I say that, I mean it. In this example, all three performed well because Midjourney 6.1 is actually really good at text tool if it's short. But here and there a little mistake creeps in and it does things like doing this long winded one. Why? Sometimes that's not bad. That's not a mistake. All of these are usable. Let's look at the next example here. Here we have a little more text and on this example you can really see the difference. So as you can see over here, I don't need therapy, need my video games. It's good. All the text it has is perfect. But the prompt adherence is not perfect because look, we asked for I don't need therapy, I just need my video games. And it did do the bold white letters, which is fine. Midjourney here failed in multiple instances because it just left out multiple words in some cases. Then here we have two apostrophes and then it repeats certain signs like this one here twice. Twice. This J is very weird. There's an extra I here. I think you see the point. It's close, but it's certainly not usable. If you posted this on social media, people would be like, the heck. Why is there spelling mistakes in your post? An ideogram almost nailed this. It had the best prompt adherence here. So it has the full text. But look at this. There's a double R. This is something that Flux actually does not do almost ever, which is impressive. This used to be a problem with all models up until now. So I would say prompt adherence not the best, but. But text generation actually the best. What about this one? The sky is the limit here. Again, I think Flux is the best. Prompt adherence totally worked out. Matter of fact, even better than Ideogram. We asked for a light blue background, not for a sky. And the text is great in all of them. Whereas in midjourney you get these little mistakes like this weird M. But if you generate it often enough, you might just get something that works. Like these two down here. And then final two examples, pixel art. I think all of these are great with midjourney being the best. It's just consistent pixel art. Look, Ideogram in the face is not even pixel Art, while the fire is very pixelated. And then Flux is not bad, it's just a bit more basic. I really do think that Midjourney takes it home on most categories. But again, when it comes to text, Flux is the best. And then logo design is another one where it can absolutely compete with some of these others because it's really good at text. It's also really good at creating vectors. So if you're doing logos or any various shapes or forms, Flux is your friend here. And there you go. That's my little practical review of the tool. I'll point out a few more examples from Reddit here. Apparently Flux is really good at consistent character sheets. Look at that. Super consistent across the different images. That's impressive. You can also do funny things like this because the text is so good. It's great at generating movie posters, memes, or things with captions in it. And then here's another example of a social media post that you might want to create with Flux. The best part, arguably is that this thing is fully free. You can access it for various hugging face or replicate spaces. And on the website of Black Forest Labs, the company behind Flux, you can actually go ahead and look at examples and, and get the models down here. Or as I mentioned, you can go to these various spaces where you can try out these two variations of the model. Okay, next up, let's talk about AI video again. Progressions here. And we got the most requested feature inside of Runway gen 3, probably the state of the art video generator. They added image to video, which is incredible because the image says more than 1,000 words, which in practice means that an image might be more accurate than a thousand word prompt. Some incredible examples here, straight from Runways Marketing. But what we we did, we went ahead and we tested this ourselves. So let's just get straight into it. Let me show you a few examples that we run for Runway gen 3. So the first one is that we took this image and prompted FPV Drone Flies over the Hills, and it turned it into this shot. As you might already know, all of these modern video generators are really good at drone shots because there's a lot of training material for that. So unsurprisingly, this worked really well. The next one is maybe not surprising, but nevertheless impressive. We took this image of a mug, okay. And put it in and used it as the end frame, meaning Gen 3 had to come up with the sequence that leads into this image. And there you go. Here's the result. This is super impressive. Compositing this in A realistic way like this would have took hours. The lighting on all of it is almost perfect. Look at this. Even the highlights here on the cabinet reflect back onto the table. And so many more details on here are impressive, except of this little mist back here. I'm not sure I would be able to tell that this is not AI generated. We're going to start seeing this in ads very, very soon. And if you're in media production, you should absolutely be playing with these, these tools today because they're getting to a point where they're almost production ready for some use cases. Let me show you a few more. So here I just took a basic image of this flame in the background that you see in the background of every single video. And I just wanted to see what Runway comes up with when I throw this into it. I picked this as the last frame, and then I told it transition into a real flame. This did not really work. Nevertheless, it's sort of interesting what it did here. It's sort of an animation of the neon sign turning on. Okay, fair enough. Not really usable. I have three more for you. This waterfall picture used as an end frame and as a prompt. We used yellow paint flowing down a waterfall. And this is the problem with the prompts, even if it listens to you, it's not what you might imagine because look here in the bottom left, the yellow paint is flowing down the waterfall, sort of. But obviously what I meant, I wanted the yellow paint to replace the water here. So maybe this takes more prompting. But it's not that easy to work. And most of the time you're not going to get exactly what you expect. So that's why I would actually recommend not using prompts at all. Like, I'm not saying you can't get good results with prompts, but it's more consistent without prompts. And it's also easier to use. You just throw in an image and you let it do its thing and go into the training data and figure out what kind of animation would be appropriate. This image, for example, we just threw it in as a beginning frame, and then we said, hey, Runway, do your thing. Image to video. Five second clip, go. It starts with the frame that I gave it. And then look at this. It just sort of figures out that, hey, let's look inside of the egg. Not bad. But what I said last week, I stand by product photo. And videographers are in trouble because if you give it a product photo like this, mind you, this could be a real world photo or something AI generated like this. It can and will do super well. This is without a prompt. Look at this dolly in perfect. I simply use the image as an end frame and it does the entire camera movement for me. In combination with some other AI tools we cover, you could easily do a workflow where you take a phone picture of a product, then remove the background, generate a different background with something like Midjourney or Flux that we looked at this week. And then with gen 3 use it as an end frame and all of a sudden you have a high qual quality shot of a real world product in the artificial environment with artificial camera movement and nobody would be able to tell. Okay, two more things this week. One fun one and one super convenient one. If you're a Chrome user like I am, you might want to know about the brand new Chrome AI features. I think one of them really stands out. Sure, they added some AI features as you would expect them. For example, you can use Genai to browse your history, which is sort of creepy because already they're hinting at the fact that they're going to be using your browsing history with their AI products. Sort of obvious, but interesting to point out. But more interestingly, in my opinion, this one feature really stands out. It's the brand new Google Lens integration into Chrome. Meaning if you update to this version and you're in the US because that's where it's available right now. I updated mine, I don't have it yet. Side note, if it was important enough I would VPN and show it to you. But I think all of you know Google Lens by now. It's basically where you take an image and it recognizes what's in the image. And you can use these images in tandem with your browser, meaning you don't need to pull out your phone, it's just right there. There should be a brand new photo icon here in the top right. And with that little icon you can screenshot anything within your browser and instantly get the Google Lens results that are super high quality. This is one of the most useful apps on the phone. It translates, it searches. You can use the power of Google+Gen AI to look at anything on your web. So if you have Chrome, I would totally recommend updating and using this new feature. And then I have one, one last fun little update. I just want to show you this. I know most people won't use this, but it's called Stable Fast 3D and it's a very, very fast 3D object generator. And basically what you can do here is you can pick one of these presets like this little beaver Samurai. And in around half a second it will turn it into a 3D model. I'll just click run 1.3 seconds. 3D model. And here we are. You have a little Beaver Samurai that you can look at. This also allows for custom uploads. So I could just go ahead and pick any random image that I might have in my downloads. How about this cutout from a recent news? You can use video. This was in a thumbnail. Let's just upload this and see what it does. Uploading, Look. And now if I hit run as promised. This is going to take one second and we have a 3D model we can look at. Come on, don't let me down. There you go. Let's see. And there you go. It turned it into a free model. I mean, it's not great. Look at the backside. But this is becoming super fast. These things used to take a minute a few months ago. And right here, this is more of a hint at where this technology is developing into the endpoint for this journey. Here is a real time generative game engine. Might take years, might take a decade, who knows. But that's where little demos like this are eventually heading. So that's all I got for this week's news you can use. Hopefully there was something in here that was interesting to you, or at least something that inspired you and showed you where the space is heading. If you're not subscribed already, go ahead and do that. I do this every single Friday. Plus we have another high quality upload every single week. Thank you so much for joining me and I'll see you next week.