Unbelievable AI News You Can Use!

Another week, another batch of AI applications that you could be putting to work today. And it's incredible what kind of variety we're getting here week by week, because today we'll be looking at one application that remote controls your computer and does things with GPT vision. Right now, I'm hands off. It's creating a training plan for me. But then we also dug up some interesting GPTs which use external actions for specialty results. Adobe has some new features and so much more. And check this out. I came up with a new name for this weekly show. From here on out, I'm going to call this AI News you can use. I don't know about you, but I really like that name more so than this week's AI use cases. And it came up when somebody asked me what I did on YouTube, I said, I always talk about AI news you can use. All right, let's get into the practicalities here. So first up, we have a big, big, big release by Google. Google AI Studio came out with Gemini 1.5 Pro. And I did an entire dedicated video on this release. This opens up new possibilities. So it was worth talking about for 15 minutes straight. The summary of that is essentially you get Gemini 1.5 Pro with a million tokens of context. You get a lot of customization, you can really easily fine tune models, plus you can now also upload videos and directly prompt on top of videos. This is something we couldn't do up until now. If you're interested in this whatsoever, I strongly recommend you go check out that video. I went into great depth on the entire interface for non developers. If you're a dev, this is a perfect primer for the interface because there's a lot of new features here. You can have Google Drive, there's different ways to prompt here. You can fine tune models and upload all this stuff. And all of this is free. If you're in the eu, you will need a VPN to access this. So that is Gemini 1.5 Pro with Google AI Studio. But moving on, we'll talk about a bit more of an esoteric app that. Matter of fact, this came out a few weeks ago, but it has been just getting popular on GitHub over the course of the last weeks. It only has 140 stars right now. And this one is called Open Interface. And to be perfectly frank, a lot of these are coming out these days. They use something like GPT Vision to scan your screen and then they use different tools to remote control your computer to get things done. This is aligned with the whole idea of AI agents actually doing the work for you and not just assisting you in doing the work. And to be perfectly honest, not a single one of these really works so well that you would want to use this in your everyday life. We talked about the Devin demo two weeks ago, that looks really promising, but we don't have access to it. And everything else in this space has been really interesting, but not that useful. And I did spend around two hours playing with this today and I tried different things. And once it gets complex, once you have complicated graphical user interfaces or you have longer actions where it's 4, 5, 6 steps, it starts tripping up. But for simple things, it works pretty well. And it's super easy to install. Actually, you just go down here. I toggle this, download the latest release, dragged it over to my applications folder, gave it some permissions, and here we are. This voice input doesn't really work for me, but I'm just going to paste this super basic prompt right here. That did work for me during my testing because I feel like you should at least know about this. Just word of warning, you do need your API key in the settings and every single action is like 5 to 20 cents, which is quite expensive considering that it doesn't get any substantial amount of work done. Nevertheless, let's give this a shot so you know what's happening in the space. I just simply said create a Word doc with a training plan for me and then I gave it some details. 80kg male, 189cm. And then I just say submit. And now we can sit back and watch the magic happen. And yeah, by the way, I'm using a very similar prompt to what he shows off, just because this is one of the ones that really reliably works in simple tasks like this. It just works. So as you can see, it opened up word right here. It always takes a few seconds because GPT Vision takes a few seconds to process and give you a result back. But it's basically a combo of the different APIs. And in a second here, it should start writing out the meal plan with GPT4 Turbo. And there you go, training plan. Look at that. Hands off. This is happening by itself. Now. You can see that it's using this button, this button. And that is because I have the German keyboard on, so you do have to switch to the English keyboard. So I'm just going to stop this and redo it with the English keyboard. All right, I switched my keyboard, going to submit this again. And there you go. As you can see, with the English keyboard, it works perfectly. I mean, this is some magical stuff, right? Yes, obviously it's super early and eventually we'll have apps like this built into the software that requires it, maybe even into the operating system. But look, right now I'm hands off. It's creating a training plan for me. This is pretty cool and I thought it was interesting enough for you to see where we're at today. So to save you time, let's just do a jump cut to the end result so you can review what happened here. And there you go. That's kind of it. It saves the doc. And yeah, now I have a training plan that was programmatically written. All right, moving on to a significant release. This one is by Adobe Firefly. So if you didn't know, this is Adobe's free to use image generator. And they added some significant features here which I think you should know about. Namely the structure reference, where you can use the composition of an image to regenerate another image. We've seen this in other forms, but Firefly is really good at hyper realistic images and they're very fully, fully commercially safe. They're the one model that guarantees you that no matter what happens, their model has been trained on safe images that are open source, that they own the rights to, and not just all images across the Internet. And them expanding it with features like this, where you can input the image here and then it regenerates a new one, but maintains the composition, maintains all the lines. That's actually pretty big because you can do things like upload a sketch that you created and then it follows the sketch and then just regenerates that as a full fledged AI image. And the way this exactly works is if you go to the firefly.com, you're going to find this structure dropdown here on the left side. And in the structure dropdown, you get to pick one of these. Let's just take this interior picture. Just because this is a popular use case, you could take pictures of interiors or sketches of interiors and turn them into realistic images. But obviously you could also upload your very own. Now, if I go ahead and just say minimalist Moroccan interior dim lighting, it should follow this image and recreate the same setup. And as you can see, it did that. Now, I wish there was a way to kind of enlarge this. Maybe I can just zoom in on this. As you can see, it's the sofa with the light in the middle and the table here too. And as you can see, yeah, this is the same Room just in a different style. It gives you alternatives here. Firefly is really one of those tools that is super underrated and isn't talked about enough. And as mentioned at the time of this recording, this tool is free and it has been since quite a while. So I don't think Adobe wants to monetize this. Over time, we'll see. But there's much more here with styles, effects, and in combination with the reference images, it's quite powerful for business use cases because you can be 100% sure that this is not going to infringe any copyright. Okay. And to round out AI imaging for this week, we'll talk about Renoise. This is a very funky one. I just want to briefly touch on it. Last week I brought up a tool that turned a cat's meowing into dogs barking. This is where we start and this is where we end up. Okay. And I was like, what the heck would you even use that for? And I gotta say, the comment section on that video, you guys are super creative. Thank you so much for a lot of fantastic suggestions what you would do with a cat that barks. But one comment really stood out. It said that, yeah, essentially this is a translator between cats and dogs. They're finally going to be able to talk to each other. I was like, yes, exactly right. AI is going to do that too, maybe. I guess what I'm trying to say is this tool is similarly useless, but maybe you'll find a use for it. I don't know. It's called Renoise and it basically changes subjects in an image. So if I take this kitten in a basket as an input, I can say, a Lego kitten is sitting in a basket on a branch and you can essentially remix that kitten to be a Lego kitten while maintaining most of the rest of the image. In the second example, we're turning the kitten into a broccoli. So I don't know if that is something that excites you, then, yeah, here's a tool that can make that happen. You can try this with your own images. Let's start by giving it access to the webcam. So there you go. That's an image of me recording this video. And now I'm going to say a broccoli with fire as hair. I'm not exactly sure where I'm going with this. Let's just see what it gives us. That's not really good. Maybe let's go back to the examples of turning a line into a tiger. I don't know. It caught my interest. I wanted to show it to you now, you know, let's move on to something else that you might just have a use for. And that's a little GPT corner for this week, because I actually found two that are super interesting. Like more and more of these interesting GPT s keep popping up over time. If you missed the news, they just announced that they started rolling out monetization for GPTs. They're working with select creators. We're yet to see how that reward structure is going to be set up. I'm super curious to hear what kind of revenue split that is going to be and how much money that's going to be generating. My guess is that GPT creators will be making tens of dollars, as Mr. Wonderful from Shark Tank likes to say. But there's two really interesting GPTs. And if you have a plus plan, I would consider using these. I pinned one of these for myself. So one of them is a photorealistic GPT and this seems to be using stable diffusion juggernaut Excel, which is really good at generating pretty humans. That's essentially what it does. It's a fine tuned stable diffusion model that creates beautiful people. So whatever prompt I give this, it's not going to be using Dall E for you. As you can see, it goes into the action right away. And this way you can actually use a superior model in certain ways to Dall E right inside of ChatGPT. In other ways Dall? E is superior, but it's just good to have choices, right? And as the name suggests, Photorealistic GPT. If that's your goal. If you want photorealistic Results inside of ChatGPT, this is your best bet. Dall? E is not very good at that. And there you go, we have an image. Not bad for the first shot, right? I could include way more detail, but essentially this is a way to generate photorealistic images with ChatGPT. And it costs nothing if you have a plus plan. It's just an action in here. Not bad, honestly. One side note is that these links might not work right away and sometimes takes up to a minute for the API to run and the image to be stored inside of your link. So if you get an error message like this, just give it a bit of time. And the next interesting GPT here is the Library of Babylon. This one is brought up inside of our community as an interesting way to discover new books. And this doesn't just use GPT4. This again uses an action to access an external database. So one thing it does really well is recommend books and a great way to do that is based on other books that you've read. Then it asks you about what you enjoy about that book and it goes into its database and gives you further recommendations. Again, if you have a plus plan, this is a great little addition that might just improve your life a little bit. I mean, getting high quality books is a big deal, kind of, and this can do it very well. And the community members tested it and they reported that it's super accurate with very esoteric and niche topics that GPT4 wasn't even aware of. So yeah, I can just say find me a book I might enjoy and then I'll just say I enjoyed Sapiens for its history lessons. I thought that's one of the best history books. I remember when I finished it I was like, wow, every human in the 21st century should read this book. So let's see what else it recommends based on that. As you can see, it's using the action to access the database in the background. And there you go. We have custom tailored recommendations that you wouldn't be getting from vanilla GPT4. So in this manner, GPT4 really is ahead. But while we are on the topic, I want to give you a quick update on the different ranking between chatbots because as you know, there's a fantastic website that, that runs an ELO system on different chatbots and it updates every two weeks and we just got to update this week by the way, if you're new to the channel also, this is currently the best way to use GPT4OPUS all the paid models for free. This is VC funded to create this ranking of chatbots. If you want to just talk to one, you just go to Direct Chat and you can pick all the models in here. But that's not the point. The point is that leaderboard update that I wanted to share with you. And as you can see, the updated leaderboard ranks cloud free Opus as number one. Same thing we said on the channel when it came out. For many use cases that are essential, this is an excellent model. As mentioned back then, it lacks a lot of the feature. But the base model is really, really strong and this arena represents it. And the way this works is essentially people rate the different outputs. So they run a prompt in here, they get two results from two different models and they rate which one's better and then based on human preference, which in my opinion for this channel is the most relevant metric. Not what the benchmarks say, but what humans actually think and how humans actually use this. Claude 3 Opus is currently king. But an interesting fact that I want to point out is that Claude 3 Haiku is actually at rank 6. Look at that. GPT 3.5 Turbo, which is the free version of ChatGPT, is rank 14. And I can completely confirm this. There is essentially no more reason to be using GPT 3.5 in ChatGPT, because you could just be using Claude 3's free models and they're way more capable. I think that's not really a controversial opinion these days. That's just kind of a fact. The leaderboard backs it up and all the people I talk to these days agree with that. But then again, OpenAI is probably about to make their next move and then that will change again. So to round this video out, I have two more things. One is that Stability AI introduced a new code instruct 3 billion parameter model. This is a tiny model that outperforms a lot of 15 billion models. So this is mostly interesting for builders. A small model like this will run very well on the phone or locally on your laptop. You do need a Stability AI membership to use this commercially. And beyond that, I want to close with a use case that you cannot use today, but that's definitely coming. And I didn't think that this was worth a dedicated video, but I do want to mention it here, and that is Sora's first impressions, where they put OpenAI Sora into the hands of filmmakers and they came up with different films. As you might know, I had a video production company before, so this resonates with me deeply. And from all these films, if there's one that you should watch, it's the first one. I mean, it's the only real story here. And it's really cool to see how shy kids actually leaned into the strengths of Sora. Creating surrealistic images. They didn't try to replicate something like in real life, but they created something only an AI model would do and they created this little story of Airhead. 100% of this is AI generated and it's a legitimate story. So just wanted to briefly mention that if you missed this, go check out Airhead, as this is going to be a new category of filmmaking. Yes, it will make indie filmmakers way more capable. It will lower the cost of a lot of that, but it also creates this new category. Super exciting stuff to me. I can't wait. Once this is out, I'm going to be using this 247 and we're going to be covering a lot on this channel, along with all the other use cases that will come out over time. So that was this week's episode of AI News you can use. I love this new name. Hope you do, too. And if you want to learn about more AI use cases, here's an entire playlist with all the videos like this that I create on a weekly basis where I keep you updated on all the new apps that you could be using for your workflows. All right? Like subscribe, comment, and all that. And I'll see you soon.