OpenAI o1 for Agents & More AI Use Cases

So this week's release has got a lot of attention across the Internet and I think it's for good reason because this is the first time where we made a meaningful step towards this agentic future that we keep talking about. It's a future where AI apps don't just assist you, but actually take over some of the thinking and decision making for you and then deliver you the results. And between OpenAI's brand new model O1 that has multi step reasoning built in and applications like Replit Agent coming onto the market that think through software architecture before or writing some code and they do that even without using the brand new OpenAI model. We're really entering a new era of what can be done with AI tools as a consumer. And then we have also innovations out of Google like NotebookLM being able to curate all the notes into an audio podcast with one click. I particularly like this feature. We'll talk more about this and so many more AI use cases that showed up over the course of the last week. In this week's episode of AI News you can use. Let's get into it. All right, so first up we need to talk about the brand new model out of OpenAI01 and instead of doing an in depth discussion here, I will make this more use case focused. How can you put this to work? Or what kind of use cases can you expect coming out of this over the next weeks? Because that's really the point of the show. It's AI news you can use, not AI news, gossip and theories. If you want a more in depth look at OpenAI's 01, I created a standalone video for that, talking about all the details, technically testing a few basic logic tests and weird translations, and giving you tips on how to prompt it. But now I want to look at how to get more out of it. So one thing I do have to mention is that this is locked behind the paywalls. So you need a teams or plus subscription to have access to this. And you only get 30 messages a week on this one and 50 messages a week on this one. This is what goes in on the background rather than you giving a prompt and it giving an answer. You give a prompt, it thinks multiple steps and then gives you an answer. Okay, so this was a 15 second summary of the whole thing. But how could you actually use this and what kind of tips to do? I have. Well, it's still super early and people are just figuring out how to use this. But first things first, I thought it was fascinating that the video that I made about 01 has some of the highest quality comments I ever got on YouTube. Now I wonder if that was due to the fact that I actually recorded this in a raw format. This whole video was one take, so maybe people were more inclined to give their genuine thoughts. But there are really good thoughts in here and I want to highlight some of them. First of all, it's Prasan highlighting here that he's an AI engineer and the biggest difference in this model is the logical reasoning. Okay, that's well known. But then he highlights that the biggest difference here is that this model can build an entire software rather than just giving you some codes that will be part of a software. And that's really the direction this is going in. And not just that, it will also consider the business implications and adjust it accordingly. And then he hints at that this will be coming over the next month. And that's exactly the theme of this video too. Because the next segment little preview is going to be about Replit Agent that came out last week and that's exactly the whole point of Replit Agent. They didn't even have 01 and they already built this this multi step reasoning agentic type of workflow where they used LLMs to design entire architectures and applications rather than just one piece of code. And now it will be working even better. So more on that in a second. But I think this top comment is spot on and this is an interesting thing we should keep an eye on. But I also like this comment. Just intuitively seeking innovative solutions for inventing and system improvements, I think all of these high level thought processes seems like the right direction and we'll get more concrete in the next segment here with repl.on what this can actually do. But I want to leave you with one last tip. Considering the fact that most people have access to this through their ChatGPT plus subscription and considering the fact that you only have 30 messages on the preview model and 50 messages on the 01 mini model, these get used up really fast, especially as you start trying this new model and seeing how it performs on some of your common tasks. Oh no, I just used up one message. My bad. I just want to highlight the fact that you don't have to use this right away. You can and probably should start most of your conversations inside GPT and then if you want to, you can always switch to O1 preview and regenerate the answer by using this. This would be especially relevant with follow up questions where you're not satisfied with the answers when you read the four hour response you're like hey, this is wrong. I wish chatgpt put more thought into this. Well, that's where you would go in and regenerate with O1 preview or even O1 mini. This is also an interesting fact. Many people on Twitter are already reporting that O1 mini actually works better for them, so don't dismiss it just because of the name. It seems to be performing even better than the preview model and some use cases. Again, I'll be pulling in all the use cases and presenting them to you in a separate video. It's just a bit too early for that right now because this just is different from any other AI model we've received so far. Okay, and linking to GPT4O I want to talk about Replit Agent. This is something that came out late last week, so it came out right after our recording cut off and one could argue that by now Replit Agent is not as relevant anymore because 01 came out and it's better at multi step reasoning and code architecture and generation. But to anybody saying that, I would point out this graph. These are benchmarking results from Cognition Labs, the company behind Devin. If you remember, Devin, around six months ago came out with this incredible demo of a code generation agent where you just told it what kind of app you want to build and Devin figured out all the steps and compiled all the code and built your full website by itself. The demos were really impressive, but we never got any access to it. And while we still don't have Devin, Cognition published this graph where they show off the capabilities of Devin, their software agent, with various AI models in the background. So this is GPT 4.0. This was OpenAI's best model. I personally would love to see Claude 3.5 Sonnet in this spot because that was state of the art in terms of code generation. It wasn't GPT4O, but it's okay for this comparison, I suppose it works. Then they switched out the model Behind Devin from GPT4.0 to 01 Mini and instantly jumped by almost 50% in the performance. This is their internal evaluation, so take it with a grain of salt. But then when they switched to the main model, the performance doubled 26 to 52% essentially. Additionally, it should be noted that all of these three benchmarks have been measured on a base model of Devin, not the production ready model. The production ready model as outlined here further fine tunes their model with their proprietary data, resulting in a further jump in in performance. But the point here is this Devin performed twice as well with 01 versus GPT 4.0 and relet agent is essentially accessible version of Devin that works today. And while as of today, Friday, 9-13-01 is not accessible through Replit agents, the CEO has already been tweeting about 01 and my guess is that it's just a question of days until they implement it and their app becomes so much better at what it already does. Okay, that was a really long intro to actually show you what it does, but I thought this context was necessary because to contextualize that all the use cases we're about to look at are going to become even better a few days from now. So what can Replit Agent actually build? We went out into the Internet and looked at various examples. These are the ones that caught my interest. So most of these things that Replit Agent built you will see are more like internal tools rather than production ready applications. So here for example we have a color palette extractor built by Heather Cooper on X and in minutes she built a tool where you upload image and gives you the full color palette and you can add extra tools. So this would be a good example of an internal tool. If you need something like this, no need to use external website, you can quickly start building a library of your own tools that do exactly what you need. Just like this two hour live stream from Karan here which shows him building an app that maps all the gluten free restaurants onto a map. Look, this is the published thing. You can actually go in and use it. There you go. Where are all the gluten free restaurants in Manhattan? Here they are. I mean let's be real, this is not going to change the world, but this also shouldn't take an hour or two to build, right? And on that same note, repl.it actually has a phone app and where you can build things on the go. And mackay Wrigley here, by the way, one of the great followers on X when it comes to gentic workflows in relation to code generation, built a stripe coupon generator on his phone in under five minutes and look, he just types in his email, gets a custom coupon code. This got me looking at it thinking like wow, I should build my own for our own products that all run on stripe. I think this one is actually super impressive because it's done on the phone and it's fully functional, linked to a stripe account already. And then the last use case I want to highlight here is building a character chatbot website. If you have interesting prompts inside of ChatGPT, you could easily set up a website like this where you could access either different characters or your different prompts in a web interface like this. Again, all of these projects only take a few hours max, and this is legitimately just the beginning because with 4.0 in the background, clearly all of this will perform so much better. And therefore all of these little niche applications, internal tools, are going to expand in terms of completely complexity and what you can do with these apps. So I'm super excited for the future of this category with a combination of agentic workflows and models with O1 that managed to actually scale reasoning just for using more compute power. That this worked is really unexpected and the future is very bright for this category of applications. So one of my personal favorite tools in AI right now is OpenAI's code interpreter or Anthropics Artifacts. I use and recommend it regularly, but what I find when I do that is that certain people get a lot of value from it and others none at all. And that's because it's one of these tools where you need to know where you're taking your questioning before you engage it. In this case, you need a little bit of a background in data analysis or at least a basic understanding of why that job even exists. So yeah, this channel is really good at showing you what tools and techniques exist out there. But at the end of the day, a lot of these use cases are going to be hidden behind some base knowledge of a specific subject. And one of the best resources to up your understanding of a specific specific subject is the sponsor of today's video, brilliant.org especially in the stem subjects. Brilliant offers hundreds of lessons and all of them go hands on with the concepts, helping you learn in an interactive way. So if you want to get better at using a tool like Code Interpreter, you could and should check out their data analysis learning path. It starts with the Exploring Data Visually course, which shows you what can be done with traditional data analysis tools. And then it goes on to explore various concepts and techniques within data analysis. And even if you don't apply any of them, you gain a little bit of understanding of what the code interpreter could be doing for you. So the next time you're inside of ChatGPT or Claude, you know what to ask for. So if you want to check out this learning path or one of many other courses, head on over to brilliant.org advantage. You'll get a free month and if you decide to upgrade 20% of the annual membership. All right, and now back to more AI news that you can use okay, so next up, there's two apps that come out of Google that you should probably be aware of, because I think these are going to be implemented into your devices and computers eventually, but right now they're available as standalone apps. I'm talking about Google's Illuminate and NotebookLM. As you can see, both of these are in the experimental phase. And I'm going to start by talking about Illuminate. What does it do? You give it a paper and it turns it into a podcast. And it's the best tool at doing that that I've seen so far. This is finally coming out of wait list. Dozens of members in our community are reporting that they just got access to this. I personally am still on the wait list, but even Google came out with an announcement that they're releasing Illuminate so you can generate your very own podcasts based on academic papers. Now, I'll note that they must have some black magic under the hood going on here, because general speaking, LLMs are not the best at summarizing long and technical papers. They just leave out certain details that might be crucial. But Illuminate seems to get it right. And if you're off the wait list, you can finally generate your own papers. Here's a little sample what that would sound like. Let's unpack a paper titled Attention Is all youl Need. What's the core idea here? Well, the big idea in this paper is that we can build a really effective sequence transduction model. And yeah, my guess is by the time this video is up, everybody should have access to this. They're rolling this out super quickly, just like they did with notebook. NotebookLM. This was also first in Waitlist, and now everybody has access to this. And I just quickly wanted to resurface this and highlight it because they did add a new feature here. And basically it's what Illuminate does to papers, but to all your sources in NotebookLM. If you're not familiar with this app, this is basically a research environment, sort of. So you add various sources here on the left side. You can add links, paste text, add documents, up to 50 sources, and then you can talk to all that and have sort of a ChatGPT clone that is based on all of the knowledge that you add in the sources. There's a few extra features here, but essentially this is a very powerful, powerful way to research documents. And I also want to highlight one interesting fact is that within the AI Advantage community, NotebookLM has been actually the one tool that has been most discussed out of all of our guides. First of all, because the guide is really good, shout out Verluca for doing an amazing job in researching and putting that together. But secondly, because people seem to find this really, really useful, especially when it comes to wrapping your head around brand new topics. You can just go in here at various links you find, at various text snippets, articles you find, and then you can use an LLM on top of all of that. It's a really good way to interact with documents. And now you can turn anything that you upload from PDFs, slides or charts into a little audio summary, all with the power of AI. And this product is obviously powered by Google's Gemini 1.5 Pro, which currently is the king in terms of context. 2 million tokens in total, and it can even take video. And that's why something like NotebookLM works really well because you can add all different types of sources that it will then use. Okay, to show this off in action, there's this introduction book right here. And I could just start typing and asking questions like I would with a chatbot, or I can actually go to this Notebook guide button and here you can generate an audio overview. And these are really the standout features that I was talking about because look, you have suggested follow up questions, you have a full summary, and if you want to listen to a podcast like conversation referring to everything inside of your notebook, well, you can do that right here. That's so much easier to listen to than just a monotonous voice reading out the summary. I really like this feature. And again, if you haven't tried Notebook alum yet, give it a shot. You might be surprised. All right, next up, I want to talk about a smartphone feature that is either in your phone already or coming in the next week. So this is definitely AI news that you can use and it's something that I personally have been hoping for since a while. It's the ability to search through every single photo or video you have on your device by its content. Not just by the name, not just by the metadata, like the location. No. If you look for Shani dancing in a red dress, then it finds those pictures and the same thing goes for action within a video. Now, this little clip comes from within the latest Apple presentation. They're shipping this to their brand new phones. But also Google Photos has announced this and is rolling this out already. Essentially you can look inside of every picture and video on your phone and search it that way. I can't tell you how many times I've been in a situation where I was looking for a specific video clip, but it was from four years ago and there's hundreds of videos on my phone. Especially when you're talking to people, it's annoying to have to go through all of that. You just want to find the moment that the conversation might be happening around and this solves it. So this is just the beginning. It might take a few more weeks before this ships you, depending on your hardware. But I suppose with Google Photos, everybody could get this soon. All I have to do is sign up for the wait list, which is over here, and then you can have access to this feature independent of your device. The privacy concerns here. Ah, sticky one, right? Big tech company, access to all your photos and videos. I'll leave that up to you. I have to be fair and say at this point that I think that Apple is the only player in this game that so far made at least the promise of something that actually solves privacy. Their private cloud compute that is independently verifiable and built on custom hardware to make the entire process safe is the only thing I've seen in this industry so far that promises what we all really want, and that is privacy with our data. I think it's just common sense not to want to give all of your pictures and videos to a big tech company just like that. But then, as per usual, I'm not here to judge. I'm just here to give you my takes. And what's new in this space. Let's be honest, privacy concerns aside, this by itself is a pretty magical feature that I hope to see integrated everywhere in a safe manner soon. Okay, next up. This is a super quick one, but I just like this little interface, this little website. You just go to riddlemethis xyz, you type in a username, and then you have to guess what the prompt for this is. We've seen this before out of Google's test kitchen, but this one updates live and you basically have to guess the prompt behind these AI generated images. But I like the fact that this is live generated. What? The prompt for this one is just hot dog. No really use for this, except this might be a fun little game to play with friends. Me and the team just had some fun testing this while researching all of these tools tool, so I wanted to share it. Let's move on to the next one. All right, next up we have an announcement out of Anthropic and they keep shipping new and highly requested features. This one is on the dev side, but I really like this because it allows you to create different workspaces for different projects. The feature is very similar to the OpenAI project feature that came out recently. You can create new projects, name them, and then allows you to fine tune models, create assistants, and do different types of testing and work inside of this project. Now we have the same thing for anthropic workspaces. Simply go ahead, add a workspace, pick a color, and there you go. We have a nice little test workspace and then they have a standalone set of API keys. So for example, if I'm going to be doing tutorials in the future, I'll have a separate test workspace for those API keys to make sure I can track the API usage separate from my automations or projects that I'm experimenting with regularly. So essentially this is just a welcome way to organize your API keys, which is very welcome because I've been using the anthropic API more and more recently. We've been cooking up a few interesting workflows that I'll be sharing with the full AI Advantage email list relatively soon. Okay, and here I want to just spend a minute on AI video generators and what changed? First of all, an interesting use case came out for Minimax that we featured last week, the free AI Video generator out of China. And it's stop motion. They seem to have trained this on a lot of stop motion animation, which has people creating these little stop motion sets. It's quite interesting and there's many examples of this across Reddit, but it seems like it really performs better at these little stop motion animations than any other model out there. So if you wanted to create a clip of Superman saving a cat from a tree, well, Minimax is your go to model here. As you can see, there's more and more examples of this. But yeah, I would say this actually works. But look, these are the limited capabilities of AI video today. I think it's way more interesting in the AI video category to actually look at the things that are coming. And that's the point I'll end on today. Because big things are coming. And by big things I mean actual use cases that integrate into your workflow. Stuff that we would love to use in our video production workflow. And I can't even imagine how documentary filmmakers, event videographers, or even commercial or narrative filmmakers will be using features like this coming up from Adobe. Because this is what the future of video looks like. Because you're going to be editing videos and if you're missing a clip, well, there's going to be things like generative extent where these AI video generators will be used to extend a clip that you already have. Color correction, visual effects, extra B roll, all of these are things that actually make sense in a video production workflow. And whereas now we're in a super experimental phase of AI video, I think this is really going to be the tipping point where it's just going to become a staple for every filmmaker to be aware of these tools and know how to use them. But to be fair, that is sort of the story of AI right now. All of these tools are being developed right in front of our eyes. And this show keeps track of how these developments go. And if something shows up that you could be actually putting to work, I'll feature it on the show that is uploaded every single, single Friday. That's all I got for today. There's a lot to try out this week. Don't forget to have some fun while exploring it all and I'll see you next week.