It Was a Monumental Week For AI Advancements!

Well, this week has been a monumental week in the world of AI. Tons and tons of big announcements. One that's fairly world changing that I actually made another video about earlier. But I don't want to waste your time. Let's just dive right into it. Starting with the news out of OpenAI this week that's got everybody buzzing, which is their new operator platform. Now, this is very similar to Claude's computer use. If you've used that in the past, you can give a prompt like could you find a recipe of linguine with clams from all recipes and and add all ingredients to grocery cart on my instacart? It then opens up its own internal browser. It doesn't actually use your browser to browse, it sort of opens up a browser in the cloud, searches out recipes for you. Looks like it found one with a high rating, figured out all the ingredients that were needed, and then put all the ingredients that you need into an instacart cart and then gets you to the point where you have to click continue checkout, but it gets you all the way there. That was just one of the examples they gave. Now what's interesting about this new operator is that it's powered by a new model called a computer using agent or CUA. So it uses a combination of GPT4O's vision capabilities with advanced reasoning through reinforcement learning. This CUA is specifically designed to interact with a graphical user interface. For example, right now, the browser that it opens for you, the operator can see through taking screenshots and interact, being able to use all the actions of a mouse and a keyboard. Again, very, very similar to what we saw with Anthropic's tool use. Just a lot less of a pain in the butt to actually get set up. Now, I have played with this a little bit and I think it's really cool, but I think Bojan here actually says it best when he says, my favorite thing about the AI agents is that they can help me get something done in half an hour. What used to take me less than a minute. That's kind of how I feel about the current crop of like tool use stuff that's out there now. Right now this operator is available, but as of this recording, you have to be on their pro plan to use it, which is their high end $200 a month plan. It's not available yet in the regular ChatGPT plus or the free version, although they did say that it will eventually be available for plus users and I believe enterprise users. Not sure about free users yet, but we could see some examples of the type of things it's capable of doing here. Book a table for two at a romantic French bistro tomorrow night at 8pm and it will leverage OpenTable to do that. Find the most affordable passes to the Miami Grand Prix using StubHub. Suggest a 30 minute meal with chicken that has at least 4.5 stars. Leveraging all recipes, it could connect directly with tools like DoorDash, Instacart, UberEats, Thumbtack, Uber, Target, Etsy, eBay, Booking.com, tripAdvisor, Hipcamp, Priceline, Reuters, associated Press, the Atlantic, Axios, and I'm sure there's a lot more integrations and tools that it sort of works with natively. Let's try an example with Thumbtack. Find someone to assemble my new furniture Tomorrow I'll go ahead and submit this and then we can actually see what it does. You can see here that it opens a little browser window. Let's go ahead and make it a larger screen. I'm actually not doing anything right now. It's browsing to thumbtack on its own. You can see my white mouse is right here on the left. The black mouse in the browser is the one that's running automatically for me. So it's at thumbtack. It types in furniture assembly into the search. Click search. Now the zip code's not correct, so if there's something that I want to adjust on here, I can go ahead and take control real quick. So I can jump in here and I can actually control in the browser. I'll go ahead and leave furniture assembly there, but I will change the zip code. Now that I've changed the zip code, I can go ahead and click this finish up button and give the control back to the operator. It asks what did you do while you had control? That's optional, but I can say I gave thumbtack the correct zip code and then go ahead and return the control to the operator. Now I'm hands off again and it will continue to work on my behalf. It's asking me how many items I need assembled. We can see over in the chat that the chat's just asking me to answer this question. So I can either take control and click myself or I can just let the chat know. Let's say we're assembling two items here. I'll just keep answering the question. Yes, I have instructions and it's checking the box on my behalf. It automatically knows to select urgent because the original prompt that I gave it was I Need it like tomorrow. And it finally found a professional for me named igor, who has 4.9 out of 5 stars, price $40. And it gave me the profile link and it asked me, would you like to proceed with this professional or should I look for more options? And I'll just say, this one looks good. Now, going back to Boyan's tweet here. If I had just gone to thumbtack myself, searched out furniture assembly in my zip COD and then clicked all the buttons myself, I probably would have done it in half the time. But one interesting thing about operators is you can open up a whole bunch of different tabs and have a whole bunch of different tasks running all at the same time. So I can have one operator going in, booking me a flight, a different operator going and booking me a hotel, a different one going and making dinner reservations for when I travel. Right. So I can be going on this trip and have three different operators all kind of going off and doing things for me. Would that have been a lot faster than me just opening up the tabs myself and then booking them all myself? I don't know. But it's an exciting era we're entering into where these AI tools can actually do a lot of the work on our behalf. They're not quite to a point yet where I feel super, super excited about what they're capable of because they just still operate a little too slow for me to get excited about it. Here's something else that's really interesting about this operator is I can actually click in the chat on the left and actually sort of rewind and see the steps that it took. So I could click here and we could see it actually went backwards and I get a little like, scrol. So I can actually watch back the entire process of it going and doing that search. For me, it recorded essentially a little video of the whole process. If I go back to my earlier one about the furniture assembly, I could click into here, same thing. I could scroll back and watch the entire process of it trying to find me somebody on thumbtack and literally revisit how it went through the process. OpenAI has also made comments about how they have more agents coming in the coming weeks. So this is just like one of their sort of agentic features that they're rolling, pulling out. And I think this is going to get a lot better over time. Right now it feels like it's just a lot quicker for me to do it myself, go to Amazon, do a quick search myself, click on the product that'll be Here, the quickest press order and I'm done this one, it almost takes me almost as long to type the prompt as it took me to go do all of those steps. But again, I think the big picture here is over time you'll have a whole bunch of these different little autonomous agents that are going and doing a whole bunch of tasks for you in different tabs, and you can sort of complete a whole bunch of things all at once. Again, this is just for the pro members paying the $200 a month and as far as I know right now it's only available to pro members in the us so hopefully this will roll out to more soon. They did say more countries will get it soon, but the European Union it might take a little bit longer due to all the regulations they have there. However, there are some other options that have recently popped up that if you do want to have an AI sort of take control of your browser and do things for you. There's this option called Brow that I saw shared by Harrison Chase here. This one's available on GitHub. It is open source. They have quick start instructions here to install it locally on your own computer using your own OpenAI API key. We can see an example here where it's creating new Google Docs. It's actually, I believe it's noticing all of the various CSS on the page and reading the CSS to figure out where to click and what to do. It appears to essentially do the same thing, but free and open source. I haven't tested it yet, but that's an alternative if you want to try something else. There's also this new paper that came out of ByteDance this week called Uitars, which is a native GUI agent graphical user interface agent model that solely perceives the screenshots as input and performs human like interactions, for example keyboard and mouse operations. This also appears to be designed to do the same kind of thing. I believe the code is available up on Hugging Face, so if I go to files and versions in the Hugging Face page we can see all the model weights here that you can download again, another one I haven't tested myself yet. Maybe I'll make a video about here's some alternatives to ChatGPT's operators and we'll test all the various options and figure out which one's better. That might be a kind of cool video to do in the future on this channel, so make sure you're subscribed for that. Although operators are probably the thing that most people are talking about, probably the Biggest news to come out of this week was the Stargate Project. The Stargate project is a new company that was formed between OpenAI, Oracle and SoftBank, with OpenAI basically being the technology partner, Oracle being like the cloud partner, and then SoftBank being the money partner to put all the money into this. And the goal of this company is to invest $500 billion over the next four years for new AI infrastructure for OpenAI. Specifically for OpenAI. They're building this for their technology. They did some press conferences around this and in those press conferences they really pushed the idea that this can be used to find new medicines to cure cancers. It's going to create hundreds of thousands of jobs to build these data centers and ideally it will ensure American dominance in AI. That's the big goals of this thing. Now I don't want to go too deep into what the Stargate project is in this video because I did an entire breakdown of it right here on this video called the Stargate project, the $500 billion AI agenda. So definitely check that video out because that's an almost 15 minute video that really breaks down what the Stargate project is as well as my concerns around the Stargate project. The Cliffs Notes of my concerns is I think outwardly they're talking about the job that's going to create diseases it's going to cure the drugs it's going to find and the leading the world in AI, that's their sort of forward facing talking points. But underneath I think their motives are a little bit different. I do think they have intentions to use this for military use, for surveillance use, and for various uses with the three letter agencies of the country. Again, I get into it more in that other video that I created. So check that video out if you really want a deep dive on Project Stargate. But when I made that other video, this little video hadn't come out yet. This is actually footage of the first data center that they're building out in Texas. Sam Altman shared this. So they're already underway with this Project Stargate. They're already working on this massive data center in Texas. And so yeah, it's happening. They're already rolling with it. This raised some questions about OpenAI's relationship with Microsoft because with Oracle being part of this Stargate project, well, Oracle's in direct competition with Microsoft and their cloud computers with Azure. But Microsoft essentially put out a blog post saying that everything they announced we're thrilled about, we're going to be a big part of it. Nothing really changes between us and OpenAI. In fact, any new things that OpenAI develops within this Stargate project, well, Microsoft's still going to get access to, because contractually they're obligated to share all of these models with Microsoft. Elon Musk, on the other hand, has not spoken too favorably about this Stargate project. I find all of this really fascinating because obviously Elon Musk is good friends with Larry Ellison, one of the guys in the Stargate project. Elon's also good buddies and part of Trump's administration. And Trump was the one who announced the Stargate project, and Elon wasn't a part of it. But anyway, when this tweet was made about the Stargate project, Elon went to Twitter to say, they don't actually have the money. Sam Altman initially replied with a very respectable response. I genuinely respect your accomplishments and think you are the most inspiring entrepreneur of our time. But then came back about an hour and a half later to. To tell Elon that he was wrong, as you surely know. Why don't you come visit the first site already underway? This is great for the country. I realize what is great for the country isn't always what's optimal for your companies, but in your new role, I hope you'll mostly put America first. I kind of secretly sort of have this hope that Sam Altman and Elon Musk at some point bury the hatchet and sort of work together, because I think these two people right now are probably two of the most important figures in the world of AI. Elon Musk was building the largest data center on the planet to service XAI. And now this Stargate project with OpenAI is building a data center that's even larger than the one Musk was building. With their brains and their teams and their power combined, I really, really feel like they can build this AI super company. And it's just kind of a bummer that they had a falling out, because I feel like together they really could build something really, really great. All right, let me show you something that's completely changing how people are approaching creative projects. So LTX Studio is an AI powered platform that's revolutionizing pre production and storyboarding. And I'm genuinely excited to share two incredible features with you. So check this out. First up is the face motion capture. I can record a quick gesture or expression using my webcam, and LTX Studio transforms it into an animated character that perfectly mirrors my movements. And what's really amazing is that these aren't just basic animations. The AI generates realistic movements that that would typically take hours to create manually, but it does get even better. Using this character dialog feature, I can make these characters speak naturally with synchronized lip movements. Let me show you a quick example I just created for this scene concept here. While I'm in the jungle, I should remind you to subscribe to Matt Wolf and Smash that like button. Notice how the lip movements match the dialogue. This is game changing for visualizing scenes and pitching ideas to clients. And the best part? These features are just the beginning. LTX Studio gives you complete control over every aspect of your pre production from storyboarding to concept development. And guess what? You can try it yourself right now. LTX Studio offers free computing time which is enough to complete an entire project. So if you're ready to try it for yourself, click the link in my description to get started today. And thank you so much to LTX Studio for sponsoring this video. We did get some other news out of OpenAI this week. More of an announcement that something cool is coming soon. So Sam Altman went to Twitter on, well, January 17th to say thank you to the external safety researchers who tested O3 mini. We have now finalized a version and are beginning the release process, planning to ship in approximately a couple weeks. Also, we heard the feedback. We'll launch an API and chat GPT at the same time. So when O3 mini comes out, you'll have access to it inside of Chat GPT and if you're a developer, you'll have access to it within their API. In a post that came out a few days after that, Sam said, big news. The free tier of chat GPT is going to get O3 mini and the plus tier will get tons of O3 mini usage. So if you were wondering if O3 was only going to be the plus and pro usage, well now you know it's even going to be in Chat GPT free. And a lot of people are speculating that they're putting out this O3 mini on the free tier. As a response to the next piece of news that came out this week, which was deep seek R1. Deepseek R1 is an open source model out of China that pretty much performs as good as OpenAI's Zero1 model. We can see the various benchmark tests that they put up on their site. These blue ones with the lines on it, those are Deep Seek R1, this darker gray that's open OpenAI's O1 model. So we can see in almost all of these charts with all of these benchmarks that DeepSeek R1 is pretty much on par and even in some cases with things like math, better than what O1 is doing right now. And you can actually go to GitHub, you can see it's MIT license, it's open source, and you can access these models and a lot of people who have been getting the new Nvidia RTX 5090s have been downloading Deepseek locally and running this on their own computers. But you don't even need to do that if you don't want. If you just go to deepseek.com, click on Start now and then select Deep Think. This will use the new R1 model and it's totally free on their website right now. This is an O1 level model that's totally free on deepseek, which again is why a lot of people think that the O3 mini model that Sam was talking about is going to get released for free in ChatGPT. Because Deepseek's probably just as good as it already. On my live stream earlier this week I did things like have it build a Snake game for me and it literally wrote all of this code in one shot here. But what's interesting is you can see its entire chain of thought. It actually thinks through the problems like a human. Okay, I need to build the snake game using JavaScript. Let me think how to approach this. First, I remember that the Snake game involves a grid where the snake moves around, eating food and growing. And you can just read through exactly how it thinks about building this out and finally it gives us all the code. And this was the game that it created. It actually works. So the green dot is my snake, the red dot is the food. As I eat the food, my Snake gets longer and if I hit the wall, game over, even keeps score. That was one shot, just one prom. I know a lot of people will say in the comments, oh, I'm sick of people building Snake games. I don't know what to prompt. This is just one of the things that's easy to test on the livestream. I asked people to give me some other suggestions of things to test instead of Deep Seek. Somebody asked me to give it this prompt here. Of Earth is 8 minutes 20 seconds from the sun. How fast is the Earth revolving around the sun? It thought for 102 seconds. We can see all of its thinking, all of the math that it's doing here as it's thinking through this problem. Quite a bit of math going on. And then finally at, at the end we can see it came out to 29.9 kilometers per second, which I asked people on the stream for me to double check and the people on the stream confirmed that it was right. I know Matthew Berman did a much deeper test into all the things that Deepseek R1 can do, but take my word for it, it is pretty much just as good as the 01 thinking model. And I think because an open source model has been released that's better than Llama, better than Mistral, better than pretty much all of the other open source models, and just as good as some of the closed Sour models. It's probably better than, you know, Cloud 3.5 Sonnet already. It's about as good as OpenAI's 01. It's kind of got all of these other model developers freaking out a little bit. All right, moving along to Perplexity because they released their own version of an AI assistant this week called Perplexity Assistant. Now this assistant is only available on Android, so I have not been able to test this one myself, but once again it looks like it's designed to take actions on your behalf. Half so help me book the best Japanese restaurant for a date. You can see it's searching for the highest rated Japanese restaurant in San Francisco. It finds the restaurant. This person takes a picture of a book and said, just finished reading this. What should I read next? And now it's finding recommendations similar to that book Sapiens Blink the Black Swan. It can use your screen for the context. So here's like a article that they're landing on and they ask summarize this page and send it to John. You can see it summarizes the page and actually creates an email to send to this John person. It can do daily tasks. Remind me two hours before the next warriors game, and it looked up the times for the Warrior game and set a reminder of warriors versus Mavericks here. This is kind of what Siri wants to be, honestly. And it looks like Perplexity is trying to build a sort of alternative to Siri or the Google Assistant and things like that. Also this week Perplexity launched the Sonar API. This is designed for developers that want to use Perplexity's API for for real time search combined with large language models. So you can now connect to an API that will do a web search and then also ping their own large language model with the details of the web search as part of the context. Which goes head to head with Google's API and OpenAI's APIs that also have similar features. But again, this is more focused for developers who are trying to build software with these various AI APIs that are out there. Google DeepMind also had a new model come out this week called Gemini 2.0 Flash Thinking Model. We can see the improvements from previous models in both math and Science going from 35% in 2.0 flash to like 63% in 2.0 flash thinking, all the way up to almost 75% with this new model of 2.0 flash thinking. Very similar results on the science side here. And we can see that it's getting these better and better results using Inference Compute. So it's doing similar things to what 01 is doing, what Deepseek is doing doing. And after you give it the prompt, it's sort of thinking for longer before it gives you that answer. So these aren't like better and better and better trained underlining models. It's just now these models, when it gives you an answer, it sort of double checks, triple checks, make sure that the answer that it's giving you is a lot more thought through. And that's how a lot of these models are getting smarter and smarter now is they're thinking through their response a little better after the prompt as opposed to only relying on their training data. We're seeing that shift right now pretty much all of the models. That's what Deepseek's doing. That's what 01 is doing, that's what O3 is doing, and that's what Google's various thinking models are also doing. And if we take a look over here at Chatbot arena, we can see that this brand new Gemini 2.0 flash thinking model is now the leader on this leaderboard here. And this leaderboard is basically somebody gives Chatbot Arena a prompt. Chatbot arena gives them two responses back. Users vote which one gave the best response between the two. And that's how this leaderboard gets formed. This is actual users voting on which of the two responses they liked better. Gemini 2.0 flash thinking is the one that's working the best right now. And I'm not actually seeing deep seq R1 in here, despite having the older Deep SEQ models on the leaderboard. If you want to try Google's newest models, you can do it by going to aistudio.google.com and then over on the right where you select your model, they have all sorts of different models available for you to choose from, including Gemini 2.0 flash thinking from 121. Come down here, give it a prompt, and you're using this new model. It's also really, really fast. I asked this old typical question, how many Rs in the word strawberry? And it was like 1.8 seconds to get me this response. And of course it was the correct one. Anthropic had a little bit of news this week, including the fact that they just secured another billion dollars from Google. This is shaping up to be pretty interesting because Google already invested $2 billion into anthropic. They just invested another 1 billion into anthropic. But we've also seen that Amazon has been d dumping billions of dollars into Anthropic and that Amazon's planning on using Claude inside of future Alexas. But Anthropic also released something new this week for developers in their new citations feature inside the API. Basically, if you're developing something and you're using their API and you're giving it some additional documents, like you're giving it some PDFs or text files or things like that, you can now actually ask the API to cite its sources. So if it gives you information back, it will now be able to tell you exactly where it found that information from the documents that you shared. Again, a feature that's available for developers right now, but if you're not a developer, you probably don't care too much about this feature. Adobe had some new features they rolled out this week as well, and all of these are available in the beta apps. So if you're a Creative Cloud member, you can actually use the beta version of these apps and get access to some of these features. Things like find footage faster with AI powered media intelligence. This is something, something that I really, really am looking forward to. I want DaVinci resolve to put this in theirs. I tend to edit a little more with DaVinci, but you can upload a ton of B roll inside of Adobe Premiere and then say, find the video of the red canoe. It'll pull up all the videos where there's a red canoe and then you can pull those onto your video. To me, this seems like it'll be really helpful for. For instance, I go to CES, I record 10 hours worth of B roll. I can throw all the B roll into a folder in Adobe Premiere and then, and then when I'm looking for a specific robot, I can type the robot I was looking at and then it'll pull up any of the videos about just the robots and I can use it for B roll. It seems like a huge time saving for my editing workflow. They also added in a caption translation feature. So if you put captions throughout your video, let's say in English. You can then also render out a version where all the captions are in Spanish, all of them are in French, German, et cetera. They rolled out a handful of other features, but those are really the main AI features. Runway AI just released a new image gener called Frames. They're still rolling this out. I don't know if everybody has it in their account yet, but if you do, you'll see. Introducing Frames. Try it now. And some of the images that it generates are pretty realistic. So if I do a prompt like a portrait of a woman with freckles on her face and black hair in front of sunlit horizon sky with a vibrant but minimal sweater sleeves, Art House Cinema. I get some pretty realistic images like this. I don't quite think it's to the same level that we're getting out of flux yet, but they do look pretty good. I was doing some tests on my recent live stream where I did various images of a wolf howling at a moon, and I really, really like the aesthetics of these bottom two. Like the colors are just, you know, they really hit that spot for me. Somebody suggested an anime wizard cast a nuclear spell that exploded in a desert. We got images that look like this. Not too bad. Another AI image model to test. If you're not getting what you want out of some of the other AI image models that maybe you're already using, this is another one to add to your arsenal. Since we're talking about AI image models, free pick, added imagin or imagine 3. I still to this day don't know if it's pronounced Imogen or Imagine. So if you like using Freepik, well, you now have access to Imagen 3 Imagine 3. Inside of that platform, Korea AI rolled out a new feature that I think actually looks really cool. You can actually train image models now inside and then sort of rotate them and map them out the way you want. Here's their actual announcement over on X. Introducing real time custom AI model models. Now you can train your own styles, characters, or products and use them in Korea. Real time. Theoretically, I can throw a whole bunch of images of my face in there, create a model of my face, and then pose that model however I want. So in order to do this, we go to Kre AI train. Let's go ahead and reuse the data set where I put my own face into it. And then down here we have a couple options. Let's go ahead and select the real time and train AI. So it took about three minutes to train my face into it. These are fairly low resolution, so I don't know how well it's going to come out, but let's go ahead and click use with real time and see what happens. So without an image on the left, it generated this image on the right using a man with a beard and a colorful background. I'm going to toss my profile pic in over here. If I right click on this and convert it to 3D, I should be able to rotate and change the angles over here. So now I got this 3D version of my head, but I can rotate it however I want. And over on the right, it actually, you know, matches the rotation.