Anthropic's NEW AI Model Now Taking CONTROL

Anthropic just rolled out a massive Update to their AI model, Claude 3.5 Sonnet, and now it can actually control your computer. It can see your screen, move the mouse, click around and even type for you. Basically, it's an AI that could take over your whole computer. Sounds pretty incredible, right? But it's still in the early stages, so it's not perfect yet. Still, the potential is huge. So let's talk about it. Alright. Anthropic has been working on this concept for a while now. Last spring they talked about building AI that could handle all sorts of tasks we do every day. Things like responding to emails, doing research, or even managing entire back office jobs all by itself. It's part of what they call a next gen algorithm for AI self teaching, which is just a way of saying they want AI to eventually automate huge parts of the economy. And that's a pretty bold goal. But now with Claude 3.5 Sonnet, they're getting closer to making that dream a reality. So what's new with this model? Well, the big feature everyone's talking about is something called computer use. Basically, this allows Claude to understand and interact with any desktop app. Anthropic introduced this feature in open beta, which means it's available for developers to start playing around with. But there's still a long way to go. Imagine it like this. Claude can take screenshots of what's happening on your screen and then it uses that information to move your cursor, click buttons and even type commands. And it's doing all of this super fast, like a human sitting at your PC. Except it's not human. It's an AI model. But before you get too excited, here's the catch. It's not perfect. Sometimes it's a little slow and error prone, and sometimes it misses basic actions like scrolling or zooming. Anthropic even admits that Claude's computer use is still kind of cumbersome. So while the potential is there, it's not ready to fully take over your desktop just yet. Okay, so why is this even a big deal if it's not fully working yet? Because the fact that model can even attempt to control a computer is a pretty massive leap forward in AI development. If it can pull this off, it could change the way we use AI in daily life. Think about it. AI that doesn't just answer your questions or write your code, but can actually use the software on your computer to get stuff done. We've seen AI tools that automate tasks before. For example, Microsoft's Copilot and OpenAI's desktop app for ChatGPT can look at your screen and make suggestions, but Claude takes it to the next level by actively controlling your computer. Anthropic's goal here is to make this AI capable of handling anything you can throw at it, whether that's filling out forms, browsing the web, or even automating complex tasks that require multiple steps. And they're not the only ones racing to perfect this idea. There are tons of companies trying to create what people are calling AI agents, basically software that can automate different tasks for you. In fact, a survey from Capgemini found that 10% of organizations are already using these AI agents, and a whopping 82% plan to integrate them within the next three years. Companies like Salesforce and OpenAI are all pushing for this kind of tech, and Anthropic wants to be right there at the front of the pack. But here's where Anthropic says they're doing things a little differently. They're calling their version of this tech an action execution layer. Which sounds fancy, but it just means Claude can break down what you want it to do into smaller actions like moving your cursor or clicking a button. And it's already being tested by some pretty big names like Canva and Replit. Canva is exploring how Claude could help with designing and editing, while Replit is using it to build an autonomous verifier that checks apps as they're being developed. Alright, now let's talk about some technical stuff. How does Claude 3.5 Sonnet actually perform? So Anthropic is bragging about how good it is at coding tasks on a benchmark called Swebench Verified, which tests how well AI models can handle coding. Claude's new version scored 49%, which is a big jump from its previous score of 33.4%. To put that into perspective, this beats some of the top models out there, including OpenAI's flagship model, which they call O1 Preview. On another benchmark called Tao Bench, which tests how well AI can use tools, Claude improved from 62.6% to 69.2% in the retail domain and and from 36.0% to 46.0% in the more challenging airline domain. In simple terms, it's getting better at doing multi step tasks like booking flights or processing returns. But, and this is a big but, it's still not perfect. In fact, during tests where it had to help with things like modifying flight reservations, Claude only managed to complete about half of the tasks successfully, and in other tests, like initiating a return, it failed about a third of the time. So. So, yeah, there's still room for improvement. Now, this might raise some eyebrows. If this AI can control a computer, doesn't that open up all sorts of possibilities for misuse? Anthropic says they are very aware of the risks here. They've taken some precautions, like not training the model on user screenshots or allowing it to access the web during training. They've also built classifiers to nudge Claude away from doing risky things like posting on social media, creating accounts, or interacting with government websites. But here's the thing. This tech is still new and there's a lot we don't know about how it might be used or misused. There's already research showing that even models without desktop access, like OpenAI's GPT4, can be tricked into doing harmful stuff like ordering a fake passport from the dark web. So what happens when you give an AI model access to your entire computer? It's a little scary to think about, but Anthropic says they'd rather find out now while the tech is still in its early stages, than wait until it's too powerful to control. They're working with agencies like the USAI Safety Institute and the UK Safety Institute to test these models before they're released. And they've built systems to monitor when Claude is being asked to engage in election related activities. This is especially important with the US elections just around the corner. They don't want AI meddling in politics. Anthropic has even said they'll restrict access to certain websites if necessary to prevent spam, fraud or misinformation. And turns out Claude has had some amusing moments during testing. In one instance, Claude was supposed to be helping with a coding demo, but instead started browsing through photos of Yellowstone National Park. And at one point it even managed to stop a screen recording mid demo, losing all the footage. So, yeah, it's not all smooth sailing just yet, but that's kind of the point of this public beta release. Anthropic wants to get feedback from developers to see where the model struggles and and what can be improved. They know it's not perfect, and they're expecting it to evolve pretty quickly in the coming months. So where does this all go from here? Anthropic is already working on a cheaper, faster version of Claude called Claude 3.5 haiku. This model is set to be released later this month and it's designed to be more efficient and affordable than Claude 3.5 Sonnet despite being a budget version, Claude 3.5 Haiku actually matches the performance of the larger Claude 3 Opus model on many benchmarks, making it a solid option for developers who need AI power without breaking the bank. Claude 3.5 Haiku will first be available as a text only model, but Anthropic plans to roll out image support later. It's going to be perfect for tasks like analyzing massive amounts of data, think purchase history, pricing and inventory records. And for anyone worried about performance, don't be Claude 3.5 Haiku scores 40.6% on Studwebench Verified, which is higher than the original Claude 3.5 Sonnet and many other state of the art models. So obviously the technology is evolving quickly, but for now it's more of a novelty than a necessity. However, the potential here is undeniable and we're definitely going to see some exciting developments in the coming months. As always, let me know what you think in the comments. Are you excited about the idea of an AI controlling your computer, or does it freak you out? Make sure to hit that like button and subscribe for more updates on the latest in AI tech. Thanks for watching and I'll see you in the next one.