Ep 386: Claude 3.5 Sonnet Updates - AI can use computers now?

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

Anthropics. Claude just released a pretty sizable update to its large language model, Claude. So now there's a Claude Sonnet 3.5 new and a Claude Haiku 3.5. So a lot of people are talking about these new models and the benchmarks, and we're going to get to all of that in today's episode of Everyday AI. But it's actually the computer use update that I think is really worth talking about, because now the large language models that we all use can control a computer like a human with natural language. So the same way you might instruct and sit over the shoulder of an intern or a new hire and tell them how to do a certain task, you can now do that with a large language model, and this is the first time that you've been able, with natural language, to do that. So we're going to be Talking about these Claude 3.5 updates and the computer use update and what that means for the future of work. All right, what's going on, y'? All? My name is Jordan Wilson, and welcome to Everyday AI. This is your daily live stream podcast and free daily newsletter helping us all learn and leverage generative AI to grow our companies and to grow our careers. So if that sounds like you, thanks for tuning in. If you're listening on the podcast, make sure as always, to check out your show notes where we'll have links. But probably the most important link is our website, your everyday AI.com. if you have not gone there yet, like, why the heck not? It's literally a free generative AI university. Almost 400 episodes, talking to the world's leaders in AI. You can go learn for free, whatever you care about. Sales, marketing, hr, education. It's all there. So if you haven't checked it out, please go do that. All right, so let's first start off as we do every single day by going over the AI News secret live stream poll. I'm currently number two, but I might be moving over to number one live stream audience. Let me know what what you are there. All right, so today's AI News, we got to do it a little bit differently because for a single day, this was in the last, you know, almost two years that I've been doing this every single day. Yesterday was one of the busiest days for just software updates and big releases. So for today's AI News, I'm Just going to break down just the AI visual tools. All right. Because I could give you the bullet points of all of these, but we're just going to give you a list. All right, so stability AI, one of the leaders in AI image generation, just released stable diffusion 3.5 big update. Genmo released Moki 1. That is an AI video tool that's open source. So a kind of a company out of nowhere released it. It's very, very good Ideogram, which we've covered on the show before by former Google engineers. They just introduced a very. What I think is a unique feature in a kind of a collaborative canvas way that you can work with their AI image generator. Canva finally added Leonardo's AI Phoenix model to the mix. So after they acquired Leonardo AI a few weeks back, so now Canva finally rolled it out. I think they called it Drop Droptober, which I don't hate. Mid Journey announced they'll have an image editing mode soon. So you won't just be able to create images in Mid Journey, you'll be able to edit them. And apparently they gave musician Grimes early access, and she's been posting about it. Runway released Act 1, which essentially you upload a video of yourself talking, like I'm doing now, and then in a character, right? Whatever style of character you want will talk as you're talking, mimic your facial expressions. So a really big step forward in what can be accomplished in animation. It's. It's wild. That's. Of all those, I think maybe the Runway One is the one that I'm most excited about. So. All right, a little bit different roundup of the AI news today. Because like I said, apparently everyone, if, if. If you're in the AI video or AI photo space, everyone just said, okay, October 22nd and 23rd. This is the day we're updating everything. All right, so for more AI news and for everything that you need, make sure you go to your everyday AI dot com. All right, y', all, let's get into it, and let's go over these new updates. All right? And I think they're pretty noteworthy. All right, so this kind of was out of nowhere, right? So now we have computer use from Anthropics Claude, as well as new models 3.5 sonnet new and Claude 3.5 haiku. All right, so if you are brand new to Anthropics Claude, maybe you haven't heard of it or maybe you haven't used it a lot. I'm sure if you're a longtime listener, you obviously know, Anthropic Claude, but it is, I'd say chat GPT or OpenAI's biggest competitor, right? So you have the tech titans that you know, have kind of developed their own AI models, their own large language models such as Microsoft, Google, Amazon and Meta. And then you essentially have the two startup titans, right in Open AI and Anthropic. And there's pros and cons to the Anthropic Claude model. So you know, I'll just say kind of right, right off the bat what it's good for. Anthropic, Claude is great at coding. It's also great at producing really human sounding language without much training. Right. Just out of the box kind of how, how I talked, how I talk about it, right? So those two, two things that people really love it for. I think Chat GPT is ultimately way better at written content, just not out of the box. You got to work with a little bit. And you know, I'm a former, former journalist so I, I feel I can judge those types of things. But you know, if, if, if you're just going in there and you're not great at prompt engineering or maybe in our free Prime Prompt Polish course, Claude is great at just producing more human sounding content. It's got a good context window so you can work with a lot of content. So that's kind of Claude in a nutshell. The downside, it's not connected to the Internet, which stinks. Which is why I would never recommend a business use this as an enterprise on the front end, right? And then like we talk about, there's always, you know, front end of how you use a tool and then a back end, you know, by using it via an API. So you know, your company can obviously tap, tap into these models and probably a lot of the software that your company maybe uses, right? Maybe your, your CRM or your, you know, I don't know, all those Alphabet soups of, of all the different, you know, software companies use, a lot of them probably either use the Claude backend or the Chat GBT backend. So this really does these updates I think will change what's even capable for your business or all the tools that you use. So real quick about the naming, maybe this is a small bone to pick but kind of confusing. All right, so essentially Anthropic has three different tiers. All right, so Haiku is their kind of smallest and cheapest model, right? So if you're paying the $20 a month, you know, you don't really have to worry about the price. So when I'm talking about price, that is if you're a developer or your company is using it on the back end. So, you know, you essentially have three different tiers. The smallest, fastest, cheapest is Claude Haiku, the middle one is Claude Sonet, and then the big boy is Claude Opus. So you'll notice there's actually now three different release cycles even within there. So as an example, Claude Sonet 3.5 already existed. This was months ago, so so they didn't change it to, you know, 3.6 or 3 point something. It's just 3.5 new, which is a little confusing if you ask me. Now haiku got the 3.5 treatment, so it was on 3 before. All right, and then you have Claude Opus, which is the biggest model, which hasn't been updated yet. So I'm sure that there's a strategy there. You know, Anthropic might be waiting for a, you know, GPT4.5 or a GPT5 to really update its big Claude Opus model. So a little confusing that actually, even though there's three tiers of Anthropic Claude, it's actually on three separate release cycles. Right. Claude Sonnet has been updated twice since three. Now Haiku has been updated once and Opus has not been updated. So a little confusing where they're at in their product cycle. But hopefully that's a good, you know, three minute recap to bring everyone up to speed on Anthropic Claude. All right, so like I talked about, let's go going over now some of the details. So, you know, for our podcast audience, I'm going to be sharing a video later, but everything else, don't worry. I'm kind of reading some of the releases from Anthropic. So essentially here, like we talked about, the model updates are now you have Claude 35 sonnet new. I actually tweeted at Anthropic yesterday, they, they weren't labeling anything, so it was hard to see if, if this was actually the new model. And then about an hour later, now it says new on it. So at least, you know. Right, okay, this has been updated and it's available now. And then you have the Claude 3.5 haiku and then computer use. So we're gonna go, we're gonna talk a little bit about the 3.5 updates first and then we will jump into computer use. But to, to whet your appet, computer use is this. So right now it's kind of only available via the API, although there's kind of now a Virtual environment that you can set up and test this out, but you have to do it via the API anyways, right? So you're going to be paying for usage even if you or your company want to try out this new computer use. But kind of the computer use 101 is. Well, number one, it's available now, it's in beta now available, right? So your company could build off this or you could try it out for yourselves. And, and here's, and here's the thing. It is literally controlling a virtual machine. All right? So in the same way that humans use a computer, you can type to CLAUDE in natural language and it will use the computer based on what you say, right? So similarly to RPA Robotics process automation, which has been around for, you know, pretty popular for decade, decade and a half, right? But the difference with something like this computer use and something like rpa, well, rpa, steep, steep uphill climb. It's, it's very rules based and it's, it's just specific tasks, right? So there's all these, you know, browser extensions where you can kind of record, you know, things that you type, but they're all very limited and rules based where this new compute use from anthropic, right. The big thing there is it, you're, you're working essentially with unstructured data, right? Where a lot of these tools, Chrome extensions, rpa, you know, for lack of a better comparison, you're kind of working with structured data, right? You're working with, with bits and bytes. You're working with very, very, very specific rules and narrow use cases where with, by working via CLAUDE and the computer use, it's kind of like, okay, this revelation of working with unstructured data, right? Being able to just type something in the kind of virtual computer and your, you know, kind of AI agent is just going to figure it out on their virtual machine and, you know, browse the web, open applications, right? That's the other thing is, yes, there's a lot of Chrome extensions and other programs that do this just within a website. But with computer use, it can do it on a whole computer, right? It can open a terminal, it can open spreadsheet, it can open, open a notepad, right? It can open different programs, you know, close them, you know, toggle between programs, right? Which right now you really can't do, especially with natural language, right? Which is again, I think a good comparison for this computer use is just how you would talk to a human, right? If you were training someone new on your team, if you were training an Intern. That's kind of what we have with computer use. Anthropic did warn, though, and my gosh, this is true. They are saying it is experimental, at times, cumbersome and error prone. Well, at least Anthropic said that because that is the truth. You know, both in my, my own very short tinkering with this and watching other people's demos, it is extremely error prone. Right. So we're going to show you here in a little bit a demo. But if you think that you're going to be able to reproduce this demo for yourself or your company anytime soon, I would say probably not. All right, so now let's get back and look at some of the specs of this new model. And hey, live stream audience, I always appreciate you guys tuning in. Right. So Jackie said she's late this morning. Don't worry, Jackie, I'm not a teacher. You're not going to get penalized for being late to class. And, and everyone else, you know, Michael was saying here, I think it's the first clear sign of the AI agent revolution. Yeah, similarly, right. On the show yesterday, we went over Microsoft's autonomous AI agents, which should be out next month. Right, but. And with this claude, it's out now, right? Very, very different in terms of capabilities and features and functionality. Right, but yeah, would love to hear from you guys, livestream audience, if you're interested in this computer use, if we should tackle it more in depth in the future. But let's just go look at the models a little bit more. So first of all, you have to look at pricing, right? And so again, this is the API, all right. And this is what I think Anthropic is hoping to be its differentiator. Because when you are using either the Claude 3.5 Sonnet or the Claude 3.5 Haiku API, you are also getting access to this computer use. Right? So now even, you know, kind of comparing the different, the different prices, it's, it's kind of apples and banana chips here, if I'm being honest. And, and this kind of tells me that Anthropic is not trying to compete on price. It seems to me that they are really trying to compete on features when it comes to their API. Right. And even y', all, I know the, you know, you might not care about a company's API, right? You're just logging in, you know, you're logging into your CLAUDE account on the front end or your chat GPT account on the front end. And I get that. Right. But whether, you know, it or not. Thousands, thousands of pieces of enterprise software use an API from one of the three big companies. Right. Open AI, Anthropic or Google. So you can say, oh, these, these updates, they don't really affect me. Yeah, yeah, they do. Right. I wouldn't be surprised if a tool that your company uses daily might start integrating into some of these features. Right. So I want you to think about that. Yeah, we're going to quickly go over here the, the, the numbers for the dorks. Right. But this does impact you. All right, so let's talk about Claude 35 Sonnet. And we're going to compare this to the pricing for GPT4O. So the pricing for Sonnet is $3 per million tokens of input and then 15 per million tokens output. Comparatively it's $250 and $10 respectively for GPT4O. So much cheaper. Right. For GPT4O, but it doesn't have the computer use capability. Right. Obviously the GPT4O API for whether you, you, for your company. Right. Your company can go in and, you know, use these APIs today, start fine tuning models. They're very, very affordable. Right. But like I said, also all of the programs that your company probably uses are going to start integrating these, these features. Right. So I think Anthropic is really trying to separate itself by this computer use. But then, you know, the, the GPT4O or you know, you could just say OpenAI's APIs, they have access to a lot of features and functionalities that Anthropics API doesn't have access to. Like real time. Right, like the real time voice. Right. Your company can use that inside of OpenAI's API. All right, so it doesn't look like Claude is trying to compete on price. All right, let's look at benchmarks. You know, we always got to talk benchmarks. And again, benchmarks are different tests. Right. Think of them as, you know, there's, you know, maybe a handful or 10 different, you know, standardized tests that you might take, you know, through grade school, high school, college, etc, large language models similarly have those tests and have benchmarks. And there's different prompting methodologies. But you know, usually this is a good apples to apples comparison. So quad 3.5 sonnet new is, you know, doing very well in benchmarks, at least compared to GPT4O and Gemini 1.5 Pro. Right. So to its two closest competitors in OpenAI's most capable general model. Right. We're not talking about the 01 more on that here in a second. And Gemini 1.5 Pro from Google. So in terms of most, Most benchmarks, Cloud 3.5 Sonnet New is doing the best, right? But it did look like Andropic did some cherry picking on this. Right? In terms of the benchmarks they decided to include versus the ones that they did not, which is interesting. You know, so one thing I would say, mmlu, we talked about that benchmark, I, I'll say that has been the most standardized, the most used, at least in, you know, 2018 through 2022. That was kind of the gold standard benchmark and they decided not to include that one, which is interesting because that's one that the mmlu, which is essentially, it really tests general knowledge across, I think it's 57 different subjects. So they didn't include the MMLU. They decided to include the MMOU Pro, which is a little bit different. And so I, I thought that one was interesting because there is no benchmark for GPT4O on the MMLU Pro. But with coding, this is one area right, where I think Claude really shines. And in the human evaluation benchmark blew everyone out of the water, right? Even the GPT4O model that had a 90.2%. Now Claude 35 Sonnet new got to 93.7, which is a pretty sizable jump there. So you know, in terms of benchmarks, we have most of them here. And Claude 3:5 Sonnet did a pretty good job at least on the ones that Anthropic decided, like I said, a little bit of cherry picking here. Their new model kind of, you know, wiped the floor in every single one except in math. All right, so math problem solving. Gemini 1.5 Pro leaps and bounds ahead of everyone else. But look at this. At the very bottom, I found this interesting and I kind of have a little bit of a bone to pick with Anthropic here because you can't have your cake and eat it too and have it be calorie free and then cake shame people. All right, so at the bottom they said our evaluation tables exclude OpenAI's O1 model family as they depend on extensive pre response computation time. Unlike typical models, this fundamental difference makes performance comparisons difficult. Okay, so if they were to keep it at that and you know, set those rules themselves, I, I would respect that and say okay, good, yeah, that's fine. You know, I do think the OpenAI's 01 model is kind of in a class in its own right. That is a, you know, previously called Q Star, then Project Strawberry, now the O1 model, right? We don't even have access to the full model. It's just O1 preview and O1 mini. It is a different model, Right. It uses kind of this chain of thought reasoning, which, you know, requires more computation, more time, and presumably higher scores on all these benchmarks. Right. So Anthropic at the bottom of their benchmark sheet, you know, kind of says, oh, we're not comparing to oh, one, because it's a different model. All right, Fair, but interesting. Anthropic seems a little, I don't know, going against your own rules here, because on their announcement blog post, they're parading the fact that Claude 3.5 sonnet essentially has a SWE bench. SWE bench verified score of 49.0. And then it says scoring higher than all publicly available models, including reasoning models like OpenAI 01 Preview. So I'm like, okay, you can't flaunt like, oh, yeah, we're better than 01 preview. But then when it comes to all of the other benchmarks that O1 preview is light years ahead of everyone else. You can't just say, oh, okay, well, we're going to cherry pick all these, you know, these, these spots where our model is better than 01. But then we're not going to put it on the benchmarks because that would look bad. Right? Because then all those, you know, green scores in your column, right? All the, all the little green things here that are, that are lit up, right? They wouldn't be green anymore. They would be green for OpenAI's 01 model. So a little bone to pick with Anthropic's approach there. You can set the rules, sure, but you gotta play by your own rules. You can't not include the world's most capable and powerful model. Right. You can't just exclude it from a chart and then cherry pick certain benchmarks and say, oh, yeah, or certain capabilities and say, oh, yeah, we're way better. Okay, well, either compare it or don't. All right, Little bone pick. I didn't, I didn't have a full hot take Tuesday this week, so I got, I got a couple spicy takes left in the tank. All right, and, and, and here's, and here's another thing, right? And I, I want to talk on both sides of this because the new 3.5 Sonnet model, I think, was actually pretty impressive. Right? I, I did a review of it yesterday and I shared this in our newsletter and on our YouTube channel. Yeah, we cover, we cover AI as it happens. You know, it's not just the podcast. So, you know, thank you to all our podcast listeners for always tuning in. But we did a kind of video review of Sonnet3.5 new on our YouTube channel and then we shared that in our newsletter. And I did notice even when not directed to 3.5sonnet often defaults to this chain of thought thinking to some chain of thought reasoning, right? So actually under the hood, it is acting much more like OpenAI's O1 model than the previous version of Sonnet. So it's like, okay, that's a good thing, don't get me wrong, because I think that means that responses will generally be better. But clearly the smart research, which again, I think is a good move, right? They've kind of taken this methodology or approach in tuning this new 3.5 sonnet. New to act with a little more chain of thought, right? Thinking through things step by step and instead of the model, you know, just, you know, next token prediction, you know, vomit all at once. Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on gen AI. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on gen AI. It takes it a little slower, thinks a little methodically and goes through things step by step. So again, not to, you know, poke at anthropic here, but I, I'm kind of, my, my, my BS meter is a little high on this whole right benchmark table because that's always the first thing people talk about when a new model comes out. Because if your model is not the most powerful on benchmarks, if you, if, if you are building. Because here's the thing, this is billions of dollars of investment and the Benchmark table is everything when it first comes out. So I have a bone to pick with anthropic that. Okay, well, your new model is kind of using some chain of thought reasoning. By default you exclude the world's most powerful model from your benchmark table. And then yet this is kind of how your model is operating on queries that require a little bit of logic, which again, I like, but I don't like how they treated it. So let's say, let's just take a look. So I have a very unofficial rubric of sorts. I have a similar set of about 12 questions when I do live model comparisons. Right. I should probably formalize it a little bit more and do some scores or something like that. But I have a set of questions that I generally ask models and I've seen, you know, I've been doing the same set of questions for, you know, about a year and a half almost since the very beginning, since, you know, GPT4 and Claude3. And I've noticed that Claude 3.5 sonnet really responded in a different way. So 3.5 sonnet new responded in a different way than normal 3.5 sonnet. Again, why couldn't we just call it 3.6? Anyways, it used chain of thought. So I have one prompt or one kind of trick question that I do a lot. It's I just woke up today with six apples and three bananas. Yesterday I ate a banana and two apples. This morning I will eat one apple and no bananas. However, I don't really like apples and one banana may turn brown tomorrow. Assuming nothing else changes, how many apples and bananas will I have tonight? So there's a lot of irrelevant information in there that I put in there specifically to trick models. All right, so Claude 3.5 sonnet new to its credit. You can see here on my screen it said let me solve this step by step. So I did this, this same prompt about eight times. All the new all in fresh windows, right? And it, it took this chain of thought step by step approach each time, which again, I like it. But then you can't separate, you can't try and separate yourself and you can't use this same language, right. Of excluding open A ones open AIs O1 model as they depend on extensive pre response computation time. Let's look at extensive pre response computation time. Why are models like 35 Sonnet more expensive than haiku? Because of its computation power. It's more powerful. So let's look at time. So I did that, I timed this response so it got it done in 6.5 seconds. It got it wrong, by the way. Right? It didn't get the correct answer. So Claude has still never gotten that, that one correct, at least in my unofficial testing. GPT4O gets it right. Obviously the O1 model gets it right. So it got it wrong in 6.5 seconds doing chain of thought, doing kind of that more computational time, time intensive reasoning, right? Which it didn't do before in the normal. It didn't do as consistently before in the kind of quote unquote normal sonnet 3.5. But then O1 mini, so not even the 01 preview, right? So the smaller O1 model got this correct and it got it done. Open a. Open. Sorry, O1 Mini, right? This new reasoning model that you can click and you can see how it thinks, it said it got it done in five seconds. I timed it. It was actually 7.4 seconds. But y', all, 6.5 seconds. Claude got it wrong. 7.4 seconds. O1 mini got it right. So. But both used kind of step by step chain of thought reasoning. So again, I know we're talking intricacies here, but I don't think you can overlook this omission that Claude Anthropic decided not to include the O1 model because presumably the O1 model, right, they wouldn't have been able to have all those green marks, right? Because you've raised billions of dollars. You have large enterprise companies. If you introduce a new model and you run benchmarks, you better be almost all green. Otherwise you know, your investors are going to panic, your customers are going to jump ship. Because that is a signal that, hey, even though we have, you know, billions of dollars in funding and you know, we have unlimited compute in the top researchers in the world, if you can't get green scores, you're screwed. So not a huge fan of what Anthropic did there. I am a fan of how they updated their model. But hey, if you're gonna essentially make Sonnet 3.5 and force it to do some chain of thought reasoning, love it. But you got a benchmark against it. All right, new model availability 3.5 sauna is available now to all users and 3.5 haiku will be released later this month so that you know, you're really going to only use Haiku if you're using it on the API because it is much cheaper. If you're, you know, a front end cloud user, you will obviously be using 3.5 sonnet new. All right, so let's now look at computer use live. All right, so we're going to do this one kind of quickly here and live stream audience, if you could let me know if you can hear this. All right? Sometimes the audio sharing works, sometimes it doesn't. All right, so let me know if you can hear this. I'm going to do a quick, quick little.

I'm going to show you a simple example of computer use today. My friend's coming to San Francisco next week and I want to take him to do some touristy stuff.

All right, so live stream audience, let me know if you could hear that and then I'll. I'll kind of explain to you what's going on. So this is a video that Anthropic released. And one thing that it says here before it gets started, this is the computer use demo and it says Claude is generating all the computer actions shown here. This demonstration was recorded in a controlled environment. Right. So that means, yeah, it's kind of cherry picking a little bit, which is fine, with some supporting infrastructure simplified to highlight the core capabilities. All right, thanks, thanks, Live stream audience. You guys said you can hear. All right, so I'm gonna let this play. It's about two minutes long. So essentially in Claude researcher here is talking through this new computer use. Let's, let's go ahead and watch and listen and podcast audience, this one's simple enough. They released, I think, three, three or four different demos of computer use. I think this is the easiest one to just listen to and understand what's going on.

I'm going to show you a simple example of computer use today. My friend's coming to San Francisco next week and I want to take him to do some touristy stuff. I think doing a sunrise hike with a view of the Golden Gate Bridge never gets old. So I'll ask Claude to figure out some logistics for us. I'll ask Claude to find a good place to see the sunrise to help me figure out timing logistics and help drop a calendar invite. So I remember when I have to.

Leave.

It'S opening Chrome, going to Google, searching, And it looks like it's found something.

All right, real quick, let me just kind of describe what's going on here. So you essentially have a local instance of Claude running and then it says, right. It's even giving kind of like coordinates, right? So it says mouse moved, you know, move to 474, 72, 682, left click, type. Best place to watch sunrise over Golden Gate Bridge. Right. So essentially launched a browser, went to Google and put in that query, but you can literally see each and every action that this computer use is performing. So I just wanted to call that out. And different things are then happening on the screen, but very much like a human would do. All right, so let's. Let's watch the rest of this and I won't interrupt now until the end.

And it looks like it's found something. Great. So how far away is the location from my place? It's opening maps, Searching for the distance between my area and the hiking location. Cool. So now it looks like Claude is searching for the sunrise time tomorrow. And is now dropping it into my calendar. And populating it with some details. And great. It looks like Claude did it. This is a simple example, but we're sharing computer use early to learn from what people build.

All right, so hopefully, hopefully that kind of made sense what was going on to our. Our podcast audience, but let me just try to quickly explain it if you didn't see every single step. So in this use case, doing some research, right? Best place to watch the sunrise. It. It launched a browser, went to a couple of different websites, searched Google, open new tabs, found the time of the sunrise, right? It said 7:22. And then it created. Opened a calendar, Created a calendar event, right? All that information. So think of this through the normal business tasks that you do day to day, right? There's probably a lot of this, a lot of research, right? Every day before I start the show, I have different websites that I read. I have, you know, kind of some dorky, like Boolean search URLs saved, and I go through and I read them and I look for certain things. I open up different web pages, I. I quickly browse them, right? To read you guys the AI news every day, right? I still keep a human in the loop, so in theory, I could program Claude and this new computer used to do these things because it can work on any application on a computer, right? So maybe I need to do that, but put information in a spreadsheet. Maybe I need to create, like I said, a calendar event because this functionality is technically not new unless you're a dork like me or, you know, if you're in it, you know, you know, like I talked about, you know, rpa there. There's kind of a lot of browser extensions that can, you know, you can record different actions within a browser and then have it run that. But like I said, the rules are very rigid. It's just for very narrow tasks and everything has to kind of happen within the browser. So, like I said it's a little more technical and it's kind of the equivalent of having to code something. Because if you get one little thing wrong in these Chrome extensions or with robotic process automation software, it's not going to work. Whereas with Claude computer use, you're using natural language, right? So in demos, it's kind of cool to see if Claude doesn't. If it runs into an obstacle, it will try something else. Whereas normally, Right. If you get one little thing wrong by using more traditional methods, you're screwed. It's not going to work. Right. That's why I really think this is kind of a line in the sand, right. If we think of structured data versus unstructured data. Right. All these other tools and processes before that have been available are pros, or, sorry, are structured data, essentially. So, you know, I'm just making a comparison there, but it's like you have to have every single thing right, like code, right, extra space, you know, something spelled wrong, nothing works, right? This is computer use. It's natural language. So you don't have to, you know, as an example, have every single thing right, right. You can have misspellings, you can just, you know, have simple instructions. And, you know, presumably the best case scenario is this new Claude computer use will figure it out for you and complete those tasks autonomously. So there are a lot of guardrails for this, right? Which is good. You, like, you can't have it, you know, create social media accounts. It's. It's, you know, so there's certain types of actions that there's guardrails against right now, which I think is a good thing. Were you guys. Were you guys impressed with that or not? Tara said, amazing. I don't know. Like, part. Parts of me are like, wow. And then parts of me are like, okay, well, you know, obviously we saw the, you know, Claude, or, sorry, Anthropic only released the most impressive of use cases, and they kind of said, like, this is a very controlled environment. You know, if you go play with a demo and, you know, Michael said, would love to see more info on setting up computer use. Is that too dorky for the show? It's definitely dorky. So, you know, long story short, you have to use an API. You have to run like a Docker. So it's super dorky. We could do that. But, you know, I am curious. And for our, you know, podcast audience, too, I know it's a little more difficult, kind of only seeing the audio, but again, just think of those tasks that you do every single Day, right? You're in the browser and then you're in, you know, maybe your, your Outlook mail program and then you're in, you know, Microsoft Word or Excel. Right. And then you're maybe, I don't know, in, in the terminal on your computer. Right. So being able to have general use over a computer with natural language, I think the ceiling is very high, the floor seems pretty low. Like, I don't think that this tech is ready for the big time right now, but that's just me. All right, so let's go over current availability. So the computer use feature is available now, but only via the API. So you can't, like, you know, I, I think and assume that essentially Claude will have a desktop app soon, similarly to how OpenAI has a desktop app for ChatGPT. And then you'll be able to run it that way. But for now you can't, you gotta, you know, you gotta be a little bit of a dork. You gotta run the API. You can do it just through obviously Anthropics API. Or you can use third party services like Amazon Bedrock, Google Cloud's Vertex AI platform as an example. So really they are making it accessible through Amazon and Google if your company, you know, uses those platforms, which so many people are either using Amazon Bedrock or Google Cloud's Vertex. So right now the implementation, so developers must create an action execution layer. Okay, so what that means, and this is kind of breaking down how this actually works. So it does it by screenshots, right? So this computer use, it just takes screenshots and then it kind of tells the computer use program, okay, based on this screenshot. So it is using kind of traditional, you know, computer vision. And it's like, okay, I took this screenshot, so now I have to, you know, maneuver my mouse to click this. So there's problems with that, right. Sometimes, and I saw this in a lot of demos, sometimes websites are a little slow to load. So it'll get caught for, you know, for a couple of seconds and then it'll have to take a screenshot again. But at least it has that like self correcting approach, right? Or as an example, maybe you have a pop up window on a website, which so many websites now do, right? Maybe the first screenshot that computer use takes doesn't have that pop up. So then it tries to go perform an action, but there's a pop up blocking it or a cookies notification. So it is working through a set of screenshots and then it sends the screenshot. It uses computer vision to analyze it and then it essentially controls a cursor in a mouse click depending on the coordinates of the screenshot. So that's kind of how it works. And it essentially translates these prompts into commands and you're executing these actions via the API. So I'll give you the hot take right now. Should you try this, right? I don't know. It's super buggy. I did a little bit. There's very. Right now. So you can. The easiest way to access it, essentially Anthropic set up a virtual environment, right? So not doing it on your own computer, like I said right now when it's in beta, the guardrails are, I'd say extreme, but that's not a bad thing. I think it's good that there's so many guardrails, right. So even in this virtual environment, right, you're, you're not going to be uploading your company's files, right. You're essentially think of it like this. It's on like a dummy desktop, right? Like if you go into Best Buy or something and all those computers there, right, those computers are in essentially demo mode. They're very limited to what you can and can't do. So the same thing when you're working on this virtual environment, it is very limited and it's heavily throttled. My gosh. So you do have to use your own API. So what that means is there's different tiers. So most people, unless you're a company that has heavy usage, you're probably only going to be a tier one or a tier two for Anthropic. Which means because it uses this screenshot and computer vision, it does eat through a lot of tokens pretty quickly. Right. Which, which is why I think this is Anthropics, you know, one of their kind of competitive advantages, or what they hope they will, because it's going to be expensive to use via the API because you're essentially having to process a lot of information via these non stop screenshots and using the computer vision. So if you do want to try this out, unless your company is on like a tier three or higher, you're going to find out pretty quickly you're going to run out of tokens, right. And you know, you essentially have to go on a token timeout. So should you try this right now? I don't know. Probably not. It's pretty buggy, if I'm being honest. I think it's just worth, you know, going to watch demos online and waiting until hopefully Anthropic does just Release this via desktop until it gets better. But right now it is buggy. But that's okay. That's not a knock on Anthropic because it is a groundbreaking technology. You can't overlook they are the first to do this. Yes, you know, we have Salesforce Agent Force, but you can't just pro like Program Agent Force to use your computer. It's only working within Salesforce, Microsoft, which we talked on the, on the show yesterday, very, very impressive Copilot Studio and Autonomous AI agents. Similarly, you can only work within the Microsoft suite of products. You can't just work on desktop, browser, you know, notes on your computer, etc. So this is a truly ground groundbreaking piece of technology from Anthropic. It's just not that good. But Anthropic didn't say, right? Like, hey, this is ready to take over the world. They did say, admittedly you got to give them credit for that. They're like, yo, it's buggy, right? This is a new technology. It's not necessarily production ready, but I think it probably will be soon because I think it's going to gather a lot of attention. So like I said, do we even need this? Do we need this, right? Like, isn't there a reason user interfaces exist? Do we need to be programming a virtual computer to do all of our tasks? I'd say as it's set up now, absolutely not. This is not a good way to work. But I think when you look at this new feature from Anthropic, you can't judge it on face value on what it can do today. I think you really have to think of what it means for your business or your company and the type of work that you do. I think more than anything else, this gives leadership teams times to time to think and time to plan and time to explore as well on how this technology, once it is a little better, right? It might be a couple days, might be a couple quarters, we don't know. But what we do know is, well, OpenAI hasn't yet arrived to the Autonomous party, which they will be, right? There's reports going all the way back to last year saying that OpenAI was investing heavily in agentic AI, right? Or in AI that can execute tasks for you like we just saw. So one thing that you have to give Anthropic credit for, they shipped, right? A lot of times even OpenAI, Microsoft, Google, right? They announce things and they might, you know, kind of roll it out a little bit. Very limited access, wait list. Anthropic just shipped it Admittedly saying, hey, it's not ready for, for the prime time, but it's available in beta today. Right. So you have to give Anthropic credit. It shipped, but I'll say overall, even like recapping everything, the 3.5 models, they're improved, but. And they're good, but I'll say they're not fantastic. Right. I don't think, you know, companies are going to be, you know, looking to switch from 4o from OpenAI to 3.5sonnet or 3.5 haiku any anytime soon. Probably not, because the 4o model, I think is significantly better. Even on those benchmarks, they didn't, you know, they kind of selectively chose benchmarks that their new models, you know, exceeded, you know, all the other models. So in terms of the 3.5 updates, not super impressed. I think 335 sonnet is much better than it was previously. For coding. There's no, there's no match. Let's. Let's put that out there right now. But for everything else as a general use model, it's, you know, good update, nothing newsworthy, but the computer use is huge. And I don't think that people are going to give this enough credit for what it can be because I think people are going to be looking at what it is today and what it is today. Computer use, I think, is a giant step forward for where AI technology can take us in the future of work. So, yes, it's buggy, yes, it's extremely throttled. Yes, the guardrails are there. But I actually like the approach here from Anthropic, so we'll have to see if it actually pans out, but, you know, we'll be covering it along the way. All right, I hope that was helpful, y'. All. A deep dive into the new Sonnet 3.5 Haiku 35 updates from Anthropic as well as the new groundbreaking computer use. Right. AI can use computers now. Pretty exciting time. So if this was helpful, I hope it was. Let me know. Hey, if you're still in the live stream, let me know if this was helpful. Do you want to see more of this? Should we do a more technical show going over computer use? I think it would be one of those. Could run into a lot of glitches, but let me know. Thank you for tuning in. If you haven't already, please go to your everydayai.com, sign up for that free daily newsletter, and more importantly, join us tomorrow and every day for more Everyday AI. Thanks, y'. All.

And that's a wrap for today's edition of Everyday. AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everyday AI.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.