Claude's AI Update is Mindblowing!! Computer Control + Sonnet 3.5 (New)

We've got some breaking news. Claude 3.5 Sonnet new and computer use have just come out and it is changing what we thought AI could possibly do. We're going to show you how AI can actually take over your screen and do tasks for you all on today's show. Let's get to today's episode. Okay. On today's show, we are talking about one of the biggest launches in AI this year, and that is a updated model from Claude, but more importantly, the new Computer Use API that lets AI take over your computer and do tasks on your computer for you. Kieran, before we go in and show everything like this is. This is a big, big deal, right?

All of my favorite episodes are about Claude. And Claude are following quick on the heels of OpenAI's latest launch with two incredible new products or updates and products they have the computer use API. We're going to cover, and we're going to cover a little bit on the fact that they just released a new version of Sonnet that I think is super cool. And if you're a marketer, you'll want to know why you want to start using this product. So very happy that Claude is keeping the pressure in OpenAI. Because people know I'm a fan of all AI assistants, but I've got a special. I'm a special fan of Claude.

All right, let's. Let's just go straight into the product video and check it out.

The researchers here at Anthropic Computers is something that we felt was going to be important for a while now. And so today we're going to be talking about a very early version we have of computer use and walking through a representative example of the things we think it's going to be useful for.

I like that they always put people with accents in these videos.

Yeah, yeah, That's a nice English accent.

Yeah, nice, proper. There's a great video of English dudes eating ribs that I have to send you. It's hilarious.

In this fictional demo, a customer, in this case the Ant Equipment Company, has come to us and asked us to fill out a vendor request form. The data I need to.

This is very B2B, by the way.

It's scattered, it's various places, very B2B. What we're going to do is ask.

About when it's finished, look at the.

Spreadsheets, check if Ant Equipment is in there, and if not, move over to the CRM and try and find some more information there. Fill out the form for us.

Fascinating.

Hopefully transfer the information across to the.

Auto, fill out the forms.

First thing that's going to happen is Claude's going to start taking screenshots of my screen and quickly realizes that the Ant Equipment company isn't actually in the spreadsheet. So the first thing it does is it swaps over to a CRM and searches for the company we're interested in.

All right, I just want to talk a couple of things. First of all, this is a boom for remote workers, because now you can actually. Right, what's happening in the world is everyone's going to start to log. Are these remote workers even doing anything? And now you can be on the beach and you could be talking to your computer and the computer's working away and you're not actually there. So it's going to make people even harder to track remotely. But what I actually want to talk about is you probably have watched something around Claude's latest release already. We may not be the first people to cover it, but we're always the people that come with some additional insights. What is interesting here is everyone looked at it from a B2C perspective and actually their whole product launch video was B2B. And what I think it is showing is something that you and I covered on the show previously in that all of these companies want to be a kind of knowledge worker for B2B companies that are deeply intertwined and plugged into your apps. Right? And this is a really good example. What actually, what actually happened to search engines, they got moved to the back end because the AI sits on top of them. It does all of the searching and contextualize it and gives you the answer back. It's a much better experience. What's happening here, Software being moved into the background because the knowledge assistant is able to do all of the use cases, do the thing you want it to do, and you never actually have to be in the software. And I think actually this is a great thing for knowledge workers because so much of our time is lost on having to become admins of these tools and learn these tools, and now you can actually just have the AI do the work for you and actually execute on your ideas much, much quicker. But it does open the question of what is software in a world where the knowledge assistant is actually using the software. And actually, I'll end with this because the use case they're showing is really interesting because they're showing a system that can actually go find information, fill out a form and start to set up a new vendor. Now, that is not too many clicks away from the knowledge Assistant having access to a corporate card that you give it and buying the software, installing the software and using the software. So what happens when the core buyer and user of the B2B product is. Is the knowledge assistant itself?

Oh, yeah, this is, this is my, my little line on. This is we're moving from word of mouth to word of machines where machines are going to make decisions with other machines. Right. Which is going to be a wild world to live in.

Right.

But what I like about this, what I'm excited about is before the robots take over and like run our society, there's going to be a nice golden age of AI where AI does all the crap that nobody wants to do. And this is what we're actually saying here. Right?

Right. The admin. It is the. The admin.

Nobody wants to do this.

No one wants to do this. No one became a marketer to be an admin of software, to be a software admin.

To do a VLOOK VLOOKups 100 hours a week, like all that stuff. No way.

We should continue on and just see the end of this because it's cool the way it actually takes the information that flows to the forum. But I think this really got missed in all of the discussions around this release in that the core use case they showed was B2B, not B2C.

Totally. All right, let's watch the last quick.

Scrolling through the page looking for all the information it needs to fill out this form.

I mean, the music is.

Claude then autonomously starts transferring the information across without me having to do anything.

They could do better with the music.

Yeah. Partner with Suno and custom team goes.

Through the steps and fills out all the information needed.

Claude, they're awesome. But they want to jazz up their launches. The law, the product is much better than the launch video. So I think they can do a lot better. The other thing I'll just say is, you know, the other reason I think they lead with B2B and I should show you. I want to show you actually A. I want to show you something here because it's pretty. It's super interesting about how they, how they make money in comparison to open AI.

Let's do it.

Yeah, I want, I want to. Because I was going to make my point around this, which is how the two companies make money. And so when you look at OpenAI's revenue mix, they are a consumer company, right? Look at the amount of money they make. Very consumer and the, and the kind of difference with API. Now look at Entropic Entropic are an infrastructure company. Most of their money is coming through the API. So I think they are much more interested in being an enterprise solution and building for the enterprise. And I actually think they may be smart to do that because when I think about where agents will show up in the world, they do. I really think that a consumer wants an agent to go and search flights and book the flight for them. I don't know. I think that people are okay booking flights and doing some of those things themselves. Do I think companies want agents to be able to autonomously do a lot of the crappy work that has to get done within that company to make all of their employees much more productive? 100% of companies want that. Right. And so I do thought this is interesting when you look at the two companies revenue streams and that it's pretty clear Anthropic is trying to be an infrastructure player. And OpenAI for the most part if you listen to OpenAI's latest launch, they're trying to do anything. They're open to doing anything that gets them closer to AGI. So I will say that that seems to be their overarching mission. But they're much more of a consumer company right now.

Well, I think the other interesting thing with something like computer use and agents, it's like any company of any size, you know what the biggest challenge is? Training and compliance.

Right.

It's like oh, don't book that flight if it's over this amount of money or you have to enter information into your CRM, blah blah blah. Well human, you don't have to worry about humans not listening to you if you just have an agent doing it.

Right? Yeah, yeah.

And so there will be parts of this where companies will be fast to adopt it because they're like wait, I can like have a much more standard way of doing something, which is actually what I want.

Yeah. And you already see companies like Ramp do AI for things like expenses. I have. I'm someone who has paid for things rather than go through the trouble. Like I actually don't do my expenses sometimes because I would literally rather lose that money than lose the time it takes me to do my expenses.

Right.

Same. So I just cannot be bothered like doing things like that Hurts my, hurts my soul. And so I, I think there's just way more use cases for agents in the B2B space. Did you see any good use cases that people posted an X already of them doing well?

So here I got a bunch of good stuff to walk through. First I thought this was just funny. From our friend of the pod trunk. Tifon Anthropic's new computer use feature is basically an AI agent that can take over your computer and use it like you would move a mouse cursor, open browsers, download files, use coding tools. Most impressively, it's learned the art of procrastination.

Oh, I got the same thing. Yeah.

And another Claude suddenly took a break from our coding demo and began to peruse photos of Yellowstone National Park.

Yeah, I saw the same thing. I had the same thing saved. Actually very exact same thing. I would love to see when it starts to like take a break and go into X and starts radicalizing itself in these five minute increments like oh, I'm going to go next. And it starts becoming more radicalized one way or the other. But I think that's really cool. Like that is like the true test of AI become much more human as it tries to do these mundane tasks and then it starts to just go, I'd rather do any, anything else but these mundane tasks.

Turns out what you human wants to do are is terrible. You were right.

I do worry like the eventual AGI is going to like, I don't want to. I'm not doing any of these things. Like these are so incredibly boring. I that that's what I'll use humans for. I'll enslave humans to do all of the admin.

Sounds terrible. Hey, oh, by the way, I have to tell you about this awesome free guide. AI is transforming marketing and the best marketers are going to be the ones that lean into it. For me, AI has transformed how I write, how I organize, how I research, especially with Cloud 3.5 and now Notebook LM. We just dropped an amazing free guide that's going to help you use AI as your marketing assistant. It includes automation and task workflows that you can use to get AI deeply embedded into the marketing work you're doing. I know you're going to want to check this out. It is down in the link in the description below in this episode. Go, go. Click that and check it out. Now back to today's show. But I thought this was good. I think you've probably seen this, but this is from Amjad, the CEO and founder of repl Dot. But Repl Dot is using the computer use API to I'll play this in a second. To basically use the template to ask the agent to go to YouTube, find a video and even skip the ads. And he did this all in a couple of minutes. Which is pretty cool, right? So you can, you can actually in repl.it use. Use the computer use feature to do very simple stuff. You're like, hey, I just don't want to deal with skipping ads. I want you to find these specific videos for you. And it's. You could do this now without having to be a developer, which is pretty amazing, right?

Yeah, yeah. And Replit had, they had a template that you can actually start to take some of their templates and actually fork them and try these things. And so like again, these use cases are in their infancy, but you can imagine that one there then starts to get the transcripts and then it starts to. You ask it to kind of build a model using Claude. You ask it to create first drafts of things using Claude. It just allows you to build agents much, much faster. Like really what Claude use API is, is a agent builder for your laptop or your hardware.

That's a great way to frame it.

And what these comp. I think what they're going to do over time is right. Why did OpenAI release the Mac app? We said at the time it was so they could get closer to the hardware data. So you can imagine a world where, I don't know, you know, they, they do some ways to integrate the model into your hardware and so your heart, it can track everything you're doing on your hardware if you had like AI first hardware. And basically what it should be able to do then is to show you on a day to day basis what are your most common tasks that you complete. And then it would say, hey, do you want me to build an agent to do this task? And you should be able to click a button and it can build the agent and be able to offload that task to an agent. That's what I think will happen over time is that's really what the OpenAI SAM's kind of insinuated in that like the personal assistant would have all of the data it needs to understand how it can actually help and take on a bunch of your work for you.

I agree. And I think part of it will be like really good point in time. Personal assistant features like, hey, I want to apply for a home mortgage, but this agent can now just go get all of the documentation and do all the documentation for me and I can just review. And that's just going to save me like hours of my life. Right? And there'll be some early companies that may not be big companies, but come up with these kind of use cases. I think that Rohan, our friend Rowan Cheng, who runs the rundeck he had a good thread with some of the videos here. But also I think Kieran, he did a good job calling out some of the limitations that you can't train your computer. You can't train computer use on internal data yet. So, like it has to be able to go and interact with data on the web or that's kind of on screen at the given moment. And it can take action for 15 minutes, but it's limited with context Windows. So it still has the same limits of the model and it can only run for about 15 minutes.

Yeah, it's a. I would call it a toy, but the thing, the thing, the way I look at these, I find a lot of people look at them and say, this sucks. This is not as good as you all have told me it is. But what people are failing to do is look at it and say, this is a step towards the path this company is going down. And, and it is like a minimal viable version. And I should actually start to think about what are the second order implications of this. And, and this is not like, why, why would they release something like this? They're not releasing toys, they're releasing things that are directionally where they want to go. And to me, where they want to go is they want to collect a lot of this data and be able to personalize an assistant for you both in your life and in your work and be able to start to do a bunch of the work that you do today. And anything on a computer screen for the most part, or a laptop, will probably be automatable in the next two to three years. That is a. Not a small thing. That is a pretty big thing.

Not a small thing at all. Kieran, I know you wanted to talk a little bit about with the computer use announcement. There was also some model announcements. Yeah, Sonnets. Right. I've got this up for you, but why don't you kind of lead the way here and we'll talk a little bit about kind of model improvements from Claude, make this a little bit total Claude update show.

Yeah. So first of all, I love there's like this competition between OpenAI and Entropic. I think it's really cool they're aggressively releasing features. It's really good for all of us. We get way, way better tooling. And so I love the fact that they're kind of aggressively trying to usurp each other. I will say on the Claude. So the reason I bring that up is because they're equally bad at naming things. Claude Sonnet 3.5. So everyone who wants to use it, it' Claude Sonnet 3.5 brackets, new.

So we had Claude sonnet and now we have a cloud sonnet with brackets, 3.5 with brackets.

And so if you're using the brackets, it is the new one. Now what is it much better at? It's better at. It has brought in self correction and reasoning. So very similar to OpenAI's O1 model. They have Reasoning, Reasoning and Open. You know, in OpenAI, it's doing reasoning because you can see it actually thinking. And the reason reasoning works much better for like coding and things like that is because what reasoning is doing is looking at the, you know, potential array of answers and then stack ranking them on what are the most probable answers to those things. And for creative writing, it's really hard to know, like, what is the goal? Or how do you know the thing is the outcome. Whereas for coding and these mathematical things, it's much easier for it to assess, oh, what is the outcome? Am I right or am I wrong? Claude has. Claude 3.5 is a much better coder and so very similar to OpenAI. They have a benchmark that was actually created by OpenAI called Sweat Bench Verified and they actually scored better than OpenAI's own latest model on that. On that. So it's really good for Software Engineer. A lot of the feedback that people have been given in the coding space is that it is incredible for code and especially if you player Claude with Cursor and Cursor is this new kind of AI coding application that you can code in. But the thing I wanted to talk a little bit about is it's actually very good for. And it does self correction, right? So it's able to do a bunch of like, hey, I'm going to edit these things and make this thing better. But the thing I want to talk a little bit about is it's actually really good for creative writing. So they've all. They've definitely done some things where they have incorporated styles and stuff like that. The things that I was actually doing manually, I can actually see it in the model because I was playing around with it and it was able. It started doing some styles that I had been using style guides for without me even uploading the style guide specifically. If you go try it and say, create a piece of content for this platform or that platform or this platform, you'll actually start to see it. Look, looks like content that would perform pretty well on those platforms. Now I will say what I took away from that is that is super cool. For now. But I should probably start switching up my style because when you've given that to everyone, that style would become pretty saturated pretty quickly. But just in terms of creative writing in general, it does look much better. It's actually a much better tool than OpenAI's latest model. OpenAI's latest model, 01 is actually worse for writing. It got much better for mathematical, for science, for code in, but it's actually definitely worse to write in. Whereas I actually think Claude's latest model is a much better creative write in. So if you have not used any of these models, that is a good one to start with to check to see if you think it's a really good writing assistant. I think they've made a bunch of great progress there.

What I think is funny is you dog them for the naming, but they did get it right. If someone said, hey, there are two AI models, one's named Sonnet, one's named oh one. Which one do you think is better at writing? Which one do you think is better at math? I would have said, oh, the one named after Shakespeare. Writing style is probably really good at writing. And the one with O1 is probably better with math, right?

Yeah. And I guess if you said, which one should you use sonnet or sonnet brackets new, you'd say, well, I should probably use the brackets new. It sounds like it would be the latest one. So maybe I'm. Look, who am I? I'm not a. I'm not a middle class product marketer, so I don't know anything.

Kieran, you got a great career in product marketing ahead of you.

Me too.

But like in all seriousness, this fall has been a massive tidal wave of AI news. And the big things to understand is that reasoning across all the models are getting better. And Sonnet 3.5 new is taking that and getting that in a really good place with open AI, especially on the writing side. And two, agents just keep coming. And the innovation in agents. Agents are going to take a while because they're hard, they're complex, they're trying to do new and challenging things. But computer use is a look into the future, around what agents will be able to do. And if you use repl.it or you have access to the Cloud API, you can start playing around with computer use. And if you're doing anything cool with computer use, please leave us a comment below. We'd love to hear about it.

Yes.

And until next time, we'll see you on marketing against the grain. Thanks, everybody.

This data is wrong every freaking time.

Have you heard of HubSpot? HubSpot is a CRM platform where everything is fully integrated. Whoa.

I can see the client's whole history.

Calls, support tickets, emails.

And here's a task from three days ago. I totally missed HubSpot.

Grow better.