Automating ALL Routine Tasks with Antrophic's Computer Use

Have you ever found yourself wishing, why can't work just do itself? Well, now it can with AI. Just imagine having your own AI assistant that can manage your spreadsheets, do all of your bookkeeping, as well as respond to emails and even test your new applications for you. Basically, this new AI tool will allow you to automate all routine tasks because it's able to access your computer. It's able to click and navigate to different links across your browser to perform a wide variety of tasks. And yes, you guessed it, it's Claude's new tool called Computer Use. Now let's roll into some demos to see how this works.

We're going to be going through a quick demo today. In this fictional demo, a customer, in this case the Ant Equipment Company, has come to us and asked us to fill out a Vendor request form. The data I need to fill out this form is scattered in various places on my computer. What we're going to do is ask Claude to look at the spreadsheets, check if Ant Equipment is in there, and if not, move over to the CRM and try and find some more information there. Once it has this data, Claude's going to then fill out the form for us and hopefully transfer the information across to the, the vendor form. The first thing that's going to happen is Claude's going to start taking screenshots of my screen and quickly realizes that the Ant Equipment company isn't actually in the spreadsheet. So the first thing it does is it swaps over to a CRM and searches for the company we're interested in. Wow.

So that is all computer generated. So it's a computer that's currently typing in this CRM and searching at the moment.

Luckily, we get a search match and Claude then starts scrolling through the page looking for all the information it needs to fill out this form.

So what's happening is Claude is continuously taking screenshots of your screen, and then it's able to translate that vision into text through technology such as OCR, which allows it to read the text on the page. And after it reads the text, then based on what you put the prompt to be, it's able to perform these basic routine actions.

Claude then autonomously starts transferring the information across without me having to do anything. And goes through the steps and fills out all the information needed and then submits the form. This example is representative of a lot of drudge work that people have to do. This is available in the API. We're excited for people to try it.

That was pretty impressive, wasn't it, guys? So after watching this demo, I decided to try it out myself and see if it's actually worth all of the hype. So there are 2 ways that you can download Claude use. You could download it and use it on your local computer, but did you know you could actually also use it on your browser? So if you download it through and use it on your computer, it is quite a hassle. You need to go through your terminal, you need to download Docker, so you do have to be somewhat tech savvy to use it. But you can use it on your desktop with just a few clicks. So the first thing that you need to do to get Quad Computer Use running on your computer is to fork this particular Replit template, and I'll include a link to this in the descriptions below. So basically, once you click on Use This Template, give it a name. I'm going to put test after because I already forked this template before. Then you can choose whether you wanted to make it private or public. I'm going to use mine privately. And after forking this template, I just had to wait a couple of seconds. Boom! You can see that now I have Anthropic Computer Use now inside of my Replit account, which allows me to run this on my browser instead of going through all of the hassle of downloading it onto my computer, going through terminal, going through Docker. So the next thing that we need to do to get this working is we need to add our Anthropic API key. So Anthropic is the parent company that developed Claude AI. So to get your API key, you just want to go over to your Anthropic dashboard, and the website is console.anthropic.com/dashboard. And you will see all of these options that are presented to you. You want to go ahead and click on Get API Key. So I'm going to create a new key, and in this case, I will call it Test YouTube Key, which I'm going to delete afterwards. Then you can select your workspace. For me, I put all my keys in Default. Then I'm going to press Add. So now I'm going to copy this key and go back to my Replit account, and now I need to put in this key. So when we look at the instructions, we can see that we have to enter the API key in the Secrets pane. And we can find the Secrets pane right here, right underneath Tools, we'll see a section called Secrets. And that's what we want to click on. And we will see that the API key value is empty. And this is where we put in the API key that we have just generated, then we're going to press Add Secrets. And it's just one more click to get Anthropic Computer Use working, we just had to press on the big green button here at the top. Which is run. And now we can see that we have a browser page that is being loaded up. Okay, perfect. All right. Now that we have the browser page loaded up, we can see that there's actually 2 tabs here. One is web view and now the other is output. So what I want to do is I actually want to put the browser view on one side and the prompt that I had entered onto the other. So I'm going to drag the output over to this tab right here so I can see now that I have my prompt that I can enter in here, and I can see the browser doing this specific task. All right, so I'm going to give it 2 different tasks to do, and let's just see how well it does. All right, so let's say I have an upcoming vacation to Paris, and I want my new AI Anthropic assistant to look up the flight prices for me. Now I'm going to put that into my prompt: look up flight prices between Toronto to Paris from December December 1st to December 9th on Google Flights. Perfect. So now the agent is going to run, and what it's going to do is take screenshots of this screen that we have just uploaded. And based on what it has sensed from the screen, it can tell where the search bar is, and that tells it where to enter in the search URL and what to do next. Oh, look, it's found Google Flights now. By the way, I'm not touching my computer right now. It's doing all this work by itself. All right, so you can see here it's put in Toronto, it's put in Paris for the locations, which is correct. And I'm not touching my screen or controlling the mouse at all. It's just doing all this work by itself. Kind of mind-blowing. And what's happening on this panel on the right-hand side is that it's taking screenshots. And by reading this, what's on the screenshots, it knows what it's doing. Okay, so not bad. It's been about a minute or 2, and you can see that it did look up flights from Toronto to Paris, but it didn't get the dates right. So it looked up from December 9th to December 10th when I asked it to look up from December 1st to December 9th. So let's just tell it that it got the dates wrong. I want the flight to be from December 1st to December 9th and not December 9th to try again. All right, let's see if this now will work. So what I will say right now is that it is incredibly slow, but I can see where this tech is going. Like, this is the next evolution of AI innovations that we're seeing right here. I think in 6 to 12 months from now, this is going to be spectacular and more jobs will be on the chopping block. Like, now not only can AI write emails, crunch numbers, but it can actually navigate through your computer to do the entire task. Before, you still need humans in the loop to perform that last mile, right? So have AI write that email, but you still needed a human to copy and paste that email into a Gmail or an Outlook. But now AI can do the entire process from start to finish. Any routine task can now be automated. This is what's coming up in the next 6 to 12 months, and it honestly makes me quite scared and excited at at the same time. All right, so, and we can see here it is modifying the dates by clicking the back button over and over again. It's clicking the back button and taking a screenshot. Mind blown by what it can do, but I can also see that it is incredibly slow at the moment. So it took about a couple of minutes, but the AI was eventually able to come up with the right dates. With the current technology right now, it is still quite slow to use AI to start automating these tasks, but Like any new technology, I'm just like mind blown that it could even do this. Okay, so let's go on to the second task that I want Computer Use to complete. I have this email from my friend Mark that I want it to reply to. So let's put in our prompt as to what we want it to do. All right. So I have put in my prompt and it is please reply to Mark's email that you see opened up in my Gmail. And I'm going to put in my prompt here. The agent is going to run, it's going to take a screenshot of the screen. And that is how it's able to know what to do. It's taking screenshots every couple seconds so that it knows where to put the mouse, what to type in, etc. It's actually quite impressive that AI can now do the next mile. It's not just generative AI where it's generating text or images, it's now able to move around your mouse on your computer and actually do the last mile of completing the task that it is made to automate. It is definitely not perfect, and it is definitely very slow, but it is definitely very promising technology. Okay, so it's giving me a message here. Would you like me to draft a professional response? Yes, please draft a professional response to Mark's email. So what you can see here is I took a screenshot of the email and it says, I can see the email. Let me know how you want me to respond. Would you like me to draft a response or would you like to tell me what to reply with? I said, you can, and then I put in the prompt, yes, you can draft this professional response to Mark's email. And boom! Whoa! Holy shit! It has sent the email. Hi, Mark. Thank you for your urgent message regarding Claude's computer use test. I acknowledge the urgency of your request. I'll begin testing it right away and I'll keep you updated on the progress. Like, AI just wrote this email and replied to this email all by itself. If you're not mind-blown, I don't know what will make you mind-blown. And shout out to Mark for sending this email. Thanks, guys. Hope you give this a try. I'll include the link to the fork Anthropic Computer used in the descriptions below. As always, make sure you like and subscribe and join my free AI training, and I'll see you next time.