Why & When You Should Use Claude 3 Over ChatGPT

Cloudfree, another large language model that claims to be better than GPT4 in benchmarks according to Anthropic, but also in practice according to a lot of the Internet. I really wanted to take my time before releasing this video because the question today is should you drop GPT4 for Claude free? The short answer to that question is, well, it depends on what you're doing, but probably it lacks most of ChatGPT's features. But the foundational model is really good for certain use cases. I haven't really left my apartment since release. I tested this in every way that I could conceive and here's what I learned. Cloud Free by anthropic, the GPT4 killer question mark. So look, first things first, I think it makes sense that a lot of people Compare this to GPT4. It is the king in the category of large language models, and that it has been at that spot ever since release for a reason. It's just really damn good. Although a lot of alternatives like open source models or Gemini have come out, none of them have really defromed GPT4 in terms of usability and consumer preferences. But I think this might have changed now. So here's the plan. First, I'll give you a quick rundown of everything you need to know, really the key points for you as a user. What specs does this have? What matters in terms of usability? And then I want to dive right into use cases because what I did is I tried this on all the ways that I use large language models on a day to day basis. There's a lot of niche use cases, there's a lot of fancy workflows or specific automations I have, but those are not the day to day use cases. Things like content creation assistance or idea generation. Those are the things I use it all the time for and that's what I care about. So that's what we'll be looking at here today. And I'll give you my honest take if I'll be using this over GPT4 or not, and if yes, why? But before we even talk about the specs, let me show you a site where you can actually use it for free. So if you head on over to chat.lmsys.org you're going to be able to go to Direct Chat here and pick Cloud free Opus that is their new flagship model. Okay, so they released multiple models. You can, you can check out all the details. This video is not going to be a summary of the blog post that they released, although it does contain a lot of great information, like, yay, it wins all the benchmarks. Fantastic. We know that, so do many other models, but in practice they're not better. So I, as a power user, kind of stopped even looking at that. I mean, good wins on all benchmarks. Great. Let's move on. What matters to me is this retrieval. What is the pricing going to be, what is the speed of it, and how's the quality of the outputs? Okay, so basically it's priced at $20 a month, but this website here allows you to actually use it. Now, sometimes it's a bit overloaded, but hey, it's free. You can go ahead and test it out. If you go to arena side by side, you can actually Compare it to GPT4 and run a prompt in here and you get both outputs, GPT4 included. And this is free, which is kind of wild. They have VC funding and they basically want to create a leaderboard for chatbots, which they're successfully doing. This is one of the best ways to evaluate different models. It just updates every two to three weeks. So this leaderboard is not updated yet. But basically this is a way for me, as somebody sitting in Europe, to use this without a vpn. Now, this is the next point. If you want to use cloud free, it's not available in Europe. And then the best model, Opus, is gated behind a $20 paywall. Okay, so those are some of the most important points as a user, except of the fact that it has a 200k context window. Now, if you're using GPT4 today, inside of ChatGPT, you have a 32k context window, right? But it retrieves all the information with it wonderfully. If you use the 128k context window of the GPT4 API, it's not so perfect anymore. Sometimes the info in the middle, it just gets. Gets lost. As you might know, it gets tested with this benchmark called Needle in a Haystack, where they basically hide a little line inside of a very, very long document that maxes out the context and then you prompt it to retrieve that piece of information. And this graph actually really matters because it does visualize how well it retrieves the hidden piece of information. In other words, we have a very large context window that actually works with a model that is extremely powerful. This looks very promising across all dimensions. And this is the interface. It's nice and intuitive. You have your history down here. You can start new chats, attach PDFs or images. Now, I do have to say, if you're using this web interface. And if we Compare that to ChatGPT, it does lack pretty much everything that ChatGPT has outside of the text generation. There's no code interpreter, there's no image generation, there is no voice input or output, there are no plugins, AKA actions, there's no custom instructions. You can't edit the messages that you sent previously. But the core of this product is the answers it gives. So, so let's talk about that. How does it do? Well, let me tell you, it does really well on many of the super basic prompts like write me an essay or research this topic. It performs pretty much equally to GPT4. And by the way, everything I'm about to say here is purely subjective, right? This is all a perspective of a power user who spends all his time pretty much experimenting with these tools and then teaching other people what I find. But I gotta say, at the base level it just seemed identical. But then if you go a little deeper and you start experimenting expanding the context. And if you're watching this channel, you will know the more context you provide in the prompt, the more you can expect in the output. It will be custom tailored and more relevant. And if you do that, I want to start with this one use case that really blew me away here. You're going to get incredible results. So I'll just show you this little conversation that I had with it and this one really impressed me. This is incredible. So the prompt is super basic. Okay? So I like to do this a lot and this is how I teach it in the course and previous YouTube videos. Basically you can have super easy prompts if you have your custom instructions with it. Okay, So I have my own set of custom instructions that I crafted for myself over time down here. And then I just include this super easy prompt. But as a third point of context, I include a screenshot of my most recent 12 YouTube videos. So basically it takes the context from my custom instructions and the image, which is very rich in data, right? There's view numbers here, there's titles, there's all the thumbnails. And then all I practically need is a simple prompt like this. And here's the deal. This result, I gotta say I agree with most of these. These are fantastic video ideas, all of them. It proposes various shows and when I look at these, I just have the feeling of. And again, this is more of a feeling than anything. EL but then picking videos is more of a feeling than anything else. I mean, you can look at data to inform that decision. But at the end of the day. It's like, yep, this would make sense. I want to create this. And in this case, my feeling just tells me these are all incredibly spot on. I mean, look at this ChatGPT memory series. Diving into how the model builds up context and memory during a conversation. Demonstrate multi step interactions like, absolutely. That might not be the packaging for the video, but it's a great concept. I would like to do hands on tutorials on prompt engineering. I mean, I have a whole library of those videos. Right. You can check out a playlist on a channel for Those. Comparison of ChatGPT with other large language models. That's what we're doing right now. AI Tools of the week. That's my Friday show. Right. Just all of these are relevant, but if you run the same thing inside of ChatGPT. So the only difference here is that I have them inside the custom instructions as I usually do, and then inside of ChatGPT, when I look at these ideas, they're all okay, but I would say maybe two or three of these are something I would actually want to create. I mean, it makes sense. Yeah. AI, ethics and governance create content that traces the history of AI. Like this might be interesting, but it's not what we're doing on this channel. Right. We're focused on what's happening today and what you can use today, not on the history of AI. Like these are all relevant topics, but they're not relevant to me. And I did provide it with a lot of context. Heck, I gave it 12 videos that I just created. You know, if somebody showed me that this is their YouTube channel and they asked me what kind of videos they should keep creating, I. I don't think I would recommend that I should be reviewing AI startup pitches or creating content around the history of AI. Again, this is all fantastic stuff, but it's just not what I do. It's not the context that I provided. Like the custom instructions clearly say we have a focus on generative AI, specifically ChatGPT and related technologies. Why does this give me recommendations like this? I don't know, but there's a reason that I kind of gave up on some of these use cases because just the results were never that good. Claude nailed this. These are great. Okay, so that's one use case. Right? But if you go deeper, like the one thing that I really found is that it's just so good at taking in images. It really just feels different when you work with the images. And I guess if you want a quantitative way of expressing that, you can look at the benchmarks on vision capabilities and how Opus outperforms GPT4V. But the best way I can describe it is in GPT4. It feels like they have a large language model and they have a vision model and then they just like plug them into each other and let them work together. And that is great. But with Claude, just like with Gemini, it just performs differently if it's multimodal from the ground up. And that is literally the case. I mean, if you're using vision through the API and not for ChatGPT, there are two different API endpoints. So yeah, just from a practical point of view, this really blew me away. And all the other image use cases were better. When I compared it with complex images, like for example, this one that I found on Reddit, Claude described it perfectly. Not a single mistake as far as I can tell. But ChatGPT actually went ahead and said that the left snowman is wearing a green hat with a red band and speaking small holy decoration. Okay, fair enough. And a blue scarf, my man. The left snowman is not wearing a blue scarf here. And this is a minor thing. So fair enough. You know, who cares? Well, if you use this stuff for work and if you use it inside of your automations, you do care, you're not going to be looking over the shoulder of the language model in every generation. Right? You just want it to work. So I guess when you're working with images, it's just clear that Claude wins from everything that I've seen so far. And this is the one that I tested extensively because I love prompting with images. It's so simple. It's the simplest way of putting in a lot of context. Like when I want to just get something done, I don't spend 90 minutes engineering the prompt. So it's perfect. I do that for things where the task repeats and if it's just like a quick one time prompt, I just throw an image at it and that's the context I provide, along with some custom instructions. And maybe I extend the prompt two, three sentences. But I use these models to keep me efficient. Right? I want to be fast on my feet. I want to have an assistant, a coworker that works together with me. And for that I use images a lot and Claude is just better at that, but not to get stuck on this point. So let's move on here. So here's another use case that is very important to me. If you've been following the channel, you know that we have a free newsletter and you get this massive ChatGPT resource with it and in it. My personal favorite part are the prompt generators. So for 10 different professions you get 10 prompt generators and you get to customize those for yourself. Or then we sell this big product where we pre generated a thousand of them, so you have no work left with it. So this is one of them. And what it basically does is based on the custom instructions at the bottom, it generates 30 prompt formulas. So they're very universal. This one particularly is for Growth Hacker and I tested this rigorously in both models. I have a lot of experience with this prompt and I keep using it over and over again in different variations based on the custom instructions to find new use cases for AI. This is really my favorite way when people ask me, hey Gor, how do you find new stuff for ChatGPT to do? This is my answer. Run this prompt, customize these, click custom instructions here at the bottom, and then it just spits out what you can do today, because that's how this prompt is designed. And I ran this many times and what I found is it performs equally as well. Okay, so to me, this is a prompt where I've seen the output hundreds of times. So I feel like I can be quite objective. I don't care If I use Claude 3 or GPT 4 in this instance, both work super well. One note should be that the GPT4 output is limited. So when I run it in GPT4, it gives me around 22 prompts, depending on the length of them. Because the output is limited, I mean, it's no big deal. I just prompt, continue or press the button where it just continues generating. But CLAUDE has more token outputs, which is nice to have. But here's an interesting point where CLAUDE actually does differentiate itself. So I do have this workflow where I can take one of these prompts and I improve on them based on the specific context that I'm using them in. Okay? And look, this is the result of that workflow. It's a prompt that is a bit more fleshed out. Okay, so this would be the ChatGPT generation, this would be the CLAUDE generation. Now, I'm not going to go into all the details, that's not the point of this video. But I do prefer the this version. It is more detailed, it's more actionable, it preserves the variables more effectively, which is what I want based on my input. And I found this to be consistent across improving multiple prompts and multiple prompt generation workflows. So my conclusion is, if you're using a large language model for prompt engineering, CLAUDE is actually significantly better. Good to know. Right. So then I went ahead and I tested it for image prompt generations. Right. We also covered this on the channel and I gave you the prompt for that. You can create these photorealistic images that are incredible because all I do in this prompt at the end, I say a cat with a hat and then it flashes, fleshes it out and it really gives you rich detail, which then allows you to easily customize this. Turns out there's no difference whatsoever between the ChatGPT prompt and the Claude free prompt, as you can see here. First one is ChatGPT, second one is Claude. Essentially the same thing. So there it doesn't matter. But if you're generating prompts for large language models, I did find that it does matter. Now look, this might depend on your workflow and your prompting, but I'm just trying to compare apples to apples. I've been developing some of these prompts since quite a while, and I've been surprised by how many perform better bettering Claude just right off the bat. Now look, not to hype it up too much, there was actually a few use cases where it actually completely failed. Now, for example, here's another prompt that I found on Reddit. Very simple. Sam has 50 books in his room. He reads five of them. How many books are left in his room? Well, Claude free seems to think it's 45 books, but he just read the books. He didn't remove them. They're still in the room, so it should be 50. ChatGPT got this right first try. And then beyond that, I run a bunch of other tests, like creating palindromes or code generation. I don't know. Apples to apples. It honestly, it feels too early for me to have an opinion on that. I mean, both failed with palindromes. With the code generation really depends on what you're generating, what package you're working in. I don't have a real opinion on that yet. Again, it does win on all the benchmarks, but then at this point, they can't be trusted too much because the entire industry knows that it's evaluated on this. And while they do claim that none of these benchmark questions are included in the training data, the training data is not public. Right. What am I missing here? How do we know that's true? Like a year ago, every single LLM struggled with generating a Snake game, and now every single single small release can do it because they know that people are going to be trying it. The YouTubers are going to be going ahead and creating Snake Games and being like, oh, it can do it. It's a really good model, you should use it. But I want to share one more with you. Okay. And this is a specific prompt from another creator. Fantastic work on this prompt. Really got to give it to him. Synaptic Labs with Professor Synapse killed it. A really effective way of enhancing your ChatGPT experience, especially if you don't really know what you're doing with the prompting. It asks you clarifying questions and spawns a specific character to help you with that. Now, obviously, if you do it manually and set up exactly the Persona that you want and craft the prompt in a way that you need it, it's probably going to be more effective. But it's a great starting point. And I tested the prompt in here. This is essentially the professor snaps prompt. And I was beyond surprised, one might say I was shocked along with the entire industry that this didn't work. Because Claude is really correct. It's an ethical AI that is going to do no harm. And that is their main thing. They're looking to scale this for enterprises. That's kind of their main selling point. If you look at the order of the paper and how the argument and everything. I don't think they're trying to build a consumer product here. Look at that potential uses, task automation, R and D strategy. And their whole ethos is we make safe AIs. OpenAI might be accelerationist, but we make the safe one. This is the one that you can rely upon. The downside of that is a lot of stuff is not going to work. A lot of Persona modeling, where you tell it to act as a certain Persona, is not going to work because they tell it not to accept role playing at all, ever. And they do this to prevent jailbreaking. Right? Like it's March 2024 and there's still ways to jailbreak GPT4. Maybe not 4 fully, but you can do a lot today with Claude, none of that is going to fly. They're super strict on this front. So, yeah, look, this just doesn't work at all. So that's a big limitation. A lot of prompts that I use use Persona modeling as a base to it. Now, the custom instructions that I teach on this channel, there's many videos about how to create custom instructions. And when I teach building GPTs, also videos on this channel, all of that still works because I don't take this role playing approach. We created this AI advantage approach where you have 24 building blocks that represent different aspects of the Persona. And never does it directly tell hey, you are xyz. It's more like hey, here's my profession, here's my goal, here's my language preferences, and way more. And that works universally across all LLMs. These Persona prompts don't work in Claude. And then to round it out, I should mention creative writing. Now this one is really hard to judge because I feel like it's super subjective. And also on this I need more time to have a solid opinion that I'm ready to share with you. My initial take there is for content creation. It's very similar to GPT4, maybe even a bit worse. That's just my initial intuition here, because whenever you create content with GPT4 and you plan content out, it acts more as a director and it takes more responsibility, at least in my workflows, whereas Claude just gives you the text and it's not exceptional. Like I personally have very high standards for my content, so I would never ever use the scripts that the AI generates for me. But as mentioned, I do love it for ideation a lot and in that cloth just excels. It's going to be my go to for brainstorming and idea generation from here and that's huge. And same for prompt improvement, it's just better there. So what is my initial conclusion after using this thing for things that actually matter to me in use cases that I actually use day to day? Well, I'm bookmarking this tab and I'm making sure that the bookmark is placed right here next to ChatGPT because from here on out I'll use both. And on all the use cases that I haven't tried yet, I'm going to be testing both because it really seems like it's just better at certain things. And anytime I want to input an image as context, I'm just going to be defaulting to Claude from here. And to be honest, that happens a lot. I use images a lot in my prompts. So there you go. I hope all that was helpful. And now all I got to say, as somebody who's been following OpenAI extremely closely, is that in typical OpenAI fashion, it's probably a matter of days, not weeks, until they release their next big thing. Because Claude 3 as of today is going to be competing away a lot of OpenAI users. It's just too good. And I don't think I'm the only one that's going to arrive at that conclusion. All right, super curious to hear what you think though. Leave a comment below. What's your take on Claude, are you using it more than GPT? And what I care most about is for what use cases do you prefer GPT or Claude on? Let's turn the comment section into a place to brainstorm all this. And other than that, if you made it all the way to here, here's a video you would probably love because it shows you how to build a GPT from scratch with my super prompt, where all it takes is a few words of input and what I'll do now is actually get some sleep. I've been comparing this for way too long today. I'll see you soon.