NEW Google Gemini Is Here! Is It Better Than ChatGPT?

Igor Larruz tests Google's new Gemini Advanced against GPT-4. It costs $20 a month, runs faster, and scores similarly on benchmarks, but lacks code interpreter and custom assistants. The real question: does it push OpenAI to move.
Questions this video answers
Gemini Advanced (product) powered by Gemini Ultra 1.0 (model).
$20 per month in the US; €22 per month in Europe.
Yes — two months free for new subscribers (as stated in the video).
About 150 countries (video states a list of 150 countries).
gemini.google.com
Chapters
All right, so Google's Gemini Ultra, also referred to as Gemini Advanced, before that referred to as Bard, just dropped. It's the newest and best model Google has released and it is the first legit competitor to GPT4. I know the term GPT4 killer gets thrown around a lot, especially on Twitter, but legitimately, this is the first model that performs as well or according to some benchmarks, even slightly better than GPT4. Now this leaves a lot of questions open and is it actually better in practice than GPT4? How much does it cost, where is it available and are there use cases that this is better for? We're going to be looking at all that and more. I spent a few hours experimenting with this and my entire team has been scouring the Internet for all the info we could find. So this is all things Gemini Advanced. Let's dive right into it. So first things first, let's talk about the naming because it can be a little bit confusing. I think there's no debating that. So what previously used to be Google's Bard is now Gemini Advanced. Now, this is powered by The Gemini Ultra 1.0 model, which is the first version of the big model they promised back at the end of 2023. Right. Apparently it's multimodal from the ground up, but this version that we got today is not everything that we were promised. This is kind of a stripped down version to get things going, but it's still very good. So, yeah, no more Bards. This is called Gemini Advanced powered by Gemini Ultra. Now, you get access to this by subscribing to the AI premium plan by Google. The this is what OpenAI calls ChatGPT Plus. That's the plan that you need to pay for. What does it cost? $20 a month in the US, €22amonth in Europe. There's a list of 150 countries that this is available in, so it's not available absolutely everywhere. I'll leave all these links in the description below, but most of the Western world can get access to this. Now, some team members reported that they have a hard time actually getting in because so many people are signing up and there's sort of a bottleneck today. So just have a little patience. It did work for me. One caveat though is when I try to sign up with my work account, which is just linked to my Google workspace and all the AI Advantage related emails, it didn't let me, so I needed to use my private account. It apparently doesn't work with work accounts. So just be aware of that, another very important fact is that they're actually giving people two months for free. So this is something GPT4 never did. And yeah, it's their competitive advantage right now. I suppose you can just sign up, get two months for free and then cancel. Just get €40 of value and a GPT4 like model today. So that is pretty amazing. So. So yeah, signing up is pretty straightforward. Just go to gemini.google.com and then you sign in with your Google account, subscribe to the AI Premium plan, get your two free months, and that's basically it. After a few disclaimers, you arrive here, which is Gemini Advanced, very similar to GPT4. So now let's talk about how this compares. So, as you might know, on all the benchmarks, this is very comparable to GPT4. Matter of fact, in some of them it's slightly better. But at the end of the day, the benchmarks don't really matter. We're. What really matters is how this actually performs on the use cases we love. Okay, so I have a few prompts that I love to test every time. One of them is just write me an essay about penguins. Another one is generating a few images. And I just use these prompts in all large language models. So I can get a subjective feeling for this. Okay, this is not any advanced research. This is my personal feeling as a power user of this. And what I can tell you so far is it's very comparable to GPT4, but different and more user friendly. It feels more like a bike with training wheels rather than GPT4. That feels more like you need some experience to really get the most out of it. This is more user friendly, there's no doubt about it, but it lacks certain features that I really love and use all the time. Okay, so let's talk about these things one by one. So first things first, if I just run this basic prompt, write the an essay about penguins. So what you'll see is that in real time, the speed of this is actually super quick. Look, it's writing way faster than GPT4 does. It's almost writing at the speed of 3.5, which is impressive. That's a good thing. Now, what about the flavor of this text? Okay, the writing style is a little different. Okay. So at this point I developed a spidey sense for what GPT written text is. I've just seen so much of it, worked so much with it that I. I just can't tell when I see it. Right. This is a little different. Now. I don't know, I feel like my personal spidey sense for GPT4 text wouldn't pick up on this right away, but it does feel a little AI written, right? But here's the first example of what I said before is that feels a bit like a bike with training wheels, a very capable bike, but it does have the training wheels, which I guess for most users is a good thing because look, it doesn't just hop into the essay. It says here's an essay about penguins that you can use as inspiration or a starting point, right? It points out what is implicit in GPT4, what I communicated so many times that hey, this is not the final result. You just want to use this as a starting point here. It helps you out by telling you in advance. And this is the case amongst all prompts. Okay, there's always this preface of like, hey, you need to know this before I give you the answer. It's very helpful. And this gets me into Ethan Mollick's article here. If you want to learn more about the emergent behaviors of this model across various tests and examples, then he wrote a fantastic article because he had access for over a month now and this is his finding. And he points this out too. It's just more helpful. It says things like this. Does that make sense so far? Is the concept becoming a bit clearer? It's like a helpful tutor, which makes this way better for learning purposes because GPT4 is a little more like an assistant that just gives you the result and assumes you're the executive in charge. And what I conclude from this, and my short experience with it, is that this is set up to be helpful for all of Google's users, which is everybody. Okay, so this is a fantastic article. It talks about kind of the sparks of AGI that were talked about when GPT4 also came out and how Gemini Advanced has this tool like this sort of emergent property of sometimes answers coming out that are just a bit too human or that I feel like echoes of something unspoken in there. I don't know, it's really hard to word, but he makes a great point here. And yeah, I remember having that feeling too when playing with GPT4 deeply and really pushing it to limit sometimes. And you just never got that with models like GPT 3.5. But this topic is a bit esoteric. So let's get back to the practical parts if you want more on that. His article is amazing and he's one of the best for follows in the AI space would strongly recommend. So what else is New here. Well, another thing that is new is that it actually double checks the answer and it uses Google for that. Okay, so you now have this button at the bottom. You can just press G and it's searching on Google and it's going to look at this entire essay and see what if this info aligns with what it finds on Google. Okay, so as you can see, it finds different articles that confirm what is said in here. Now this is a fantastic feature, right? Like, if you're using this for research, double checking everything you find is kind of a must. Even if you prompt super well, you're usually looking at somewhere from 80 to 90% probability of all the facts actually being right. There's just always a chance of hallucination. If you eliminate all hallucination, it won't be creative anymore. It's just the inherent property of these language models. But here it uses Google search to double check this. Now, how good is this? I don't know, because we have a similar thing with Bing. You can kind of do this using Internet browsing. That doesn't work really well. I think only time will show how good this double checking is. But I really like this content concept. Now you can rate it and say if this was helpful or not. I guess it's a good feature to have. And talking about features that are in here, this interface is way richer too. Okay, so one point of view saying that's worse because you have more clutter. But the other point of view is, well, it's more user friendly. It just shows the options. Right? So these are all things that I would manually prompt all the time, right? Change the style to more casual, more professional. Make it simpler, longer, shorter. This is all things we talked about on this channel a year ago. Like varying the styles is literally one of the first videos I did in 2022. So now you just have this little dropdown where you don't need to be aware of the fact that you can do that with prompts. You can just say, make it shorter. Okay, so while that is basic, there's some functionality here which I really enjoy. So I've been using the ChatGPT app regularly for the voice input and output. It's actually really nice to kind of have that assistant in a voice interface. You can have conversations way more naturally and fluently like that. This has a built in. Okay, so I could just press this? Absolutely. Here's a shorter version of the Penguin essay and I could use voice to input. Right. Make it more concise. Nice to have. I reckon that OpenAI is going to add this shortly, as they kind of have to now. And this is the thing that I'm excited about here. A lot of these things will prompt OpenAI to take steps further ahead. So as we wrap up this video, I'll talk about the direction this product will probably be taking and the direction that OpenAI will be taking from here on out. Because, hey, we have competition now. But before we do that, let's talk about the final differences here. Here. What else should you know? Well, okay, so let's talk about the things that it doesn't have, because there's actually quite a few. One of them would be code interpreter. I use that all the time. This just doesn't exist. Now, around the release, they did mention that this will be coming later on, but as of today, it's just not available. And another thing is that the image model is very different here. Okay, so this is not Dall e free as with OpenAI. And look, it just refuses initially. Then I simplify it and remove the text. And here we have an image of a cat with a hat. Now, I dare to say that Dall e free is actually super superior in this, but I guess a lot of that is subjective. Also, it's not the most important criteria. Right? Okay, so another thing that is obviously missing here is GPTs. I really learned to enjoy the fact that I can set up these chatbots with preset instructions, knowledge bases, and actions in some cases that just do a specific thing for me. This is not a feature in here yet. Now, will this come over time? Maybe. Probably. Let's see. But as of right now, it's just not. And another point would be context length. What is the context length of this? I actually went ahead and tested it myself. Okay, so the way I did it is very simple. I first had a conversation around the token limits, and it just. Just doesn't inform me. It told me it doesn't know. It can't reveal it. I didn't find any exact information on the Internet, so I just went ahead and ran a little experiment myself. So I started by saying, my name is Igor. Now write an essay about penguins. Okay, so it did that. Then multiple times I said, another one, another one, another one. And then sometimes I edited the prompt and said, what is my name? And it always got it right until I got to a point where it just forgot my name. So, as you can see right here at the bottom of the conversation, I say, what is my name? Unfortunately, I can't determine your real name, though I told it in the beginning. So obviously here it forgot it. Right. So then I take the entire conversation went over to the OpenAI tokenizer pasted in here, and turns out 4100 tokens. This is pretty much the same as GPT4, so its memory is about the same length. Now you know too. Okay, so another thing that I tested is a more complex prompt. So this is our prompt generator. If you ever wondered what can I do with these large language models? Well, as most of you know, I put together a template with 10 different jobs and prompt generators for those jobs. Okay. Okay. You can get that for free if you sign up to our weekly newsletter. And basically what the prompt generator does in this case, it creates 20 use cases for growth hacker 20 prompt presets. Okay. So you could customize the growth hacker to anything else. And there's a bunch of other presets. We also sell the business blueprint, where we have over 1000 different assistants and GPTs and prompt generators for them. Right. So this is a very complex prompt that doesn't just use few short prompting, but also a lot of intricate prompt engineering. And what it does is it creates two 20 prompt templates. Now I compared these because I did this a lot, and I gotta say, they're equally as good. So with a complex prompt like this, it works very well, better than any other weaker model that I've seen so far. Even models like llama 70B didn't get this right. GPT4 was the only one where this worked really well. Now Gemini advance is another one. So yeah, from my point of view, I will confirm this is a GPT4 competitor. It lacks a few features here, it has some minor advantages there, but overall it's really good and you can get it for free for two months. So this is not a bad deal, no matter how you look at it. Now there's one more thing which we should point out, and that is the perks that Bard already had over GPT4 right beforehand. The model was not as good, but it did have these extensions. And I'm not talking ChatGPT plugins, which were like community developed and they usually did one thing. And let's just be real, most people didn't even use them. But here it integrates into Google flights, it integrates into Google, it integrates into Google Maps and YouTube workspaces, hotels. Now these are some of the best apps on the Internet, right? YouTube goes without saying. Maps and flights are used for travel planning all the time. So things like pulling YouTube transcripts or data from YouTube is going to be way simpler and way more fluid. It has been with bart. It's just the base model was not good enough to be used all the time. Well, that model doesn't exist anymore because now we have Gemini Ultra powering this Gemini Advanced AI. All right, so that leaves one question open, and that is, where is OpenAI heading and what is it going to do to the entire space? Well, this is my take. Yes, my turn. So OpenAI has clearly been going in the direction of autonomous agents. Okay. If you're familiar with projects like Auto GPTs and Baby AGIs, I've been talking about this since six months. How this is the clear direction that they've set with the release of GPTs and the GPT Store. Although they are in their infancy, it clearly sets the direction of, hey, we want to build autonomous agents that are going to do stuff, okay? They're going to be going around, clicking buttons, doing things. And Gemini Advanced is taking a bit of a different direction here. It's trying to be your helper, not just your assistant, but really your tutor. This is going to be there to plan your trips, to help you learn things. And as they roll out more of the multimodal functionalities, which I'm very excited about, more and more use cases will be unlocked, especially as it integrates into their native ecosystem of just incredible apps that already exist. Right? It's all going to work together. It's going to be double checked with Google Search. This is a powerful competitor for GPT4. And finally they're out. And hey, I'm down for some competition because that's going to drive OpenAI to release more cool stuff sooner as we finally have a second player that can match up to GPT4. So if you want to follow all this, but you have a busy life, well, that's why we have our weekly newsletter. Once a week I send out an email that summarizes all the new happenings and subscribe to the YouTube channel for updates like this, which go a little more into depth. And at the end of the day, I'll just be spending all my time exploring these and sharing what I find so you don't have to. All right. And if you want to learn more about this action oriented direction that OpenAI is taking, check out this video because here I teach you how to build your own agents. And literally one prompt that I created, it's pretty cool. See you there.










