Does Google Gemini Stack Up To ChatGPT-4?

This is one of the biggest days in AI Google has rolled out its Gemini product. We're going to break down everything that is in Gemini, how it's changing search and what you need to know all the details and our breakdown coming up next. Breaking news in the world of AI Google. Gemini is here and it is shaking things up. It's showing us what the future of search might really look like. And, and it's not chatgpt. But even with this amazing launch, Google has missed the mark at a couple of key places. So stick around to the end of the show where we're gonna tell you what's missing from Gemini. I'm your co host Kip Bodnar, CMO at HubSpot, joined by my co host Kieran Flanagan, CMO at Zapier. And this is marketing against the great. Let's get to today's show.

Kieran.

We have real AI from Google. We have real AI from Google. It launched this week Google Gemini and it launched in three flavors. Can you tell everybody the overview of what is happening with Google Gemini, which is Google's large language model?

These are my new favorite launches, which is the surprise launch. This is my new favorite product marketing strategy is just like surprise launch things. I'm not, I don't think anyone expected it. I don't know if you saw last, I think it was last week there was this really good meme going about from Google. It was this video, whether I don't know if it was fake or not, where it was these four people on a zoom call and one of them was telling the other one, have you got my report done? I asked for it by EOD and that person is like what I thought I had until end of December, which is like five weeks time. No, like I need it today. Right. And everyone was joking. That really is how work gets done at Google. It's like hey, we need to plan about the plan, about the plan and at some point we'll do some work. Well, Google proved everyone wrong. Where they have Gemini came out yesterday. I thought the videos, the launch videos were really good. We're going to get into some snippets from those videos. The couple of big aha things for me on that and I'm not going to get into the technicalities. Like you could go read the comparison against GPT4 beats GPT4 on many different parameters. But there's a, there's like three things I took from the launch videos. First of all, this is built with multimodal ground up break down for everybody.

Watching what what multimodal from the ground up means. Because I think that is the most important part of Gemini, if you ask, right.

It means they pre train data on different modalities from the start, right? So text, so, so text, audio, video.

Audio, all images, all the different formats of information was trained from the very.

Beginning from multimodal from the very beginning. And then it was fine tuned with multimodal, multimodal data as well. And if you actually look at the launch video and we'll get into some of the snippets, the result is incredibly impressive. Now when we get our hands in it, does it look like the launch video? We had the launch video for GPT. It promised to sketch to website and that never came to fruition. But I think this multimodal from the ground up is really important because that really is the future of AI. One of the other things is like somewhat technical, but it's one of the first model to beat human experts in a bunch of tests. And why that's interesting is for the most part we thought of AI as an assistant and like GPT4 is probably a very good assistant in many different places or very good intern in many different places. But we don't think of it as an expert within those topics. Like better than the experts. Well, Gemini claims to be better or performs better in a multitude of different categories better than the experts. I think that's really important. And then the other thing is, and we'll get into these is like there's just some use cases and when you look at them you're like, wow, this industry is going to be transformed over the next two to five years. So education is one. There was a really great use case of a woman given Gemini these kind of workbooks that students get. And Gemini just like fills out all the answers and gives the logic to all of those answers and then goes back and corrects itself and gives itself a grade.

Another example I saw that's in the videos. We'll link all the videos in the description so that everybody can watch them if they haven't seen them. Is like the guy had a map, the guy from Google had a map and was like, let's play a game.

Yes.

And so the AI just started saying, all right, well this country is no. Has won the most world Cups and gave a couple other facts and the guy just touched his finger to Brazil on the map and the AI knew it was the right answer.

Yeah, let's pull that one off.

And you're like, that's one of my Favorites. This is like live on the fly. Like, you could just tell it, you want to learn stuff, tell it what you have, and it'll find a way to teach you in whatever setup you have. And that's like the education point you're making here, right?

I just want to show that one, because I think this one is when you really grasp how multimodal is completely transformative even from what we've come to expect. All right, so this is the game creation. This is the map. And so what he starts to do is like, ask to play a game, K game. And this, the AI comes up with the game, right? Guess the country, creates the game. He wants it in emojis, wants to use emojis. And then he starts to actually, like, do the emojis, guess the country. And then he points to it on.

The map to your point, you see? And then it's like. And the AI knows that is freaking amazing. And then the AI does the guess the cup game, the visual little street game where the. The ball or the paper is under the cup. They do it in real time on the video. And the AI can tell it's the left cup.

There it is. Can you. Can you imagine when we have these built into glasses and you're sitting there.

Like glasses and phones? Like, how is any casino ever going to make money, right?

Like, how do you make.

It's bananas. The amount of opportunity in the next, like five years is crazy.

It's like a known thing that poker tricksters do or like poker fraudsters. And so you're able to like, mark cards or you're able to actually provide some sort of deck of cards where you can see through, like, it's like translucent and you can kind of see through to the card. I don't know what you do when you're. You're like rocking AI glasses because you're going to be able to do so many different things that it's going to.

Be impossible to know games like poker and stuff. Casinos are basically. There's going to be no Internet connectivity in some rooms, right? They're just going to be dark rooms, essentially. It's going to. It's going to be wild. And so there's the multimodal part. The other thing that was interesting, Kieran, as. Because as we're kind of sharing everybody now the capabilities of Gemini. So a couple other things I wanted to share is a coder. Scott David Keefe said that basically Bard, Google bard, yesterday, Wednesday, December 6, started incorporating the new Gemini model into Google Bard. And it's toasting GPT4 in terms of code quality and coding, which is very interesting. One of the other things that Gemini is very good at, it's really strong at like, technical issues. So both coding and it has access to a ton of scientific research, not just the text, but the graph data and everything. And because it's multimodal, it can take all that graph data and make it accessible in the model. And so he's basically just saying much higher quality code and much faster to render with Google gemini, Google Bard versus ChatGPT. So I thought this was very interesting to show some of the technical prowess of Bard versus the multimodal. We're here trying to lay out what is Bard really good at. Bard's really good at multimodal because it's from the ground up. It's really good at technical use cases, coding, scientific use cases. And Kieran, I don't know if you saw this. You ready? Did you see one of the people who was real busy coding on Google Gemini?

No.

Sergey Brin, co founder of Google. Oh. He was coding listed on the Gemini technical paper and one of the, one of the folks from the project said that he was basically there every day.

Everyone's getting.

Products are good when founders get involved.

Baby, I love it. This is what, this is how tech should work. Everyone getting back to the craft.

Everybody's. Kieran's so happy. He's like, I want to be in the details, I want to be in the craft.

This is my happy space.

This is what you're seeing with some of the best products right now. The people are deeply in the craft of the product. And so this is a Google product that has the Google, Google founders fingerprints all over it, which is very different than what's happened over the last several years with a lot of the different Google products.

Exactly. I want to give one more quick example of a use case. So I'm going to play this. It's going to be about 40, 40 or so seconds. I want our audience to just listen to it and then I'll actually describe what's going on. Because this is a really good example of how you can create incredibly powerful custom software just off the bat, like with, with Gemini as an assistant. So they, they wanted to take 200,000 new scientific search papers that had been created, look through them for some sort of some set of data because they wanted to update some charts. Right. So in the normal world, you would have to go through all of these things yourself, you would have to extract the data and then you would actually have to rebuild the charts. And that would have taken an inordinate amount of people and months and months and months and months of time.

Right.

Let me show you in this kind of little clip how she does it. So this is, first of all it's gone through, it's read all the documentation and through natural language, it's understood what is the important data to pull out of those documents. So it found 250 documents and then just pulled out the data seamlessly into this structure for her. But because Gemini is multimodal, not only can it reason about information from text.

It can also reason about figures.

So let me show you something really neat with our refresh data set. So this is the original graph that she wanted to update with this new data. This is up until 2019. Because it's multimodal, she can just easily ask it to take the data that they've retrieved and update the chart. And you can see now it goes to 2022. Now that looks like just one, you know, a before and after chart. That is, that new chart is the, is, is the result of 200,000 papers being researched, the data extracted being normalized and then actually the chart being rebuilt. That took her a couple of queries.

Well it, first of all, it's a mind blowing use case and it shows how much better access to information we can have in the future. The other thing it really shows you is that while this model is multimodal like we've been talking about, this model is also a coder by default, right? Like it didn't redraw the graph, it wrote the code to plot the graph. Right. It didn't regenerate an image it was obsessed about. It wrote the actual code. And so because it has that code, you could redraw that graph by just the other way, by having the AI edit the code and you could have 20 derivations of that graph if it was something important. Right? And like that is what's crazy and it's going to cause the whole world to go into stupid deep dives on everything. But that's a, that's a show for another day, you know. But like the instantaneous access to like up to the date perspective on data. And an issue is wild, right?

There is just no limit to the, to how we can use knowledge. Like with, with AI, like used to be the ability to retrieve the knowledge and was like probably, probably one of the biggest barriers. And then actually some people struggle with like how do I use that knowledge? Right. But actually the retrieval of that knowledge in a way that you. That is actionable, that you can actually use it. Even with search engines, it's still quite hard. This makes knowledge much more searchable and much more actionable. And I think because of that, we're going to see some really incredible use cases that we were not able to see in the past kind of get spun out of it.

I agree. And so there's. There's a couple other things. In just a second, we're going to move on to the future search. And I got a video I want to show, but there's a couple other things that I want to show beforehand. One point that I want to make here that I think is really important for everybody who's watching the show to understand. And this is a retweet from the legendary Robert Scoble on Twitter. But all of Gemini was built on Google's homegrown TPU chips, so they don't need Nvidia's GPU chips. And so there's a whole different vertical integration that Google is running, then Microsoft and OpenAI is running. Nvidia is not the stopgap for Google, which is very, very interesting. And it shows, like, why this is important is because this is a big company advantage that Google's actually using. Like, hey, we're one of the biggest companies in the world. We have a lot of capital, we have a lot of technical experts, and we have built this from the ground up. And so this Gemini is not just an awesome AI innovation. The whole stack Gemini is built on is it amazing cloud computing and cloud AI story. And I think you're gonna see a lot of things coming from Google's new hyper AI, hyper computing capabilities. I thought this is a very important point to just punctuate here before we go on.

This is why Sam Altman likely wanted to start his own chip company. Right. That's the company that's been rumored he's wanted to start with. Johnny Ives was one of the rumors of why he actually got into a bunch of trouble with the board. But it makes a ton of sense that you want to own the full vertical stack.

The last thing about Gemini that we want we need to cover before we talk about search, Kieran, is that it comes in three different flavors. Right, Right. And so this is a tweet from our friend Rowan Chung. But there's a lot of tweets, but this is kind of the core Gemini image going around. So they have their ultra version of Gemini, which is their big enterprise full API model, their Pro, which is their more shrunken down kind of basic use case model. And then the thing that I think is the most interesting, Kieran is this nano model which can run locally on your devices like your phone and doesn't need connectivity to the Internet. And because it's a small scale Dell model to run kind of on edge devices, it's going to be more focused but it's going to be able to do very powerful, focused use cases without having a ton of Internet connectivity, reliance, latency, all of those things. And so I think the last thing for everybody to understand about Gemini, it's coming in these three very different flavors.

Right. Some great product market in there, pricing and packaging.

It is from the product marketing with bad branding which we're going to get to in a minute. We there's a lot happening with Gemini. We had to break it down for you. But let's get to the real, let's get to the real topic of today's show, which is the future, the definition. If you're watching this show, you get a lot of traffic from Google search engine and you are like, what the heck is going to happen to my Google traffic? I think the most important thing that needs to happen is that this is the must watch video of this entire show that we're sharing as well as of the entire Gemini announcement. And so this was shared by Paul Rotzer, who friend of the show, runs a big AI research company. And this is from Google AI. This is their video on the future of search. So let's roll. And so what we're showing you is what the next generation of Google search could look like. It is generating a custom experience based on the prompt.

It's a bespoke interface to help me explore ideas. It's got lots of ideas, it's visually rich, it's interactable. Now, none of this was coded up. It was all generated by Gemini. Gemini uses a series of reasoning steps, going from broad decisions to increasingly high resolution of reasoning, finally getting to code and data. First, Gemini considers does it even need an ui? Is a text prompt best?

Okay.

This is a complex request that needs lots of information to be presented in an organized way. Gemini then tries to understand if it knows enough to help. There is a lot of ambiguity. I didn't see what my daughter's interests are or what kind of a party I wanted. So it had asked a clarifying question. When I said we're thinking about an outdoor party and my daughter loves animals, Gemini reasoned it had enough information to proceed. But it made a note that there was still ambiguity about what kind of animals.

So I think there's, there's three core things in that clip, right? When we first saw ChatGPT way back when and if you don't believe us, go all the way back to that very first show. There's a couple of things we call that or three things that I really, really resonated with me my first experience with it. And I think same to you because we, we talked about in the show, we said that this was a better search experience. Even though at the time it was clunky, it was hallucinating, but you could just tell the user experience was going to be orders of magnitude better long term for a large portion of search. It was going to allow search to be multimodal, right? Again, back then we were promised to multimodal. We didn't get it from the launch video. You could see the sketch the website. And then we said though, I think the future of all of this is AI will build one of one of apps, right? Like it would actually build you an app.

Look, hold on, we have to take a big victory lap.

Here we go, big victory lap.

Like we've got a bunch of stuff wrong. We said the biggest, most formative aspect of AI was going to be its ability to code build disposable one off bespoke web experiences to solve problems, right? And we are seeing it come to life in this, this demo video from Karen. Like this is the first real kind of like, oh crap. Like this is the future type of user experience because it's a multimodal user experience. It's not that chat GPT text only stuff, right? It is like, oh cool. It's a bespoke visual UI that I can interact with via text. Like that is like nothing we've been using to search and discover information before.

If I was like sitting here and I've got my bulk of my traffic in the blue links, I'd be like, hey Kip and Kieran, what does this actually mean? And so I was thinking about this earlier and there was three things I came away with. First of all, it's changes how we actually search. It changes the actual function of us, the searcher and how we search. Because with Gemini being ground up multimodal, we are going to be trained that text is probably one part of how we search, but probably not as important as like a video or an image or all these other ways voice that we can actually search. So first of all, and then in the launch video, you saw this incredible video where he's like sketching a sketch and at three different intervals he asked like, what is this video? And it was like a circle, then it was something else and then it was a duck. Right. It's incredibly good at the, at the multimodal part. So how we search is fundamentally going to change. It's not going to be just through text, it's going to be through text, video. So sketches, voice, all of these other things. The search ux, which I think is the biggest thing is going to change. Right. We are used to getting the 10 blue links with Google plaster and a bunch of more things around it over time. But right, predominantly it's the 10 blue links with some imagery and the knowledge boxes, things like that. What this is actually doing is like inferring what is the best UX experience for you based on your results and it's coding that up for you and iterate in through code. So you may search for something and I may search for something in the future and neither of the output looks similar. So how do you ever know how to optimize for that when the AI is actually creating a one to one custom interface? I can't sit there and go, it's my keywords and my site speed. Like it's going to be so much more complex to know how it's doing that. I'll let you come in and I have one more.

Yeah, yeah. So just I want to build on that last part. Right now the onus on Google is on the searcher still. So it's like, oh, I want to figure out this thing, I search for it, I don't find the right links, I need to type in a slightly different keyword phrase or change my query a little bit and then see what the new list is. See what the new list is. And so if you're an SEO and you're a marketer, it's like, oh, I need to be relevant to that kind of core topic and I know there's gonna be all these permutations. Well now whatever you query, you're just going to get the right results for. And if you have a bad query.

Exactly.

The UI through the AI agent, through the user experience is going to basically be like, well I need this information, what's this answer? And let me clarify this. And it's going to give you the perfect result versus you having to like keep querying and querying and querying.

Exactly. That was my third is that role of the role of the actual searcher. So the first was like how we search changes, then the UX of the search experience Changes the role of the searcher changes. The searcher is like an input but not the driver of the search. They like input something. But the actual Gemini is able to infer better than the searcher what they really want and it's able to craft the prompt better. It's able to say, you need to add this, this and this for this to be a better query for me to be able to build something for you. And so the searcher becomes like an initial input into the query, but actually the query gets built in a much better fashion through Gemini and Gemini is also able to infer what you actually meant. So you may say one thing and it may return something else completely different. That's still a better match than what you thought you wanted. Right. So it's not like what we've been used to is like if we add the keyword, it will give us the thing relevant to the keyword. Now if you add the prompt, it will give you something that is a better fit for the prompt than you had thought of. Right. And again, it makes it much, much harder to know how it will do this. All that to say, like, if you listen to any AI researchers, they actually themselves don't know how AI answers. It can't predict how AI is going to answer. So how you, the marketer will ever be able to predict in the same way we were with search, through optimizing some core things, what it's going to give you back is going to be impossible.

It is going to get so much harder to forecast search traffic. That's one of the things we want to, I think, get across to everybody. It's going to be unpredictable. Kieran, I wanna call a couple shots here. You ready?

Let's do it.

I think one of the biggest factors in SEO in the future is going to be the diversity of content formats. Because we are still a very text heavy world in most businesses and most industries. And I think the future of winning search is having content across all modalities so that it's accessible to the AI, so the AI can remix it and make it part of its ui. If you just have text, you're not going to be included as much as if you have video, data, audio, imagery, et cetera. Right.

And so I. You should have a media strategy. Multimedia, you have to. Media, you have, you should have a media strategy, not a search strategy. Yes, yes, you have a media strategy, not a search strategy.

The next part of this, which I think is wild, is you have to obsess about making all of your information public. And accessible. Right. Like you're gonna have to start thinking of like, oh, what parts of the, of my business data is it possible to make public so I can get included in these results? What parts of my process, my, my technical instructions, all of those things can live maybe publicly in an easy to digest way in the way they haven't. Because it's not just about some, some headers on a page and some text and some good inbound links. Like that world is dying. Right, Right.

I agree. I think so. And I, I think the one thing I would love to ask some of the Google developers, they would never answer is how, how free are you from the shackles of having to make that custom UX experience work with or without ads.

Right.

And so I don't know if they, they've added some special things in there to say like actually build a custom ux. But there's some parameters, some guardrails that will still allow us over time to figure out how to integrate ads. Because ads is like 58% of their revenue, it's 160 plus billion dollars a year. I believe they're trying to move fast and be disruptive like the innovators dilemma disrupt themselves before others do. But I can't not think that they don't have some guardrail in there that doesn't just say build the best custom UX experience, it's build the best custom user experience within these confines.

So I watched that video and we were talking a lot about how searches can change and that could be scary. I watched that video and I thought two things. Kieran and I rewatched that search demo video like five times. It's probably my favorite rewatched demo videos ever. First of all, there's going to be new ad units that don't exist today that are going to be way more contextual and have the potential to be really powerful ad units. And there is going to be arbitrage and results opportunities for people who move fast on those. I think that is like for sure. The second thing, the same thing's going to happen on the organic side. I'm now more excited about organic search than I was before because I believe that with these custom experiences you're going to be able to just build a much better media strategy. To your point Kieran, to just be a part of queries and get in front of people that you would have never ever been in front of before. And I think that is awesome.

Yeah. And I think the big thing is, even though we're saying the death of the temple links. Google is still surfacing up third party content within that chat UX and I think over time you'll your chat, your UX experience through search will be a blend of the chat respondent and then this kind of mixture of video and imagery and other things that you can select yourself. Right here's like what I think and here's some other things you can go if you want to go further in that thing. And so to me because it's building that custom experience and trying to blend it really neatly into search, it can build some really great custom ad units because you can actually tailor them much, much better to the query because the AI can probably figure out well I'll build it in this way and this is the best way for AI ads to be integrated when I build it in this way. And I don't think like the the kind of most extreme version of search being disrupted is if everyone just does it through the natural language interface and expects the answer to come back through clear and concise one answer through a chat panel that gives no room to any brand to do anything with. But I think Google's is better for marketers, better for brands because it still keeps some of the third party publisher commitment. Like try to send traffic to your site. But I do think like what traffic? Maybe, maybe everyone gets half of the traffic they used to get. Like in any case scenario like I think it's that sizable of a difference.

And we're just talking about Google search. There's also news and all the other Google products. But we're going to spend 60 seconds in a minute to talk about what Google got wrong with this Gemini launch. But I'm going to make one wild prediction Kieran is that if we get a friendly regulatory climate Google will buy Twitter from Elon because they need that real time data. They need that if they buy it.

I don't know. Do they? I don't think so. Why like the brand of Twitter right now or access.

It's got to be a friendly regulatory climate.

So I think that's yeah but I'm more immediate.

I think a lot will I think.

The employee Google revolt they need the data. I'm just, I do agree they need.

The data prediction that I want to look just raise so I can come back to in the future.

Rock just raised the 135 or they're raising 135 million pre seed. I think Elon probably thinks he'll buy he want like if you know, if we know anything about Elon he's Probably thinking, I'll buy Google.

Well, that's, that's fair. I'm just, just putting it out there. Okay, what'd they get wrong? They got two big things wrong with this launch, Karen. The first is the branding is really confusing. You've got Google Bard and Google Gemini. What is what like? It is a weird confusion. Is it just confusing to me and it's fine for you, or do you think it's confusing?

Look, I think I give them some leeway here. I think when you were trying to move fast, what I suspect they're doing is they have autonomous teams trying to create things in silos, not being beholden to the core company, being able to launch things in front of customers. And actually because of that, it's all going to be a mismatch of brands and a mismatch of experiences. The opposite end of that is doing what everyone makes fun of Google for doing, which is trying to make everything like, gel together, taking a ton of time. And so I actually think go out there, have a worse experience than usual, make it a bit confusing, but actually find the things that truly stick and over time, like, start to tell the story in a much more coherent way. So I give them a little bit, I give them a little bit.

I'll give them a little bit of leeway with you. The one thing I won't give them leeway on, which I think is the actual big problem in this launch is that they, they do not have the same trust and in with developers as OpenAI does. The API for Google Gemini doesn't come out for another week. They're not leading with developers. And I saw a great tweet yesterday that I didn't, didn't save to show it's my fault. But the point was basically like a lot of developers are hesitant to build on Google because Google will try things and shut things down.

Right.

And everything that Google has tried where they haven't been the market leader at launch to their different social media products and everything, they've ended up shutting down. And so developers are a little bit more afraid to build on Google. And so in the short term, I do think that is a real challenge for them.

Yeah, I think they need to be much More developer first. OpenAI have done a great job there. I think developers trust OpenAI, even through all of the Sam story and kind of thrash, the actual APIs apparently were working great. So I think to really, truly win in this space, they need to be developer first. Now the API is coming out in seven days, but they could have led with the API as part of like the core launch and not make it feel like a secondary launch and I think that's what they need to do.

I completely agree. Okay, if you like the show hit, subscribe that we covered a lot of ground. If you have follow up questions on Gemini, I think we need to do a follow up show. Leave us comments on YouTube. Let us know what your questions are about Gemini, future research, Bard, et cetera. This has been crazy. We are out of time. We did this as an emergency show. We're trying to get this up as soon as possible. Thank you all. We'll see you next time on Marketing against the Drink. This data is wrong. Every freaking time. Have you heard of HubSpot? HubSpot is a CRM platform where everything is fully integrated.

Whoa.

I can see the client's whole history. Calls, support tickets, emails. And here's a test from three days ago.

I totally missed HubSpot Grow better.