Gemini implications, Bard vs ChatGPT, SI AI Scandal (AI Corner Dec 2023 Week 2)

All right, so we are going to talk about— here's our AI corner. So I'm going to start it off first and then we're going to go back and forth. So first and foremost, what I find interesting here, this is a big one. Gemini was just released earlier this week as of this recording. And the way Gemini is being marketed is it's stronger than GPT-4. It's multimodal, which means it can track— it's able to look at images, look at audio, it can look at video. And I watched some of the videos. I don't know if you've probably seen some of them too, Neil. But for example, with podcasts, yeah, they had a fake demo, but with podcasts, you can go back and if I'm listening to a podcast in Hindi, for example, it can actually be, I can say, hey, can you translate this for me? I don't understand it. So there's a lot of implications for creators as well because now maybe it's real-time translations that can happen. And I can see even a higher proliferation of content using Gemini. The thing is, it's still very early days right now. The demo that they showed is kind of fake. There's a lot you can do with it. I can show you 2 balls of yarn, for example, and say, hey, here's one in pink, here's one in green. What can I do with this? And it'll show you the different things you can make with it and give you a picture of it. And then you can have conversations with it in real time. And you can even plan your kid's birthday party for you as well, which I think will be very helpful for Neil. So, I think it's exciting. I think this is a good discussion for us to have here in terms of implications.

I think right now what's happening is you got Bard, you got ChatGPT. Grok is getting traction. I'm not sure how good or bad it is. We'll start running tests on it. But the reason Grok's getting traction is because it's Elon. At least people are talking about it. And it's going to be a crazy race to see who has better AI. But I think who's the winner right now won't necessarily be the winner 3, 4, or 5 years from now.

Yeah.

Because there's just so much crazy advancement that's going to happen in the next 3, 4, or 5 years.

Here's what I think. I think the 10 blue links that we're used to for the last— we've been used to for the last 30 years or so, I think that's going to change. Not immediately. I think it's going to start to change quite a bit. They're going to need to figure out because here's the thing. I don't know about you, Neil. When I find myself using Bard and ChatGPT more to ask questions because I have full-on conversations, right? Remember that thing I showed you about the private jet? with the, um, the, that, that tweet, right? And I was like, explain, I was like, explain this in a full in-depth post, right? And it wrote it out all in depth. It explained everything, you know, and I'm just like, man, um, I think it did a really good job. Like last week I was asking about like put options and things like that, right? And it's hard to do it in Google. It's just the user experience is easier when it comes to just working in a chat. And I think Google will figure out Are they going to add links in there? Are they going to add like ads in there? Like, I don't know how it's going to look, but you can bet that I think they'll get there.

But we're not seeing— we have a lot of analytics data and Search Console data, over a million websites. We're not seeing search traffic or impressions, you know, from Google really go down.

Too early right now.

But it's not that. What we're seeing is more people typing in longer tail stuff. The thing you would use ChatGPT for is not what you would end up using Google for in many cases. I think what Google's trying to do with Bard is get it so what you would use ChatGPT for, you would also use Google for.

I just think the experience is going to change. It's too old. I just think it has to change at some point. It's too risky for them to change it out the gate, like as a company, but something's got to give.

I don't know.

So hey, by the way, we'll come back to this clip in like 10 years and we'll see who's right.

No, but they've been changing the experience of Google a lot. Yes, it's still search, but they've been slowly tweaking it based on user research. If you look at what you're using ChatGPT for versus Bard, you can't have a chat-like interface for everything. If you're just looking for, you know, red running shoes, that ChatGPT interface is terrible.

Is it?

Yes. Try going to figure out what type of running shoes or buy— try to buy something based on doing research on ChatGPT versus a Google search. We've tested this a lot. You get your answer on what you're looking for way faster in Google if you're just trying to buy something like transactional. Than ChatGPT. On the flip side, if you have detailed in-depth questions, I'm not talking about long-tail search phrases that are 2, 3, 4 words, technically 2 words wouldn't be, but like 4 or 5 word searches. I'm talking about, you know, when people use Google, typically they're searching for 1 word phrases, 2 word phrases, 3, 4, 5, 6, et cetera. But it's not really going to like 10, 12, and/or whole sentences. And what we're seeing is the people who are using ChatGPT and Bard, They're typing in really in-depth questions or prompts to get a specific answer. And that's a very different use case than Google. But what we're also seeing at the same time now, because again, we have a lot of Search Console data, people are starting to type in longer tail searches within Google that we've never seen before. Like we're seeing people type in sentences and this was very rare years ago. We have data that goes back a long time.

They're not mutually exclusive, right? So I would say, sure, I do believe that people are probably typing in longer queries, but I'm just looking at the experience right now in Bard. Like, it's going to get better over time. And my base case is this is going to change a lot in the next 5 to 10 years, not immediately, but it has to change. Like, it's just, it's time to change. So that's where I'm coming from.

Yeah, and I agree with you in which search is going to change because for a lot of queries, just like when you type in, What's the weather in Los Angeles, California? I don't care to go to weather.com, just show me the weather. And I think for certain queries, the interface is going to change because it's not transactional. People just want the answer. It's a very different type of query. But on the flip side, if someone wants to buy a red bike, you know, I don't think the ChatGPT interface, and we've done a lot of tests on this, it just takes longer to get what you're looking for.

Yeah, like I said, it's too early right now and I, my base case is still it's going to change. I don't disagree that, hey, the data is people are typing longer queries. Also, their search generative experience, right? People are following up with it as well. But still, it's like Brad Gerstner put it in a real good way. So he has a crossover fund called Ultimator Capital, and he's like, look, the monopoly that Google has had in the past, it will no longer be a monopoly. We're talking in the next 5 to 10 years. So that's where I'm coming from. I think there's a lot to be excited about with Gemini. I'm just looking at it, right? Like there's, there's, okay, for example, here's another one, Neil. If Gemini over like a lunch break can do, it can look at 200,000 scientific papers and it will be able to tell you like, hey, here are the key insights and it'll help you. You can have a graph, it'll help you extend the graph too based on the data. So there's gonna be some cool stuff that you can pull there.

Yes, but what you're talking about, because you mentioned how you think—

I've moved on.

Yeah, but that itself is a very different use case from what most people use search for.

Yeah, but I'm not talking about the same thing. I've moved on.

You know what I mean? No, no, I get that. But I'm trying to tell people this is why I think both will exist just in different formats. And I do think whether it's Gemini or Grok or ChatGPT, you know, it's going to— they're going to continually level up AI and make it more better for end users. You know, Peter Thiel talks about last mover advantage. Google wasn't the first search engine, right? There was a lot of others before them, but yet they're the biggest. Apple wasn't the first cell phone maker out there. If you look at Facebook, they weren't the first social network, right? TikTok wasn't the first one either. And a lot of people are last mover's advantage. They learn from everyone else and they figure out how to make it better and adapt to what people really want and need.

Yeah, let me be real, really clear from my side here. So I think 10 blue links are not going to go away completely. I think there's going to be less and less use cases and I think it's going to adapt. Like, look, you know, Apple's got the Vision Pro coming. Like, that's still like a 5 to 10 year interface in the future, right? ChatGPT, I do agree it's not for everything, but I think we're going to get more used to that UI, right? It's just, you know, things change over time. So Neil, let's move over to your topic. What do you got?

10 blue links. A lot of search queries actually don't have 10 blue links anymore too.

You get what I mean? It's just like a bunch of links.

No, but I'm saying they're already moving to what you're saying, right? Like they're already adapting to what you're going towards in which they're adapting to what users want. And that won't always mean 10 blue links. They slowly have been starting that for years and you're going to see more and more of it just like you're saying.

Got it. All right. What do you got, Neil?

So we did a—

your Twitter one?

We did an interesting test on ChatGPT versus Bard to see which one produces better content.

I'm going to pull it up.

And we analyzed more than just that. We ended up analyzing duplicate content first, and then we started asking people more and more questions. So in essence, what we did is we took 1,000 prompts and we ended up putting it into ChatGPT to have them create content around those topics. We did the same thing for Bard. Of course, you adjust it based on what Bard prefers because the interface, it's a little bit different on when you're doing prompts for ChatGPT versus Bard. But in essence, we had the same 1,000 articles produced. And what we ended up finding is, is we took the articles, we ended up running it through Copyscape, right? And we did this on what I call the private internet. We didn't just publish it all to the web, even though you can get it on a live URL. We weren't trying to do this for SEO traffic. But what we found is when we ran it through Copyscape, Bard had 85.1% unique articles, ChatGPT had 81.4%, and the duplicate content on ChatGPT was 18.6% versus Bard being 14.9%. In most cases, it was only a sentence or 2 that was duplicate. And in some cases it was, you know, a paragraph, but that was more so rare. And then we started asking people, which one do you prefer? So we asked individual people to read the article from ChatGPT and then we showed the counterpart from Bard. Now we did it in both orders. Sometimes it was the ChatGPT one first or the Bard one first. And we also didn't tell them that it was written by AI. So they didn't know it was actually written by ChatGPT or Bard. And when we ran this, And, you know, Bard won 188 times out of 249, and that means ChatGPT won 61 out of 249 times. In other words, people prefer the content from Bard much more than they're preferring the content from ChatGPT.

Neil, what do you think the takeaway is for this for people?

The big takeaway is to test multiple platforms. Because just because ChatGPT is getting all the buzz doesn't mean that Bard can't be better or even Grok. And who knows what's going to happen 6 months from now or 5 years from now or even 3 months from now? You got to continually test because one platform could get better than the other. All these guys are really, really wealthy, not rich, wealthy. You know, I define wealthy as someone can end up buying a sports team, And it doesn't work out, and you can just keep.

Wait, wait, wait! Who are you talking about here?

I'm talking about the corporations: Baller, Twitter, you know, or Elon, Bard, you know, Microsoft being ChatGPT. But if you look at these organizations, they're like extremely wealthy, right? And I know some of them are owned by individuals, but like it doesn't matter. They can blow ten billion dollars and still have more than enough money to do whatever they want with. And for that reason. you could bet that they're going to do whatever to try to create the best system. And it's not about the cash, it's about winning for them. So keep trying and iterating. But the big lesson, you know, from all of this, and Eric has talked about this and I talk about this, it doesn't matter if you use ChatGPT or Bard or anything, make sure you continually modify the content and have a human in there because then you're adding the E, experience, expertise, authority, and trust. And that's what makes the content really shine.

Yeah, actually, I mean, the human in the loop thing cannot go understated enough. I think people are over-automating right now and they're losing the human aspect, right? And so we're not saying, hey, we're not being speciesist or anything like that. It's just like, it just works better at the end of the day. And you know, it is what it is. Anything else on this one, Neil?

The last thing I want to mention when we're talking about AI is imagine yourself reading CNN, BBC, New York Times, et cetera. And now imagine reading those sites, but all the content is written by AI. Some of it you're going to like, you're going to have no issues with it, but AI is creating a lot of regurgitated content based on what's already out there in the web, at least in its current form today. And when you want to just put tons of AI content in your website without modifying it, you're just going to have a lot of crap content. And in our last episode, Eric ended up showcasing the stats on Single Grain on how he's generated over 17,000 clicks in a matter of months. Through AI, which is really impressive. But Eric calls it AI-assisted content. He's not just letting AI just go free and create the content without human intervention. It's not good enough right now to really shine and get you tons of traffic without the human intervention.

Dude, okay, let's move to this next one over here. So, and by the way, that was a little experiment that we were running, right? And now we're looking at tripling it down. So the way you want to look at experiments is like, hey, if the graph is starting to go a little parabolic, then maybe you should double and triple down on it, which is what we're trying Neil, did you hear about this, the Sports Illustrated AI scandal?

No.

Okay, let me tell you. I was just reading this. The amount of useless stuff you guys do is staggering. So basically, they canned 2 executives. I think one's like a president and someone's like pretty high up there, but they were doing some stuff. Okay, so look, one week after Sports Illustrated suffered an embarrassing artificial intelligence scandal, the latest leader of the company that operates SI laid down the law for his new employees. So you guys do a lot of useless stuff, stop doing dumb stuff. PowerPoints, he declared PowerPoints are illegal, blah, blah, blah. I want to get into the AI scandal. So the AI scandal here was that there were people that were using AI to create fake articles using fake people. And then Sports Illustrated tried to blame it on somebody else. But then I think everything's still traced back to the executives at the end of the day. And so this just goes to show you that, yeah, you should have a human in the loop and don't go dumb. Don't be doing dumb stuff.

I'm confused. So what did they use the AI to do? Create like—

They used AI to create, it's like fake content. Like it wasn't even like relevant content. And then they used fake authors as well. And then it was like on the Sports Illustrated domain.

Fake sports content, right? Not just random content to get SEO traffic. You're talking about like fake sports content, they're making up news using random authors just to get traffic.

Correct. Here, let me get a little more details here. So, the browser blocked. Okay. So I'm reading it and—

That's going to hurt their brand.

Yeah. Well, their stock dropped 20%.

I didn't even know they're publicly traded.

Yeah. Well, you know right now. Okay. So, oh my God. CNN is like the worst reporting. I'm trying to pull from that website. It doesn't work. But anyway, you guys get the point. Um, these, these publ— this is another thing with publishers. They all suck at getting to the point. And so therefore, we're just gonna end this episode. So please don't forget to rate, review, and subscribe. This helps us grow, and we'll see you tomorrow.

Bye!