How ChatGPT's Search Algorithm Works, AI Recruiters vs. Humans & The Future of Traffic | Ep. 3031

Why don't we move over to how to rank highly, how ChatGPT search algorithm works. You want to know how ChatGPT's algorithm works?

I kind of already know, but yes, let's talk about that. But before we go into that, I started integrating ads into my YouTube based on your feedback.

How's that going?

You remember we were talking about our podcast a few weeks ago or a month ago and you're just like, dude, we get a lot of leads for Single Grain and Carrot by me just mentioning, you know, in the middle of the recording. Hey, if you want us to do this for you or you want us to do X, Y, and Z, just go to Single Grain. So we started doing it. I have no data yet from it because only one video, I believe the Perplexity one, has it.

That was doing well.

Yeah, it's doing well. Um, and then the next one that comes out next week has it, of course, as well, because all of them have it now. But, um, by the time we record next week, I should have more data for you.

Yeah, that's— that I guarantee it'll be a good outcome. And plus, like, Even if it's— even if it's a small lift, it doesn't matter because you're making the content anyway.

Yeah, it's free. Small lift is better than no lift.

Yeah, yeah. I'm curious to see what the data looks like. Okay, so guys, let's— I want to talk about how ChatGPT's ranking model works. And then this— I'm going to share screen here. So Chris Long shared this. He's actually a pretty good curator. So how ChatGPT's re-ranking model works. And so I think what's worth calling out here is it says at the heart of ChatGPT's retrieval system is a re-ranking model. with the cryptic name Rett-RR-SkySite. And it's a sophisticated post-processing layer that takes initial search results and completely reorders them based on quality signals. So number one quality signal here is freshness is king. So perhaps the most significant finding for content creators is this setting, use freshness scoring profile equals true, right? And so there's a bunch of other findings, but we've kind of been talking about this where I think Ahrefs cited that 95% of ChatGPT results have been refreshed in the last 12 months or so. It's still kind of traditional SEO, right? And then this guy shared it over here. So you can see this use freshness scoring profile is true. And then he cites some other stuff. And then you have all this other stuff here, down here. Configuration reveals a sophisticated filtering pipeline with multiple checkpoints, enable query intent, source filtering. But I think probably the thing that's worth calling out, I'm not going to go through all this other stuff, is the freshness piece. What's your take?

Yeah, the freshness piece, we've seen it as well. Freshness is very important and people miss that with ChatGPT and even Google. We're seeing freshness be a super important metric, but the problem that we're seeing with freshness is marketers are looking at it as, oh, let me just change one sentence and update the timestamp and I'm good to go. That's not what they consider fresh. What they consider fresh is you updating it and making it valuable to what a consumer would ideally want to see in today's world. But on the flip side, if you have a piece of content that's performing well and you want to update it 2 months later to keep it fresh, but you're just going to change a sentence because there's nothing really that you need changed and it's already good to go and as up to date as it can be, you don't need to update it for the sake of it just to change the timestamp.

Yep. So speaking of ChatGPT, by the way, remember Brian Balfour from Reforge?

Uh-huh.

So he did a podcast on Lenny's podcast. I was listening to it when I was, I think, at the gym last week. But the question is, is the new distribution channel ChatGPT? And before we even answer that question, when you think about Facebook back in the day, organic, anything you posted on Facebook would get a lot of views, right? And then eventually you'd have to pay for it. LinkedIn, now it's kind of trending in that direction. Strong organic reach, it's kind of moving into paid, right? So you always get really strong organic distribution in the beginning. eventually you have to pay for it. And I remember talking to an executive at Microsoft and he even said that, hey, we're actually considering being able to pay for some results inside of like an LLM, right? Now, for the moment with ChatGPT, sure, you know, with all searches, Google's still 42% and ChatGPT or LLMs, I think it's only like 0.19%, but it's growing quickly. The question is, is the new distribution channel ChatGPT and I would argue that I think it's a strong new distribution channel for people to consider, and there probably is good organic reach to think about right now, but eventually you're going to have to— it's going to become more pay-to-play. And same thing with Snap in the very beginning, or when Stories launched on Instagram, they tend to favor these new features. And so I think that it's definitely worth paying attention to, and eventually it's going to kind of whittle down, but it's in the early stages of growing really quickly. It's still small, but it's growing quickly. Go ahead, Neil.

Yeah, I'm just curious to see what will happen over time. But yeah, and then I think everything, just like you and I have seen, it all becomes pay-for-play models. These guys all try to push you because it monetizes that way. It's just like Google AI mode. They're just pushing, their whole goal is to push ads through it. It's launched recently in 180 countries, but all Anything that's good in marketing, if it keeps, or anything that's good from an engagement usage perspective, all these platforms will eventually figure out how to push you towards a model where it monetizes and that does really well for them. It doesn't mean organic's dead, it just means you got to pay to play.

Yeah, organic eventually becomes paid. That's just funny.

Yeah, because even SEO, you know, so many people hit us up and be like, yeah, I want to do SEO because I'm not paying for it, it's free. And It's in my best benefit to be like, yeah, it's free, but I don't want to lie to people. And I'm like, technically yes, from a perspective of not paying Google per click, but you are paying people, whether it's in-house or an agency to help you get rankings. And not going to lie, you got to maintain it and keep doing more or else other people outrank you. And that does cost money. So even though you're not paying the platform directly, you are spending money on it.

Anything good has a price.

Okay.

All right. So I wanted to take a moment to tell you about my podcast co-host Neil's agency called NP Digital. And they work with a whole host of global companies or global organizations. Also, Neil has SEO tools such as Ubersuggest and AnswerThePublic. All you have to do is go to npdigital.com to learn more, and we'll see you on the other side. So by the way, remember we talked about the 8,000 bots versus— 8,000 people versus 1 chatbot? So there's a study here on how AI recruiters crush human recruiters, and this is from an AI researcher, I believe. And so AI in HR. In an experiment with 70,000 applicants in the Philippines, an LLM voice recruiter— so this is the AI recruiter— beat humans in hiring customer service reps with 12% more offers and 18% more starts.

Wow.

Also better matches. So 17% higher 1-month retention, less gender discrimination, and equal satisfaction. And then it says down here, the break-even point, including all software and inference costs, was 8,500 interviews. And you can see over here, we're looking at the graph right now, the AI interviewer in the blue bar graph versus the human one in the gold, let's call it that. Fraction receiving job offer, it was higher for the AI interviewer. And then fraction started job, higher for the AI interviewer. And then fraction meeting 1-month retention rate, higher for the AI interviewer. And Look, this part is the most interesting to me. So when offered the choice, 78% of applicants chose— they choose the AI recruiter, and we find evidence that applicants with lower test scores are more likely to choose AI. What do you think about that, William?

Yeah. Yeah, that's right. Who do you prefer? Do you prefer the AI or the human?

Human.

No, you're supposed to prefer the AI. That's okay.

That's okay. It's— William has a heart.

Yeah, Bugatti. Daddy's working right now to try to get you the Bugatti.

Bugatti.

I need 10 more years, but can Daddy finish this? Okay, 7. You can finish 7 calls. Okay, that's how I get to work. He says I can work for a Bugatti.

Yeah, Neil's working hard for a Bugatti. There's always a purpose, guys. So yeah, what's your take on this? Because we remember we did the one on the HR bot, the 8,000 people versus 1 HR bot. In this case, AI recruiters Um, and 78% of people, the applicants, preferred the AI recruiter.

You're talking about the IBM story, right?

Yeah.

Yeah. In the past, that did really well on your Instagram, by the way. It did.

It's funny how—

Um, I, I'm with you on this. I, I think AI can do a lot of this stuff better than humans. I think in some cases AI is better and in some cases humans, but in this case, yeah, I see the value in it. Although on your, um, HR one with IBM, People started leaving comments about the long-term effects and how the company's not looking at that and they're being more short-sighted. I'm not sure if it's true or not.

I mean, people are always going to say stuff like that, so I don't know. Like, yeah.

Yeah. I take it with a grain of salt.

Yeah. All right. So that's that. We got one more to do and then we'll be on our way.

So let's see. Oh, here's one. You know, I know we were talking about the profound traffic. Increase or decrease. And I know people have had similar issues for Google over time and whether there's increases or decreases. Everyone's talking about their traffic increases or decreases, but I don't know about you, very few people start correlating it with their revenue going up or down. Like you see all these posts online, oh, look at my Google traffic, look at my ChatGPT traffic. Oh, it's—

I don't even look at my traffic anymore. Do you? I don't look at it.

No, I, I technically my team does, so I don't have to, and I get a weekly report.

Yeah.

But it's all correlated on our end. Our data analytics team correlates it to revenue because I'm just like, who cares if your traffic went up double, but you didn't make any more money? And who cares if it went down half and you didn't lose any more money? I care on what traffic and what visibility is actually causing revenue to go up. And I want more of that. And if it starts going down, I want to fix it. Right. But I'm not talking about the traffic part. I'm talking about the revenue part. And I think marketers got it all wrong. For some reason, none of them really focus on revenue. They all want to look at traffic and that's cool. But if traffic in most cases, and you and I know this when it comes to content, it's not a one-to-one. Going up doesn't mean you're going to make more money and going down doesn't mean you're going to lose money.

Do you want to know what is super irritating to me that's kind of like somewhat related to this is because I've been spending a lot more time on recruiting. So when people make it over to me and I'll talk to them, I'll just, one of the questions I ask is, What are 2 of the hardest problems that you've solved? Please share numbers, right? And I don't know why, maybe this is an agency world thing, but it's always like, oh, Neil, I increased the traffic by 400%, or I increased this by 200%, I increased revenue. Even if you increase revenue by 100% or whatever, but on what? Like on $1? So is it $2 now? Like, I don't know what it is. I get so mad.

Oh, I created 364% growth. Okay, what was your baseline? Oh, we were doing $10,000 a month in MR. All right, that's not much of a growth for me.

I'll tell you what, guys, like you listening to this right now, the hiring bar is going to continue to get higher. Like if you're a founder, you have to continue to raise the bar more and more. And so when you're asking these people these questions, they have to report on a baseline and they have to have some type of numerical fluency. And we're constantly talking about traffic and how much percent they increase things by. You cannot afford to hire these people because They're not putting enough time to invest in themselves. And another question we ask Neil is, hey Neil, how much are you investing out of pocket in AI right now? Even if someone tells me $20 a month for ChatGPT Pro, amazing, right? But if you say $0 a month, we cannot hire you, right? Some people, by the way, remember I told you some people in Brazil, $500 a month is a lot of money in Brazil, right?

Yes.

They're spending $500 a month out of pocket, their money on AI stuff, right? And that's how serious they are because they're looking to enhance themselves and just do better.

Dude, I'm with you. But yeah, you and I have the same pet peeve when it comes to interviewing and talking to candidates. And I don't care where you are, you know, like whether it's Brazil or India, if you're not investing to improve yourself, then why would you try to improve the business that you get hired by? You probably won't care.

You know, you can't have 2 types of cultures in your company. Someone's either AI-forwarded or not. So if you hire a set of people and they're kind of anchors and they're not for it, and you have one group that are, they're going to fight each other the whole time. And then, you're not going to be able to move in the right direction. Whereas if everyone's pointing in one direction, you're going to move in that direction, right? I was listening to Brian Armstrong from Coinbase. I think he was talking on the Stripe podcast. So that's actually a pretty good podcast, Cheeky Pint. So you had the Stripe founders interviewing the Coinbase founder, right? And he's like, dude, he was like, yeah, we were very serious. We said, if you're not into AI and we don't see your usage, we're going to fire you, right? There's a group of people that didn't use it. So I think he had them come in on like a Friday or Saturday or something like that. And some people had legit excuses, like some people were coming back from a vacation, but then he fired the rest. And so he was just saying that that was very heavy-handed, but he's like, that's what's required. And keep in mind, Brian Armstrong back in 2020, he had the guts to say, hey guys, we have one mission and we're focused on doing— we're focused on pushing crypto forward. Like whatever politics you have, whatever you have on the outside, it's already hard enough to have one mission. We cannot afford to have this other stuff. Everyone was pushing back against it. It was like, oh my God, we're going to lose like half the company. 5% of people quit, right?

So I think sometimes— His stock did better. His stock's done well over the years. I know right now it's down a little bit compared to what it was a month ago, but who cares? Overall, it's growing in the right or going in the right direction.

Yeah, but I think where we're landing the plane here is you have to have the guts to make a call when it's unpopular. And Brian Armstrong was okay doing it because it's funny, you had Google make a call like maybe a year ago saying we're not a political company, But that's already like years and years after Brian Armstrong did that, right?

And it's years and years after Google has had videos where they laugh about political stuff and the videos get leaked.

But think about it, dude, how hard is it to already have one mission? It's— but to fight on all sides, it means like the people are getting distracted and maybe don't have enough to work on or whatever. But if you're running a company or if you're a leader, that's what you got to focus on, right, William?

Yeah.

That's right.

That's right. You know, Uncle Eric's the one who gives you those cookies. The last time you ate cookies, it was from Uncle Eric. What do you say to him? Thank you.

You're welcome, William. I got more cookies coming for you, but, um, Neil will decide. Your father will decide how many you get, so it depends on how good you are.

And Emma too.

All right, yeah, 2. Hey, I don't control—

He's saying Emma too. Emma too. Yeah, of course.

Emma too. Oh yeah, Emma too. Emma too, of course.

Hey, thank you. Thank you!

Talk to you later, William!

Take care, everyone!

We're done. Bye!