Google Exploit Data Discussion with Mark Williams-Cook

Hello and welcome to this week's episode of Search with Kander. We're in a lovely little cozy studio setting. Hey, Mark.

I like it.

How are you?

Very happy with the new digs.

You'll be seeing this in more detail next week, folks, because we'll have three special guests. I know I've been hyping up for a little while. We'll have Arne Helmens, Bergum Kaya, and Yama Simsek in the studio with. Basically the other side of where you guys are now with the camera. We were kind of sat here, give or take, but we had a whole 5-person studio talking all about predictions for 2025, recapping 2024, and all that fun stuff. And this is kind of the plan, right? To be a bit more cozy in our in-person podcast in the future.

Yeah, I just felt like it was nice in there, but we're 2 big guys and we were kind of squished together. And it's nice to look at you when I talk to you.

Thanks, mate. Yeah. Yeah. We had, we kind of like angle the microphones a little bit and have a little conversation and stuff.

Yeah.

Um, this week the plan is we were gonna kind of talk about some new stuff as we usually do, but we thought since your talk from Search Knowledge has blown up so much, I think it was 12,000 views or something like that, something crazy on the Search Knowledge YouTube channel. We've been covered by Search Engine Land. Lots of people have been doing recaps on social media and all that kind of stuff. We'll dive into a little bit of details about the Google exploit that we discovered, Mark's talk, some of the stuff we didn't cover, answer some questions and all that kind of stuff. But before we get to that, here is a word from our sponsors. A huge thank you to SE Ranking for sponsoring this week's episode of Search with Candid. SE Ranking is the company behind the powerful and intuitive SEO platform trusted by millions of agencies, businesses, and SEO professionals since 2013. With a robust toolkit, SE Ranking empowers SEO professionals around the world to excel at executing their strategies. Key features include the rank tracker, which provides accurate and up-to-date keyword rankings, as well as the competitive and keyword research tools that offer deep insights into market trends and competitor strategies. The platform's AI-powered SEO insights leverage advanced machine learning to deliver actionable recommendations, streamlining the process and letting you outrank your competition. The backlink checker and monitoring tool enables you to analyze backlink profiles and monitor their status, ensuring you've got a very healthy link building strategy. SE Ranking have launched their AI Overviews tracking feature. It's a brand new tool that allows you to track AI search engine snippets that will help you see how you stack up against your competitors in Google's AI enhanced search features. The tool supports you in identifying your rivals in the snippets, analyzing their content and detecting new opportunities for strategic decisions. Click the link in the video description down below or in the show notes if you're listening to this episode. And a huge thank you to SE Ranking for sponsoring us here at Search with Kander. Optimizer is the modern PPC management software that is built by PPC people for PPC people. Optimizer contains everything you need to create, optimize, and automate high-performing paid campaigns across multiple platforms. As I just mentioned, we're going across platforms, and I think this is one of the best time-saving options and features when it comes to Optimizer. If you are advertising across multiple platforms including Google, Microsoft, Amazon, and Facebook, then you need to be using these cross-platform tools. You can create reports, you can create shareable dashboards. You can share budgets and alerts across all of these platforms. If you are individually managing each of these things, you could save yourself a lot of time by using Optimizely's cross-platform tools. For monitoring, the alerts let you know when budgets have been exhausted or KPIs have been reached or missed. And when it comes to the rule engine, this allows you to set up custom rules to automate so many time-consuming and manual tasks that are part of your paid campaigns. You can easily automate rules. Basically, if you can write your strategy or tactic down, you can automate it. It's that simple. This can be especially helpful in things like e-commerce when it comes to stopping overspending of a particular ad. This is something I know from personal experience. I have had an ad that has overspent, and if I had used Optimizer, that would have saved my budget and save my campaign in that instance, and I wish I'd used Optimizer at the time. You can also adjust bids and choose which products or ads are showing based on things like the weather and pretty much anything else you can put into a spreadsheet. It is fully customizable and automated, so you can choose when ads start and which products are showing depending on the time of year, depending on the weather, and so many other factors. It's all automated and makes the process so much easier. To sum it up, Optimizer is the award-winning all-in-one PPC toolkit that makes managing paid search easier, faster, and less prone to error. It puts you back in the control of your campaigns by allowing you to create and deploy your own automations across multiple ad platforms. You can start your 14-day free trial by going to optimizer.com. That's O-P-T-M-Y-Z-R.com. You can also click the link in the show notes. And if you want to see Optimizely in action, you can book a personalized demo with one of Optimizely's experts by clicking the link in the show notes as well. So you can get a 14-day trial or book a demo at no cost to you. Like I said, go and click the link in the show notes or the video description down below. And a huge thank you to Optimizely for sponsoring Search with Kander. So for those who don't know about your talk at Search and Orange, Mark. I know you titled it Doing Advanced SEO Through Conceptual Models or something like that. Conceptual Models, yeah. And you had a little, and what I learned with Google exploits in brackets there. In classic SEO talk fashion, you keep the little sexy bits in the parentheses. So very briefly, without just doing your talk all over again, kind of give the listeners and viewers some context.

I still think it's a bad name for the talk, right? Conceptual Models. And I brainstormed this several months ago with people like Chris Green and some people in this, uh, some SEOs in this WhatsApp group being like, here's what I want to talk about, what do you think would be a good title? Here's what I'm thinking. And we came up with classic SEO-y kind of clickbaity type titles, which didn't feel right. Um, And I think it was Chris who sort of said to me like, yeah, it just, that is quite accurate though, what it's about. And I was like, yeah, but it sounds boring. And he was like, yeah, it kind of does. So—

You really could have gone like, what theme park taught me about SEO or something like that.

Exactly. So yeah, um, the, the talk was called, uh, Conceptual Models, um, in SEO with Google Exploits. And it was essentially how I think about SEO in terms of building a working model of how search engines work and focusing on the why search engines do stuff rather than necessarily the how, and talking about how you can build one of those models in your head. Because one of the issues we have is there is so much information out there about SEO. There are so many conspiracy theories.

Yeah.

There's so much contradictory information. One of the problems is how do I decide, you know, if this is a ranking factor or if this is, and how they all fit together? Because that's the other big thing that the point of the talk is, and the exploit shows us, there are different ranking factors, but they all sit on top of each other and rely on each other.

Yeah, yeah.

So it was really how I think about SEO, how I add to this mental model I've built up over years, and the Google exploits um, were meant to be axioms. Now I know as a physics person, you, you'll be comfortable with that term. I thought everyone was, but then everyone was like, what the hell does that mean? So axioms are kind of like things that are, um, you can take for granted as true that you can build. So these are foundational truths that you can build from. And the idea was if we get things directly from Google, so we know this metric exists and it's within this range and we can see it does these things, We're not guessing at that point. And that's again one of the difficulties about doing SEO, right, which is that it's a guess sometimes built on a guess. Yeah, built on a guess. And when you start getting 3 or 4 guesses high, you can maybe go off course.

Yeah, I've seen like those error margin kind of graphs where it's like, oh, this one's 10%, and you stack another 10% on top of that and another 10%. Exactly. And it gets much and much bigger.

And I think That's something really important as an SEO community. Sometimes we don't think about enough, which is like, how big is my error margin and how far, how far am I from something I know is a fact?

Yeah. Yeah.

Because if you start building—

Every time you're making decisions, you're 4 or 5 steps away and you could be in trouble.

Exactly. And then you have nothing that's based in reality. So that's what the talk's about.

And also theme park and video games.

And theme park and video games.

Yeah.

So framed in, framed in video games because I do love a video game.

I've noticed. Yeah. Yeah. Funnily enough, it was funny seeing people react. And I know you commented on this earlier when we were chatting about it, the fact that theme park is brought up. And I remember that from PlayStation 1 back in the day. Obviously you're much more of a PC gamer than I am.

Yeah.

But then a lot of people talking about, oh yeah, I love like Hospital or like Theme Hospital or RollerCoaster Tycoon or whatever, like the later versions. And I know there's been a bit of a revival of these kind of sim games happening. So I think that kind of helps definitely like resonate and stuff, right? You've got to have a bit of that in a, in a kind of talk to translate it to people and make it more kind of, uh, workable for them and stuff like that, right?

Yeah, so I, I explained kind of how I think about SEO in terms of video games because for me that wasn't even like a, a narrative device. It was legitimately how I think about things.

It's not just a bit.

No, it's not a bit. Um, and you know, some people do genuinely laugh at it when I explain to them, I honestly—

Don't oversalt your chips, that's how I do SEO.

Yeah, but I like getting good at those types of video games, not like COD, but like games like Theme Park. Legitimately, the way that you approach learning about those games, those systems, which is what they are, and seeing I do this thing and this thing happens, and I change these variables and this happens, and then short-term, long-term impacts, cause-effect. That's legitimately how I got good at other business things in life, whether it's running a business or whether it's doing something like SEO. So it wasn't, yeah, it wasn't like a narrative device. It was honestly like you can frame these things like SEO in a useful fashion in the same kind of framework.

Yeah. Yeah. I think that's a, that conceptual model kind of thing I found really interesting because again, coming from my science background, you mentioned I have a physics degree. That's always how I kind of put things together in my head is going in with taking your existing knowledge and kind of building an idea of what to expect. I know you do the tennis ball example, right? Dropping the tennis ball and you will kind of guess how high a tennis ball is going to drop from how high you drop it from, but you don't necessarily do all the calculations. I talk about this with my cats. I can literally watch my cats like doing the calculations. They'll stare up at something and be like, do you reckon I could jump that? I reckon I could probably make it over that. Maybe like doing little head tilts and like trying to do it. Like they're not doing maths, but there's clearly an idea of like—

They're collecting information.

Exactly. I've jumped this high before. That's a slightly different angle over there. Oh, that bit's a bit lower. That bit's a bit softer. That's a bit harder. It's that real kind of building a model and an understanding in your head so you can then make judgments based off of that. And like you said, as long as you don't go too far away from the root cause of it, the truth, then you can kind of get a pretty good example. And someone like yourself who has 20 years of experience, you can kind of get an idea of, oh, I've come to you with this migration problem. You're like, it's probably this, it might be that. I know where to look first so you don't waste your time. Oh, I'm just going to crawl the whole site and see what happens. this— sorry, Martin Split— this might be a JavaScript problem or something like that.

Definitely.

Probably not. But you have that idea and that will really— I think that really helps me even from my relatively short experience of kind of without leaping to conclusions and stuff like, oh, I roughly know where to start. If you have an indexing issue, it might be this. Or if you're having a crawling issue, it might be this. And that kind of thing, I think a lot of people kind of underestimate in a lot of ways when it comes to taking your existing experience and building that and using that as part of your workflow and all that kind of thing, right?

I mean, I think anything like that does require a fair bit of experience to be able to, because it's almost in a way like gut feeling in a way. But for people that are maybe fairly new to SEO, the point was that you won't understand what's going to happen with the search engine just because you've really studied like vector embeddings, or, you know, you've studied some really specific bit of information retrieval or something. It's like, those are great things to know, and absolutely they will shape and impact other things and how you think about them. But any of those things in isolation is not going to give you the, that helpful model of you drop the tennis ball and it's going to bounce. this high. And that was one thing I wanted to, to kind of get across, because there's a futility to that.

Yeah, definitely.

Sometimes as an industry we do, you know, and again, there's a balance, right? I'm 100% for people learning about everything. I'm like, yeah, learn to code. I'm like, yes, absolutely. Like, research all these different, um, kind of branches of stuff like information retrieval. And, you know, like, I was years ago even in Perl trying to like program my own search engine from a book that's telling you how to do it. And it was a useful experience, right? But none of that actually replaces the overall bigger picture of, okay, well, this is actually how this is going to work if we do this thing.

Yeah, definitely. And to kind of dig into the, the exploit data a little bit, I know there were some pretty big numbers in there. It was like 2 terabytes of data Was it 90 million plus queries, something like that? It was a lot of, a lot of stuff going on. I don't know. I was chatting to Arnoud Hellemans, like I said, who'll be on the show next week about it a little while ago. I know you've had conversations with Arnoud as well about it. And it was, I think, pretty shocking for a lot of people that, that, that amount of data was in that. And I know obviously we reported it to Google when we got the maximum bounty, I think it was, that was based on the— risk of manipulation and the like amount of data that had been leaked. We then donated that to charity. Shout out to Brave Futures, our charity partner. And it was really cool to see people's reactions and kind of almost like the hype building because we're like, yeah, we donated this stuff to charity. I know you'd already done the talk twice previously, right? You'd done SurfConf in Vienna and also done Search and Stuff in Antalya. And then I don't think those guys posted to YouTube as quickly as we did.

Yeah, I don't think it went public. I know SERPConf have got their— it's called SERPflix, which you can pay a bit and see the last SERPConf, all the talks. And I think the previous ones are free.

So it's more like a portal kind of thing. Yeah.

So I don't think it went out public, public.

I don't know. You and I were just talking about that. So like, should we publish this? Like, I feel like this is going to be a big thing. Maybe we shouldn't. Maybe we should.

It was weird just because of the, like, Search in Norwich is a fairly small event. It's like, what, like 50 people there?

Yeah.

So it's weird that 50 people came and enjoyed it and were like, that's cool. And then it goes to 10,000 people in 24 hours.

Yeah.

Things I didn't get to mention that I think are really interesting about Google and the exploit and the kind of structure in general. I don't think we've even had this discussion yet.

Probably not, no.

Is so the data came about from a Google endpoint, right? And as far as I understand, so you have to take everything I say now as this is my understanding from, you know, things that I've seen and, you know, this is how I think this is, this is set up and working. So to me it looks like there are several Google endpoints, and by endpoints I mean these are places that surface information to Google's front end. So there is, for instance, um, so we know PageRank is still used by Google, right? In some form. Okay. We got no data about PageRank in this exploit because I don't think PageRank is a metric that needs to be surfaced to the, to the front end.

Yeah.

So firstly, I would say this, you know, whatever we found, didn't find, this isn't the entirety of Google's algorithm.

None of that previous algo leak that everybody freaked out about.

Yeah, so, and the, again, so if we stick within the realms of axioms, axiomatic things, things we know, the data that we, the impact of the metrics that we saw were at least definitely impacting frontend features. And why that's important is that a lot of front-end features, and by front-end features, I mean things like People Also Ask results, featured snippets, map packs, things like that.

SERP features, things you're going to see actually on the SERPs, right?

Yeah, exactly. Are generated post-ranking. So that means that Google has a query and it generates a list of sites for whatever location that's going to rank for that. And then before that SERP is kind of comes out of the oven, um, Google sometimes seasons it and it's like, oh, I'll add a rich snippet in here and a PAA here. And to do that, it still needs information metrics that they have collected during that big kind of calculation. So it would seem logical to me that most of the metrics that are surfaced through an endpoint like this are surfaced because they have some kind of usefulness at that post-ranking front-end stage, right?

Yeah.

It doesn't mean they are not related to anything that is happening in the backend or other impacts on ranking. Absolutely. But it's just that we need those to do, you know, these are helpful numbers to do, to do this. So the most obvious example was obviously the site quality one.

Yeah.

And the very obvious thing we saw as I mentioned in the talk, was there was a specific threshold whereby if sites were below this threshold, they did not get, say, People Also Ask results.

Yeah.

And if they were above it, they did. So that was, um, one thing I think was interesting in terms of understanding— one, understanding what type of data you get in that endpoint, and 2, understanding that's how Google works. That's a thing that happens because one of the points of the talk was, um, there— this threshold exists. And you have seen loads of guides online about, say, like, here's how you win a featured snippet.

Yeah.

But nobody has mentioned, like, there is this qualification to have one. So it doesn't matter how good your content is, if you don't hit this threshold, no, you can't have it.

Yeah, I think that was a— and that's definitely seemed to be something that has resonated with people when it comes to Like you said, that idea you can churn away, you can pour your heart and soul into the— put all the schema in the world and all that kind of stuff and try and get a featured snippet or a PAA feature or whatever it is. But if you don't meet that minimum threshold, that's not going to happen. And it was something funny enough, I had a client call earlier and I was talking to them about that where they are— had previously been in featured snippets when I chatted to them previously and were now also being featured in AI overviews and stuff. I was like, hey, we're, you know, This is how the SERP landscape is changing. And I mentioned like, oh yeah, Mark did this talk and kind of briefly, very briefly discussed it. I think, um, one of our account managers sent them the link afterwards, like, go and watch Mark's talk, learn more about how it works. Good luck. And it was an interesting discussion of like, oh wow, do you think like certain competitors would then, you know, fall under that threshold? That means we can, for want of a better phrase, and I hate to outrank them with certain features and stuff. I'm like, Yeah, in theory. And then I know there was a lot of discussion around how YMYL stuff turns into that. This client, as an example, is, is in definitely YMYL kind of area. Do you think that kind of thing is something people should be thinking about? Does this relate to the kind of YMYL stuff? Do you think they're more— I know we always say like they're more critical or more kind of aware of what's going on with YMYL stuff, or do you think that's a totally separate conversation?

So, I mean, one of the questions we've had about the leak was like, are you just going to release all the data?

Sure, yeah.

The answer to that is no. But there's a couple of good reasons for that. I mean, one is, to be honest, I don't exactly know and haven't looked into even like if there's any legal issue in doing that.

Yeah.

With Google because it's all, I guess, their data. Um, and even, even if there isn't, um, we had the content warehouse leak, um, and I was a little bit disappointed with, I guess, the overall response to that. Not— so there were some individuals who like wrote about it and championed it and stuff, and that was all great and fine, but the wider SEO community pickup, I thought, was, um, outside of the kind of method I gave in this conceptual model approach in that we were just finding variable names and then Judge Dredd summarizing that, well, this is a thing.

Time to change your strategy right now.

Because, yeah, because the word sandbox is here and I don't know if that's in relation to crawling, indexing. anything, but now the Google Sandbox exists as the legend.

Conspiracy theory confirmed. Yeah, exactly.

So, and I almost felt like as a, as a community, it like lowered our actual knowledge about SEO because that from what I saw, and again, if we just look at things that we know were factual from that data, it to me looks like in general, everything Google had been telling us about search was correct. I mean, the example I gave in the talk was quite, uh, you know, verbose about the CTR, predicted CTR thing, ranking post-ranking. And I'm aware that's very nuanced, and I'm being kind with the, you know, giving them benefit of the doubt. But it is technically correct, and it's a very big complex system. So to go back to your question, like, should we be thinking about this for YMYL? I think it's useful because we have, um, and there's still more stuff we could publish about this actually in terms of like query classification.

Um, that was probably my— well, we'll get on to that in a minute. I think that was the most interesting thing for me for sure.

So one thing that surprised me was just the amount of classification, fairly granular, that Google did like at a query level. And we've talked previously about query-dependent and independent ranking factors. And to me, it just seems like, yeah, this actually makes sense in that if there's fairly quick, easy, cheap ways to classify queries, they can go down this kind of decision tree and land in a bit of the algorithm that's most suited for that kind of query. Like, okay, well, this query has like explicit local intent. It's YMYL. Yeah. Um, Google has told us specifically if something they think is YMYL, they weight ranking factors differently. So the important thing that I think we learned from that is that this is one reason why we can't just be like, well, you need to do X to rank, because there's so many, uh, specific cases based on the type of site, the industry, the query, the locality, the language, um, the SERP behavior, all these things like impact how, how a site ends up ranking. Um, and that was kind of one thing I wanted to, to get across, you know, that apart from all these classifiers, we— and then you've got these factors that stack on top of each other. Um, and we've seen like AJ Cohen post about—

Yeah.

Um, like rings before and, um, working from cheaper to more computationally expensive kind of systems, which all makes perfect sense. Like if you can determine cheaply and quickly that something is spam and crap, why would you waste, you know, expensive limited computer resource doing some kind of expensive analysis?

As we always say, especially at the scale that Google is doing everything, right? And we talk about how expensive like the LLM side of things is and AI overview is and all that kind of stuff. If you can do it cheaply, especially for billions, trillions of queries, that's totally going to make sense. Absolutely.

So yeah, so I don't, I don't necessarily think it's something on a granular level you need to, to, to think about. Um, however, one thing that we are going to publish from this is we're going to try and build some of like these classifiers to be able to say, okay, well, we think Google thinks this is a YMYL query, for instance, or we had those refined query semantic classes, the 8 of them that we published, which I thought was quite interesting. Um, so I think the, like, what's actionable from that, what's actionable from that, I think, is we can do some studies. And we're doing one at the moment with SE Ranking as to how they correlate to specific SERP features. So we can see, okay, well, if we're producing this kind of content, for these 50 queries, we know they're classified as this, and there's almost always a video or whatever. Like, we can help plan what we need to do to make the best possible content. Because again, Google's like— Google's got all that feedback, and that's why they have like their tanogram and why they know people want images or they want video or they want a featured snippet here. Like, that's useful to us as people that are making content and websites to know, okay, Google's got this data and they've told us this is what people are looking for. So this is what we need to make to be in the running.

Yeah, there's almost that kind of— you're almost, funnily enough, building like a conceptual model of the SERP then, right? And trying to understand without literally doing SERP analysis. And I know, you know, there's quite a few tools out there. Keyword Insights, Sistrix do some really good like SERP comparison kind of stuff. Even I know Sistrix have like the literal screenshot of And again, this is something the industry is talking more and more about, of how much of that actual screen space are you taking up in the AOV with your brand, with your name, all that kind of stuff. Having an idea of roughly what kind of features you're going to see from this particular query or this particular side of things, I think is a really interesting way to kind of get that ball rolling in a particular direction. When I'm doing kind of keyword research and stuff, you're obviously thinking about search intent, you're thinking about how relevant it is to this brand and to this client and all that kind of stuff. But having an idea of, without going and manually searching every single time, it's like, oh, should we be thinking about video content rather than a blog post? Should we be thinking about focusing on really high quality images if it's going to serve a carousel at the top instead? Having that kind of initial thought process, I think, can point you in the right direction really easily. And that's what I think— that's why I found those kind of 8 categories so interesting because it helps kind of build that idea and kind of steer you in the right direction without having to go and manually check it yourself kind of thing.

Yeah, the— for me, the biggest surprise thing was like the how the predicted CTR change— the predicted CTR changed when we were updating like titles. So I think I mentioned in the talk, obviously that's something Google's been doing for a long time with Google Ads. They give you a, like, here's how we think the ad's going to perform in terms of predicted CTR. And while I'd always optimized for CTR being an SEO, I'd never really thought about it kind of the predicted CTR being a ranking factor. But it does make sense.

Yeah.

In that if Google can say, well, I think people are going to like this, Um, which again for me is then an area which, especially for like big enterprise sites, where that's ripe for a machine learning automated iterative approach of let's optimize this.

Yeah.

Because that's again a thing that is ripe for actual optimization, which is let's get as many people to click on this as possible. Within the— again, without metrics running wild— within the realm of we aren't, um, like mal-adjusting the expectation. Because obviously you don't want to be like, click here for the best thing ever, win a million pounds and all your dreams come true. And it's like, oh no, it's just an infographic. Um, so that's something I think that I focus on more now. Yeah, which is like actually we need to be doing consistent widespread tests. And if we've got enough data, like in Search Console, you know, using Hugging Face, stuff like that, it's more than possible to build predictive models to be like, this is actually what you should maybe try. We think this will give a better click-through rate. And of course you human review it as in that will give a better click-through rate. Yes, it's still, um, descriptive of what's on the page.

It's not just clickbaity or outside of the brand or whatever. Exactly.

So There's so much nuanced stuff that I think is important in terms of where you put your focus. And again, in terms of advanced SEO, I think that's one of the most important skills, which is you— whether you're in-house or whether you're working with a client, where do you apply your focus? Because your, your resource is always going to be limited, and there is an almost unlimited spectrum of stuff you could be doing. You know, like for years, you know, I've always said to people like spend very limited time twiddling with meta descriptions, right? Because Google's going to rewrite them anyway. Like, there's much better things to be doing with your time. So this for me just again adjusted a few dials on, okay, it's probably actually worth spending more time on this.

Yeah, we often have that conversation. I know we've talked about this when it comes to audits, when it comes to just on-page stuff like that prioritization of resources versus impact, right? And I know we've mentioned this many times in the podcast before, Meta descriptions being a perfect example. And I think that's maybe something— I don't want to badmouth other agencies and stuff, but I've definitely seen it plenty of times, right? When you get that— I've worked at other agencies that I've seen this happen where you get the like, oh, just rewrite all the meta descriptions, an entire site-wide meta description review. I'm like, why? This is going to take me an entire 2 or 3 days. There's hundreds of pages.

I mean, at least A/B test them.

Right. Yeah, yeah.

Don't just change them.

Yeah, exactly. It's like, oh, that seems better. Like you're doing like half-assed keyword research while you're doing it kind of thing. It's like, that's not the best use of my time. And I know that's definitely something a lot of people fall into the trap of is just twiddling for the sake of, well, I need to make a change, so I might as well make a change. It's like, let's just do all of it all at once. And I wonder if that's— hopefully we're getting better at that. At least the people that are doing good SEO.

You've just made me think about something because you said twiddle. I think I said twiddle. Twiddle, and then you said Twiddle, and it backed me. So which is the, the Twiddler framework.

Okay.

Um, so the, the Twiddler framework, I think it was 2019, I, I first heard about this through some Google leak thing, but it got mentioned, I believe, somewhere in the DOJ stuff. And for those that don't know, the, the Twiddler framework, or, um, is essentially a bunch of standalone kind of programs that Google runs, as far as I understand, post-ranking. Um, they're called Twiddlers individually, right? And they're kind of like little micro programs that can run in a vacuum, and they can use metrics that are calculated during ranking. So I'm assuming like stuff we get from these endpoints.

Yeah.

Um, to make adjustments to ranking, right? Um, So I believe some of the post-ranking stuff like the DOJ example where they re-rank results based on user interaction data if it's wildly out of their expectation, I think that's like a Twiddler framework level thing.

I'm never not going to giggle at Twiddler.

Yeah, I know, right?

Every time.

But again, like, I wonder how many SEOs kind of know and think about that and think about that in terms of the, the stack? Because the long-term value for SEO comes from your— so the long-term value for your like actual SEO comes during ranking. All of the post-ranking stuff is like temporary.

Yeah, sure.

Um, or is a second-tier impact like people know your brand so they might select you over time, but then If— so we had a question actually on the YouTube channel, which was something along the lines of, to paraphrase, like, um, I don't understand why, say something ranks in Google and they have a predicted CTR, but nobody clicks on it. Like, surely Google is going to change the ranking of that. How are they not using ranking on individual sites?

Right.

And I was like, that's a really good question, right? And I thought through this. So the— I mean, my understanding is that Google have some kind of like predicted click-through model that they use to help adjust rankings of sites. And if the user data is wildly off from that, they will temporarily re-rank things.

Right. Yeah.

Now, I'm not sure that situation could ever really happen where, um, the model is so wrong that something is, say, ranked 2nd in Google and nobody clicks on it because people will just click on whatever's there, right? So Firstly, I don't think that could happen. But if we went on the hypothetical that it did happen, the basis for that happening, ignoring the fact, of course, that lots of other things determine where the site actually ranks apart from—

That's a very small part of the pie. Yeah, exactly.

Apart from that. But assuming that did happen, it would mean that the model was very broken, that it let something so wildly wrong in. So I guess what would happen then would be that we would see a temporary re-ranking in that other sites would be getting higher click-through rates wildly than expected. And then because the model is so wrong, the aggregate data would slowly change.

Right.

So the predict—

Like a counterbalance itself.

The prediction would slowly move. And so that site would slowly go Bloop. Yeah, like that's what I would expect to happen. So because again, like I'm not saying that's definitely correct, but what I'm saying, what I'm saying is that that explanation at least fits with all of the facts that we have. And I class things that Googlers tell us plain as day— this works like this, this doesn't work— that's fact, right?

Yeah.

And one of the things they say is we don't use CTR directly in the ranking phase. So, you know, the only other way is to explain that where, or at least this person was posturing, was, well, they, they do this in ranking. And I'm saying there are other explanations that work, that fit with all the other evidence, which again is part of the point of the talk, right?

Yeah.

Like, don't subscribe to stuff if obviously there are other explanations. And again, like, the other thing I have to always, you know, keep in mind is like, I am not qualified to the level that these engineers work at. And just because I can't think of a way that something works, right? And I've had these discussions with SEOs where I'm like, well, you know, they say it doesn't work like this. And they're like, well, I can't see how it'd work any other way. It's like, well, yeah, because you're not massively qualified and experienced in doing that really specific, like, engineering coding job.

Yeah, you're going to spin me off on a whole anti-expert discussion right now because that's literally like a conversation I've had with my dad once. was like, ah, quantum physics doesn't seem real. It doesn't matter. Pretty sure it does. I'm pretty sure it's like one of the most tested scientific theories in human history.

I don't get it.

Exactly. That's exactly what I don't understand. It was like, read a bloody book then. Like, I don't know what to say. I did a degree. Like, that kind of stuff matters. And I think you're totally right how that kind of like permeates into SEO as well. Like, if you don't understand how something works and obviously at the very beginning of the episode, you're saying the how and the why that matters, right? If you don't understand how something works, maybe don't just jump to conclusions. And like you said, you're building evidence and you're building your decision off of the stuff that you don't know. So maybe you're building your house on a block of sand or whatever, right? Rather than the concrete of this is the fact, this is the axiom you're going from. Well, I don't understand it, so I just leap to this conclusion.

And the difficulty with marketing or SEO as a profession and like I am fully including myself in this. Oh, I'm not talking down to people because I don't have a degree. Is that you can get into this field basically with no qualifications.

Absolutely.

Um, and not even that qualifications are necessarily important, but, um, sometimes I feel that having met people like yourself that have been through that process, you learn how to think critically. Yeah, like, you know, cite sources and all this, which I think is important. But no, you can be like a very well-paid, well-respected SEO.

You can be really good at SEO and have no backgrounds. Yeah. So no, like, because, and it's interesting, basically every SEO I guess over the age of like 30 at this point, it's like, how did you get into SEO? It's always the first question. I was like, well, I was a dog groomer for 15 years and then I built a website and I guess I was an SEO. Every— nobody I know And funny enough, speaking to people, you know, 10 years younger than me, 15 years younger than me who are getting into SEO now, take like Ryan from SEO Testing who has been building websites since he was 16 and he's still 10 years younger than me now with the same amount of experience that I have 10 years later. It's like, oh God, it's crazy. And speaking to people who are in their early 20s like, oh yeah, I've just been— I've lived on the internet. I just exist on the internet my entire life. So it's inherently part of this thing.

Mm-hmm.

just being aware that Google exists from such a young age and, well, poking and prodding. And how does that work? Why does that thing— if I search for this, why does this thing happen? Whereas I know that's definitely true for you and definitely true for me as well. I remember life before the internet. And so I—

I do.

Didn't have that kind of initial switch on of like, how does stuff work? Like in Google, how does that work? I remember probably like 10, 15 years ago, I was like, how does that stuff work? Why is this ranking? Why does this matter? And realizing there's an entire industry around that kind of thing. I find it fascinating when people, like I said, having conversations with my dad, for example, and just being like, well, I don't know how any of it works. Like, and then as we've said many times before, like, I hate this new Google thing though. Like the AI, everything pops up and like, well, it gave me the wrong information, I think. And we obviously talk about how people are trusting AI in one sense, but then a lot of people are not trusting in AI or losing trust in certain technologies and stuff like that. It's a really interesting conversation for the future of search, the future of technology and data and how we all see it, right? I think there's a, hopefully a shift back in the other direction after so many years of this anti-expert stuff, although lots of political stuff going on. Hello, second, second term for Trump and all that kind of stuff. We're going to get more sunscreen and bleach and all that kind of stuff to— fix COVID and all that kind of thing. But I'm going— coming back off of that tangent, I think having that fundamental basis, even if that's just from experience like yourself, like not having a science degree, I think that helped me in a way to kind of get in with, like you said, with that critical thinking side of things. But you can build that from your experience. You can just—

Yeah.

I know we always say this, like build your own website, start doing your own tests, all this kind of stuff. Get, get your hands dirty and do the work. Yeah, exactly. Do the work and find out for yourself. Exactly. Yeah. See what happens. I've totally been doing this and I mentioned this on last week's episode when I was talking about the top 10 episodes of 2024. I've been playing around with YouTube all year. Does this title work? Does this thumbnail make a difference? I've been, you know, you can A/B test, I guess, 3 YouTube thumbnails to see which one gets the highest click rate and all this kind of stuff. It felt like I was doing SEO for the first time. I was like, oh wow, yeah, I'll just add a word on the title here. After 2 days, I'll change the title back to this thing. Like, best. Exactly. Yeah, yeah. Funnily enough, I was like, top 10 podcast episodes, top 10 episodes of my podcast. The greatest and best. Exactly. Best SEO podcast.

Emoji emoji.

Fire emoji.

Can you use emoji in link— in, uh, YouTube titles?

No, you can't. No, no. But people replace it with their own face and you get the like Shocked face, like human emoji versions. That is obnoxious. But yeah, back on track. Sorry.

Maybe if we round off on a technology angle, I think an interesting thing to end with would be protobufs. So do you know much about protobufs? I can't remember if we talked about it.

Very little.

No. So this was key to the kind of exploit. And I found some interesting, like, Kanda lore while, while doing this.

Oh, nice.

So basically Google made this, um, I don't know if I'm gonna get all the terminology right here, but I guess like a standard called, called Protobuf, which is a, um, really efficient way of communicating data online.

Right.

In terms of like, uh, byte size and speed. So essentially how it works is, say I want to send you a whole bunch of, of data, um, we will have— both of us will have a .proto file, which is like a, a key, if you like, for our little pigeon speak. And if I— and I say if I write like AB1 on my bit of paper you can— I'll just send that to you and it sends really quickly because I'm just sending you like this, these few characters, AB1, and you check your little .proto sheet and it says, well, if they write AB1, it actually means this whole, say, like function.

Right.

Right. So that way you don't have to transmit like the whole thing. Um, it does mean that it's unique for like, say, each endpoint or each thing has its own .proto, so its own like set of shortcuts. And the kind of the core of what we found with the endpoint is the Protobuf is how I believe a lot of these Google endpoints are talking to each other, um, because it's again really fast, quick way to send the data. Um, and we essentially just saw a network request that looked weird and it was sending this odd data, and it— you could modify in the query string the format request, right? And just basically request the data in JSON format. Um, to which Google was just like, okay, and then dumped like a load of data. And if you try and do that now in the same place where the exploit was, you get like not authorized.

Makes sense.

To do that, which I guess is like what it's meant to be. So that's just some kind of background on roughly how we came across that data. And again, some more picture as to, I think, what Google's doing and how it's sending around that data. But interestingly, I found we had actually used, um, Protobuf on a client project, um, several years ago, right?

Right.

Where we had a really old client legacy, like really old system that they used from another company that's like their industry standard. I won't, I won't name it, but one of the issues was it's just an incredibly slow, inefficient system. So we needed a way to really efficiently send data between that and the website because the website needed that data.

Right, right.

And as actually part of that, we contributed to the open source part of Protobuf. So there was a, there was, um, a mistake, I believe, with the, um, secure socket stuff on Protobuf where it wasn't closing off connections or something. Um, and it was our team that kind of found that in the PHP version and submitted that. So I was like, oh, that's cool. Like, it all came round and I was like, that's interesting. Our like sort of dev team was involved there.

Nice.

But yeah, um, there's a whole kind of thing about Protobuf Online and it was just, again, I'm— you can probably tell from how cautiously I've stepped around Um, that topic. It's definitely not my expert area, and I've done very light reading around it, mainly because I don't think there's a huge amount of benefit to me, like, understanding it deeply. It was more—

The effort versus impact thing we were talking about earlier, right?

It was more just like, okay, let's see what this is and what it's used for and understand the context of why it's used. But, um, I think a lot of SEOs won't have even heard about it before, and it was just interesting to me that that's And it was made by Google as well, this, like, this standard. So, um, a little interesting thing to, to read about.

Yeah. Awesome. Well, thank you for joining me in our little living room setting here, Mark. It's been very nice, very cozy to sit down and have a little chat and pick your brains, learn some stuff we, like you said, we haven't even discussed before.

Yeah, we haven't had a chance to talk about it.

I kept getting people contacting me after you'd done SERPConf and searching stuff being like, oh, when's the data from Mark's talk? When's Mark's talk going to be available? I'm like, I don't know. I don't control any of that.

You hadn't even seen it.

No, I'd seen like 3 slides because you'd shown me a GIF and I've been like, ah, that's funny. Yeah, cool. Look, theme park. I'm like, cool. I have no idea. I hadn't seen the talk. People were like, when's it going to be out? When's it available? What do you know? I'm like, I don't know anything. I'm learning something as you're learning right now, viewers and listeners. So yeah, uh, we'll be back next week. Like I said, at the end of the year with the big 5-person panel episode. By far the biggest production we've ever done, which is very scary but very exciting. I think it's going to come out really, really well. Very excited for everybody to watch, to listen to what Arnoud Begum and Jammer have got to say. And Merry Christmas, I guess. This is coming out a couple of days before Christmas, so if you do celebrate, Merry Christmas. We'll be back next week with, like I said, a big episode. There's no breaks here. We're then straight away back in January as well, so You'll get a lovely kind of watch this on Christmas Eve kind of thing. And then gather around the tree. Exactly.

Fire up search for candy.

Exactly. Then you'll get like a New Year's Eve one. We'll have a, have a lovely time. And then we'll be back again in 2025. I've already got guests. I'm recording the first one of '25 tomorrow, in fact, with Jamie Indigo. That's a, that's a pretty good way to kick off 2025, right?

I'm so pleased it's finally happening. So I saw Jamie's, um, Well, whole section, the SEO section on the Almanac.

That's exactly what we're talking about.

Yeah. And then I was like, oh, Jamie's been on the podcast, surely.

I thought the exact same thing.

And then I was like looking through, I was like, no, Jamie has not been on the podcast. She must be.

I literally did a site search of like site with canada.co.uk Jamie Indigo.

We must have done that.

We've mentioned her a couple of times. Like, Jamie's never been on the show. And then literally I think we had that conversation and Jamie texts me like, why haven't I been on the podcast?

It's so weird. And I messaged up. I was like, that's so cool you did that because I was literally like saying, Jack, why has Jamie not been on?

Exactly, exactly. Dream guest. I'm really looking forward. Jamie's one of my favorite people to talk anything, basically metal, D&D, SEO, JavaScript, all that fun stuff. And yeah, we'll be breaking down basically the SEO chapter of the Web Almanac, how the Web Almanac is put together along with Dave Smart and the rest of the guys who do the SEO side of things. And there's some really, speaking of really interesting data and stuff, there's some juicy stuff in the Web Almanac. So that'll be on the 6th of January, if I remember correctly, that's the right date. Uh, and yeah, kicking off 2025 with Jamie Indigo, which is very, very exciting. Until then, Merry Christmas. Thank you so much for watching and listening throughout 2025. We'll be doing more of this, chilling on the sofas and hanging out and stuff in 2025. So we'll see you then. Bye-bye.