AI and SEO: This Is What You Need to Know! with Mark Williams-Cook

For me, that's what I'm talking about in relation to this hype cycle. I think more people are realizing now, and it's certainly been my experience when I've gone around to other companies who are like, oh, we've set up this AI workflow to do this. And yeah, it kind of looked great. But then actually, when we got down to it, it was making these errors, and it didn't work so well. So we had to take it offline. But our other competitors doing this and this, and then I go and speak to them. And basically, they're the same. Everyone's kind of peering into each other's gardens being like, oh, they're doing something amazing.

Yeah.

That reality, I think, is still appearing for some people.

Search engines are reinventing themselves, and what's coming will fundamentally change how brands get discovered online. Mark Williams-Cook, my guest on today's show, is the director at Kanda and founder of Also Asked, an intent research tool. Mark has spent 22 years in SEO, from agency director to affiliate marketer to SaaS founder, which gives him an unusually clear view of where the industry is actually heading. In our conversation, we explore how AI agents are replacing traditional search behavior and why companies like OpenAI and Perplexity are racing to launch new browsers to compete with Google's Chrome. Mark explains what happens when the user interaction data that powers Google's rankings simply disappears. He shares his agency's pivot towards building multi-site brand presence rather than obsessing over keyword rankings. We discuss why he's skeptical of prompt tracking tools and also break down the difference between pre-training and post-training in large language models. We discuss the MIT research showing ChatGPT users experiencing brain atrophy, the hidden costs of AI convenience, and why understanding what base models think about your brand matters more than chasing individual prompt results. So without any further ado, on with the show.

Mark, it's so great to have you on the show.

Really excited to be here, Stefan. Thanks for inviting me.

Yeah, so I'm a fan of your Also Asked tool. I've, as you know, written about it in The Art of SEO, and I'm just a really happy user of the tool. And I love what you post on LinkedIn about SEO and AEO or GEO or whatever you want to call it. And just AI generally and where the industry is heading. So I'd love to start with maybe a little bit on your take of where the world is heading, the industry and user behavior.

Wow. Start with the small stuff, huh? Yeah. Why not start with the easy questions? Okay. So I think that it's helpful to define changes in terms of both technology and how they impact and shift user behavior because the two kind of lean into each other, right? And we have to be aware of both of them. The thing you mentioned about this discussion around, is it SEO, is it GEO, is it AEO? It's an interesting conversation. My viewpoint on that is that a lot has changed and there's a lot of new and different things we need to think about as marketers in terms of a lot of the practical, pragmatic things that we are doing. If you were doing SEO in a way that I personally deem as a good, correct way, I don't think too much has changed. For marketing in general for brands, the evolution that I think is the most interesting is this. So in the maybe say 1950s, if you ask someone what brand was, I think a reasonable definition would be, it's what the company tells you they are about, right? They run TV ads, they put billboards up everywhere. And it's just this game of you get seen everywhere with this phrase and this tagline that associates you with that. And then we slowly moved into the internet and people being hyperconnected and brands shifted from being less what you told people and more what en masse they posted reviews on Trustpilot or discussed in forums and said, you know, this brand, the quality is great or the quality is terrible. So it became harder for you to control that as a brand. And that's where traditional kind of SEO, if you like, has sat. Where I think we're going is the definition of brand will essentially be what these AI LLM likely based systems think, to use the word usefully for want of a better word, think about your brand, how they understand it. One clear thing we've seen over the years with technology is if people can get a similar result, a similar output for less effort, they will do it always. It just wins, right? And I selected this year the conveyancer to help me with my house move by using Perplexity to gather loads of user reviews and give me summaries of pros and cons of different companies that might help me and made my shortlist from that because it was a lot less effort than what I'd previously do of Googling loads of things and bringing it all together manually and kind of checking it. So I think that's something that SEOs, marketers have to consider in general, that paradigm shift of people are going to be having conversations with agents that perform these tasks on their behalf.

You know, one analogy I really like, I forget where I heard this from first, but search is like a mall and AI is like a concierge. So if you go go shopping in a mall, you're, it's kind of on you to go from store to store until you figure out who has what and what you're going to buy. Whereas with a concierge, you can make a whole set of instructions and they could even order something to be delivered to your room.

Exactly. I think that's a really good analogy. I'd used one about a similar, like a supermarket, whether you whether you go in yourself, because supermarkets are designed in a specific way, right? Whereby all the essentials like bread and milk and things are normally at the back. So you have to walk past everything else. And then when you get to the queue, you have all the, normally like in the UK at least, all the high sugar impulse stuff. They're designed to get you to see more and spend more. And that's what we do with our websites, right? They're designed for conversion. They're designed for action. And you could argue that in some ways that's anti-user in terms of that system's rigged to benefit the company, not the user, right? And it's especially prevalent when you do things like you try and book flights and then all these extra charges creep in and they're trying to give you dark patterns so you upgrade and things like that. So I think in that way, an agent solves some of these pain points for a user. They reduce friction by putting, as you say, that concierge effect in, which I think is super interesting in terms of the strategy, if you like, of how do we market ourselves and how do we want to be seen? Because you can't rely on those, I think it's fair to call them almost tricks, to get that extra conversion. The other side I think is interesting is how the technology is changing. That's just fascinating to watch for me. Companies like OpenAI, who at the minute are just burning through money at a terrifying rate. I think it was like $6 billion last year they lost. This year it's on track for around $12 to $15 billion they're looking to lose. And then there's just some astronomical amounts in the future. They've just said, yeah, we're going to burn through this. Yeah.

Like a trillion dollars or something.

Yeah.

Yeah, on what, $15 billion of revenue? That's just— and some podcaster and VC guy interviewed Sam Altman recently and called him out on that. And Sam got really ticked off and said, well, if you want to sell your shares, I could help you find a place to, you know, to dump them.

I saw that. I saw that.

Really rude and reactive. Yeah.

So I think that's, I mean, what I think will happen there is from a technology point of view, when I was thinking about this, it's clear to me that the LLM-based, if we call them that, LLM-based searches have moved towards doing more kind of web searches in the background than previous iterations because there are some pretty sharp edges to using LLMs with knowledge cutoffs, hallucinations, that kind of thing. Now, they don't have their own ranked index of the web. We know they're doing searches in the background with Bing or with Google or scraping Google or Bing API, et cetera, because these companies have been around a few decades. They've got a lot of experience in ranking web documents for queries and battling spam, all this kind of stuff. And none of these platforms can do that effectively now. That's easily demonstrated, right? What I find particularly interesting is we know that if we use Google as an example, Google uses a lot of user interaction data, especially on their SERP, to work out if a result is good, if it is relevant, and lots of other things like the people also ask questions. That's like a feedback loop for Google to understand the the query, right? And we've seen from their internal presentations that came out in the DOJ when Google was like, yeah, we're really bad at understanding if the content's good. We just rely on user signal feedback. So my question is this, what happens if the majority, for argument's sake, people start using this LLM-based search, which is in the background doing that web search for them, and they're not going, they're not clicking on Reddit or you know, any other sites. So that means that wellspring of data that's vital for modern search to perform well just dries up.

Right. Because the financial incentives for companies like Reddit go away when there are no users visiting the site and clicking on the ads.

Yeah. And I mean, for Google as well, to understand, like, people, there'll be way fewer people clicking on their SERP because ChatGPT is doing it kind of for them. So they don't know anymore, Oh, was result 3 better than 4? Was 5 better? So it wouldn't surprise me if we start to see a degradation of quality, but then that's what's powering those LLM systems. And that stumped me for a while of, well, what's the long-term game plan here? And then you see Perplexity, OpenAI quite aggressively start to push Atlas and Comet, their own browsers. And it became quite clear to me that For 2 reasons. One, that's the best place to get user interaction data from the browser because it gets around as well a whole bunch of privacy issues as well. And secondly, we've seen Chrome now has a bunch of little AI models inside it, LLM stuff inside it. If some of this inference can be run locally in the browser, That could potentially be a huge cost saving for these companies if we can run some of this stuff locally using the machine's power rather than it going onto the bill of OpenAI. It also solves a whole bunch of problems around the agentic side of— you've seen now there's a lot of talk about, well, OpenAI really struggles with stuff like rendering JavaScript. It just doesn't seem to do it a lot of the time. If you're using their browser, Your browser is going to do all that rendering for you locally and you don't have to pass bot checks. So that to me, I think that's where we're heading into like new browser wars territory. I think that's going to be the new big kind of thing they're going to be pushing for us.

Yeah. I don't know. Uh, Google has way so much of a head start that Comet and, uh, and Atlas are Terrible browsers.

Well, someone would describe me, they're as bad as they're ever going to be.

Yeah, right.

And you're like me, right? You saw, you remember how dominant was Internet Explorer back in the day, right? It was just, it was like Internet Explorer, right? And then a tiny bit of Netscape and Opera. And that looked on top, you know, like you couldn't topple it. It was like 90% market share, right? Back in the day, it came with Windows. And then Firefox came and Chrome came and they really took that lunch away. So I'd never say never, but yeah, Google, I mean, I think they just posted their, was it first, was it their first quarter of a, it was something like their first quarter of a trillion quarter, financial quarter or something. It was just a gross amount of money they were making. And when you think about that, the infrastructure, the technology they've got versus companies that are just burning through investment. I think Google has such an advantage, like you say, apart from the market share dominance, the technology dominance, the money to keep going, they can just hold their breath longer, right?

Yeah.

And I think Atlas had a lot of Chromium. Atlas was Chromium, right? And it had Gemini models inside it as well. So they're building on Google tech.

Yeah. But then the security and privacy issues because it's like both browsers are open to getting hacked or injected and things like that. So you're taking a risk. You might get your identity stolen. You might get, you know, you're— you might get doxxed or who knows what if you're not careful. I just— I don't— I don't want to take the risk.

Yeah, I think So for the other thing I'd discussed previously with some people was for agentic to be a thing and for it to happen, I think it needs quite a lot of information about you. Because I want to be able to say, you know, find me some new running shoes. I don't want to have to say, find me some new running shoes that are size 11, that are for men, that are these colors, that will ship to hear. It would be tiresome for me to have to manually prompt all of these things all of the time, to the point where it'd be quicker just for me to Google the few companies I like and do that. So I think that's going— I think it's going to happen. And I agree with you from a security aspect. And I think that's, if anything, the direction we've been heading in for many years. that more of our identity is online and potentially stealable. And I wonder how much the younger generation— it makes myself feel really old here now, but how much the younger generation cares or thinks about that, because wherever they join in, that's like the new norm, right? For me, even things like, it took me a while to warm up to stuff like password managers until I really thought about it. It's like, so you're telling me I put all my passwords in one place and someone else looks after them? This sounds really— I don't know about this. It's happened to a few password keeper companies, right, that they've had data breaches. So yeah, I agree with you, but I don't necessarily think that will stop anything.

Yeah. One of my clients just came out with a book on privacy and how to protect it, and the book's called Privacy Crisis. This is Chris Parker. He's the founder of whatismyipaddress.com, which is an SEO client of ours.

Yeah.

Yeah, they've been going strong for 20 years now, 26 almost. Yeah.

Definitely used that website before.

Yeah, well, I'm sure because they've got tens of millions of users. It's like 16 million or 14 million unique visits a month.

I had no idea so many people were that interested in their IP.

It's not just— it's like things like, you know, lookup tools for all sorts of things relating to your, your internet connection and, you know, VPNs and all sorts of things. Anyway, so this book, Privacy Crisis, is about protecting your privacy and not being a hermit in the process. So I think this is going to be a big issue for everybody, like how much privacy are you willing to give away?

Yeah.

And autonomy, because and also brain power. Because you know, I don't know if you saw this, but MIT did a study of ChatGPT users compared to a control group and found that their brains were atrophying. So you you you don't use it; you lose it. And so the less critical thinking skills, the lower the brain flow to certain parts of the brain. The more you use ChatGPT.

I think privacy versus convenience is the age-old thing, right? And unfortunately, that would likely be driven again by workplace-type pressures of if you don't use these tools, can you be, as sick as it sounds, as competitive as other people in terms of usage of your time? And yeah, that's the brain atrophy thing. So again, to make myself feel old, when I first started driving, it was the paper atlas, like, work out where I'm going ahead of time, plan some roads, kind of memorize them, and then work out, look around where I am. And that was fine. I didn't think much of it. And now I use Google Maps on my phone for for everywhere. If I'm just going to—

Even the same place you always go, right?

Yeah, because it does the traffic stuff as well, right? Tells me if there's roadblocks, like all this. But as a result, I have absolutely noticed my sense of direction now has massively deteriorated in terms of my awareness of where I am and how to get to places. And I only figured this out when I moved house recently. And I'd been living there. This is really embarrassing. I've been living there like a month or so, and I realized, because it was quite a bit further out from where I was, I didn't know the way without Google Maps because I was so subconsciously following that, thinking about other things just on the road, what's around me, not thinking about the direction I was going, which roads I took. So I actually had to consciously think, right, I need to pay attention to where I'm going now and map this out in my mind. So it doesn't surprise me at all if we start using LLMs for cognitive tasks that, yeah, it will atrophy. Use it or you lose it, I think is super good wisdom for anything in life, right?

Yeah. Yeah. Like just being aware of what direction you're going. Am I going north or east or west or what? You know, like you should know this stuff. You should be able to add stuff up in your head without getting your phone out to Start up the calculator. So let's talk a bit about where your agency is directing its efforts and how you're future-proofing your agency as, you know, some accounts, some agencies, some different freelancers and so forth are struggling. And they're, you know, having to pivot or take a job or do something, uh, not ideal. Like, what, what, what are you doing and how do you see, uh, this, uh, this working out over the next, say, 2 or 3 years?

At the moment, the— from speaking to other agencies, freelancers, etc., the impression I'm getting is At the moment, it's people that copyright, that produce content are in the toughest market at the moment. So the people that I've been helping to find other opportunities, they're almost invariably in those kind of roles. And I can see why, right? We're really in this Gartner hype cycle of kind of inflated expectations. I feel we're hopefully now just coming over the peak of that in terms of what AI can do. And this was really crystallized actually for me this week, I think it was. We've seen now for over a year or so all these AI workflows and multistage things using LLMs, these business kind of flows. But then this week, Sam Altman's really proudly announcing that their flagship GPT-5 model can finally not use em dashes reliably if you tell it. And to me, I'm like, a few months ago, he's saying you can talk to it, it's like a PhD-level expert on any subject you want to talk to it about, but at the same time, you can't make it not use an em dash. So my question is, if it's involved in 5 pieces of this chain of critical business stuff that needs to be correct, how is that going to work? So for me, that's what I'm talking about in relation to this hype cycle. I think more people are realizing now, and it's certainly been my experience when I've gone around to other companies who are like, oh, we've set up this AI workflow to do this. And yeah, it kind of looked great, but then actually when we got down to it, it was making these errors and it didn't work so well, so we had to take it offline. But our other competitors doing this and this, and then I go and speak to them and basically they're the same. Everyone's kind of peering into each other's gardens being like, oh, they're doing something amazing. So that reality I think is still appearing for some people. In terms of our agency, in terms of Kander, so we've never really done much in the way of content production. We've always been in terms of the content stuff we do. So we're doing from a marketing point of view, basically only SEO. So organic SEO, that's all we do from that point of view. And the tenets to that are still similar to what they have been for many years, which is we've got a very strong technical team. I think that's more important than ever for a variety of reasons. So again, looking into the future, I think making sure websites are usable by agents is, and by that I mean agentic agents, not kind of browser agents or whatever. So there's a whole bunch of tests you can already start doing on e-com sites. Like you said, at the moment, Atlas, et cetera, they're very bad. But you can start to get an idea of, I think that's going to change quickly where that's going. And again, the absolute basics of does the main content and links, everything work without JavaScript? All these things are still important. My prediction that I got horribly wrong 10 years ago when I was having similar conversations was, oh, I think technical SEO will become less important because search engines are going to become better. Right. What I didn't bank on is as an industry, we just made things so massively overcomplicated normally with like client-side massive JavaScript frameworks to do the most simple things. that tech SEO has flourished. We do a lot of work with content strategy with clients, and I think that's going to be around for a long time still. The actual content production has changed significantly in terms of we do use AI to do things like historically, if there was an expert in a client's organization that we wanted to get something from, we might provide them with some kind of brief to collect information from them, right? And it's very difficult to get people who aren't writers to do this kind of thing. They might just basically verbatim answer the questions. What we've been able to do now is like this, have more casual conversations with them and let them naturally lead the conversation. And we get some really interesting stuff. Then you can use AI to do a single pass of basically, can you put this into a document, into an article? And then we can edit it from there, which is massively improved the output in terms of the actual quality, let alone the speed. Digital PR is probably the largest portion in terms of billing of what we do. And it has, again, has been for a long time because links are still 20 years on really important. But from an LLM AI search point of view, to me, that's a multi-site problem. So we've changed things like our keyword research process now. So we take traditional keywords, we can personalize them with persona descriptions to feed them into LLMs to get related, to get the kind of prompts people might use, combine that with other search data, and then we can make a prediction on whether grounding is happening within those LLMs. And then we can see what grounding searches are being done. And then those grounding searches tell us, okay, these are the 20 sites or our target that we want to be mentioned on and we want to say these things about us. So how we're actually doing the same things has changed quite significantly, but that demonstrably works. We get on those sites and you can change what ChatGPT shows about you in days sometimes, which is very satisfying compared to traditional search results, which is you do the work, you build the links, and then you cross your fingers for 3, 4 months to hope when the tide comes back around of Google that it's worked.

What if it's Reddit or Wikipedia that you need to get a link and a mention and you've got terms of service and conflict of interest guidelines that you have to navigate and all sorts of stuff?

So the Reddit thing is a really good question, right? So I don't believe the right thing to do is what I've seen many people saying, which is, oh, your company needs to be on Reddit and you need to be posting here with the idea that your community is there, which seems obvious, but I've got a Reddit account now that's old enough to drink. And so I've been around there for quite a while. And I know what happens when a lot of companies try and come onto Reddit because there is a certain etiquette and culture there that again doesn't blend well with what we were talking about earlier from the website point of view, which is like anti-consumer interest kind of thing. So for our clients, if they have an audience on Reddit that is strong, then the advice I give to them is outside of SEO, let's not be the person that's climbing through the window at the dinner party to try and come into the table and talk about us. Let's give them the spark. Let's be the thing they talk about. So what are they interested in? How can we make a story? How can we publish data? How can we show something about the product? How can we tell a story that will make this community talk? And where else can we put it? So we don't even need to post it. That to me is the scalable way to do this because the LLM kind of stuff is, you know, you need to work with social teams, you need to work with CRM teams, with sales teams. Because what I said to begin with about brand being the impression left in AI, this is going to be across all channels it has access to, it's going to be multimodal as well. So this isn't anymore an SEO discussion about, you know, we need links from this website. So I think there's a much stronger case for bringing together different teams now and working on being the source, the thing people want to talk about, which is a different strategic direction because it's not easy and sometimes requires companies to do things they haven't done before. But I think it's more sustainable than going on Reddit and trying to force start those conversations, even if you're being open about it.

Yeah. So are you doing link building and citation building and digital PR for your own agency?

For Candid?

Yeah.

Um, so we, we occasionally do, um, like we get, well, we get requests to do like comment on, on things. I do a lot of podcasts. I do a lot of events now, but in general, um, We don't do much kind of outbound digital PR stuff. If opportunities come up, since we have a digital PR team working like all the time for clients anyway, we see opportunities come up naturally and I might get a Slack being like, oh, do you want to give your comment on this? Because someone's asking for this, which I'll do if we have time. But in fairness, so the previous SEO agency I used to run, we used to rank number 1 for SEO agency when it was like a more national search term. And the leads I got from that were very poor from an agency. Yeah. So, and I think, yeah, there's certain markets where, you know, I don't want to do SEO for someone that is figuring out if they need SEO or how SEO works. That's not where we are as an agency. I want to do, I want to work with all organizations and clients who are like, okay, we know we need SEO, we understand what it is, we're bought into it, we just want to find someone that's a good fit for us and will align with what we're doing. That's who I want to work with. And it's fairly rare in my experience that those people are going on Google and searching for SEO agency or similar.

But what about ChatGPT? What if they're going on to ChatGPT and saying, I need to hire an SEO agency and I have a $20K a month budget, uh, and, and here is, you know, the kind of areas of focus and the, the, the business case and so forth. Find me an agency.

So I think this is a more interesting question, right? And actually where it aligns more with what we've naturally been doing anyway. Um, because for instance, you know, I'm mostly known through like the stuff I do on LinkedIn, right? All of that kind of activity I can see has been picked up by ChatGPT. So if you ask like in the UK about, you know, top SEO experts and stuff, a bunch of the times I'm on those lists without really having to specifically try and do anything because it's been more of this multi-site focus. Whereas I think the, you know, as a website, our actual with Canva domain is fairly weak in terms of actual rankings. I haven't tried to make it to rank for anything for the reasons that I've come about. So this is more, as I said, for the AI side of things, having this multi-channel strategy of just trying to be in different places. And that's what we get through podcasts. I do a lot of conference speaking. I publish things on my social media, like data from Also Asked that gets covered on search engine land, things like that. So you're kind enough to put me in an article and a chapter of your book. So all these different things, all these different touchpoints, I think are what rise the tide.

Yeah. Let's actually talk about Also Asked. How did that come about? And what are some of the use cases for our listener or viewer in this day of AI search and so forth? What are we wanting to use it for and how does it get used with AEO?

Okay, so the idea, so I've been using People Also Ask data, which is when you do a Google search now, most of the time you'll get a kind of concertina box of normally 4 questions or so, which says People Also Ask and gives you some questions that if you click on, they expand, you get more questions than an answer. I've been using that data for over a decade for SEO. And I gave an SEO talk in 2016 where I was just doing a normal SEO talk and I was talking about how we use this data. And we, at the time, we used just like some command line Python stuff to scrape it and do some stuff with it. And I assumed most people were doing something similar. And after the conference, I had a bunch of people come up to me, oh, can I have the script to do that? It's really great. And I was like, sure. So I shared that, much to my folly, as then I was getting support requests of, oh, it says I'm missing this package, or this version of Python's not right. And then that made me realize there's obviously a barrier to entry there, right, of people that aren't comfortable coding. So I said to the team, okay, well, maybe their space will do an online version that does the same thing in the background. So we made the alpha version of Also Asked, which was wildly popular. We were doing hundreds of thousands of searches within a few weeks, which was problematic because we had to use proxies and it cost money. So in the end, after a few months, I just had to turn the alpha off and say, thanks for the feedback data because it was costing literally like over £1,000 a month to just do the basics. And I thought this would be nice and quick. We built this in like 3 months, another 3 months we'll have the paid version running. Oh, my sweet summer child. My first foray into SaaS. It was pretty much 2 years before we had the kind of final one that had passed things like security checks that was scalable, all the things I didn't know about. Like if you sell a digital service product globally, you are liable to charge the correct amount of tax locally, whether it be by state or country. All these things had to be thought of. So we got that launched, which was great. So now we have a way that you can put in any search, any query that you might put into Google. You can specify a country and a language, and now you can actually specify even a city level because there is city-level personalization quite prominent for a lot of searches in Google. And Also Asked will return a map of, okay, for this query, these are the 4 or 5 most closely related in terms of what I call intent proximity, meaning if someone asks this question, These are the other questions they are very likely to ask. And then that spirals out to another 2 or 3 levels. And we can also return things like now which websites actually are selected for those answers, what title they have, if it's an AI overview, because they're quite common now in PAAs. And then if it is an AI overview, in the data export, you can get which sites have been cited and such. So it's like a intelligence tool from that point of view. And in answer to your question about what is the use case for this, the original kind of primary use case was, I know that one of the things Google looks at in terms of their own measurement for have we done a good job is time to result, which is essentially the point between when a user starts kind of an exploration journey for I have this intent to when it is satisfied. Paul Haar did a brilliant talk about that at SMX. West again, probably like 10 years ago, talking about how they do this. And to me, it's obvious, if you have this list of questions from Google saying, well, when someone asks this, normally their next query is this, that a lot of the time it makes sense to have those things answered in the same document, in the same piece of content you are writing for. cut down the time for the user. It's almost kind of what AIOs or feature snippets are doing in a way for saying, well, I know you're going to ask this, so I'm going to find all the answers and cobble them together for you. And I had someone literally last week at ISS in Barcelona come up to me and say, oh, we started using Also Asked and I was applying this for all of my content and we saw results within months. And it's so obvious now that we didn't think about it that way, which Really makes me happy to hear that. So the primary mistake I think people make is trying to just answer the questions like they're FAQs. That's not the way I approach this data. It's about just saying these are the kind of things people are worried about. So for instance, that can be used from a brand perspective. So a common example I give is if you do a search for something like Revolut, which is one of the challenger new online banks, common questions are around, like, is it safe in the UK? Like, how much money can I put in there? How much is protected? So people are really worried about the security side of things, and that's super important to know as a brand. So on our homepage, we need to give people some reassurances that their money is safe and how that works and where we're governed, et cetera. To answer the last part of your question, how is it useful in like an AI world? We specifically use the People Also Ask data to map out the rest of a conversation. So I spoke earlier about how we can synthesize, and it's a really neat way of using the LLM data, the conversational type prompts people might use, because that's exactly the data that LLMs have been trained on. They're trained on specific forums of someone saying, oh, I'm a 38-year-old man and I'm interested in running, what trainers should I buy? Also, I'm a vegan or something like that. So it's a good way to get those initial prompts. However, we know that people don't use LLMs like search in terms of they're doing all these one-shot type queries. It's an ongoing conversation. So once we have those initial prompts, we use the Also Asked API to say, okay, if someone's asked this, What's likely going to be the next questions that they ask? And then we include that in our map of these are the things we need to be visible for to appear in these AI searches. And it's always to me funny and predictable when I see, so I always use running shoes as like an example. And the questions I know off the top of my heart is people asking like how much they should spend on running shoes. And then it's obvious to me that For a lot of people, running shoes, like actual running shoes, are a bit more expensive than people realize because then there's a whole other branch of queries around basically how much faster will expensive running shoes make me? Because people are interested in their 5K time. They're like, well, if I spend €200 instead of €50, will it make me run faster? It's just cute to understand all those things, but that's what's on people's minds. It sounds silly. But that's the information people want. So understanding that is critical. And there's all other ways to finish up. We've got a built-in now script with Screaming Frog. So if you use the Screaming Frog spider tool in their custom JavaScript library, they send it out now with an also asked script.

Okay.

And what you can do with that is you can crawl all of the content on your site and it will do things like scrape the header tag or the page title and send that off to Also Asked in the background. And then it'll use the ChatGPT API to say, does the content on this page answer all of these questions? If it doesn't, then let me know which questions are potentially unfulfilled. So you can just press crawl on a site, come back, You've got 1,000 pages of content and then you can see immediately these are the questions where I might have kind of content gaps. And that's particularly useful because these PAA results change so quickly. Like when we have an election in the UK, it's a really good example. They sometimes change like every 5 or 6 hours depending on what people are searching for. So that's something else we use them for is looking for intent shift because intent is not static.

Yeah. Well, also when you are using, say, ChatGPT or Google for that matter, and any of the LLMs, you get these query fanouts that will vary on each search or each prompt. Like every time you prompt with the exact same prompt, you'll get a different set of, of subqueries.

Yeah.

most people don't realize that that's happening behind the scenes, but you can actually see what those subqueries are. So there might be some sort of analogy here between query fan-out and the People Also Ask and the Also Asked tool.

Yeah, absolutely. So that's a really interesting point, and I think you're right that most people don't realize that the query fan-outs are not stable either if you do 3 searches, because I believe for AI mode, it's Gemini that's generating the query fanout. But it's not quite that simple because I know they use a custom Gemini for AI mode and they've talked about how it leans on their knowledge graph. And because Google is just a collection of different microsystems that talk to each other, right? But certainly that kind of data, I feel is some of the best we have in terms of when people search for this, this is likely what they're going to search for next. In combination with trying to ask, we've got a Gemini grounding API we can use where we can do things like give it questions and it will tell you if it's going to use grounding and it will tell you the web searches it will try and use. So they're all good things to do.

Yep. And just for our listener who's not familiar with grounding and what that means, just a quick explanation of that and RAG.

Sure. So when you do a search on an LLM-based system, a prompt, there will be a decision made as to whether that needs grounding. And what grounding is, essentially, I'm going to go and check the answer against the web, doing a web search. And the reason it does this is it's got to do with the kind of bell curve of probabilities of the things it thinks it should say next. So to make that clearer with an example, if you gave it a simple query like, is the is the Earth round? Okay, there's going to be a very steep bell curve of training data being like, yes, the Earth is roughly a sphere, it's roughly a globe. So it's highly unlikely that it's going to expend the money and resources to do grounding, to do a web search in the background. It's like, no, I got this. And it will just tell you. At the other end of the scale, if you say, what happened in the news today? So the LLM is going to look through its kind of training data and be like, okay, if I've got this prompt, what's likely to come next? And there will be a very flat kind of bell curve. And it's like, oh, I don't know, this could be anything. It generates a probability that it'll need to do a web search. And when it gets above a certain threshold that are different for all the LLMs, it will go off and do these web searches in the background that it thinks it needs to find out this information. It'll very quickly review the content on the documents it gets back from an information retrieval system like Bing or Google. And from that, it will try and— it does RAG, which is Retrieval Augmented Generation. So it's still generating from its model, but it's leaning heavily on the data it's retrieved to generate an answer, and it's looking for some kind of consensus if possible. Hallucinations are still possible. And again, especially where you've got particularly novel searches where there's very low training data, they're still quite likely. I did a webinar last week where I got, just to give you an insight, I got asked by, I think it was a school, how can we write our content to prevent hallucinations? Because students have been using ChatGPT to find out when various deadlines are for submitting work, and it regularly gets them wrong because obviously there's nothing really in the model about that specific school for this academic year for these deadlines. So it's just, it can't find a consensus. Maybe it can't find what it needs to on the web searches, so it's just making stuff up. And again, a lot of people aren't aware of that.

Yeah. Well, hopefully that's becomes less of an issue over time. I think the hallucination rate is somewhere around 15% at this point. Is that right?

Yeah. So there's a, for ChatGPT, they have a standardized, it's called SimpleQA test, which they publish on their technical card, which gives the hallucination rates. And there was a paper recently done by OpenAI that essentially said, yeah, then they're never going to go away completely, hallucinations, at least from the base model, because it is kind of a feature, not a bug, because that's the generation part, right? The spice that means it's not the same thing every time. But again, I would be humble in that this isn't my expert area in terms of if we tried to spend an hour explaining how transformer architecture works, I'd very quickly be out of my depth. So maybe there is a solution in the future, and it's my shortcoming that I don't see technically how it would be possible.

One more question before we move topics in this area of AI. So can you differentiate for our listener pre-training versus post-training? Sure.

So when we're dealing with LLMs, we have what's called the base model, which is essentially most of the documents on the internet that they can beg, steal, or borrow, depending who you speak to. Yeah. And there is a phase called pre-training, which basically has a bunch of steps to clean up that data. So this isn't as well published, like, specifically as far as I'm aware. Like, OpenAI hasn't said, this is how we do our pre-training, this is what we look at, this is what we don't look at. So this is based— actually, a lot of my knowledge is based on a chap called Andrew Karpathy, who's posted some really, like, 3, 4-hour-long explainers on LLMs because that's his expert area. But this training phase is essentially where they'll do stuff like they'll try and filter out personally identifiable information, they'll categorize by language, for instance. And what I found particularly interesting is from the sources I've looked at, they will tend to strip out things like code. So the HTML code around the content, because what we're doing to, you know, the, the basis of the LLM is it's doing token prediction. So tokens are chunks of characters basically that commonly occur. So I think GPT-4, um, is in the around about 190,000 different tokens they have, which are again strings of characters. And they're basically, this whole architecture is designed to work out the probability of if you have this set of tokens, what the next most likely set of tokens is going to be. It's essentially super, super, super clever predictive text, which is why, again, it doesn't know if it's right or wrong. And it doesn't make sense to me if you're trying to— and this is, I think, the beautiful thing for me from a maths point of view about LLMs, which is that by mapping out by breaking our actual language down into these chunks of characters and just mapping out all of the billions, probably trillions of probabilities between these chunks of language, there is somehow knowledge encoded in that, which I think is a fantastically beautiful thing from a maths point of view, that we can have all these probabilities and just kind of pull the string and that you've encapsulated a lot of human knowledge on there. So it doesn't make sense to me why we would include stuff like HTML code if we're trying— what we're trying to do is encapsulate the knowledge of people with the LLM. So we go through this pre-training phase, which is this reading all the documents on the internet, filtering them down. So essentially we've got one Bayeux Tapestry-style endless document of the internet. And that goes through this process of tokenization and working out the relationships between those tokens. What you end up then with in the base model is essentially an internet document simulator. It's not a chatbot. So if you said to it, tell me some good places to go in France, it would just continue writing what it thinks the next most likely thing would be, which would probably be like a forum post. It wouldn't answer you like the conversational kind of things we have now. So then the way they get these models to behave and act in the way we want is to essentially hammer them with then more training after this pre-training is done. So they will hammer them with conversational type data for responses to try and get the token prediction to be this back and forward way. And one of the ways they can do that is they introduce these tags in the training data That's kind of like agent-user. So it doesn't understand, because I don't want to anthropomorphize it, but it sees this pattern of this kind of back and forth, which is then when we get a model that's like a chatbot where you can say, where should I go in France? And it understands it kind of needs to jump down in the token prediction to the answer part, if you like. It's also how some safeguards are built in. So things like why it's more tricky than it used to be, but certainly still not impossible to, for instance, get ChatGPT to tell you how to build illegal stuff that you shouldn't, that's information that's normally protected. But then there's been cases where people have asked it using leet speak. So for the younger people, it doesn't really exist still now. That's like replacing some of your characters with numbers, like U would be J00, hero. And because the training data doesn't account for that, the chatbot immediately gave all these instructions about how to construct bombs and stuff just in leet speak, because the guidelines it was given are very soft. There again, it's about token prediction. It's not understanding you can't talk about this subject. So that's the pre-training and then the post-training, which comes in, again, various kind of flavors. A lot of it as well is kind of manually curated in terms of human conversation. There's a lot of effort that goes into that last bit. But the actual LLM that we see as well, that you actually use, there's also a system prompt, which is fired when you begin those chats, which the analogy for that I use is, it's like, say you've trained to play a football game. Right. So all the training you've done is kind of like your pre-training, if you'd like. And then you've maybe done some training that's specific on scoring goals. That's your kind of post-training bit. And the system prompt is what the coach tells you as you go onto the pitch. Don't forget to do this. So it's in the front of your memory. So the system prompts are kind of fascinating because you realize how dumb LLMs are. So it will say something like, today is the 18th of November. If anyone asks you about this date, refer to it as today, would be the kind of things that go into the system prompt so it seems more natural. And they can be quite big. And again, they'll give instructions about how to answer, tone of voice, what not to answer. And I think this is what they were tweaking with GPT-5, because famously they made it a little bit less sycophantic. Which some people were sad about because GPT-4 obviously was kind of their friend and then GPT-5 was a bit harsh. So there's that system prompt and a few of them have been leaked before, which is again really interesting. And then lastly, you've got the context window, which is really this short-term memory for the conversation you've been having, which will affect further token generation. So the example I give for that is if you ask ChatGPT for a vegan recipe, it will give you one. If you then say, can you recommend me some running shoes? It will say, sure. And I know you're a vegan, so here are leather-free, cruelty-free running shoes and brands, because that context window has affected the answer. So it's layers upon layers upon layers.

Right. Yeah, really great explanation. Thank you for that. And I know we're out of time, but I have one quick last question where you do this as like a, a lightning round. Like, what are your favorite tools for SEO and for AEO?

Uh, so my favorite SEO tools, um, I'm always using Sitebulb Cloud, uh, for my auditing, uh, now for a long time. Loved using that. Um, Keyword Insights, um, as well for kind of clustering. It's a tool I'm using regularly. Um, From an AI, AO, LMO, disco, what we're calling it, my favorite approach is a tool called WAIKAY, which is W-A-I-K-A-Y, What AI Knows About You, by Dixon Jones, who's—

Yeah, I've had Dixon on recently to talk about it.

So, okay. So yeah, I just like the approach they have taken in terms of it's more focused on understanding what the base model thinks about your brand. Which I think is much more helpful strategically than coming from the top down looking at prompts for a whole host of reasons. I know Dixon's recently put prompt tracking in because it's what everyone wants, but I'm like, to me, if you understand what the AI knows about you, that is what's going to generate answers to these prompts. So that's my focus. That's the main tool I've been using for that. In terms of other kind of AI tools, it's mostly the standard SEO toolset I've been using. And I'm not a fan of myself, like wide-scale prompt tracking. I just think for the amount of time, energy, and money it costs, how quickly the AI results shift, I just don't see the value in it myself yet. That may change.

So when you're talking about that, you're referring to like Brand Radar from Ahrefs and Profound and so forth?

It's just, yeah, the way of saying, okay, well, where do we— does AI cite us for like these 2,000 queries, right? There's so many, I think, issues around that point of view in terms of things we mentioned like personalization, the fact that the answers are non-deterministic, so we have to really run them multiple times, which is fine, to understand where we sit on that bell curve, and that the answers, the citations do change over time. I think that kind of thinking to me is of the age, the paradigm of we're very used to doing a search and having a search engine say, this is a linear list of documents that are most relevant to this query. And it makes sense to track the rank of those things in a way, even though, again, it's not a perfect science. For me, how I at least try and conceptualize what LLMs are and how they're working, it's more of this kind of soup. It's not this linear, these are the things that are most relevant. I would compare it in a way to if I asked someone about their favorite, again, running shoe brand, they wouldn't just give me like a list of kind of brands maybe that they like, they would talk about it in a different way. And that's the metrics that we're starting to use are things like, if you understand what your brand is about, what you want to be known for, how you want people to understand your products, and you can describe that, And then you have a conversation with an LLM and you can score how well married they are. I think things like that are more important because I think there's a really exciting future actually for especially smaller, more specialist companies. I think there's a discovery problem with search at the moment because it's dominated by the biggest players. But I think for all the people that have like very, I want vegan running shoes and I want them to be green, That these kind of systems have the opportunity to say, oh, well, I know about this very small company that's actually only 22 miles away from you. And I very clearly understand that's all they do. They're this company, they're completely aligned with you. Why don't you buy from them? As opposed to now, it's like, that's difficult to find. I'll just do a search on Amazon. So yeah, that's where I'm at with that. And I'm open-minded about it. This isn't like a dogmatic stance I've got on it. It's just that the amount of effort sometimes that goes into tracking seems too much of the resource versus let's make our product better, let's make our marketing better, let's spend some money on improving things for customers.

Yeah. Awesome. All right. Well, I know we got to wrap this up. So if our listener or viewer wants to work with your agency, if they want to learn from you and follow you, Where should we send them?

So I'm most active on LinkedIn. If you just search for Mark Williams-Cook, I think I'm actually the only Mark Williams-Cook on the internet, which is scary in a way. It's very easy to find me.

Congratulations.

Yeah. I run also—

You're canonical then.

Yeah, I'm the canonical one for now. So I run Core Updates as well, which is coreupdates.com, which is a newsletter every Monday I send out, which is just what I think is the important stuff that's changing in SEO. So if Google's tweaked the font they use on the SERP, you won't find that in my newsletter. It's just the boiled down, these things might affect the way I think about things. And also we run a podcast called Search with Kander that if you want to kind of learn more, that's again weekly. We interview people a bit like this, go into subjects in more depth. But yeah, I'm always open if people do want to just chat to me about SEO. I say don't Don't be afraid. It's not like you're going to get an invoice. I like talking about it. And yeah, always, if you think we might be able to help you, you can certainly have a conversation about that.

Awesome. Mark, it was a pleasure and thank you for sharing your wisdom and insights and prognostications.

Thank you for having me. Pleasure was all mine.

Yeah. Awesome.

And thank you, listener.

So go out there and make the world a better place. We'll catch you on the next episode. I'm your host, Stefan Spencer.

If you want to hear more conversations like this about the future of search, AI, and marketing strategy, then I recommend episode 458 with Rand Fishkin, episode 500 with me, Stephan Spencer, and episode 525 with Eric Enge. Also, I invite you to check out the show notes, transcript, action-based checklist, and all the URLs mentioned in this interview of Mark Williams-Cook at Marketingspeak.com/530. If you enjoyed this conversation, please share the episode and more importantly, the entire show MarketingSpeak with a friend. It'll really help this podcast grow and reach an impact. Thank you.