From RAG to Riches - Mark Williams-Cook | SnS Antalya 2025

Thank you to everyone, Jammoh, and all the organisers. I'm really happy to be here for the second year. Really excited to see some faces from last year, and also it's really cool to see a lot of my friends from the UK, from Europe here as well this year. I've said it before, and this is probably my favourite SEO event of the year, so I'm really, really happy to be here. And I'm going to talk to you today about keyword research in the AI era, because I've been having lots of fun changing what ChatGPT says, what AI mode says about us, and I wanted to talk about it a little bit. I've called it keyword research, and I know people now say, oh, it's not about keywords, it's semantic and all this. So you have to forgive me about that. At least when I say keyword research, we know roughly what we're talking about. So, my name is Mark. This is me. You can see the resemblance. I have a little bit of AI fatigue. I see AI kind of everywhere now. AI images, LLM this, LLM that. So, I've actually resorted to drawing my own pictures in MS Paint, which you'll see around. I find it therapeutic. It's like the digital equivalent of going to live in the woods. So you'll have to, like, forgive me. But I'm not anti-AI. I actually did my first qualification in machine learning 9 years ago, and I think AI is great. I think it's hopefully going to solve a lot of the problems that we've created for ourselves, not just in SEO, although we do create a lot of problems for ourselves as well. I'm not necessarily so sold on LLMs. And as the kind intro said, I've been doing SEO for over 20 years. I put this on the slide, right, because it sounds like a cool thing to say. Cool. But I haven't worked out— this either means I'm really good at SEO, or actually I really suck at it because I've had to do it for this long. So, I'll let you make your mind up about that. Who's seen this graph over the last few weeks? Get out of the way. Some of you? Is that all? Okay, so this graph is some data from a company called PromptWatch. that was showing how often various sites are cited in ChatGPT. And obviously, the interesting thing on this graph is the Reddit citations absolutely crashed. And that happened around the 10th of September. Does anyone know what happened around about this time or the reason why this has happened that we think? Number 100. Number 100. Thank you, Jordan. So around about this time, Google changed that we couldn't get 100 results all at once. And we know ChatGPT does use Google. They do scrape Google or use services to do that to get results. So this was super interesting to me, right? So as soon as they lost the ability to get 100 results straight away from Google, the amount of citations of Reddit, which we know appears a lot in Google, crashed. Which brings me to my thesis about geo, which is that it's mainly SEO, maybe. When I put this slide up, what I want to say to you is, in my opinion, I think what we do for AI search is very similar to what we have been doing in SEO for many years. How we do it is not necessarily the same, which is kind of what I'm going to talk to you today about. But for me, that's nothing different because how I did SEO is different from 5 years ago and very different from 10 years ago and very, very different from 15 years ago, right? How we do things has to change. But this is what we're going to talk about today. So our traditional keyword lists that we love from SEO, our big lists of things we'd like to rank for, are kind of useless now because this isn't how people search in AI. We also have to tackle other problems like personalization. We have to tackle problems like ongoing conversations. So, people aren't doing these zero-shot searches in LLMs. They're having these full-blown conversations. So, these are the problems we're going to tackle. Our traditional search model was relatively easy to understand. We had people, they did searches in Google or Bing or, as I later said, wherever. And the optimisation bit came in understanding what they were typing in and trying to rank for that. And that's actually harder than you might think, right? Because 2 people in this room might search for the same thing but actually want something different, because to them, the words mean something else. Or conversely, there's lots of different ways to search for the same information by using different combinations of words. So that's one thing we had to kind of deal with when we were doing traditional SEO. Now people are just having big old conversations with things like ChatGPT, and sometimes ChatGPT is behind the curtain going and doing searches itself to verify its information. And I like to think of these almost as universal intent decoders, because when you have these conversations, one thing my research has shown is the searches that happen behind the scenes, there's quite a lot of parity to them, meaning there's a consistency in how they search for that information, which we don't actually have with users, because users are mad. They just search for the craziest things. I'm sure you've all done keyword research and found really entertaining things people search for. So, the optimisation bit has just moved a bit further down the line. And what I'm going to talk you through is an exact process that we've been using to influence these LLM results, and I've got loads of tools to share with you so you can do this as well. So this is our starting point. We have our list of traditional keywords which we've deemed fairly useless, and the first thing we want to do is try and apply a layer of personalization. Why is this important? Because if you go to ChatGPT and say, hey, I'm vegan, can you give me some recipes? ChatGPT is Of course, like, sure, here's your recipes. However, if in a week you go and say, can you recommend me some running shoes? It's remembered that you're vegan. So, it's going to recommend you cruelty-free brands or products that don't have leather in. So, you can't just take your running shoes keyword and assume that's gonna apply to everyone because we're getting personalization. And one of the ways we've been adapting to this is actually using the ChatGPT API to build prompts to enrich our traditional search terms with persona data. So we can actually pass a prompt that says, if I was, and then we can have 1, 2, 3, 4, as many as we like different personas. So if I was a middle-aged vegan white guy who's just starting running and I'm worried about my health. That would get passed along with the prompt and I would tell ChatGPT, I'm trying to find, and then I start putting my keywords in. So we're asking ChatGPT, if I was this person and I was looking for this information, what might I ask you? And ChatGPT will give you for each search term maybe 4 or 5 different variations of how someone might phrase that question. Now, I've always told people, never do keyword research in LLMs. This is not keyword research in that we are not thinking about what people are searching in Google for. What we are trying to find here is how specific people frame conversations, which is very reflective of the training data. The training data is full of specific forums where, you know, it might be a running forum or it might be the vegan subreddit where someone is like, hey, I am a beginner runner. What shoes should I buy? Should I look at buying? So the kind of results you get back when you give ChatGPT persona and search information like that is very, very helpful. The next thing we want to consider is where is the conversation going? Because traditional searches, they've moved away from this, but tended to be more kind of one-shot. You do a search, and then if you need more information, you go and start— you like start again and do another search. Whereas these conversations are building on top of each other. So I use People Also Ask data for this. I'm sure some of you know Also Asked is our tool. So just to get that out there. It's just an easy way to get this data. I don't care how you get it. But what it's going to give you is what I think of as the closest intent proximity. So if someone is asking about running shoes, they will ask about cost. And there's really fun stuff in this keyword research. Like, when people ask about cost, it's really common for them to ask how much faster expensive trainers will make them. So, they're actually trying to like justify, oh, there's €50 trainers and there's €200 trainers. Hmm, how many minutes will the €200 ones take off my 5K time? But these are all the questions that are likely to come up in that conversation. The reason I've recommended AlsoAsked there, not just being our tool, is we have an API. It's super easy to use. It literally writes the code for you. You just choose what language you're using and it will generate the code for you. Because what we have now is we have input traditional keywords. We have input some persona descriptions. We've used AlsoAsked and we have a list now of candidate prompts, the kind of conversations people will be having with LLMs, with AI mode, with ChatGPT, and the follow-up questions they have. And where we need to go from here is work out if those searches, those prompts, are grounded. So grounded is when these tools are doing retrieval-augmented generation. They're going off to check their answers. It's probably one of the most important parts, in my opinion, of this. So when you do a search on ChatGPT, it will make a decision as to whether the query needs grounding. It will score it, and there's various thresholds at which they decide to do a web search. If they do not do a web search, you will get what I call an in-model answer. And you're kind of out of luck there, because the only way you can really affect these is through the training data. It's very, very slow to do. The results are generated primarily from this language graph, and it's a lot of effort to kind of make any real gain there. Fortunately for us, a lot of searches do actually get grounded, meaning it does a web search. These are super fast to influence. Basically, as soon as you're appearing in the search results, you can influence what the AI agent is telling people. And one of the things the AI search surface does and value, as I'm sure other people will tell you here, is recency. It's one of the biases. So actually, we've had examples where we've published things and then within 3 or 4 days we're in the ChatGPT or in the AI mode answer. So we have this threshold and essentially we have either a query that is what we refer to as solved information or like we used to talk about with Google QDF, query deserves freshness. It deserves up-to-date information. Now, if you're a bit nerdy in when you're using ChatGPT, you can go into the, uh, into the response in the DevTools, and you can actually see something called search prob, which is their scoring for if they are going to do a web search for the query you've typed in. There is different thresholds. So for the latest ChatGPT, if you are using the free version, because it's different if you use free or paid, the threshold's around 0.65. That will trigger it to do a web search. If it's below that, it gives you an in-model answer. I assume this incurs extra cost for them because the paid versions tend to do more grounding than the free versions. So you can get that information yourself for any query if you like, or Some of you may know Dan Petrovic, very much worth a follow on LinkedIn. He's doing a lot of work kind of in the machine learning space. He has built some models which he's put online which essentially predict if the query will need grounding or not. So he's got one for OpenAI, and you can go in and paste in a list of queries. So something like, what do red blood cells do? That answer is not going to change. Or understanding of it probably isn't going to change radically. So it's pretty much got a 0% chance of being grounded. You're getting an in-model answer. Conversely, if you say, what happened in the news today? That definitely needs grounding. So it's like 100%, this is going to be grounded. He also has a tool to do the same with Gemini. You can actually use a Gemini API. I'll talk about that later, but this is essentially a free way to do it. Or if you are super nerdy, is anyone here a member of Hugging Face community? A couple of people, 2, 3, 4. So, it's a really great AI community. He's actually uploaded the trained model onto there, so you can download it and run it yourself locally. So, if you want to process thousands of keywords, it's a really easy, quick way to do it. What we're going to get from this process is we've taken our list of candidate prompts, and we can run them through these tools, and then we will have a list of the prompts that we know need grounding. So, these are the ones that we can affect. So, what this step in the process has done is saved you a lot of time not trying to affect the visibility for things essentially you're not going to be able to change in any reasonable time. So, you're working out what not to do in this step. So, we have our prompts that require grounding, and the next thing is to find out what are the web searches that are happening in the background. Now, again, if you are a bit geeky, you can find this out in ChatGPT. So, when you are having a conversation, if you go to the Network tab, Look at your conversation in the list. You'll find something called search model queries. And this is— it will list the queries, the web searches it's doing in the background when you're asking questions to it. There are, I believe, I think the Surfer extension now can do this for you. So, you don't have to go and dig around yourself. You can probably vibe code something as well to output this easily. But those are available to you. Gemini is a little bit easier. They actually have a grounding API. Has anyone used the Gemini grounding API? 1, 2. 2. Okay. So, what you can do with the grounding API is actually pass a query, and it will tell you if it's going to be grounded, and it will give you the response again of the traditional web searches that it is doing in the background. So you can do this again at scale. If you've got your 1,000 traditional keywords at the start, you run them through personalization, through Also Asked, you maybe end up with a list of 10,000. That list gets smaller when you work out which ones need grounding, and then you can run them all through these tools again, and what you're going to get is a list of things you need to rank for in traditional search to appear in those AI answers. So, the example here is if you asked Gemini who won the Euro 2024, these are the actual queries that it's going to trigger in Google that you need to be there for. It's traditional SEO. So, we've essentially gone through here and got 3 things from it. So, we've got our candidate prompts, we've cut them down to ones that are likely to require grounding, and then we've gone one other step and we've literally got the list of things that we need to rank for to affect AI mode. And from here, you can do hopefully what you've all been doing for years, which is the actual SEO to rank for those things. So it might be, okay, we don't have any content for this, or we need to optimize it. Or the other thing to consider is if you're trying to appear in AI Answers, it's no longer just about your website ranking. You don't need to do that. These tools are going to be looking at the top 10, top 20, top 50 sites. You just need to be mentioned as much as possible in those. So, to take the example of running shoes to run with it, that was not intentional. I apologize for that. If there's, for instance, like review websites, send them products, get reviews on there. If they're doing roundups, that's where you need to be to be mentioned. Parting thoughts on this. LLM visibility is a multi-site focus now. It's not just about getting your site to rank. It's about knowing these topics we're interested in trigger these queries, and these are the sites that appear when those queries happen. They're the ones you need to be visible on. I think traditional search is going to be around for a long time. These AI surfaces are not information retrieval platforms. They're not doing what traditional search engines do. They're leaning very heavily on companies that have an index of the web that is already sorted, ranked, and doing all the smart stuff they've been doing for many years. If anything, with ChatGPT-5, we've seen more grounding than ever. I think what they're starting to realize is there is big downsides to large language models with knowledge cutoffs, with hallucinations. So they're positioning more as an assistant for— you're telling it, I need this information, and it's saying, cool, let me find the right tool to do that for you, whether it's writing a Python script or whether it's doing a web search. There is like a hardcore version of this process. So again, Dan Petrovic has started building models that are Closer to how query fan-out has actually happened. As far as I know, I don't think he's kind of battle-tested them yet. So, this is what we're using at the moment. As Alayda said, we're all kind of still learning together. And lastly, no, we can't fully account for personalization. I've shown you how to enrich kind of the conversations with personas, but the way I feel everything is going with like privacy and cookies and stuff, It's not gonna get better. It's gonna get harder, but this is, I think, a good— the best way that I've seen to do it so far. And here's the good news. If you're like, oh shit, that was a lot to take in, that whole process is gonna be online for you for free. So, all you need to do is paste in your list of key phrases, paste in your API keys. It's gonna be live on queryfan.com in about 2 weeks. And it will do everything I just said for you. So even if you weren't listening at all, just write this down. And then when you go back to the office and they're like, what did you learn? You'd be like, this, this thing, we need to do this. So this tool will basically allow you to paste in your key phrases. It will go, you can put in your personas, it'll mix them, cycle through them all. It will go check with ChatGPT with AI mode what's being asked. It'll get the expanded conversations and it will give you a lovely table of these are the prompts and these are the queries you need to be visible for. I'm on LinkedIn. Come say hi. Don't be shy. I love talking about SEO. As Bigum said, feel free to stalk me around the venue as well. I'll be here for a couple more days. Thank you so much for your attention. Thank you.