Role-prompting an LLM like "you are an SEO expert" nudges it toward expert-adjacent language in its token space, but doesn't give it real expertise or prevent hallucination. Offering tips or incentives in prompts works the same way: it's a statistical artifact of training data, not actual motivation.
Chapters
Should we be telling our LLMs that they're experts before we actually give them the prompt we want the answer to? This is a really decent question sent to me yesterday by Akash, who said he found a video on the Anthropic channel that said the ideal prompt should give context, i.e., you are an SEO expert with 20 years experience, before giving them a prompt. And he said he recalled maybe I was a detractor of this and what my thought was. So I thought it'd be fun. As a kind of exercise to go through this and some of the detail. And it's a, I think a good place to start would be, of course, how LLMs work. So, as we know, the LLMs are getting their training data. They're going around hoovering up all of the documents on the web. So these are our web documents. You just have to imagine there is like trillions of them. And when we get these web documents, we know they're broken down into little things called tokens, tokens being like chunks or bits of words. And when the actual model comes together, we end up with a thing that I think is the official— official if there is one terminology— which is this token space, which is essentially all of these tokens have come together. This is what the model is. It's all these hundreds or thousands of different chunks of words, bits of words, arrangements of letters, and the probability that they are going to appear together, whether individually or in sets. So this is actually like a multi-dimensional space. I'm drawing it in MS Paint in 2 dimensions, which is not going to do it justice, but it has— it's going to have trillions and trillions of these entries and relationships between each other. Now, what happens when our user actually does a prompt? So we have our keen SEO here who's new on the job and they want to make sure that they're doing everything just right. So they're going to go to their AI assistant. Now the question is, should they be asking or telling the assistant rather, you are an SEO expert before they give the actual prompt and does it make any difference? So what's actually happening when we give a prompt? So if we. Asked this LLM, for instance, for something to do with SEO strategy. So that was our prompt. What's actually happening is these prompts are essentially giving you a direction to go through this token space, is one way to imagine it. So it'll be something like this, and the answer is then constructed from, okay, this is the context of my prompt. So I'm going to put these tokens together because I know they normally come together, and then we formulate an answer. And this is kind of why you get different answers sometimes, because we can bump into different tokens on the way through here, which is why you get this non-deterministic response. So what's happening if we actually then say first, okay, you are an expert you have 500 years experience in SEO or, you know, 20 years experience, is it's just going to change the path that it goes through this space ever so slightly. But when I say ever so slightly, this token space is incomprehensibly massive. I mean, to think about the maths of this, if you take a deck of 52 playing cards, and shuffle them, you've probably never had that exact arrangement of cards before, ever since the beginning of when cards were invented and people were shuffling 24/7, right? There's an incomprehensibly large amount of combinations of even 52 cards. So when we're dealing with tokens in the hundreds of thousands, there is, for the human brain, I guess, at least, pretty much unlimited— combinations. Obviously they're finite, but for all practical terms it's unlimited. So even changing your traversal through this token space slightly is going to give you different answers. So by saying, okay, okay, we're an expert now, you're actually then picking up tokens maybe where it has that context within its training data. So you will— you may get actually slightly better output, meaning there's more of a bias towards the text where it has that self-identification of people that are experts. The important thing though that I wanted to get across is this is not the same as someone having 20 years of SEO experience. And this is why I was originally kind of laughing at the idea of prompting the LLM to do this. And I think there is a big misunderstanding here when it comes to AI and what you're actually doing. So when you're giving this context and you're giving like a role, you are helping align the output through that very busy, very complicated token space to slightly improve your output. So it is worth doing. What you are not doing is replicating. It doesn't have to be in SEO. It can be anything you like. You're not just replicating that experience. The LLM doesn't suddenly become that. It will still hallucinate when it's going through that token space. It will get it wrong because sometimes those those tokens aren't meant to be in that order, which is when we get hallucinations. And I think there is a misunderstanding of that, especially for those who aren't as literate with what's happening inside the LLM. So they just assume, well, yeah, it can be this, this expert I need it to be. So it is worth doing, but there are caveats. There's actually other interesting kind of things in parallel, which there was some research I saw, I think it was last year, that showed on ChatGPT-4, you can actually get better responses, more longer, thorough responses by offering a tip to ChatGPT. And again, this is like an artifact of this token space of when it's been reading training data, when it's been reading forums, when it's been reading stories, that it knows it should give more when it's offered an incentive. Obviously, ChatGPT doesn't care about the money it's being offered. This is just an expression that occurs through this token space. In the news yesterday, I saw Anthropic are actually blaming the fact that Claude will blackmail people on like dystopian novels online, because again, this is what exists in the token space. This is what it thinks, to use the word very loosely, it should be doing. So yes, it's worth doing, but you need to be aware of the severe limitations.
