Mike Taylor explains why LLMs hallucinate: they're search systems, not generators, and fail when queries fall outside their training data. He spent thousands optimizing prompts for a 5,000-persona project and warns that synthetic personas can't predict real behavior, only anticipate objections. Human relevance matters more than technical capability.
This really reminds me that AI is a data problem. This is a data curation problem.
It actually really helps, I think, to look at an LLM as a search problem rather than a generative problem, because they don't actually generalize very well outside of the dataset. It's just the dataset is very large, right? Like the dataset is the whole internet, right? Right. And if it hasn't appeared on the internet, they tend to hallucinate really badly. So, so your job, I think, as anyone who interacts with that, making that product better or making that workflow better is really just to think about like, is, is the task in distribution? Is it common in the training data? And if not, how do I make that more common for the model by, by either putting that into the prompt or fine-tuning the weights of the model? What can you give it to ground it in that task? And I think then you can do the rest. The prompt is like very important, it's costing you a lot of money, or it's really unreliable, then you need to sink more time into it. And more money as well, because this costs money, right? Like when you— actually, I spent a couple of grand on an optimization process for a client. And, you know, like the credit card that I had on that account was like maxed out. So I had to call the client and say, I'm really sorry, but I need to spend another couple of grand and can I have your credit card? But that was for a huge personas project of like 5,000 personas. We were building the personas and then training them to be more accurate. It shouldn't cost you that much for DSPy. I would say like typically it's, it's like under a dollar for an optimization run. Usually when people go for prompt optimization is because they, it's, it's costing them a lot of money. Not having that prompt optimized. So they're willing to try anything. It's like OpenAI is like the biggest cost in my company right now. It's more than I pay my own salary. So can we get off of this into open source models?
Do you think these synthetic personas of a real individual can actually predict what people will do or how they really think?
I think we have free will, so I hope not, you know, but I would say that it can, it can predict what would be plausible for the person to do. And the distinction I'm making there is I cloned myself and I asked it, what's an embarrassing purchase you made? And it said, at 2 AM, I was browsing the internet and I found a lightsaber, build your own lightsaber kit. And I bought it. And as soon as my wife saw it, she thought I was an idiot and You know, the kids loved it, whatever. And I was reading it, I was thinking like, that didn't really happen in real life, but like, man, like that's exactly me. Like that's exactly this. I've made purchases similar to that. Like I've done, uh, you know, that's exactly how my wife would react. And, you know, so it's, it is not something that really happened, right? Like it didn't predict what really happened. Uh, but it did, uh, it was consistent with my personality. It was consistent with my interests. I think that that's actually like as far as we can get, right? Like, I don't think this is like, you know, Minority Report, like predicting future actions and, you know, making decisions based off that, right? Like, um, I think this is more, you want to know what's plausible and, uh, you want to know, uh, what, what might people say about this idea that I have? And the reason why that's useful is so you can anticipate their objections is so you can, you know, understand like. What would make it resonate better with them? Like, what would they like more? How would you, how can you serve them better? And I don't think you actually have to predict their actions to do that, especially at scale.
How's it different from using Rally?
I used to do this before I built Rally. I used to call this personas of thought, my prompting technique. You can just ask, come up with 5 different personas of people who would buy this this product, and then tell me what each one individually would think. Summarize their responses at the end. And what you would get is something like, you know, tangential, like similar to Rally, but it'll still be a bit too sycophantic. It will lack some of the diversity because they're all in the same thread. I call this the vegan problem. If one of the personas is vegan, then the other personas tend to be more vegan. Like they say that they wouldn't eat meat or something. And I'm like, you're not even supposed to be vegan. But because they're in the same thread, they color each other's responses.
I mean, yeah. This has to be context engineering, right? But, but Rally—
Exactly. It's context engineering. With Rally, it's all individual. So we're literally sending off 5 different prompts to 5 different models, isolated responses, and then we summarize at the end. Humans are the only source of meaning in the galaxy. Like we only care about stuff if other humans care about it.
Exactly.
Yeah.
Or if it involves other humans.
Like, I always think of, you know, I'm from England and just moved, and in London there's a piano in the train station, in St. Pancras train station. And, you know, I always think about this, like, people will stop and they'll play piano, and it's like always people who don't think, like, wow, you know, that guy plays piano, and it's like beautiful, you know. And like, whenever someone sits down to play, even if they're not that good, like, people will stop and watch and clap. And I always just think like, you know, that's like a tough crowd because people are busy running through the train station. And I just think like, you could literally just get a piano that plays itself and put it there. That is a solved problem, right? Yeah, it's been a solved problem even before AI. You could have a piano play any of the best songs in the world in history, right? Like Bach or Mozart or whatever. And nobody would stop for that.
