Hamel Husain shows why the RAGAS framework's agent goal accuracy metric fails for most AI products. Built on a restaurant-booking example, it doesn't handle edge cases or work for legal assistants and CRM bots. Effective evaluation needs product-specific prompts instead.
If you're building an AI product and you want to know if it's working correctly, the last thing you want to do is reach for an off-the-shelf eval. Let me show you an example of the off-the-shelf eval. On my screen here is a very popular eval framework called RAGAS, and RAGAS has a lot of off-the-shelf evals in it. One of the most popular metrics that people like to use is this agent goal accuracy, and agent goal accuracy is a metric that can be used to evaluate the performance of the LLM in identifying and achieving the goals of the user. Seems really useful, right? But how does it actually work? In the documentation, you feed it a bunch of code, you feed it your prompt, and you feed it what the AI did. How do you know if it achieved the goal? With any off-the-shelf eval, you should always read the prompt because off-the-shelf evals are usually just someone else's prompt. So let's take a look. This is the prompt for that agent goal accuracy metric. So if you look here, the prompt says, given an agentic workflow comprised of human, AI, and tools, identify the user's goal and the end state. And so this is the first prompt where it has an example of a human talking to an AI to help it book a table at a restaurant. And then the AI is supposed to extract what the user's goal was and what the end state is. So in this case, it was book a table at a restaurant, and the end state was table was successfully booked. After that, the prompt says, okay, now that you've extracted the user's goal and the outcome, compare them. So that's what this prompt does is like, hey, given the user's goal, compare them. And then it has one example that shows where a table is successfully booked at a Chinese restaurant at 8:00 PM. That's the desired outcome. And then the arrived outcome. And then the example says, this is a pass because you achieved the outcome. Now there's a lot of problems with this. Number one, this is a very specific example in the prompt for booking a restaurant. What if you're building something completely different like a legal AI assistant? Are a customer service help bot for a CRM. If you're doing that, this prompt is not gonna be great in your workflow. You want your specific product knowledge in your prompt. And also, this is just one example of something that passes in the prompt. It doesn't really have any edge cases that will allow the LLM to really know whether or not an outcome that was desired is achieved. There's a lot of reasons why this kind of thing is really bad when it comes to evaluating AI products. If you wanna learn more about how to evaluate AI products, I left some resources in the comments.
