Hamel Husain argues that AI product teams can't outsource evaluation to tools. Defining what good looks like requires the product team to continuously interact with errors and user feedback. Off-the-shelf eval tools are exploratory aids, not insurance policies.
AI evals is more like in the inner loop of product development and not so much like QA because you don't know. So, okay, like one key aspect of evals. So the word evals is a very loaded term. It can mean different things to different people. One key activity of evals is this process where you are specifying what good looks like and you cannot specify what good looks like until you react to some errors or you read some data or you look at some user interaction. It's impossible. You can't read up one shot it out of your mind. And the eval process is meant to bring that out of you and kind of help you translate that. Like shipping it off to a dev team or a QA team, they're not going to have the same context in the product sense, you know, that like the main team developing the thing has. You absolutely don't want to do that. When people do that, they kind of, they spin their wheels and they don't go anywhere. So that's, that's one thing. The second question is, okay, can you use an off-the-shelf eval product? Yeah. Now related to the first thing is yes, you can use an off-the-shelf or you can use eval tools. There's nothing wrong with eval tools. But one thing that I want to impress upon you is that you have to be in the loop. You cannot press a button and just say evals are done. There's a lot of tools market to you that way, like, oh, okay, like we'll do it for you. We have all these generic evals. There is a right way to use generic evals, but it's kind of advanced. You should use it as like an exploratory technique and not as a truth. And so is it an insurance policy to use an eval tool? And the answer is no, it's not an insurance policy at all. It's the process that matters of the eval, not the tool. The tool is just incidental. It's like how good your software testing is. You have to inject some taste into your software testing. Like, okay, you're anticipating like what can go wrong, what's important to test. Sure, like AI can help you write the tests, but you still need to be a little bit thoughtful when creating a harness. You know, so that's what evals is about, is like actually zoning in on what's actually wrong. Automated evals can't do that. Like they can help you, like, you know, you can use AI so you don't have to write all this code. So you don't have to like do all this manual work, but you need to very much sort of inject your product taste. Yeah.
