Hamel Husain explains that giving agents evaluation skills isn't enough. You also need to teach humans what makes a good eval and customize the skills for your specific use case. Generic tools hit a ceiling; the real work is training people to recognize quality.
So how do you think about constraining or steering the agent to be able to do these things well as a human partner? Do you think about equipping them with skill tools in MCP servers that you've built and maintained or agents markdowns or yeah, how do you coax agents to perform well at these types of tasks?
So we have skills and those skills are open by the way, and I've been making updates even as recently as yesterday. But it's just like anything else, it's just like coding or anything else. You need to educate the human and you need to give them some skills. And the human needs to understand the process of like what good looks like. What are we trying to do? The skills can only help you so much. There's an upper limit to how skills can help you because, I mean, at the end of the day, it's a bit generic because it's meant to apply to a lot of different use cases. But you have to put a little bit more thinking into it when it comes to customizing it for yourself. It really comes down to like learning what good evals look like, plus using the skills and then customizing the skills for yourself. Yep.
