Hamel Husain shows how to evaluate an AI curriculum planner for PE teachers by embedding trust signals into the product itself: expert provenance, proof other teachers use it, alignment with best practices. If you can't find a clear way to evaluate it, the product probably has deeper problems.
One example that came up recently is one of my clients were building an AI curriculum planner for physical education, K through 12, and they were trying to figure out how to eval it. And the answer is we need to think creatively. Put yourself in the shoes of an educator. You want to know, was this plan created by an expert? What other teachers are using this plan? Is this plan based on best practices and expert knowledge? You need to infuse all of that stuff into the product so you can give the end user trust in what they're seeing. And it just so happens that that will also help you eval the product as well. So the takeaway is, if you can't eval your product, your product probably sucks, and you need to think about creative ways that you can provide your user signal so they can assess whether or not the answer is trustworthy. And that also happens to be something that helps you with evals. I hope that helps.
