Hamel Husain argues that AI products fail because engineers evaluate them instead of domain experts. Engineers lack the specialized knowledge to catch errors in legal, medical, or recruiting AI systems. Putting domain experts in charge of evaluation and development is the fix.
Stop blaming engineers for AI quality issues. It's really a product process. And what I mean by that is oftentimes engineers don't have the domain expertise to know if something is wrong. If you have a legal AI assistant, your engineers don't have legal expertise. You're building a medical AI assistant, your engineers don't have medical expertise. Even if you're building an email assistant for recruiting, your developers aren't recruiters. They're not the ones writing emails. What you need to do is make sure your domain experts are in the loop and they're the ones doing evals. You can't outsource your evals to engineers. This is the number one cause of failures in AI projects. So what you need to do is expose the right tools to your domain experts so they can be involved in the development loop.
