Hamel Husain explains why AI evals aren't like software tests. A 100% pass rate means your eval is too weak. The point is finding where your AI fails so you can fix it.
It's really important to not treat your AI software development like traditional software in one specific area, and this is testing. In traditional software, what you do is you write tests and you expect 100% of those tests to succeed. In AI product development, what you want to do is have evals that stress test various aspects of your system that might be weak on. And what you don't want to see is a 100% pass rate. That means those evals are too weak. You always want some signal on where your AI is not succeeding so that you can iterate against that.
