Hamel Husain breaks down evals as data analytics for AI models: collect data, define metrics, monitor performance, run experiments. It's a systematic way to measure and improve language model applications, not the intimidating thing people think it is.
What the heck are evals? For folks that have no idea what we're talking about, give us just a quick understanding of what an eval is. Eval is a way to systematically measure and improve an AI application, and it really doesn't have to be scary or unapproachable at all. It really is at its core data analytics on your LLM application in a systematic way of looking at that data and where necessary creating metrics around things so you can measure what's happening and then so you can iterate and do experiments and improve.
