Hamel Husain warns that tweaking prompts without systematic evaluation is like whack-a-mole: you fix one problem and another pops up. Instead, build simple, repeatable metrics to measure whether changes actually improve performance.
If your AI app is not working the way you want and you just keep tweaking the prompt, you're wasting your time. It's like playing whack-a-mole, just keep hitting different problems that pop up. It is a never-ending vicious cycle. You're just going to spin your wheel. And what you really need to do is set up some simple evals that you can iterate against so that you know if you make a change whether you're actually making progress or not.
