Hamel Husain outlines three ways to kill an AI project: never look at your data or logs, constantly chase new tools without measuring anything, and add unnecessary complexity with frameworks you don't need. He's being sarcastic, but these are real patterns that sink teams.
If I'm sabotaging an AI project, here are the 3 things that I would do to completely destroy it. And by the way, I know a lot of y'all are doing that. The first thing I would do is make sure I never look at data. And to guarantee that I can't look at data, I wouldn't even bother logging the data. I don't know what my customers are saying. I don't know how the AI is responding. I don't know what's being retrieved. I don't know what tools are being called. I don't know anything. I want to make sure that I stay in the dark. And instead of data, what I'll just do is I'll just put my finger in the air and measure the AI that way or use vibe checks, really the same thing. The second thing I would do is make sure that I'm constantly bike shedding the latest tools. So don't have any evals, don't have any metrics, just bike shed what the latest tools are. A new RAG database comes out, use that. A new agent framework comes out, use that. Use the latest and greatest thing that you see on Twitter at any given time. And the third thing I would do is make sure that you absorb as much complexity as fast as possible. So when you're first developing your product, use the most complicated agent framework you can find. Even better, use a graph database. Your data looks like a graph, right? So put it all in a graph database and you should absorb as much complexity as possible. So use an agent framework in a graph database, even if you don't need one, because you never know. In conclusion, make sure you do these 3 things: never look at your data, always try different tools without evals, ingest as much complexity as possible.
