Teams waste weeks tweaking prompts without a system to measure what actually works. This video shows why prompt engineering alone fails and introduces an open source tool with dashboards to test and fine-tune AI agents properly.
How do you know that your endless tweaking of prompts is actually making your agent better? We had the same problem, so we built an open source solution just for this. So on the left here you can kind of see the prompt version A, prompt version B and see an example query what's your turn? Policy? And the responses that would come from each and a subjective quantification of quality to help with your analysis. In addition, we've created some dashboards that help with fine tuning the agent so we could actually see traceability in the improvement based on queries and response time changes. If you're interested in learning more, let me know. I'll drop a link to our open source repo in the comments.
