Dr. Violet Howard shows how to evaluate AI agents in n8n by testing them against a dataset, comparing predicted outputs to expected results, and measuring accuracy, token usage, and runtime. The demo loads six examples, runs them through an AI model, logs results to Google Sheets, and calculates aggregate metrics to prove whether your workflow actually works.
If you wanna build and sell workflows, you have to learn how to evaluate them. I thought it would first make sense to talk about what is evaluation. In my mind, the simplest way to put it is validating your hypothesis with objective proof. Think back to when you were building an AI agent or a workflow and you weren't happy with the results. What you do is you think about, okay, if I do X, it's going to make the output better. And so that's your hypothesis. You go ahead and you make that change, you run it again, and you basically subjectively judge, is it better or worse? All right, so now we're actually in N8N and we're gonna look at how we do an evaluation flow. It's gonna pull in our dataset, which in this case, we're gonna use this dataset of 6 examples with our expected category and our expected priority. So it's gonna pull in those examples. It's going to feed them through the AI model, which is supposed to tag them, you know, the different categories and the priority, set outputs, which just means it's gonna write the AI's answers back into the Google Sheet, and then it will set the category and the priority which we will be able to check in the evaluations tab. And then I'm going to go to evaluations and just hit run test. And so this is basically going to send those 6 different items in our dataset through this model. It's going to collect an average of how many tokens were used, prompts total, and then also how long it took. And then it's going to give us an average of the priority and category score correctness. So Looks like that already finished up. If I clicked into it, you can see it processed all 6, and you can see which ones, you know, were right and which ones weren't. If you want to watch that full breakdown, click on the play button right here, and I'll see you guys over there.
