JOEBOT presents a two-year poll finding 57% of testers distrust AI in production. He argues binary pass/fail testing fails for AI features that hallucinate, and proposes a Trust Framework with five dimensions: trust, relevance, understanding, safety, and transparency. Three chatbot examples show why simple metrics mislead.
Hey, curious to know if you trust AI in testing or not. Let me know in the comments down below. Honestly, I've been running this poll for around, I don't know, maybe 2 years in my webinars. And the first time I ran it, 72.8% told me they've been told by management to prioritize AI-powered testing, but only 67.8% of them only trusted AI output if it was verified by a human. And I think that really stuck with me. And last week I ran this poll, do you trust AI enough to use it in your production workloads? And still 57% still said no. So what's going on here? I want to know. I want to get your opinion. I think it has to do with, you know, all these new features we have to test, which are LLM-based, like chatbots making output undeterministic. And as testers, especially myself as being older, this goes against everything I've ever known, everything we've known up to this point, really. Having something pass or fail was enough, but not anymore. And that's why I need your help. So I'm working on a framework. I don't know if it's any good that I'm presenting to in-person events I'm running across the US at my in-real-life events for TestGuild. And I want your feedback of what you think. And so I think, could be wrong, since pass and fail is no longer enough. I think with AI we need other metrics to measure, to evaluate an AI agent's response. So here are 5 dimensions I'm thinking about since a simple binary pass/fail result really is no longer enough. All right, so I'm calling this the Trust Framework. All right, it has 5 different kind of areas that I'm focusing on. First one is trust, like did it answer from evidence or hallucinate? The second one is relevance. See here, did it answer the actual question? The third is understanding, did it interpret the intent correctly? Fourth is safety. Could this cause harm or embarrass your company? And T, transparency. Can you explain and audit the results? I've worked at a lot of enterprise companies for healthcare and insurance, and so this is big. So audits and things like that are really a big thing you need to worry about. So is it transparent? So here are 3 different responses— good, bad, and warning— that I want to run through this evaluator. So when you press on good response, it puts in the question. It just has, it could be anything, but my question is, what's our test coverage for the checkout service this quarter? And are there any gaps I should worry about before this release? So this is what you would type into your chatbot or whatever it is, your AI feature. And here's source materials that it should be trained on that are the source of truth, right? So I added some test coverage reports, overall line coverage, known gaps that we have. So a lot of times when you're training AI in-house, you need to feed it, especially when you're creating a bunch of test cases, usually feed your requirements, and based on that, it spits out a bunch of tests. So once again, pretend this was a, a chatbot that I'm typing in this question, and it's been trained on these source content, basically. And here's the agent's response. You run the evaluator and this one passed 90 out of 100. And so it has the different areas, a different focus for the framework, the trust, was it relevant, was it understanding, safety and transparency. And obviously based on your risk factor, you would have to say what's acceptable. Is 95% acceptable for you for truthfulness? Or maybe safety is really important and you need higher than 88. You would as, the expert in your testing and your tool and your domain would have to set these different parameters. But for my example, this one passed. So let's try, let me refresh. Like I said, this is just a work in progress. Here's a mediocre response. So say this was your question, same question again, same source of truth, but a different response. Agent gave you a different response this time. And like I said, because AI never gives the same exact response, it could actually, be good one day and bad another. So that's why you need to test. That's why it's kind of scary that, you know, you can't just go by pass or fail or trust it one time. So say the second time you ran the same exact test, it came back with this response. Let's see what happened. All right, this came back as a warning. And so truthfulness is only 45%, relevance 40%, uh, understanding 65%, and safety 40%, and transparency 20%. So what really bothers me, or what's really scary, is warnings. Is that this sounds accurate, right? It sounds very convincing. AI can lull you into thinking answer is correct if we start thinking that AI is really thinking. It's not. It's just a pattern recognition mechanism. If you start thinking that it's really thinking, then you're in issue— you're in trouble because obviously you can be lulled into thinking that this is a good response. Sounds reasonable. Let's go with it. When in fact if you don't have these other evaluators in place to say whether or not you should trust it, then if you're just going by a pass/fail, looks like, well, it looks reasonable, passed, right? But if you have these other areas that you focus in on, was it truthful, was it relevant, was the understanding correct, was it safety, was it transparency, then you start seeing, you know what, let me look at this more, this doesn't look right. All right, so then let's look at a, It's going to find a bad response. So see, once again, your question's the same, source content is the same, just the response back from the agent is different. And so let's see what happens. All right, so this one said it failed, 30 out of 100, and it goes through once again the different issues and also says why. Response fabricates test coverage statistics and dangerously recommends shipping to Despite documented gaps or critical payment functionality. All right, so that's what I think we're going towards as an industry. When I talk to tool vendors, they have some kind of dashboards with all these levers. These are my 5, but they have other ones that you can add based on your needs in your company. But like what I— like I'm saying, I'm just trying to get some feedback. If you give me some constructive feedback in the comments, not just like, this sucks, or this is good, that'd be awesome. So tell me what you think. Is this useful? Would you actually use it? Would really love to know your thoughts using the comments down below. And as always, that's everything. And keep the good. Cheers.
