LLM Testing. Free Test Tools, AI Test Management and More TGNS174

What are the four top free new tools every tester should check out? How can AI powered test management tools help QA teams balance rapid software delivery and quality assurance? And is your traditional test strategy falling apart with large language models? Lando on this episode of the Test Skill News show for the week of November 9th. So grab your favorite cup of coffee or tea and let's do this. Hey, before we get into the news, I want to thank this week's sponsor, Zaptest AI, an AI driven platform that can help you supercharge your automation efforts. It's really cool because their Intelligent Copilot generates optimized code snippets while their Plan Studio can help you effortlessly streamline your test case management. And what's even better is you can experience the power of AI in action with their risk free 6 month proof of concept featuring a dedicated ZAP expert at no upfront costs. Unlock unparalleled efficiency and ROI in a testing process. Don't wait. Schedule a demo now and see how it can help you improve your test automation efforts using the link download down below. All right, so let's kick things off. Four powerful tools you definitely should check out. So I just dropped a new podcast episode where I go over the top free tools testers need to try and they're all available on Test Guild. Com and I go through a full breakdown of each tool. But let's check out each one right now really quick on the website. The first one is the Test Guild Tool Matcher. This helps users quickly find the right testing framework or solution by answering a few targeted questions and no venture bias, no endless research. The next one is the Test Risk Calculator, which gives managers and leads a way to identify and prioritize high risk areas in their testing tool pipeline using Data driven scoring. The third, which is one of my favorites, is the Accessibility Scanner, which makes it easy to check out any website for accessibility gaps in seconds, offering plain language explanations and quick visual feedback to catch common compliance issues early. And last is our Automation Assessment Quiz which helps teams benchmark their automation maturity, highlight where they stand and where they can approve. Next, together, I created these tools for the testing community for a clear picture of your testing overall health from two selection to risk accessibility and overall maturity. All without a price tag. So definitely check them out down below. We'd love to hear your thoughts and how I can improve them as well. All right, here's a company I haven't heard from in a while, so glad to see they're still around. And it's sauce Labs that just introduced a new tool for AI for Insights. Right, and they're calling this sauce AI for Insights, which is designed to transform how engineering teams handle test data. It was launched last week and this tool addresses the overwhelming volume of test data that often slows down release cycles. And this is something we see at the guild all the time. People always struggle with test data, and according to Source Lab CEO Prince, the problem isn't generating test data, but interpreting it, which has become specialized knowledge. And this new AI solution aims to democratize quality intelligence by providing instant answers to natural language queries about test results. The company reports that teams using this beta version have seen dramatic improvements, including 99% faster identification according to them of root cause and debugging time reduced from hours to minutes. The system provides real time analytics, rule based insights and visual outputs for charts and clickable links to relevant test artifacts. Next up is our webinar of the week. If you're looking to get hands on with AI in your testing workflow, BrowserStack's upcoming webinar with the Test Guild might give you practical insights that you can use right away. So in this webinar, Raul is joining us to go over how AI powered test management can help QA teams balance rapid software delivery with quality assurance. Going to go over many things, but one of them is going to demonstrate how BrowserStack's test management platform uses AI agents to enhance the entire testing workflow, from planning and creation to execution, maintenance and validation. And they tell me that their AI solution is built with true AI engineering for context and accuracy rather than simple GPT wrappers. So you definitely want to attend live to see some demos of AI applications and testing workflows and learn how real time analytics can help your team identify issues earlier, improve testing decisions. And as always, I highly recommend recommend you join us using that link down below. Register now. Even if you can't make it, make sure to register down below because you'll receive a recording of the event in a day or so. All right, so now let's zoom up from tools to strategy, because sometimes the biggest testing challenges isn't technical at all. It's usually soft skills or how you can help lead change in your organizations. David Ingram explores a crucial perspective shift for testing professionals in this article, Test Leadership as Change Management. David goes over how testing problems are really about testing itself, but about managing change. QA leaders don't simply evaluate software. They assess how the team adapts to new processes, tools and methodologies. And this insight becomes really relevant. As you know all this AI technologies that's coming out there is being implemented into our pipelines. And according to the article, every QI initiative is fundamentally a change initiative. Whether introducing automation, shifting quality left, or redefining what done means, leaders are asking people to alter their behavior and sometimes their professional identities. And David references two different models to illustrate how teams experience predictable emotional reactions during transformation from denial to acceptance. The article also emphasizes that resistance should be treated as valuable feedback rather than opposition. And for effective change management, David recommends evolving team members early, starting with pilot projects, normalizing iterations, maintaining transparent communication, and positioning a new technology as a a partner rather than replacement. Really great point, and you can read more about it using the links down below. And as AI becomes central to modern applications, your traditional testing approaches might be falling short, and here's why. Testing LLMs requires a completely different mindset. What is it? Let's check out this next article from awesome Testing, which outlines how traditional quality assurance practices must evolve when testing large language model applications. And the author explains that LLMs have fundamentally changed testing by introducing non deterministic outputs that regardless of what you do, the same prompt can often produce different responses in the article introduces a three pillar approach to LLM testing. First is test deterministic parts of the system with mocked LLM components. Second is implementing end to end evaluations using benchmark tests that measure business value rather than technical correctness. And the third is establishing robust testing in production through user feedback loops, AB testing and comprehensive observability. And for a practical implementation, the article demonstrates how to use Wireshark to create deterministic mocks of LLM APIs, enabling traditional pass fail testing for the application's core functionality. And then explains how LLM as a judge can evaluate system outputs on criteria like relevant and tone by scoring responses from 0 to 1. And the author recommends tools like OpenAI, Evalves, Prop, Fu, and TrueLens for benchmarking LLM applications, emphasizing that quality becomes a continuous measurement rather than a binary state. All right. I always felt that being able to properly test something, I need to know how to do it myself. That's just me. That's my approach. So I found this next article how to create your first AI agent, and this is by Debbie o', Brien, who's the leading development advocate for Playwright at Microsoft. She shows how to build a simple AI agent using NAN with chat based triggers, then wired to practical services. And in the demo, the agent answers questions like Today's weather, current YouTube subscriber counts, and a bunch of other things as well as email the summary of the conversation. All right, whenever we talk about self handling, a lot of times we were thinking functional automation, but how about if it's for DevOps? Well, this next tool just might help you Komodar has announced new autonomous self healing and cost optimization capabilities for its AI SRE platform. Designed to simplify operations for teams managing kubernetes environments at scale, the platform uses agentic AI technology that can automatically detect, investigate and remediate issues in cloud native infrastructure. And this new autonomous capabilities addresses growing operational complexity as industry research shows 88% of test leaders poor rising stack complexity and last up is the growing opponents of not only automation and functional testing, but also security testing. And one way I gauge this is by acquisition. So here's a bug crowd just acquired Mayhem Security if you don't know, Mayhem Security is an AI offensive security pioneer founded by elite hackers and this move aims to create a comprehensive security testing platform that combines AI driven automation with human expertise. And this integration will provide customers with automated protection during deployment and human driven adversarial testing of deployed software. And Mayhem Security brings specialized AI capabilities for API testing, code security, dynamic S BOMs and reinforcement learning. All right, for links of everything of value we covered in this news episode, head over to those links in the comments down below. So that's it for this episode of the Test Skill New show. I'm Joe. My mission is help you succeed in creating end to end full stack pipeline automation awesomeness and as always, test everything and keep the good. Cheers.