Testing Skyscrapers, AI Drift, Playwright Agents That Promise to Do It All

Is the Testing Pyramid holding your team back? AI agents are writing, planning and even fixing your Playwright tests automatically. But do they actually save time or just add complexity? And why are your AI agents tests passing today but mysteriously failing tomorrow? I know in this episode of the Test Skill News show for the week of October 12th. So grab your favorite cup of coffee or tea and let's do this. Hey, before we get into the news, I want to thank this week's sponsor, Zaptest AI, an AI driven platform that can help you supercharge your automation efforts. It's really cool because their intelligent copilot generates optimized code snippets while their plan studio can help you effortlessly streamline your test case management. And what's even better is you can experience the power of AI in action with their risk free 6 month proof of concept featuring a dedicated ZAP expert at no upfront costs. Unlock unparalleled efficiency and ROI in your testing process. Don't wait. Schedule a demo now and see how it can help you improve your test automation efforts using the link download down below all right, so let's start with something that might make you rethink your entire testing strategy. What is it? Let's check it out. This by Andy Knight, writing on Automation Panda, who argues that the Testing Pyramid is an outdated model that no longer reflects modern testing capabilities. Andy, who previously wrote and supported the Pyramid, now calls it an antiquated scheme that deceives testers. The Testing Pyramid originated when UI testing was difficult test due to outstandardized browsers requiring testers to build their own frameworks Around Selenium webdriver test execution was slow and flaky, leading teams to label UI tests as bad. They favor unit tests for their speed, reliability and measurable code coverage. Andy, though, identifies three major changes since the Pyramid's inception. UI testing tools including Playwright and Cypress now provide greater stability through automatic waiting and faster execution, while Selenium continues improving with the BIDI protocol. And he also goes over how traditional API testing can be replaced by internal unit tests for domain logic, contract tests for service handshakes and UI tests for end to end validation. And test orchestration now enables continuous test execution for every code change, pull requests and even local end to end tests before commits, making fast feedback an important area that is more valuable than test type quotas. Andy proposes the Testing Skyscraper as a replacement model. Unlike Pyramids this narrow towards the top, skyscrapers have multiple levels of varying sizes where each floor serves different tenant needs. This allows testers to architect strategies, meeting Their specific requirements build tests at any level deemed necessary for business needs, skip testing levels as calculated risk, creates empty flaws until needed change, and use modern testing tools to scale strategies upwards and onwards. Another great read by Andy, and you can check it out using the link down below. All right, speaking of modding approaches, if you're using Playwright, you need to know these three new AI agents that promises to automate your entire testing workflow. All right, this topic has been covered by a few people. First one is Karthik who goes over the three new key agents, and also Kalash who blogged about this a few days ago as well. And they all talk about how Playwright has released documentation for three AI agents that automate test creation and maintenance, Planner Generate and Healer, and these agents can work independently, sequentially or in a chained agentic loop to produce test coverage. It goes over all the different agents. For example, the Planner agent explores applications and produces test plans for scenarios and user flows. And the output is a markdown test plan saved to the spec directory that is human readable but precise enough for test generation. And that's where the generator agent comes in which transforms this markdown. It's executable Playwright test and once the generator agent is done, the Healer agent agent handles failing tests by replaying the failing steps. Expect the current UIs to locate equivalent elements or flows and suggests patches such as locator updates, weight adjustments or data fixes. And I think this approach in these three agents kind of addresses what Andy was talking about with the testing skyscraper. Using modern tools to help you with modding testing, and this can help you as well. All right, staying with this AI theme, here's a new tool tracking one of the hottest problems in testing concurrency bugs. I found this on my LinkedIn feed by Jay, who talks about this new FISB autonomous testing tool. And this is the model based testing solution. And one of the reasons why he created this. He talks about how even with AI generated code, using tools like Copilot to write tests remains insufficient because testers must still think through scenarios and review generated code by using fysb. Autonomous testing requires testers to describe the design in a python like language to show how their code implements that design. The tool handles the rest, including testing for concurrency issues. This tool supposedly claims no other autonomous testing solution addresses. This tool targets distributed databases and storage systems, APIs and microservices with evolving state concurrent services where operations order matters and stateful systems too complex or handwritten test suites. All right, I'm not sure how many people use this tool but if you do, just so you know, Test Wheel has just released a new plugin. They released a Jenkins plugin designed to simplify test execution within your CI CD pipelines. This plugin allows developers and QA teams to orchestrate and execute test suites across multiple environments and device labs directly from Jenkins, reducing typical setup time from hours to minutes. Michael Morello, marketing manager at Test Wheel, states that the plugin addresses frustrations with manual configurations, fragile tests, and and incompatible tools. The plugin includes four main features One click integration with Jenkins jobs intelligent test filtering using AI to automatically select to run only tests affected by recent code changes. The plugin also handles environment variables, device configuration, and secret credentials between Jenkins and Test Wheel. All right, so let's shift gears to strategy. This next post is all about observability the three Pillars for Quality Engineers by a guide explaining how observability helps quality engineers answer critical questions about system performance and production incidents. This article breaks down observability into three components, logs, metrics, and traces. So it goes into detail what are logs and how Quality engineers can use logs and test automation to document execution details in performance testing to track resource usage, and in integration testing to monitor API calls between services. They also go over metrics and what you should measure to help with performance benchmarking, SLA monitoring, capacity planning and setting automated quality thresholds. And finally, they go over traces and how traces can help with end to end testing in microservices systems, identifying performance bottlenecks and mapping service dependencies and pinpointing failure locators. And the article goes on to demonstrate the three pillars, how they work together using an E Commerce checkout scenario. All right, next up is our Webinar of the Week. If you have anything to do with performance, you definitely need to join us on this webinar with Perforce with our experts from blazemeter and Shane Evans and also the one and only Scott Moore, we're going to cover how to shift performance testing left and integrate it into your CICD pipelines. This session is going to center on helping teams ensure their applications can handle production level traffic and demands, which you should always pay attention to, but especially as we go into the holidays. So in the webinar we're going to cover how blazemeter can be used to simulate realistic user traffic from multiple global locations, validate performance across different testing protocols, and integrate continuous testing into your CI CD pipelines. Scott and Shane will demonstrate strategies from shifting performance testing earlier in development cycles, improving application stability under load, and identifying bottlenecks before they affect your users highly recommend you register even if you can't make the session, because I'll send the link to the recording after the fact, so don't miss it. Register now using the link down below. Hope to see you there. Next up is about Agentic drift. So here's something that I think should be on every tester's radar. IBM is warning about a phenomenon called agentic drift that can silently degrade your AI agents over time. And and this one's crucial if you're testing AI powered systems. This article talks about the testing challenge with AI agents called agentic drift, which occurs when AI agents change behavior as underlining models update training data shifts or business context evolves. An agent that performs correctly today might produce degradated or incorrect responses tomorrow, creating problems for traditional software testing methods built on deterministic logic. So IBM's solution is the IBM Agentic Testing Framework. This framework uses large language models to evaluate agent responses against natural language expectations rather than exact strings. All right, for links of everything of value we covered in this news episode, head on over to the links in the first comment down below. So that's it for this episode of the Test Guild new show. I'm Joe. My mission? Help you succeed in creating end and full stack pipeline automation awesomeness. As always, test everything and keep the good. Cheers.