Playwright Repair Agents, Cypress AI Assertions & the Future of QE

Are your AI-assisted tests giving you a false sense of confidence? Is the traditional QA role about to be replaced by an agentic engineer? And the creator of Selenium just showed up with something new and you could see it live for free. Want to know more? If so, find out on this episode of the TestGuild News Show for the week of March 8th. So grab your favorite cup of coffee or tea and let's do this. All right. First up, big news for the TestGuild community on the West Coast. The TestGuild IRL California tour is coming and it just got a major addition. Jason Huggins, the creative behind Slime and Napium and the mind behind the latest testing tool, Vibium, is joining the lineup for both stops in San Francisco and also Orange County around LA. And we asked Jason if he could do a hands-on session, we'll see if he will, so you can walk away with some actual things you could take away right away. Both events are gonna run between 4 to 8, 4 to 9. The first hour is all about networking. It's completely free and attendance is capped at 100 seats. So this is gonna fill up fast. This is an in-person only, no recording, meaning that if you're not in the room, you're missing out. You don't wanna miss it? Grab your free spot now using that link down below. Hope to see you there. And one thing I learned from IRLs is that the way we test software is changing. And so I was happy to hear that Manoj Kumar contacted me about a new blog post he just wrote about this. And in this post, he draws a clear line between how teams currently test software and what testing AI systems actually demands. And the gap is wider than most organizations have acknowledged. In this post, it goes over how traditional testing assumes predictable behavior and predefined expected results, assumptions that don't hold for probabilistic, non-deterministic AI and LLM software solutions. This article's central theme is that most enterprise AI systems are still being tested as if they behave like deterministic software when we all know structurally they don't. And the practical implications are spelled out with concrete examples. For example, an AI-powered customer support assistant, the same query about a rejected refund can produce an accurate response one run and particularly correct response another, and a response that references a policy clause that doesn't exist in a third, all of them fluent and well-formed, a regression suite that only has a single canonical answer doesn't meaningfully evaluate the system. And because of this, the real question becomes whether the behavior is consistently within acceptable risk boundaries. What do you do? Well, the article calls for a shift from verification to evaluation, from asking whether an output is correct to asking whether behavior stays within defined risk boundaries over time. And in other news, Cypress has just launched Studio AI in public beta. Cypress has launched Studio AI in public beta that targets one of the most common gaps in end-to-end testing— tests that pass without actually validating what users see. So Studio AI extends Cypress Studio by analyzing what changed in the DOM between each recorded step and generating smart recommendations. So AI-generated assertions, suggestions that reflect meaningful visible UI differences after an interaction. So in plain terms, as you record your test interactions, the tool watches the page and tells you what it thinks you should be asserting. Each smart recommendation includes a short explanation of what is being validated and the generated Cypress code. You can individually accept, reject, or modify recommendations, or perform bulk operations when reviewing a series of changes. And smart recommendations can cover visibility changes like element existence, text context updates, form input values, CSS classes, ARIA attributes, and URL or page title changes. And on the selector side, Studio AI favors stable attributes before falling back to less stable options. It does require Cypress version 15.11 or later and a Cypress Cloud account. All right, so the next thing that caught my attention on LinkedIn was an article written by Dan all about the autonomous testing tipping point. And in this post, Dan makes the case that the industry has reached what he calls an autonomous testing tipping point, and that quality engineers are the people best positioned to navigate it, not be displaced by it. Dan also says it starts with a pressure most testers already feel firsthand. Systems are growing more complex and distributed while delivery expectations accelerate, budgets shrink, and team headcounts get cut. And his position is that most organizations are still treating automation as a long list of brittle scripts requiring constant maintenance, and that LLMs and GenAI tools have exposed something that scripted automation never could the gap between explicit requirements and what he describes as implicit and unknown information about a system. And he goes on to make a distinction drawn between current AI tools and autonomous testing tools that's worth paying attention to. He separates tools that follow instructions blindly from a tool that can relate an instruction into contextual settings, scrape systems for previously unknown information, and take further actions based on what they uncover. This core argument Autonomous testing only works when humans define the context, boundaries, and constraints, and value-based objectives, and when humans can interpret the signals back. The dependencies, he argues, makes it structurally impossible for these tools to replace quality engineers. And being replaced is something I hear a lot of testers being afraid of. So here's this next article from Vernon all about, now what if the robot could do my job, what do I do? And as you know, Vernon always doesn't hold back about something, and a lot of people in the industry are quietly feeling but not saying out loud, and it's worth hearing. So Vernon does describe feeling befuddled and puzzled about a series of converging events, getting heavily involved in his company's AI testing strategy, participating in panels like at Automation Guild on how automation engineers are using AI to get ahead, and watching a company let 40% of its workforce go with AI cited as a reason. And the result of all this angst was a set of questions he could not ignore. If he automates away as much of his job as possible, What does that leave? And will it be valuable enough to make a living? What really makes this piece useful is that it doesn't spiral into doom. Instead, Vernon walks through a structured way of getting unstuck, and he shows how to run a full career inventory listing skills, experiences, character traits, industry insights, achievements, and values built up over 20 years, and combine that with an honest assessment of his relative position in the market and what actually motivates him. The output of the process gave him a direction and broke the paralysis. In his short answer, stay put, get your head around the new reality, and start adding value where only you can. The longer answer, he says, is coming in the next post that we'll probably cover as well. And another industry expert, the awesome Melissa Tondi, also wrote a blog post as well that caught my attention around the same kind of theme. And it's all about how the software industry is about to make and repeat a structural mistake it has made before. This time, though, with AI. And Melissa's thoughts center on what she calls the agentic engineer, a role she argues has been taking shape across every major technology disruption from Y2K to cloud migration. The pattern is consistent: new tools create new specialties, and eventually generalists emerge who orchestrate those specialists rather than replacing them. And her case is that QA professionals are uniquely positioned for the orchestrator role. Precisely because they already sit at the intersection of requirements, user experience, and technical execution. The warning in the article is direct though. Most organizations today are just bolting AI into existing siloed roles, giving AI engineers an AI-assisted testing tool, giving DevOps engineers a pipeline copilot, without questioning whether these silos themselves still make sense. Melissa calls this the same structural crutch that kept QA locked to inspection-only functions for years. Now being recreated in reverse. But the agentic engineer model she describes works differently. The agent becomes the specialist running regression continuously, monitoring pipelines, validating data, while the engineer becomes the general contractor who builds, orchestrates, and governs those agents. Good news: the domain expertise doesn't disappear. It becomes the blueprint the agent is built from. So speaking of AI agents, this next article caught my attention. And this was posted by Ivan, who talked about MukTest that has launched a beta release of a MECO, a model context protocol-based test repair agent built specifically for Playwright automation teams dealing with ongoing test maintenance. And how does it work? Well, the core problem this targets is one most automation engineers know well: tests break faster than teams can fix them. The underlying intelligence is built on 6 years of Mudok's data across hundreds of thousands of automated tests. And the benchmark numbers are the headline here. Compared to Playwright's native healer agent, they claim their agent completes repair tasks nearly 5 times faster and, and delivered significantly higher repair success, averaging 5 minutes per repair with 70% automated success rate versus 25 minutes and a 34% success rate from the native healer. And in performance testing news, I found this new project on GitHub that makes large-scale JMeter load testing easier to run across the cloud, and it's worth a look for teams doing serious performance testing. The project includes a ready-made Azure template that sets up distributed JMeter testing environments using Windows virtual machines in multiple regions. One machine controls the test while others generate the traffic. All connect over public internet addresses, so there's no need for complex private networking, and setup is automated. Script installs JMeter on every machine configures settings, opens required firewall ports, and ensures load generators start automatically when machines boot. Testers can launch runs from the command line with simple controls for user load, warm-up time, and test duration. All right, for links to everything of value we covered in this news episode, head on over to the links in the comment down below. So that's it for this episode of the TestGuild News Show. I'm Joe. My mission is to help you succeed in creating end-to-end full-stack pipeline automation awesomeness. And as always, test everything and keep the good. Cheers.