Testing Pyramid for AI Agents, Playwright Vibe Testing and More

What if Playwright tests could be created, not coded? And what if that tool just removed the paywall to prove it? Are you testing AI agents? All wrong. And could a Chrome extension replace your entire Playwright setup? Find out in this episode of the Test Guild News show for the week of January 18th. So grab your favorite cup of coffee or tea and let's do this. First up, before we get into it, I just saw this on Product Hump. Blink IO is ranked number three. Let's get them up in the charts. But if you if you don't know. But they have a big announcement as well, they open free access to their Playwright test automation platform, targeting testers who want to build browser tests without extensive coding. The platform centers on two concepts the company calls Zero Touch Experience and Vibe testing, which they describe as a tent driven test creation. And According to Blink IO, that platform allows testers to build Playwright test in minutes using a web recorder that captures user flows. The system generates what the company describes as real Playwright code with no vendor lock in, and which is awesome because it means your tests can be exported and run independently of the Blinkio platform. The tool also includes automatic healing capabilities designed to maintain tests when application elements change. And they also emphasize that Vibe testing allows testers to describe test intent rather than writing explicit step by step instructions, with the system translating those intentions into executable real Playwright code. And you can check out the full platform now. No cost to start. Testers can sign up and begin creating Playwright tests immediately without payment information. And what's cool is their free tier provides access to their core features, including the web recorder, automatic healing, and Playwright code generation. So if you haven't tried it yet, Blink IO has removed all barriers for you. They have this free tier you definitely should check out for yourself using that special link down below. All right, so the awesome Angie Jones is back in the news, all about a new framework she just introduced for testing AI agents that challenge the traditional testing pyramid. And in this post she argues that the classic unit integration UI model breaks down for agents because they violate a core testing assumption. The same input no longer guarantees the same output. Instead, Angie proposes an agent testing pyramid organized by how much uncertainty you're willing to accept, not by test type. The base layer remains fully deterministic using traditional unit tests for agent foundations like Retry, Logic, Turn limits, tool schemas and extensions. These tests also rely on mock providers with canned responses to stay fast, cheap and predictable. Answering one core question was the software built correctly? The middle layer, called reproducible reality, replaces classic integration tests because agents depend on external tools and MCP servers that change independently. And she gives an example how her company block records real interactions once and replays them forever. The same approach applies to LLMs using a record and playback test provider, making multi turn agent behavior testable without handcrafting responses. And above that, she places probabilistic performance as measured through structured benchmarks. These tests check task completion, tool selection and artifact quality across multiple runs. Regression is defined by declining success rate, not exact output changes. One run means little, but trends reveal everything and at the top sits vibes and judgment, covering summaries, research explanations and creative outputs that resist strict assertions. And anything by Angie Jones is a must read, so I highly recommend you check it out using that link down below. Alright, I actually found this next tool on LinkedIn and this is by Amit, who posted about a new tool that I haven't heard of called Playwrightr, and it's positioned as an alternative to the Playwright MCP server. What to do? Well, let's check it out. So instead of running through a separate browser instance Playwright, it works as a Chrome extension that controls your existing browser tabs through a WebSocket connection. The key difference here is context window efficiency. The tool uses accessibility snapshots instead of screenshots to understand web pages, while reducing the typical data set to language models. Playwright exposes the full Playwright API through a single execution tool, which means test engineers can write standard Playwright code instead of learning a new set of browser automation commands. The tool supports advanced features like setting breakpoints and running JavaScript live, editing page scripts without reload and inspect and CSS styles similar to Chrome dev tools. There's also a visual labeling system that overlays clickable reference labels on interactive elements, making it easier for AI agents to identify targets from screenshots. All right, another tool that caught my attention is from Roy, an automation expert who messaged me on LinkedIn, telling me that he just released a new update to Test Blocks, which is a visual test automation tool built on Blockly and Playwright. He also told me this update introduces a refresh user interface, the ability to create variables directly from values and variable autocomplete to speed up test creation. Test Blocks lets testers build API and web tests visually using drag and drop blocks instead of writing code under the hood. Tests are saved as JSON files and can be viewed in Git and executed in CI pipelines using a command line runner. All right, all this talk about Playwright reminds me I forgot to tell you about the webinar of the week so we're hosting a webinar tomorrow featuring Blink IO all about what we talked about earlier. They're testing approach called Vibe Testing for Playwright Automation. The session introduces what Playwright describes as an attempt first method for building Playwright tests. Instead of writing code line by line, testers describe what a user journey should achieve and the behavior that matters. And then blink IO's AI test engineer translates that intent into executable Playwright code and this webinar will cover what Vibe testing is and why. Blink IO believes intent driven Automation can help accelerate quality, how product QA and engineering teams can align on shared definitions of behavior, and how AI handles dynamic UI changes, timing issues and maintenance challenges automatically. And the session, as always, is going to include a live demonstration from a tent to automation scenario to execution as always followed by live Q and A. But not only can you try for yourself, but you can see it in action as well and follow along using that link down below. Register now. Hope to see you there. So Jason Arbin flagged this next article for me, which suggests it could be very useful for testing work and hinting that this might actually deprecate all testing frameworks. Will it? Well, let's check it out. He's talking about Cowork, a new desktop application from Anthropic that extends Claude code's capabilities to non coding tasks. The tool allows users to grant Claude access to specific folders on the computer where it can read, edit and create files. And according to the announcement, Cowork can handle tasks like reorganizing downloads by sorting and renaming files, creating spreadsheets from expense screenshots, or drafting reports to from scattered notes. And unlike standard Claude conversations, Cowork operates with greater autonomy. Once given a task, it creates a plan and works through it while keeping users informed of progress. And the initial release includes a set of skills designed to improve document and presentation creation. And when paired with Claude in Chrome, the tool can complete tasks requiring browser access. And the company is currently treating this as a research preview. I don't have access to it because I don't have Claude Max, but if you have Claude Max, you definitely could check it out. I might upgrade so I could try it as well. So but if you have it and you definitely want to see the future of automation, possibly does it replace testing frameworks? Well, let me know in the comments down below. Next up is some performance news. This is posted by David from Abstractor that they've released BlazeMeter MCP 1.1 adding skills and help, two features designed to give AI agents direct access to Blaze Meter's official documentation instead of relying on brittle web searches. Originally launched as guides in late 2025, the system evolved alongside Anthropic's agent skills and it now ships fully aligned with that model. If you don't know skills package structured task specific knowledge about BlazeMeter's features spanning 15 areas from performance testing and service virtualization to troubleshooting, scripting and billing and help layers in more than 500 official docs, split between step by step guides and practical answers to real world problems like slow responses, HTTP 500 errors and authentication failures. And I love this approach because it helps probably to reduce hallucinations and outdated device by grounding agents in authoritative docs, making AI assisted testing workflows more reliable from the start. This next one is all about testing AWS microservices. This is published by Arun, a senior performance and resilient test architect. All about multi availability zone testing for AWS microservices based on real world work with applications running on Amazon. So Rune argues that performance testing and resilience testing can't live in separate silos if you want to understand true end user impact and the article outlines how to combine peak load tests with controlled failure scenarios, potentially taking down one availability zone out of three and validating that traffic shifts to the remaining zones within one or two minutes and the recommended flow is straightforward. Run steady state peak load for 15 to 30 minutes, take one AZ down, observe behavior under load, then restore the zone and verify the system returns to baseline. Partial failures such as taking down individual nodes or pods are encouraged to simulate more realistic conditions. Definitely a great approach. You definitely should check out for yourself as well. All right. And for links of everything value we covered in this news episode, head over to the links in the comment down below. So that's it for this episode of the Test Skill News Show. I'm Joe. My mission to help you succeed in creating end to end full stack pipeline automation awesomeness and as always, test everything and keep the good cheers.