https://www.listennotes.com/e/05a3e0c4551c4b0fb4247c27343e12a6

Are you falling into this testing trap? Is AI powered testing just an illusion? What is Javis Appium? Find out in this episode of the Test Guild News show for the week of June 22nd. So grab your favorite cup of coffee or tea and let's do this. Hey, before we get into the news, I want to thank this week's sponsor, Zaptest AI, an AI driven platform that can help you supercharge your automation efforts. It's really cool because their intelligent copilot generates optimized code snippets while their plan studio can help you effortlessly streamline your test case management. And what's even better is you can experience the power of AI in action with their risk free 6 month proof of concept featuring a dedicated zap expert at no upfront costs. Unlock unparalleled efficiency and ROI in a testing process. Don't wait. Schedule your demo now and see how it can help you improve your test automation efforts using the link down below. All right, first up, Prijesh, a testing expert, examines a common trend in software testing the when compelling marketing narratives like automation is fast and modern and manual testing is slow is outdated shifts from influencing choices to driving overall testing strategy. Andy breaks down this article of what happens when businesses adopt tools and platforms largely on buzz rather than fit. And he also goes over that while automation brought speed and regression testing, it also introduced maintenance burdens, bloated suites, flaky tests, false positives and unchecked risk areas. The issue lays in making automation a goal, not a tool. And he also includes a real example from a fintech startup. And he also flags the rise of low code, no code tools that promises to democratize testing. And he thinks that these further the narrative that anyone could test with minimal skill and goes over how this furthers the narrative that anyone could test with minimal skill but risk masking hard testing decisions behind simplified interfaces. So are you falling into these traps? I think software testers should scrutinize automation narratives like these and select tools that align with their mission critical risk, not just those with the flashiest marketing. And something you need to be even more aware of now in the age of AI. And you can read more about this in the links down below. So speaking of traps, another one is thinking that AI is really thinking when it's doing automation or testing. And here's a deep dive from research from Apple that talks all about this important research. And this is all about how the research team as Apple has published a study critically evaluating the capabilities of Frontier Lodge reasoning models. And it goes over how the study introduced a controlled puzzle based framework to test reasoning abilities under increased levels of complexity, offering new insights into how these models process logic beyond just first answers. And rather than relying on math benchmarks, which are often contaminated with training data and difficult to control, the researchers used four puzzle environments, the Tower of Hanoi, Checker, Jumping, River Crossing and Blocksworld to analyze reasoning performance and trace accuracy. The framework allowed them to measure not just the outcome, but the quality and evaluation of the model's intermediate reasoning steps. So what did they find? Well, for simple problems, AI is good. Regular AI without thinking mode often does better on easy problems. For medium problems, thinking AI helps. The AI shows that thought process can be more helpful. Here the hard problems, though, both failed. Once the puzzles get harder, both types of AI break down and they stop thinking properly and give up even though they still had space to write more answers. It also highlights some weird behaviors that like sometimes the AI finds the right answer early but keeps guessing and messing up. And even when the right steps are given to the AI, it still messes up following them. So I think this highlights that if you want to create tests that just use AI to make decisions and solve problems, you can't trust that the AI is really reasoning. It might look smart, but it could be faking it or mimicking patterns. So something you definitely, definitely need to be aware of. Nick, find out more about the study and read the whole thing down below. Now, after all that being said, let's check out some AI OpenAI has just published OpenAI testing agent demo, a public monorepo showcasing a front end testing agent powered by the OpenAI computer using Agent Model and the Response API. And this demo demonstrates how the agent orchestrated by a Node server, uses Playwright to launch a browser, navigate a simple E commerce site, and executes automated test cases defined via a Next JS front end. I think this demo gives testers a live example of how to use AI driven UI testing and how to integrate model controlled actions with Playwright in a configurable front end environment and also showcase a potential shift towards agents that follow written test scripts and manipulate interfaces autonomously. But it remains experimental and as we've seen in the previous result, it's not really reasoning. So you can't really trust this. So if you're going to go down this road, make sure you're always testing the AI test. All right. One of my favorite frameworks back in the day, Serenity has just made a new release and Yan announced this on LinkedIn. The release of Serenity JS version 3.32 and this release introduces two key changes aimed at improving Playwright test integration. So some of the key updates is the worker scope actors in before all after all hooks. You can now use the actor called within those hooks to initiate actors once per test worker. If you don't know what actors are, this is something you should definitely take a deep dive on as well. It also goes over the support, setup and teardown tasks like starting services or validating API statuses shared across all testers in a worker has a cleaner and richer reporting environment and a bunch of other things as well. So if you're using Serenity JS definitely a worthwhile way to upgrade and if you haven't checked out actors, definitely something you should research more as well. And they have that all in the Serenity documentation also. All right, this next one also caught my attention on LinkedIn. I'm terrible with names, but Software development Engineer and Test Venkatesh recently published a LinkedIn post showcasing a custom built full stack automation framework that integrates GitHub, Copilot, Playwright, MCP Server and Trace Viewer to streamline end to end testing and you actually uploaded to GitHub where you could try it for yourself. And this framework automates scripting, test execution results, visualization and trace login across both the UI API layers. And and it also goes over how the use of GitHub Copilot supports AI assisted script generation while Playwright handles automation across web user interfaces and APIs and the MCP server is used for centralized orchest and execution and the Trace viewer enables detailed debugging via visual test traces. So if you're a tester and you want to start dabbling in testing exploring AI Accelerated Automation to check out how tools like GitHub, Copilot and Playwright can be integrated and orchestrated using MCP Server, you definitely probably want to check out this GitHub resource using the link down below. All right, I found another post by psy on his LinkedIn profile. This is probably the fourth time in four weeks I featured something that they've released recently on LinkedIn and this is a new tool designed to simplify mobile test automation using natural language and it's been Open source on GitHub. In this project, Jarvis Appium was jointly developed by Sudharshan, Srini and Sai and it offers an MCP server that connects Appium test automation with AI powered capabilities. They also link to a video demo, but if you go to the GitHub you can see how it allows testers to describe mobile test scenarios in plain English which are then converted into executable Java or Testng test code. This tool also includes AI native element detection to optimize locator strategies, and is compatible with Both Android UI automated two and iOS X UI tests. The key features of the tool is the integration with Goose, which I've been hearing a whole bunch about. I definitely have to get Angie Jones on the show to talk about it on the podcast. But anyway, if you don't know, it's a local AI agent used to coordinate workflows and support autonomous testing, and Javas handles test generation and execution, while Goose manages orchestration, forming a combined system intended to reduce manual efforts in mobile Qi processing. All right, this next article is a cool approach about documentation. It's by QA specialist Ivan who details how to eliminate the need for traditional API documentation by implementing self documented testing payloads within a TypeScript based Quality Assurance framework. And this article offers practical guidance to improving reliability, consistency and maintainability in API testing workflows, particularly in high complex environments like Fintech and Crypto. And it outlines a strategy for using custom TypeScript types to address four persistent issues. It also makes a case for embedding documentation directly into code through strongly typed definitions, effectively removing the need to reference external API docs during test development. And if you're using Playwright and you want to incorporate accessibility testing and have a dashboard, I have a resource for you. You have any? Hit me up on LinkedIn to let me know of a new tool called Axe Playwright Report which is available on NPM for testers using Playwright. This library allows users to embed accessibility testing into Playwright workflows by leveraging Deque's Axe core engine and enables detection of accessibility issues in real time using UI testing and some of the key features that can help you find accessibility issues during Playwright test. It also has a really cool dashboard report all right, in the links of Everything value we covered in this episode, head on over to the links in the comments down below. So that's it for this episode of the Test Skill New show. I'm Joe. My mission is to help you succeed in creating end to end full Stack pipeline automation awesomeness and as always, test everything and keep the good cheers.