Top Testing Moves This Week: Playwright Migration, AI Skills, Shift-Left Pitfalls TGNS162

Are you stuck in a test migration maze? Can you really shift left or are you just shifting into chaos? And is Copilot's new browser the ultimate shortcut for animation engineers? I know in this episode of the Test Skill New show the week of July 6th to grab your favorite cup of coffee or tea and let's do this. Hey, before we get into the news, I want to thank this week's sponsor Zap Test AI, an AI driven platform that can help you supercharge your automation efforts. It's really cool because their intelligent Copilot generates optimized code snippets while their Plan Studio can help you effortlessly streamline your test case management. And what's even better is you can experience the power of AI in action with their risk free 6 month proof of concept featuring a dedicated Zap expert at no upfront costs. Unlock unparalleled efficiency and ROI in a testing process. Don't wait. Schedule a demo now and see how it can help you improve your test automation efforts using the link download down below all right, let's kick things off the question I know many testers have on their mind and that is should you finally make the leap from Cypress to Playwright? Here's what you need to know before you jump Check it out so on Medium maclesh recently outlined an approach for migrating test scripts from Cypress to Playwright using AI powered prompt engineering, and this article describes how testers can use a single, carefully crafted prompt to automatically convert Cypress code to Playwright, leveraging AI models like GPT4. And this method involves providing context rich instructions to the AI specifying input Cypress code and desired output, the equivalent Playwright code, along with a detailed breakdown of the transformation rules. Aklash shares example prompts and suggests using tools such as ChatGPT or similar large language models to perform this conversion. He also emphasized that this approach can help teams transition to Playwright with without manually rewriting large test suites, potentially saving you a bunch of time. However, he also notes that testers should still validate and maintain the generated code to ensure it works as intended. Check it out for yourself in the special link down below. Let me know in the comments if it works for you. So next up is the webinar of the week, so Shift Left is the mantra, but most teams get stuck in the weeds, so find out why and what you can do to fix it. In our upcoming Webinar, with Blink IO happening next week on July 15th, Shift Left sounds good on paper, but in reality, from my experience, most efforts fail due to scale and developers are expected to own automation, but they lack the time, tools and governance to do it well. The results? Fragile test code, duplicate efforts across teams and automation that's too flaky to trust, and 90% of end to end tests are never really wired into pipelines as quality gates. We're going to go over things like why shift left efforts often fall short in real world environments. What's holding back scalability, reliable end to end test automation, how gen AI can help your teams build production grade automation without the overhead, and how you can leverage AI test engineers to help you write, maintain and heal tests so your developers don't have to. So if you haven't already, I highly recommend you register for this webinar happening next week using the link down below. Hope to see you there. All right, so let's talk about mobile Generative AI is making big promises in mobile app testing, but is it ready to actually help you ship with confidence? Well, in this recent Medium article by Amrit, he examines how generative AI can be applied to mobile app testing, outlining its potential to speed up and strengthen the testing process. And this piece explains the traditional mobile app testing often struggles with a variety of devices, operation systems and frequent updates. Generative AI, known primarily for creating text and image, is now being explored to generate test cases, automatically, simulate diverse user interactions, and even help with UI validations. And this article highlights that generative AI can analyze app requirements and produce detailed test scenarios, reducing manual efforts and broadening coverage. It could also simulate real world user behavior, which is critical for uncovering issues early. However, they also note that generative AI solutions still need careful human oversight to validate outputs and avoid false positives or or irrelevant test scenarios. Amrit emphasizes the team should evaluate the reliability of their AI generated test cases and integrate them carefully with their existing automation frameworks rather than fully replacing traditional approaches. But I think it's a good point. Maybe generative AI can help some of the things that software test automation engineers need to do when they need automate mobile apps. But the best thing is to use your mind and work with the AI to make sure the output is correct and it's actually saving you real time and not just adding more overhead. So definitely check it out for yourself and let me know your thoughts. All right, so onto one of my favorite topics and that is tooling. So Copilot's new built in browser might sound like a dream for automation engineers, but how can it really help you in your day to day workflows? So GitHub has just announced that its Copilot coding agent now includes an integrated web browser, and this new capability allows Copilot to autonomously search the Internet to gather relevant information, retrieve documentation, and find code examples in real time while assisting developers and testers. And this browser feature is is designed to enhance Copilot's ability to solve more complex coding tasks that require context beyond local code or pre trained knowledge. For example, the agent can now look up obscure API references, check library updates, or verify best practices directly from the web without human prompting. And GitHub notes that this browsing behavior is controlled to reduce security risk. Inquiries are visible to users to maintain transparency. The update represents a significant expansion of Copilot's functionality, shifting it from a static code completion tool to a more autonomous coding assistant capable of live research and self directed learning. So just another thing Software testers need to be aware of what the developers might be using, and you may need to start considering the potential for inconsistent or unverified sources to impact your final code base that you're testing. So as we zoom out, AI isn't just about prompts anymore. There's a new essential skill that you'll want to build to stay ahead as well, if you haven't already. So in a recent article by Phil, he introduces the concept of context engineering as a new discipline aimed at improving how large language models perform in production environments. And Phil explains that while prompt engineering focuses on crafting instructions for single queries, context engineering involves designing and managing the broad set of information such as documents, user profiles, and dynamic data that an LLM uses to generate responses. And this article emphasizes that Context engineering helps LLMs make more accurate and relevant decisions by shaping the data they draw from rather than just refining the prompt. And Phil also aligns several strategies to context engineering, including retrieval, augmented generation, embedded based context selection, and using context agents to personalize outputs dynamically. For software testers and developers working with AI based features, understanding context engineering is crucial for building more reliable and user specific applications. And this approach also addresses common issues like hallucinations and inconsistencies in LLM outputs that are a major concern in critical systems. And while we're talking about skills, here's a reminder from the field. Deep technical system knowledge is becoming a make or break factor for testers. On the QT blog by Richard Bradshaw really emphasized the importance of having a strong technical system knowledge for quality assurance professionals. And the article explains that testers who understand a system's technical architecture, including hardware operations, operating systems, network dependencies, can design more effective tests and uncover issues that purely functional testing may overlook the article also points out the gaps in technical understanding can lead to missed defects, especially in areas like system integration and performance. And it also encouraged testers to move beyond black box approaches and engage more deeply with underlining system components. And it also highlights that as a system becomes more complex, technical awareness becomes essential for preventing costly failures and maintaining product reliability. The article also suggests that investing time in learning about system internals can also improve collaboration with developers and system architects, and by doing so, hopefully strengthen overall software quality all right, AI and security is always in the spotlight and OWASP just dropped a new AI testing guide. So let's unpack what's inside and why it matters to you. Check it out so OWASP has published the AI Testing Guide, a new resource aimed at helping software testers and security professionals evaluate AI enabled systems. This guide addresses unique challenges in testing artificial intelligence and machine learning models, which differs significantly from traditional software testing. This document covers critical areas including data quality and integrity, model robustness, adversarial testing and explainability. And it outlines specific strategies for assessing the security and reliability of AI models, emphasizing the need to test not only the code, but also the data pipelines and model outputs. This guide also highlights potential risks such as biased training, data, unexpected model behavior and vulnerabilities to adversarial attacks. It also encouraged testers to adopt a comprehensive risk based approach that accounts for both technical and ethical considerations when working with AI systems. Alright and for links of everything valley we covered in this news episode, head on over to links in the comment down below. So that's it for this episode of the Test Guild News Show. I'm Joe. My mission is to help you succeed in creating end to end full stack pipeline automation awesomeness and as always, test everything and keep the good. Cheers.