SimpleQA, Playwright in DevOps, Testing too big? TGNS140

Are your automated tests too big? How can you use AI powered automation on Android app testing? And have you seen the Guide to Azure DevOps pipelines in test plans with Playwright? If not, find out on this episode of the Test Guild News show for the week of November 3rd. So grab your favorite cup of coffee or tea and let's do this before we get into it. If you haven't already, please make sure to subscribe to Our Test Guild LinkedIn News Show Newsletter that I have a link for down below to never miss another episode. So first up is all about large language models and it was posted on LinkedIn by Tyrique, where talks about how OpenAI recently announced the launch of Simple QA. What is Simple QA? Let's check it out. So Simple QA is a new feature designed to enhance the accuracy and reliability of answers generated by large language models, and Simple QA aims to refine LLM outputs by focusing on straightforward, concise answers rather than complex multi step reasoning. OpenAI's move reflects a growing awareness of quality issues surrounding AI systems, especially with the increased use of LLMs in professional settings where precision is critical. And as you can see, industry leaders including Tyreek King have expressed optimism about OpenAI's initiative, viewing it as a first step towards a broader industry wide emphasis on quality control for AI and LLMs. Also, I can't believe how fast time is moving. I just want to give you a sneak peek Behind Automation Guild 2025 so if you already signed up for Early Birds, you should receive an email on when you can stop purchasing tickets. If you haven't, definitely check out our website where you can find out more about what is going on at the 9th annual online event and I also started picking some of the sessions we have 16 so far. The first batch, I have about two or three more batches we'll go through and you can see a lot of the sessions here and you'll be able to see the type of sessions that are going to be available. The 2025 Automation Guild that's happening February 10th to the 14th. Would love to see you at the next Automation Guild. You can find a link for it down below. Register now. Hope to see you there. All right, a question that comes up very often is should I invest in open source tools or commercial tools? And what's the difference? Well, I found this article on LinkedIn that I think is going to help break it down a little bit more for you. This is posted by Eric and it's a link to a recent white paper by Query Surge, which is titled A Comparative Analysis Commercial Software versus Homegrown Utility versus SI Frameworks and this explores the difference between the three different types of software testing solutions and the analysis considers factors such as cost, performance, scalability and ease of maintenance. I know from working in enterprises that commercial software solutions often provide robust support and scalability, making them a popular choice for for large organizations, and I think these solutions are generally well documented and offer a comprehensive suite of features. However, they can be costly and may not be fully aligned with your specific business needs. It also goes over how homegrown utilities created and maintained in house offer high customization levels and can be tailored to specific processes. While they may save costs initially, these solutions often require significant ongoing maintenance and in house expertise, which can be resource intensive and the last type is system integration frameworks, on the other hand, combined elements of both commercial and homegrown solutions. They provide a balanced approach offering scalability customizations, but typically require specialized knowledge to implement effectively. So which one should you choose? Well, you can read more about this in the comments down below and let me know your thoughts in the comments as well. All right, next up is all about a community survey around test automation. Dorothy Graham just released her ebook on the survey results of test automation practices and reveals persistent challenges in test automation. What are they? Well, let's take a deeper dive. This new community survey conducted between August and September of 2024 has uncovered many insights into the current state of test automation. The study, which gathered response from 206 software testing professionals, reveals that while automation adoption is widespread, fundamental challenges persistent. Key findings show that 80% of organizations are now using open source automation tools, with over 90% implementing system level and end to end testing. However, the survey identifies concern gaps in implementing and training. Notably, 20% of practitioners receive no formal training in their Automation tools and 25% lack training in testware architecture. And the survey also exposes a critical weakness in automation design. Nearly 40% respondents would need to replace 75 to 90% of the total code if they change automation platforms, indicating potential issues with test maintainability and architecture. So some key takeaways is you may want to prioritize architecture and design in your automation projects to avoid costly rewrites. You definitely want to invest in proper training despite management pressure to deliver quickly. Definitely focused on creating maintainable test agnostic testing frameworks and document and communicate automation ROI to manage stakeholder expectations as well. Definitely an ebook. You should check out anything that Dorothy has released I've been a fan of since the 90s, so you should definitely check it out as well and you can find it in the comments down below. Also, on my own in house survey I found one gap is around training around Playwright. So I found this guide on to use Azure DevOps pipelines and test plans with Playwright and this was written by Abigail who provides a comprehensive guide on using Azure DevOps pipelines. Abigail elaborates on key features such as continuous integration and continuous delivery, which helps in automating workflows, increasing efficiencies and reducing manual errors. She also emphasized the importance of configuring pipelines accurately to facilitate seamless integration and deployment processes. And the guide provides step by step instructions on setting up pipelines, configuring triggers and deploying applications across different environments. And another post that caught my attention this week was by Kailash, who posted a blog post he wrote for applitudes, which unveils the advancement in their automated testing platform, which unveils the advancement in their autonomous testing platform and some key features Kailash highlights is the platform's ability to integrate seamlessly with popular CICD pipelines and its capacity to scale test operations for large organizations. This ensures that software defects identified early in the process, minimizing end user disruption. In addition, the use of machine learning technology is emphasized, enabling testers focus more on critical analysis and less on repetitive tasks. And he goes over a little bit more about all these benefits and how you can use it as well to help you with your testing efforts as well. Have you wondered how to use AI for AI powered Android automation? Well, this recent blog post by Damon Marinatra delves into the impact of AI powered automation on Android app testing. As mobile applications grow increasingly complex, traditional testing methods really struggle, usually to keep up. And the article breaks down how using AI may be able to help you do testing even better of mobile apps with your automation. AI driven automation solutions promise not only faster test execution, but also can reduce human error, ultimately leading to more robust and reliable applications. This is a breakdown of a proof of concept AI techniques which are harnessed to create automated test cases that adapt to various scenarios, considerably reduce the time and effort required for generating and managing tests. And by intelligently identifying potential issues before they affect end users, AI powered testing tools can ensure higher quality digital products. This blog highlights several advancements of cooperating AI and into the testing workflow. Have you ever felt maybe your test are too big? Well, you're not alone. This next post by Rich Jordan breaks down are you testing too big? Isolating your blast radius? And he explores the challenges software testers face when dealing with large scale testing. Rich argues that testing overly large scopes can lead to increased risk and inefficiencies. He advocates for minimizing the blast radius or the extent of potential negative impacts from testing to assure more precise results and to safeguard system stability. And he goes on to emphasize that breaking down tests into smaller manageable units allows for quicker iterations, easier debugging, and more targeted understanding of system behavior. And this approach not only enhances the quality of software testing processes, but also improves overall productivity by reducing the complexity involved in troubleshooting the large scale system failures. Recently I've seen more and more discussions around performance testing and how it's taken center stage as an essential practice for assuring application reliability. And this next post on LinkedIn, highlighted by Carlos, emphasizes the growing importance of performance testing and platforms popular among developers. And if you're not sure how to get started, he also post a link to Grafana K6 cheat sheet, which is everything a performance engineer needs to know. Last up is Security News. I always love coming across new tools. This is a new open source application security testing framework I just learned more about, and it's by Go Security. Go Security is a cybersecurity firm and it's recently announced the introduction of Reaper, which is an open source tool designed to enhance the secure development and testing of applications. And Reaper offers comprehensive features that support various stages of software development including code scanning and threat detection and vulnerability management. And for links of everything value we covered in this news episode. Heading over to the links in the comment down below. So that's it for this episode of the Test Skill New show. I'm Joe. My mission is to help you succeed in creating end to end full stack pipeline automation awesomeness. As always, test everything and keep the good. Cheers.