Top Gift For Testers, 70% Problem, Test Coverage and More TGNS144

Do you want to know the perfect holiday gift to give that special tester in your life? Have you heard about the 70% problem when it comes to AI assisted coding? And what does test coverage even mean? Find out on this episode of the Test Guild News show for the week of December 15th. So grab your favorite cup of coffee or tea and let's do this. Just a heads up, this will be the last new show for 2024. Thank you everyone for your support. I'll be away celebrating the holidays and taking a long Cruise celebrating my 25th wedding anniversary. So I'll be back mid January. But before we get into it, if you haven't already, let's end the year strong by subscribing to Our Test Guild LinkedIn New Show Newsletter that I'll have a link for down below to never miss another episode in the new year. All right. And also, don't forget to get the lowest price of the season to check out Automation Guild 2025, our ninth annual online conference dedicated to helping you succeed in creating all things automation awesomeness. It's the perfect gift for that special automation tester in your life. We also offer bulk discounts if you have an education budget that's use it or lose it that most people do at the end of the year. And this is a great time to grab your ticket now and accelerate your 2025 automation efforts and start your year strong. Don't miss out. Register now using the link down below and hope to see you there. All right, so what is the 70% problem? Let's check it out.

So a new analysis by Adi Osmani, a engineering leader at Google, reveals a significant disconnect between AI powered software development speeds and actual product quality. And despite developers reporting increased productivity with AI coding tools like GitHub, Copilot, Cursor and Klein, the investigation finds that software quality isn't showing corresponding improvements. The report identifies what researchers call the 70% problem. AI tools can rapidly generate initial code, but struggles with the crucial final 30% needed for production ready software. And this includes handling edge cases, error management and maintaining code stability. And I think this study highlights a particular concern for testing teams, as an AI generated code often lacks robust error handling and comprehensive test coverage. So senior developers were observed spending significant time refactoring AI suggestions and strengthening type definitions, steps that junior developers frequently missed. And the article also points out that AI tools are creating a new category of testing challenges. Teams using AI for rapid prototyping often produce what's termed house of card code applications that work during demos but fail under real world conditions. Common issues include confusing error messages, overlooked edge cases, and accessibility problems. The investigation found that successful teams implementing a trust but verify approach, combining AI generated code with rigorous manual review processes and automated testing of edge cases. So I just think this points out what a lot of industry experts point out, that testers are more important now than ever with AI and may be able to get you halfway there. But really the thinking part, the human part, is so valuable, it's always going to be there. And if you're a tester, I think you have even better prospects for your future in your career in 2025. If you have anything to do with Cypress, you want to listen to this next announcement? It's by Sebastian wrote on LinkedIn that they just released a new version of 1.40 of the Cypress AJV schema validation plugin. The plugin, which is open source, simplifies API testing for validating API responses against JSON schemas, swagger documents, and open API specifications. It leverages an AJV JSON schema validator known for its speed and reliability, and the tool integrates seamlessly with other popular Cypress plugins. So if you're doing anything with Cypress or API testing, you need to know about this plugin. Definitely a worthwhile plugin to check out. See if it can enhance your API testing workflows. All right, Kristen Giacovone just released a new article as well, and it's all about what does test coverage mean? So in this recent blog post, Kristen goes over the complexities behind test coverage and its implications for Software Quality Assurance Authority examined. And in this post it delineates test coverage as the measure of how much of a system's functionality is exercised by test in place, and emphasize that while high test coverage can indicate thorough testing, it does not necessarily ensure product quality or defect. Free software the article outlines various methods for measuring test coverage, such as code coverage, climate coverage, risk coverage, and others, each providing different insights into the software testing status. It highlights the common misconception of 100% test coverage guarantees zero bugs, clarifying that and instead reflects the dullness of the test executed against documented requirements or code paths. So software testers, you have to understand that test coverage is a tool to assess the breadth, not the depth, of testing. It should be used alongside other quality metrics that Kristen points out to ensure comprehensive software assessments. You can learn more about this using the link down below. Also on LinkedIn, as I was scrolling, I found a new package I haven't heard of before that I think it's kind of unique for a test case that a lot of people struggle with, and it's a new NPM package Temp Disposable Email Released by Parasync, this package offers an efficient solution for managing temporary email accounts specifically designed to streamlining email based workflows such as account signups, password resets and email verifications. So TEMP Disposable Email enables testers to create unique disposable email addresses with ease, retrieve and verify email content, and automatically clean up accounts post testing. Notably, the package integrates seamlessly with popular testing tools like Playwright, Cypress and Node js, so making it a versatile addition to the test suites. It supports both es modules and CommonJS, ensuring compatibility across various project setups, and some key features include configurable polling for retrieving emails, the ability to extract email content for validation, and automated cleanup processes to maintain a really tidy testing environment. Really cool solution. You definitely should check out if you're doing anything with email testing right? And follow the money or acquisition news we have SmartBear has acquired qmetry to enhance test management solutions. So if you don't know, qmetry is known for its test management solutions, this acquisition is poised to expand SmartBear's portfolio. SmartBear has acquired a bunch of other companies and so they now have really extensive end to end kind of testing solution and they're integrating qmetry's offerings to provide comprehensive test management capabilities to the users. So the reason why I bring this up is for some reason the past month or so more and more testing companies have been bringing out new test management solutions for their portfolio. So obviously it's a area of need within the community, so just something to keep your eye on if you're struggling with test management. A lot of these companies is now starting to integrate better solutions within their existing testing portfolio, so definitely something to keep your eye on and try out for yourself. In other test automation industry news, I came across this article how Lambda Test partners were to enhance their no code mobile app testing with real device software support. So LambdaTest has announced a strategic partnership with Audify to augment its no code mobile app testing platform by introducing real device testing capabilities. And this collaboration aims to enhance testing processes by allowing users to perform more accurate and reliable tests on real devices, providing an essential tool for testers to emulate genuine user experiences more effectively. I first learned about Audify this year and I'm really excited to have them as one of our sponsors for Audio Automation Guild and they have a really cool no code testing functionality So I think it's a really good fit between the two. Clearly another growing trend that we're seeing going into 2025 is the growth of Playwright and other companies starting to leverage Playwright themselves. This is how Artillery has just announced Playwright Test Run reports and now supported an Artillery Cloud. You don't know Artillery Cloud is a platform known for its load testing capabilities leveraging Playwright and it's announced full support for Playwright's test Runner Reports. This new feature allows users to monitor Playwright tests in real time. You detailed HTML reporting complete with traces and screenshots and automate pull requests comments via GitHub Actions integrations and it's available to all Artillery Cloud users, including those on the free developer plan with no additional costs, which is really cool. We're doing anything with Playwright and you want to really get some use out of your functional test. To help with performance testing as well, here's a solution you should check out as well, using the link down below and up Next is how OpenTelemetry has made a huge announcement. They just announced the integration of Generative AI technology into its observability framework. Going to the article, this integration aims to enhance the capabilities of software testers by automating data analysis processes. Open Telemetry widely adopted for its open source observability solution. We'll use Generative AI now to provide deeper insights and streamline error detection, hopefully leading to faster issue resolution. And the incorporation of AI is expected to reduce the minimal effort required for data interpretation. And incorporating AI is expected to help reduce the manual effort required for data interpretation. So OpenTelemetry is awesome and now including generative AI is really going to help. I think the testing community if you haven't looked into OpenTelemetry, here's just another reason to do so now as we head into the new year. All right, for links of everything of value we covered in this news episode, head over to the links in the comment down below. Down below. So that's it for this episode of the Test Guild new show. I'm Joe. My missions help you succeed in creating end to end full stack pipeline automation Awesomeness as always. Test everything and keep the good. Cheers.