AI Prompt Engineering for Testers, Performance Pyramid and More TGNS109

Have you seen how to do prompt engineering with Cucumber and AI to create automated tests? What is the automation pyramid model for performance testing? And how can you use a new AI bot to assist in edge testing? Find out in this episode of the Test Skill new show for the week of January 21st. So grab your favorite cup of coffee or tea and let's do this. So this week I had the pleasure of interviewing two of my favorite automation testing tool entrepreneurs, Guy and Tal, and they told me all about what they're seeing as a perfect use of generative AI with automation. So they showed me their latest creation, Blink IO, and they went over how it can help prompt engineering in the testing world. Leveraging cucumber feature files to get AI working for you and your tests. And due to generative AI, the concept of prompt engineering really is gaining traction. You've probably heard a lot about it. And this innovative approach involves integrating artificial intelligence intelligence into the testing process, particularly through using cucumber feature files, a tool used in behavior driven development. And just like anything, the key to harnessing AI's potential in testing really lies in effective communication between the testing process and the AI system. So prompt engineering is the heart of this integration. So this is really an innovative solution using Cucumber to help testers and automation engineers to create prompts and directives with with cucumber feature files that are able to be understood by an AI system. And these prompts can instruct AI to perform various tasks like generating test cases, prioritizing scenarios, it can help identify risk areas, or it can actually help you automatically execute tests under a lot of changing conditions. And I like this approach because it kind of creates a direct bridge between traditional testing practices and the advanced capabilities of AI. And it allows testing teams to enhance the efforts with AI driven insights and actions. And the result is a smarter, more adaptive testing process that combines the expertise of a human tester that will always be needed with the efficiency of AI, leading to improved efficiency and hopefully software quality. I highly recommend you try it for yourself by using their Try now option and see how it works and then let me know your thoughts in the comments down below. All right. Last year I had SQL as one of the top skills I thought testers would need in 2023 and a lot of people laughed at me. Well, this next article goes over what I think is an important topic and that is how to optimize your SQL queries. This is by Navneet Kumar and he shares his journey of optimizing SQL queries that he says achieved a staggering 23% performance improvement. And he said the initial problem was the query was taking more than six seconds to fetch data. It involves a bunch of different joins and where clauses and sorting. And I can relate because when I was a performance engineer a lot of times performance issues were related to a poorly optimized SQL query. And he goes over how he identified several key issues contributing to the slowdown, like improper indexing, unnecessary joins in a bug related to empty string comparisons. But the most significant improvement was realized by rewriting the ORM generated SQL queries. So just a really great case study on how to you can help your performance optimizing SQL queries and definitely check that out in the comments down below. Also on my LinkedIn feed I noticed a post by Jason Arbonne talking about Checky AI and how it's introduced a new AI tool, the Edge Bot. What is it? Let's check it out. So this bot focuses on edge testing, which helps you do exploring of uncommon inputs or actions that are on the fringes of typical user behavior or often uncovering issues that developers may not even have anticipated. And the Edge Bot part of the Checky AI co test pilot Chrome extension really is a cool use of AI assisted testing, but also inspires human testers to think more creatively to come up with even more rigorous tests. And I really love this theme of collaboration, not replacement, of using AI and human testers together to create a more effective software testing plan. Another development in AI you should definitely check out and you can find that in the links down below. And staying on the theme of AI, I found this next article that talks about a new feature by Practitest. We had them on the podcast a few months ago and they said something cool was coming our way. And here's the new feature I think they were talking about. And this is how Practitest, which is a leader in enterprise grade software test management, has introduced its latest innovation, the Smart Fox AI Assistant. So the Smart Fox AI Assistant helps you with manual test creation by automating this traditionally labor intensive task and not only speeds up the process, but also enhances the clarity and precision of test steps based on user preferences. And besides the Smart Fox feature, they also introduced a Test Value score. I really love this. It's a feature that uses advanced machine learning algorithms to evaluate and score each test impact on the testing process. And according to them, this tool helps enable QA teams to retire up to 40% of irrelevant tests from their testing cycles, optimizing and streamlining their workflow so to learn more about a lot of advancements in AI, not just to functional testing but also with test management, here's one example of practitest that you should try out for yourself as well. Also on LinkedIn one of my automation friends, Tony sent me a link to this latest article that talks about a new visual language model called COG Agent and this article goes into detail on how Cog Agent was designed to help test automation, particularly in GUI based applications. This model boasts an impressive 18 billion parameters how excels in deciphering and navigating graphical user interfaces with precision and he goes into detail on Coagent's dual capabilities of processing both low and and high resolution images which allows it to interpret and interact with GUI elements at a high resolution. And this advancement is critical in scenarios where traditional language models fall short, such as intricate GUI navigation interactions and really goes into detail with a lot of examples on how Cog Agent is really useful. It's open source and definitely check it out and let me know your thoughts. I also came across an AI approach to generating code. So this novel approach to code generation was designed for code contests which significantly enhanced the performance of large language models in solving complex coding problems. So Alpha Codium which comprises competitive program problems with platforms like CodeForces and the result was impressive. Using Alpha Codium, GPT4's accuracy increased from 19 to 44% on the validation set and this method outperformed previous solutions like AlphaCode while requiring a significantly smaller computational budget. And the Alpha Codium approach involves a test based multi stage code oriented iterative flow and this method differs from traditional natural language process techniques, focusing instead on understanding and addressing the unique requirements of code generation tasks. And this process includes generating additional data such as self reflection in public test reasoning and enriched public test with AI generated tests, lots of AI and another open source solution which I'm excited about that you could check out as well. Do you use Playwright? How about BitBucket or do you want to know how to integrate Playwright with your CICD pipelines? Well I have an article for you and this is an article by Kailash who goes over Playwright with bitbuckets CICD pipelines and the article goes over how this integration streamlines the process of end to end test and parallel test execution while capturing crucial data like video and screenshots upon test failures. And the BitBucket pipeline is an integrated CI CD service within BitBucket automates the build, test and deploy process and cloud based containers. It features code collaboration built in, continuous integration, issue tracking and it goes into detail on how to set it up. So so really detailed article that if using Playwright, if you're using BitBucket, if you want to learn more about integrating it with cicd, here's an article for you. So two of my friends in the performance testing arena just came out with the new model. I think it's going to help you your performance testing. And this is by Leandro and Frederico and they go over a new model that Abstracta has introduced as a really cool approach to performance testing. And this model obviously was inspired by the famous Cohen Pyramid for functional automated Tests and is designed to optimize performance testing strategies and agile software development. And this automation pairing model for performance testing emphasizes the importance of automating various types of software tests, including load tests, which are crucial for preparing systems for high demand scenarios. And this model also helps in understanding how systems behave under heavy concurrent user access, preventing potential performance issues. This pyramid stands as a guide for defining a performance testing strategy to particularly in systems with microservices. It provides a detailed look at each layer of the pyramid, from unit performance testing to client side performance, offering insights into the advantages and disadvantages of each. All right, for links of everything of value we covered in this news episode, head on over to the comment down below. And if you haven't already, make sure you like and subscribe to the test Skill new show share with a friend or coworker because it really helps spread automation awesomeness to the masses. So that's it for this episode of the Test Guild News Show. I'm Joe. My mission is to help you succeed in creating end to end full stack pipeline automation awesomeness. As always, test everything and keep the good Cheers.