249: Next Generation Agile Automation with Guljeet Nagpaul

Hit it.

Welcome to the Test Talks podcast, the place to go to geek out on software testing. And now your host whose mission is to help you succeed with test automation, Joe Colantonio.

Hey, it's Joe. And welcome to another episode of Test Talks. One of the hot new topics in our industry is continuous testing along with AI and machine learning. So today we'll be test talking with Gujit Nagopal, all about next generation agile automation. We cover topics like codeless test automation, improving agile quality management and more. Guljeet used to work for Mercury back in the day as head of the ALM practices, so he really knows his stuff. So listen up and discover what has changed since that time and what you need to know now in today's rapidly changing automation testing landscape. Check it out. Test Talks is sponsored by the fantastic folks at Sauce Labs, the cloud based automated testing platform that eliminates the need to maintain your own selenium grid and test infrastructure. Try it for free today. Visit testalks.com and click on the sign up now link under the homepage sponsor section.

Hey guilty. Welcome to Test Talks.

Hey Joe, great to be here. Thanks for having me.

Awesome. It's great to finally have you on the show. You one of our sponsors at the Lasso Automation Guild and I've been really. You did an awesome job in the roundtable. So I thought it'd be awesome to have you on the show to talk a little bit more about your all your experience around automation testing, but specifically about next generation quality assurance for Agile test automation. But before we get into it, can you just tell us a little bit more about yourself?

Sure. And appreciate, you know, the opportunity in the Gale conference. I honestly thought it was a great forum and what I really uniquely liked about it was kind of the community feeling to getting everyone together and the agenda, how you kind of drive it and the topics were, you know, very oriented to kind of making things better rather than kind of focusing on a certain tool or.

A certain technology and kind of bottlenecking it. And that's why we felt from an accel queue standpoint that was kind of a perfect forum to.

Be part of the community. At the end of the day our goal is similar to make things better for testing, for test automation and kind of really get aligned to the agile quality assurance mode.

Awesome. So guajit, you mentioned about making things better. I believe you started in Mercury Mercury Interactive back in the day with Windrunner.

That's right. I'm always shy in introducing myself. I think you asked me.

Again, my name is Guljeet Nagpal Yes, I did start really back in the days, in fact even before it was called WinRunner, if you remember the early early days when it was Test Runner and we were the first record and play tool. So I spent many, many years, almost over a decade with those guys and we got acquired by hp. So really seen the evolution of these tools. I was leading the complete ALM practice for North America for the ALM toolset. By the end of it we had a whole suite of products, not just the automation products, but the performance products.

The test management solution, which is now alm, the service virtualization and the whole ALM suite. So it was an exciting journey and learned a lot from that experience and really excited to be now part of Excel Q which is kind of truly the learning of what we learned from how these tools have evolved in the past 20 years as well as how technologies and methodologies around our development has evolved in the past so many years, aligning with more continuous delivery, faster releases and so on. So I'm excited to kind of talk about that today with you.

Yeah, so that's, that's the first question really is what did you learn from that experience starting off from really at ground level, automation testing tools that you thought, hey, if I ever started my own company I would do this or this is a problem I've seen for 20 years and it can be addressed. Now is there anything that, that caused you to decide to go with the new company that you started to, to address any specific issue that you've been thinking about all those years?

Yeah, absolutely. I can, I'll try to stop myself not to talk about it all day, but keeping it, keeping it kind of short and quick, you know, unfortunately, you know, testing still remains to be a bit of reactive. Right. We've seen so much innovation and automation and deployment release build these life cycles to align with the Agile or the continuous integration or DevOps mode, whatever we call it. At the end of the day you want faster releases. But testing has suffered a lot of overhead and reactive and there is some truth to it. Our goal was not just to automate a test case in silo. Automating a test case or an end to end flow has been done in, you know, past 20 years. And yes, there is some magic to that also. Right. So that you automate that in the most efficient manner, making it kind of most modular, most reusable, lower maintenance and all that good stuff. But also the goal was to take a step back and kind of, you know, identify, you know, Various bottlenecks or inefficiencies across the complete life cycle. Right. Things like test design, they still remain very manually driven. Right. You know, no matter whether you're going for commercial tools or you're doing open source, you know, you're reinventing the wheel of frameworks and then kind of chasing your automation code and the code maintenance and then API testing unfortunately still remains to be kind of in a completely different world. So you know, I'm talking about a lot of these challenges like test design, framework overhead, API testing, kind of not being able to embed as part of your end to end flow as well as it still remains to be very specialist function. It needs really senior development skills to be able to produce a good automation sound code. So our goal was to take a step back and look at all of these challenges, not just test case automation in silo and have a platform which connects all this, but then at the end of the day also connect it to your business process, connect it to your actual application flows, not have it more technically driven. So you know, these are kind of some of the key things. We really wanted to take a kind of a fresh and a principally different approach. Does that make sense?

Yeah, absolutely. And I know a lot of people say, well we have Selenium, it's open source and it does pretty much what I want. But you know, are there anything you see wrong with, not wrong with that approach, but lacking in that particular approach that maybe a solution can help them with?

Yeah, I mean Selenium is, I mean super powerful. I mean it's, I, I always kind of think of that analogy when it comes to someone asking me about Selenium. I love Selenium by the way. It's changed the way we look at automation. It's changed the way how we can scale up automation. Look, these testing tools which have come through in the past so many years, they look kind of fancy and I hate to use the word gimmicky, but almost gimmicky in terms of how easy the test automation can be. But, but if you look under the hood, if you really want to scale up your automation, you can't scale it up unless you have the power and flexibility. And that's what Selenium brought to the table. Right. To really handle more complex business rules, to handle various technology stacks, to give the power back in testers hand. Right. But it obviously comes with a pretty huge overhead of custom frameworks, automation design, chasing that code maintenance. And unless you write a series of perfect world frameworks, you know, the output of Selenium is going to drag you down, is going to drag your ROI down. So you know, again, our goal was to kind of bring that power, bring that flexibility, but with the ease of use, so that an automation engineer, you know, can, can focus more on their throughput and their efficiency rather than handling the overhead in their, in their times and effort which they spend. So, you know, the goal was to kind of take the best world, take, take the best of both worlds. Right?

Awesome. Absolutely. And so I'm just thinking back in the day with Windrunner and like you said, Test Runner before that a lot of these solutions were promoted as just record and playback and obviously we know that that caused a lot of issues. Nowadays it's always we have a codeless solution that's AI based. What's different between the promises made with a recorded playback solution maybe 20 years ago, I don't know if it's been that long, but almost 20 years ago, and a solution now that uses a Colis type AI based approach.

Right, so that's a great question. Right. So we all saw how our web technology evolved. Right. So even before going to test automation tools, I'll kind of quickly touch on that. Right. Our web technology, if you think about it, in the time frame when these tools were starting to come up, was so simple. Right. I mean we used to have, you know, simple HTML pages. There would be a Java code sitting behind it or a JSP page if you, if that rings a bell. Right. And you know, our content was not as dynamic. Right. Our technology frameworks were pretty limited. Right. And record and playback did a fairly good job back in the days to handle kind of that, you know, that, that technology. Right. And of course our methodologies around our development were very waterfall driven and we had kind of defined and long phases to get to that regression maturity. And then these record and playback tools, the moment you get into complex validation, still had a lot of limitations even back in the days in terms of then having to script your way out of handling any complexities in terms of business rules. But then as kind of our web technology evolved, our methodology evolved. It needed a lot more dynamic handling in terms of the content on the page, maybe the front end getting.

So dynamic as well as the technology stacks constantly evolving with custom controls to angular to Node JS to what have you. We have pretty much a new flavor every month. The need was obviously from a record and play standpoint and scripting your way out of it would not be able to handle it. And then, you know, then came a wave of these AI based tools which kind of talk about can kind of magically handle a lot of that, which is good, but at the end of the day you still need certain flexibility giving back in the power of developers hand so that they can actually kind of, you know, handle those complex business validation rules. Right. Which still need some level of, you know, language in terms of being able to handle that within the automation logic. Right. So again, you know, the goal was to take the AI approach to make things simpler, but at the same time give the platform back the power to the testers similar to what they're kind of getting used to in terms of Selenium world. Right. So in, in our mind the goal was to leverage AI to make things simpler, not to take kind of the whole testing process, the automation process, and kind of spin magic around it. Because at the end of the day our applications are complex and they need to be, our automation tool needs to be able to handle those complex business validation rules.

So also with the modern test tool, a lot of teams like you mentioned earlier, are trying to go towards continuous delivery, continuous integration. Are there features you've seen lacking in the industry now that a solution like yours help address to help make that particular transition easier for an organization trying to move faster and faster?

Yeah, totally. I mean, you know.

Interestingly, we talk, we talk about continuous delivery and continuous integration and I so often talk about this fact that, you know, if you have, you know, a lot of tools and a lot of organizations kind of have a misconception that if they just align their automation suite to a CI tool, that's their job's done. Right. They've achieved continuous testing. Right. But it's really a lot more than that. Right. If you think about it, automation has been always limited to regression automation, which means there is this notion other than all the other things I talked about in terms of bringing efficiencies, making it faster and all that good stuff, there still remains to be a big.

Blockage, so to say.

Which basically points to that you shouldn't be automating too soon in your application lifecycle. Which means if your development's going on or your screen's not baked enough, your application's still going through too many changes, let's kind of limit the automation. But that's one of the biggest blockers to, to bring that automation within the sprint. Right. So one of the kind of the unique approaches or unique differentiators, how we've approached it is almost brought that virtualization which we hear a lot in the performance world or in the web services world down to the functional world. Right. And it's something which is completely.

Innovated from our platform standpoint. Right. Which is basically being able to functionally virtualize your application so that even while it's under development, start writing your automation logic, start actually writing your automation test suites so that you really get ahead of the curve. Right. In terms of bringing that in Sprint automation and not suffer a couple sprints lag, which you usually do. Right. So it's one of the aspects where you can kind of really align to that continuous testing and then eventually having your testing aligned to your continuous delivery.

Can you give an example of that? Like you're stubbing or mocking something that you're interacting with that may not be developed yet?

Pretty much. Pretty much. So as long as you have a fair bit of idea in your head how your wireframes would look like, it's pretty much being able to abstract down to the last element level detail, so that you can think through and write your automation logic and your automation scenarios while your development's happening, abstracting all of those elements and the page logic down to that level of detail. And then there is an automated reconciliation process. Again, like I said, that's where we've leveraged AI along with a lot of other aspects. Right. Where to kind of make that process a lot more efficient and faster. Right. But yeah, pretty much like you said, very cool.

So I think I'm a big proponent of using any tool that helps me create software better, faster, with higher quality for the users. But it seems like almost people are like locked in on it has to be an open source solution because they're afraid if they don't use an open source solution, they're going to be locked into someone else's ecosystem. So with your type of solution, do you force people to have to change the way they develop software? And if they start using your solution, then they're locked in using that forever.

That's a great point. And I always think of that analogy, Joe. You know, you make a customer buy a printer and then they got to buy the ink for the rest of the time.

So, you know, we are very aligned to the next generation or the next evolution of how technology is consumed. And we've already seen it in the past five to eight years. Right. You know, technology consumption has become very community oriented, very open source oriented. And that's the way it's going to be. And we're totally cognizant and aligned to that aspect. Interestingly, even though our automation is natural. English Language driven. It doesn't vendor lock you into any kind of proprietary language. We are completely open source aligned. You don't like the tool, the next month you can just walk away and continue using your Selenium. So it's complete transparency to open source and precisely to the point you mentioned. I kind of hate to bring it up in a salesy mode, but that's pretty much how our alignment is that you don't like it, walk away, continue doing how you were doing it without this whole migration to worrying about vendor locking overheads and all that.

Awesome. So I'm just curious to know how this would work. CC1 has an existing framework.

That they want to add more functionality to or they want to leverage, say Excel Q. How would that work? Does it work on top of an existing framework or would they have to start from scratch using your framework?

We pretty much are aligned to open source, so maybe getting started on our framework or moving away. Either ways, we use a whole bunch of frameworks under the hood. Like I was saying, I think we do web automation, APIs, mainframes, what have you in terms of enterprise support, like enterprise applications applications, maybe Salesforce, SAP or what have you. Right. So there are a lot of frameworks in play, but all of them are pretty much open source online. Right. So if we have customers who we've moved over using a migration framework from their current Selenium as well as we have customers who really like to see what kind of Java code is being generated behind the scenes from our framework and kind of, you know, see if they can actually take that and continue doing that even without our tools. Right. So both ways we kind of offer that flexibility. Right. Because at the end of the day, you know, we want the customers to feel confident and we want to be confident that, you know, they're using the platform because they like it, not because they're locked into it.

Absolutely. You know, you just mentioned Salesforce and erp. It just reminded me of something that's one of the reasons why I used to like A tool like WinRunner or QTP is it would have certain add ins that could give you built in functionality to test these things without having to write code from scratch. So that seems like a benefit that out of the box go into Excel queue. If you're testing these type of Salesforce type applications, does it have baked in features to help them with that already so they can start testing without putting any overhead in?

Yeah, I would have loved to. I'm kind of almost itching to share my screen and show how a universe driven model works. Because you know, after a point I can imagine talking doesn't do the work. Right. You've got to kind of see it in action. So. But I'll try to answer that without kind of showing you.

Right.

But we have a universe driven approach, right. Which is basically like an application modeling. Right. And with a lot of these, you know, ERP applications or kind of, you know, these enterprise applications, we have partnerships where we pre built these universes. In fact, with Salesforce, since you mentioned, we were actually at Dreamforce and we are very close partner with Salesforce. They actually went through the evaluation through their technology innovation board for the automation. Right. And we've pre built universes for Salesforce. And the other thing is Salesforce and some of these applications, if you think about it, Joe, are very configurable, right. Which means they're pretty nasty in terms of their dynamic content and the way their workflows can be configured. Their fields can be.

Changed. Some customers may have five fields, some may have 50 fields and so on. Right. And then, you know, they're using dynamic iFrames, embedded IDs, maybe Pegas, Salesforce of the world. And that's what kind of, you know, I'm going a little bit more in detail, but that's kind of the sweetest part of our platform which is being able to handle these, you know, dynamic contents from an element identification standpoint kind of seamlessly. Right. So that's what made us so close to. Apart from just building their assets as part of our pre built universes, our kind of ground up approach to dealing with their dynamic technologies made us so kind of closely driven with them. Does that make sense?

Absolutely. Yeah, that's a good point. So I guess another thing that people would struggle with is I guess the question here is how does AI come into play with any of these automation solutions, say for Salesforce or any other automation someone's doing, does it automatically do element recognition for you or like how hard or how much AI is baked into the solution?

Totally. So AI these days, Joe, has been used so loosely that I've almost limited myself talking about it in the first go. I like to kind of handle the core concepts and that's why I really appreciate you kind of asking me what, what. Let's take a step back. What are the things you're trying to solve? Right, let's look at that first. So you know, like, like you rightly said that that's, you know, that's how we look at AI, like you know, in different Aspects to, to make things a lot more efficient. Right. And those different aspects being, you know, one is natural language automation. Right. So we've kind of coupled the recording, you know, and identification of screens with this natural language interface, which kind of brings the power to the tester to be able to handle more complex stuff and still give them the flexibility, but without programming. So that's one aspect where kind of our AI kicks in. There are a lot of different aspects in natural language processing, as you can imagine. Right. We want to make it more intuitive for the users to adopt the natural language as well. So that's one aspect. There are other aspects like you mentioned on element identification to make it. At the end of the day, what is the biggest pain point in element identification? Right. It's just not robust. It's just too flaky. Right. It just needs a ton of maintenance. And if, if we can leverage AI both in the identification design process, but also in the runtime process to make it more robust, given our developers love to change attributes every release. Right. So again, bringing in certain algorithms and flavor of AI to kind of deal with both design and runtime robustness of automation and then also looking at other aspects like data driven testing has been around forever, but if you think about it, it's still a short in the dark of how many permutation combinations of data you should test your process with. Right. So bringing in certain AI based algorithms to kind of auto generate your test cases for you. Right. And then certain elements of AI and change management. So across the lifecycle. Right. That's how we looked at it. That. All right. Where is the scope of bringing in certain algorithms which can bring in that efficiency and automation to the overall process?

Cool. So I definitely see the benefit for AI. Another issue I see about automation in general from frameworks that people build, is how are they able to make informed decisions based on the output? I mean, they may be able to have a lot of tests and they have a lot of output, but they don't necessarily know how to make decisions based on that is what I found. So. So does this solution include anything to help you make more informed decisions on where you are with your release and whether or not it's good enough to ship?

Very true, very true. And then you talked about Selenium and I actually almost missed that aspect other than the framework. Right. Is the whole governance and tracking aspect of it. No matter how good you can create your automation, it's no good if I can't relate it quickly back to my upcoming release and imagine how less of a time I have to create a perfect, or to create a regression suite for an upcoming Sprint within my Sprints, or have multiple versions of my test plans and test strategy driven by whether it's smoke that sanity, whether it's module oriented, whether it's for a Sprint release, for a fire fix or what have you. So again, going back to kind of that challenge and being able to relate that so suite and those quote unquote scripts to your actual business processes or your user stories.

From your requirements management standpoint. Right. So other than kind of the tracking and execution, I feel that's kind of been one of the most critical missing pieces. Maybe if you talk about commercial tools which had like this glorified handoff to test management, or if you talk about Selenium, where all of this is kind of, kind of really broken down in terms of multiple handoffs and so on. Right. So that, that, that, that's one aspect where we kind of again approached it principally different. We kind of baked in this whole test management, test planning, test strategy, execution, tracking as part of automation itself rather than traditionally what we've seen it being a completely, you know, separate tool and a separate function and a handoff driven thing. Right. Because we know every time there is a handoff there is huge inefficiencies which come into play. Right?

Absolutely. And I work for a large enterprise. This is a hard thing because what we do is we have our open source tools, we do outputs, we create some sort of fancy parser that reads it, puts it into alm and then massages it. So you're able to get some sort of data management can work with. So it sounds like this, this takes care of a lot of that for you out of the box as well.

Totally. And the reason is we are, we've gone with the application model driven approach. So almost your application is live modeled into a visual kind of flow. We call it the universe. Right. And I know it's hard to visualize, but that kind of drives the entire quality life cycle, not just the automation assets, but also being able to intelligently drive your test plans and track them in terms of your coverage. Right. So it's not a script and a theoretical aspect to coverage, but it's almost like looking into your live application flows and figuring out, all right, what is the right coverage across this flow. Right. So really being able to link that back to your business process and your application flows.

Well, you just made me think of something. I don't know if your tool does this because it is model Based it sounds like or does some sort of modeling. Is it able to let, you know, I know you have a login screen and I know you have these three other screens you're gonna have to interact with, therefore you should write these types of tests or does it do anything like that?

Yeah, totally. I mean it does, it does path analysis so it can actually analyze paths and it can even kind of go to the level of having your data model. So if you think about it, a lot of these modeling tools, and I've kind of worked with them a lot in my HP days as well as, you know, when we were, you know, back in the days, there were a lot of these model based tools, but all of them unfortunately were all for manual testing and manual driven. Right. None of them had kind of automation baked into the trait and they had a beautiful concept of application modeling, but they only were one dimension in terms of being able to handle the UI aspect of the modeling. Right. But another critical aspect which we these days let go is kind of the data aspect. Right. If you think about it, our screens these days are limited, but the amount of data which the scenarios, which the screens can handle is where, kind of where the complexity lies. Right. So we've kind of taken a two dimensional approach, right. One is the modeling on the screen side and then other is modeling on the data side so that from both aspects you have path analysis, right? Both on your application flows as well as the data models. Where once you have a test automation script or a test automation flow, you will have multiple data driven aspects to that test scenario. And that's kind of where the data modeling efficiency kicks in, where it can marry these data models and, and come up with the number of permutation combinations you should really be testing a scenario with. So kind of our platform integrates both the models so that you kind of get more comprehensive coverage when you're driving your tests.

Okay, Gulji, before we go, is there one piece of actionable advice you can give someone to prove their test automation efforts and let us know the best way to find contact you or learn more about accelq.

Sure. You know.

I don't want to sell my product in that one piece of advice. All I would say is don't jump into selecting an automation tool or a technology.

Whether you may be evaluating newer avenues or whether you may be going into automation as a new profession, or whether you're looking to adopt more and more automation for your organization. Look at it from a perspective of which solution kind of brings in the most robustness in automation, which can give me obviously the speed to align with my continuous delivery. And then lastly, most importantly, don't just look at web automation in silo. Have a solution. These days the application architecture is complex enough to be selecting a solution which can handle your web to your file, interactions to your APIs to your database, all in simple flow without preferably having to hire an army of developers to accomplish that. So look at these core aspects before you kind of jump to select a solution or a framework or a tool, what have you. Yeah, you know you can find me on, you know, on LinkedIn as well as the easiest way would be just go on www.excelq.com, sign up for a trial, and that way you'll be able to try it firsthand in terms of a lot of things we talked about. I know until you get your hands dirty on it, we don't feel comfortable about anything. So have a go at it and you know, you kind of relate to a lot of what I talked about.

Thank you Kaljeet for your automation awesomeness. Links to everything of value we covered in this episode. Head on over to testtalks.com 249 and while you're there, make sure to click on that sign up for our free trial link under the Exclusive Sponsor section to learn all about Source Lab's awesome products and services. So that's it for this episode of Test Talks. I'm Joe. My mission is to help you succeed with creating Automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Talks podcast. Head on over to www.testtalks.com for full show notes, amazing blog articles, and other automation awesomeness.