Revolutionizing Test Automation: AI-Powered Innovations with Guljeet Nagpaul

Get ready to discover the most actionable end to end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Calantonio, host of the Test Guild Automation Podcast and my goal is to help you succeed with creating automation awesomeness.

Hey, it's Joe and welcome to another episode of the Test Guild Automation Podcast. And today we'll be speaking with Galji all about next gen AI evolution and test automation, some new features at Excel Q and a bunch of cool things about testing platforms and where the future is leading us. If you don't know, Gilg is the Chief of Products and Strategy and founding member at Excel Q. He is an accomplished leader with experience ranging from software engineering to leading platforms and marketing Strategy. He spent 10 years at one of my favorite companies, I always say this Mercury, where he was the North American head of ALM practices. During that time, ALM captured the highest market share and Mercury was acquired by hp. So you could tell he knows his stuff. Really decided to have him back. I think it's the third time he'll be joining us, so it's going to be fun catching up. You don't want to miss this episode. Check it out.

This episode of the Test Guild Automation Podcast is sponsored by the Test Guild. Test Guild offers amazing partnership plans that cater to your brand awareness, lead generation and thought leadership goals. Get your products and services in front of your ideal target audience. Our satisfied clients rave about the results they've seen from partnering with us, from boosted event attendance to impressive roi. Visit our website and let's talk about how Test Guild is could take your brand to the next level. Head on over to test gil.info and let's talk.

Hey Guilty. Welcome back to the show.

Hey Joe, how are you doing? Great to be back.

Awesome to have you. So last time we spoke, I mean officially in an interview is a few years ago actually and I think a lot has changed so I thought we'd dive into a few things before we do. I always ask is there anything in your bio that I missed that you want the Guild to know more about?

No. You were very generous. You know the only thing I would point out is my stint at all those other companies that helped me learn about our field and experience is now kind of XLQ over signs, all of those. So yeah, you know we are going pretty strong in eight plus years and leader in the market so. So yeah, as much as I always used to shine on my earlier career, XLQ is just taking over in terms of how we are leading the Market.

Absolutely. And I know there's a lot of great innovations going on at Excel Q that I think even recently that have been released. So I want to dive into that. I guess the first thing since last time we spoke, it's been a while is the rise of ChatGPT. AI seems to be accelerating. I know it's not true AI, but it seems to be really catching fire now and a lot of testing tools is trying to integrate with it. So just want to get your views on maybe generative AI, ChatGPT and how do you see it impacting testing and test automation tools?

Yeah, it's caught all of us, you know, the storm, especially from a layman standpoint, you know, ChatGPT kind of really caught the scene, you know, in people who've kind of just heard about AI off and on and really never got into experiencing what impact an end user or a layman can have in terms of that technology. So ChatGPT, I think more from anything else, I would say more than the generative model of revolution that it perceives to bring. I think more so it revolutionized an end user to understand what impact it can have on our lives. But if you really think about it, machine learning algorithms have been around for over a decade. We've leveraged it pretty much since the inception of XLQ from 2015, 2016 onwards, and that's been the normal trend. We started out with AI algorithms which were more traditional, which would do data set analysis, give you combinatorial analysis. Then we transformed to machine learning algorithms which would learn from data sets and then help you do predictions, right? Whether it be the neural network algorithms or the canon algorithms or sequencing algorithms, right. They helped us do prediction. So they would learn from data sets and they would help us do trend analysis predictions. They would help us with, you know, correcting our, for instance, in the automation world, you know, object interactions, right? Whether it be self healing or so on. So it was a natural transformation going from those machine learning algorithms to then leveraging that learning or deep learning to generative models. But if you really think about it, right, some of the generative models, which are not language models, which a layman experience have been around for a few years now, Like, I don't, you know, if you notice, one of the chief architects of Tesla Autopilot last year came out and said 70% of the code he writes is written by Git copilot, right? So, you know, that's an algorithm, a Codex algorithm, which is also a generative AI algorithm, but not a language model algorithm. So it has kind of existed in our technology world for a few years, but the end user never experienced its impact. Right. So generative models have various different forms. Right. The reason ChatGPT throws so much light is it's a language model which just converses with us. So it just makes our searching and our behaviors different. So we all got caught up in that storm. But in reality there will be different versions and different variations of GPT models which would be either VAN or GAN models or Codex like models or like Google's Lambda models. Right. Which would do different kind of generations, whether it be image generation, video generation or code generation, like a codecs model does from Git. So, you know, it's a natural evolution. But yes, it has definitely brought it in the forefront of how this is going to impact kind of, you know, the world in the coming decade.

Absolutely. So like you said, you started off Excel Q has been AI powered for years and years and years and I just think this brings it forward more. Do you see ChatGPT or the adoption of it impacting codeless or codeless solutions? Because now people can just get code generated for them. Or how do you see that when someone says, oh, I don't need col list anymore. I don't know if everyone ever says that, but. Or is it more like, oh, I, I can use codeless now because now I see how it's happening and it seems like it's going to save me a lot of time. I don't know if that made any sense.

Absolutely does. I think it is. You know, the generative models are becoming so mainstream that it, it is going to come into both codeless and code word. Right. So just like in testing world, in our world, how these two worlds still exist parallel and you know, decently, healthily, right? Where we have this whole code world of test automation and the codeless world pretty much existing in parallel and the generative AI and code generation models are going to become more and more mainstream. Even if you look at Codata, Git Copilot already has like 10 competitors which do a pretty decent job in generating code. So both worlds are going to adopt it and sooner than later because all these algorithms are getting more mainstream, they're getting more componentized. So companies and products which are coming around it don't have to reinvent the wheel by writing these models. So it is certainly going to come in. The question would be, you know, the efficiency of generation, whether it be in the codeless world or it in the code world. Because at the end of the day Even if you look at GPT models, it. It all comes down to how much context we can bring to that generation, right. Whether it's going to be how much noise it's going to create as opposed to how much efficiency it's going to give us in long term scalability. One thing's for sure, Joe, the basics of design can never go away. And that's the strength that we need to look at, whether we look at the codeless world or the coded world. Right. And that's kind of where we've always shined and that's why we really differ from the market, because we understand the reason open source got so adopted in test automation was it needed that core design sound approach. Because without that you can't scale up. You can patch code, whether you write it or you generate it, you can patch it as much as you want, but it's going to become garbage, it's going to become throwaway. And if it becomes throwaway, it doesn't matter that million lines of code was generated or someone wrote it. Right. So that context, whoever brings it, is going to come out on the top. Right. Whether it be in the coded world or the codeless world. I'm trying to be agnostic, just given the spirit of your question.

Absolutely. So, you know, so using a tool, certain tools, do you think it can help you with design better then? So if you start off with a solution that's in the cloud that can scale that, maybe you could train the algorithm on test data specifically to your solution, making it even more intelligent than just using ChatGPT, that's more generic. Are those the type of things that will help with design?

That's right. So you know, bringing in more predictability with learning application behaviors. Right. And then kind of helping the generation, which is in context to a sound design, which can then scale up as opposed to trying to generate a dump of code that will see a whole bunch of people, players do. Right. So that's kind of where the industry would evolve and you know, how players can kind of really differentiate themselves. Right. And you know, we are uniquely positioned in that space because that's the basics which has always set us apart. Right. In terms of how that design first approach or that don't repeat yourself approach. Right. Which basically is the key magic to automation. I mean, you can build a bridge without having a sound design, it's going to fall. Right. So that context of how you bring in the generative AI, whether it be in any part of the life cycle of automation. So if you look at the if you dissect the automation life cycle, you know, you have the design, you have the development, you have the test data generation, you have environment preparations and then you have the governance and you have the analysis and predictions so that you can really test smartly. At the end of the day, you know, what is our end goal with all of this is I want a most lean and mean test. I want to test the least amount, to get the most amount of coverage so that I can really produce a good quality product. Right. And you know, if that context is put across the life cycle in leveraging generative AI, Right. That's kind of what can differ the approach, right?

Absolutely. A lot of solutions I see out there focus mainly just on web testing. What I like about Excel Q is you have basically a full platform that tests all the things. I guess where you really shine is maybe ERP software testing like SAP and Oracle. So you already leverage AI to help with that type of testing, is that correct?

We do. You know, we've always leveraged AI, you know, more like I said on the machine learning front where, whether it be deep learning to, and natural language processing to bring codeless to more accessible work. You know, if you think about codeless, you know, we used to call it scriptless a decade ago and the reason it was scriptless was it would generate, you know, a recording dump of script which would be transparent to the test engineer. Right. But we all saw in the past, you know, 15 years with the onset of open source frameworks that that was not scalable because real world complexity needed that human touch. Right. So, you know, XLQ was always in the forefront of leveraging natural language processing and machine learning algorithms to be able to bring that human touch, but yet not bring the syntax based programming complexity and have code injections because a combination of codeless and code actually makes things worse. Most test engineers don't want to deal with that. Right. So we were always kind of ahead of the curve, so to say, in bringing those two worlds together. But I think where we are gearing towards would be more generative models which can be used kind of in a git like fashion. Right. Where test engineer can literally, you know, interact with the application but not just stop there for generating that codeless automation, but also being able to leverage generative AI to actually recommend tests, recommend test data sets. Because if you, I'm sure you've heard in the past so many years, test data is probably one of the things which slow down test automation the most. Right. So there are just so many areas that we are looking at, you know, and believe it or not, everything's just kind of around the corner in terms of, you know, how we plan to progress through this year.

Very cool. And I think this also points to a larger trend I've been seeing every year I release my trends for the new year. One of them was actually going from automation tools to automation platforms. So I don't know if you've seen the same thing. And if so, you know, what would you consider an automation tool versus a platform?

Yeah, that's a great question, Joe. The difference between a utility and I always like to call sometimes tools are nothing but glorified utilities. And they may be really good in doing what they do. If you look at a very specific area in our test automation, web has become very mainstream. But if you look at, for instance, database testing, ETL testing, or the way API testing started out, if you really think about it, those automation tools were mostly just utilities, right? You know, they would do a request response, verification and they would do it really well. And because that practice was starting out, whether it be testing databases and complex ETL procedures, or whether it be testing APIs, those utilities kind of made sense and it felt like they're solving a purpose. But as our applications and our architecture has gotten so componentized, these omnichannel interactions, whether it be microservices, whether it be non relational databases, whether it be different types of enterprise service bus, whether it be a user journey that starts on web to mobile, to going to a microservice that have become really integrated and these utility testings just no longer scales an end to end integrated automation behavior flow that needs to be validated, right? And the idea behind a tool and a platform is you can't just put together multiple utilities and call it a platform if these are individually testing your behaviors. Whether it be mobile or API or database or green screens are still well and active in most enterprises. There are a few dimensions to differentiating a tool and a platform. One is under the same hood organically being able to handle across technology stacks. But most importantly, you know, not something that just throws out a script at you. Whether it be codeless or coded, it doesn't matter because efficiency lies in managing that script, reusing it. Just like when we write code, right? If I go back to our code word, you know, you want your code to be as componentized, as modular, as object oriented as possible. No matter how fancy, you know, technologies we use, right? At the end of the day, the heart of object oriented programming is still what makes Our code, you know, reusable, easy to maintain and scale up to the application changes we need. Right. The same stands true for automation and that's been ignored. So that's the difference between a tool and a platform which something which can manage those assets, modularize them, version control them and then bring in that modularity and business relevance to it. So that's kind of where we've always built the platform towards. Right. And it's a journey. I'm not going to say Xcel Q is done with that journey, we've been long on that journey. But it takes time to evolve a tool and then add bread through it. That's the third dimension. Right. You can just live with non functional testing. I mean, you know how important accessibility testing is. Right. Shift left performance, which is catching those performance bugs sooner than better. Yes, I can stress the system and then test it out with my non functional testing, but wouldn't it be better if right in my Sprint tool I can find those bottlenecks so that I can improve the architecture of my application? So you know, that's the third dimension of a platform where it can add, but you know, the important thing is building it kind of all organically under the same hood. Right. And that's kind of where we've been in our journey on Nice.

And also I guess is it like the go to place for the whole team to get insights on all things testing related where before it almost seems like siloed with these different utility automation tools. Is it everything feeding in? Because it's feeding into this one place as well. Be able to test all the things in one place. You're able to actually bubble up insights that you probably, you probably could do, but you'd have to do a lot of coding and a lot of connecting different platforms together to get that information.

Yeah, very true. And that's kind of comes back to our earlier discussion where we say why are we doing all this? For doing it to do the most lean and mean test. And the way you can have it is if the governance is tied very closely to what you're testing. So to your point, can I go in and leverage AI to really slice and dice all the assets that I developed to tell me what I should be testing for this next print or for that next release. So bringing that. Right. And if you think traditionally test management has always been other quote unquote utility or a platform. But if you think about it, that doesn't make sense. If I have automation living in a separate world and it's governance living In a separate world, no matter how fancy I make the reports, it can't work well together. Right. So I won't say it's still an easy problem to solve because, you know, whether we like it or not, we live in an agile world where user stories need to be equally intertwined to the test assets and then, you know, you have the defect lifecycle that you're managing and then that whole thing needs to be part of a CI CD workflow. So there will still be a lot of interactions. But the idea at least the way we have progressed, at least fitting into this ecosystem, even though we would want Xcel Key to be that one stop shop, but the reality is the ecosystem of how much JIRA is leveraged or Jenkins is leveraged is to be able to really closely tie with these ecosystem tools, but at the same time for all things test become that one stop shop. So it's still an ongoing kind of evolution to fit into the ecosystem, but yet have a one stop shop. But that's kind of been the whole principle behind how the platform is differentiated itself.

That's a good point. I'm not going to pick on hp, but they tried to make everything closed off almost from open source and that's what they lost. Though I think it sounds like, yeah, you want one place, but you don't want to be. You want to be able to integrate anything that may come up in the software development lifecycle.

Right on. So, you know, a specific example there is we don't force Agile Scrum masters to get out of their jira. So we've tied up with Atlassian so closely and I'm taking a very specific example that if I'm a Scrum Master, I don't have to leave my dashboard screen. I should be able to see for my story within my Sprint how many tests were run, what failed. Is it an environment issue? I don't want to go to Excel queue because I'm a Scrum Master. I just want to live in my world. Right. So that's the key, like you said, I mean, and, and we've seen Microsoft evolving from the company. It was close to an open ecosystem. Right. So yeah, we care about that aspect in terms of community and how the community would adopt a tool at the end of the day. Right. As opposed to trying to kind of take over all the process areas.

Absolutely. So, you know, you said it's a journey. Even Excel Q is still going through a journey, but as you're adapting to all the new releases, all the new things that are coming Out. I think you released a pretty big version called Skybolt. So can you talk a little bit about Skybolt, how it maybe is set up to help you integrate with the AI and the new things that are coming up in the future?

Yeah, it's, you know, like I mentioned, the generative models have been around for a few years. We've been tracking it pretty closely. You know, the usage of codex like models or, you know, other generative models. Even long before kind of chatgpt language models came along. Right. So Skybolt was a step along that direction. In fact, it's been now almost a quarter, it has been released and well adopted. So even before we heard all of this about ChatGPT and it kind of positions us really strongly in this market. Right. You know, the new introduction in Skybolt was really the stepping stone towards leveraging all these different AI models. You know, we came up with more of a studio environment to designing test as opposed to a logic editor environment. Right. That's kind of where traditionally the industry has been, whether it be codeless or coded word. If you think about it, it's always been about let's generate scripts. Right. Whether it's a codeless script or a syntax based script. So we've moved towards more of a design studio environment which kind of, you know, emulates the underlying application behaviors and helps kind of create process flows instead of still looking at, you know, and getting caught up at the object level. So if you think about traditional automation, it's still fighting the battle of hey, how can I best interact with this object? Is J, is XPath the best way or the jQuery the best way or should I write my own DOM based interactions? So the automation traditional world. And we fought it also. We fought it for the first few years of our inception and how we evolved. Right. But with this new introduction, we are able to kind of almost become a layer above that so that those process flows almost make it transparent to the end user to more like how an end user is looking and interacting with the screen as opposed to trying to fight the automation technologies which interact with. Right. So we've been able to advance our platform to that level and remove all the flakiness which comes with our traditional automation. Because you know how flaky that traditional automation is. If a DOM changes or a developer changes the framework, everything just breaks. And this is more common, even more so in these days when applications have three, four releases a year. Right. And you mentioned ERP applications like Salesforce, Oracle, saps of the world. Right. Workday servicenow so with for that we also released something pretty cool called Live to be able to keep live release alignment and then also have behavior process flows which can be used in the design studio. So other than that, Skybolt also increased the breadth like I was talking about a different dimension of platform. We added virtualization, accessibility testing, shift left performance testing. So pretty excited to see our customers kind of adopting, you know, Xcel Q also for these non functional needs as we kind of, you know, expand on that front.

Love it. And like you said, you know, you're using AI pretty much since you started, so you're not playing catch up. And I think what reflects that also I think you pointed me to a Forrester report recently that just rates a bunch of tools in different areas for platforms. Nixel Q I think based what I looked at is I think it's in the number one spot or it did pretty well on each of these areas. So how did you anticipate like because you're not playing catch up, it seems like everything that you would look for for a platform you already have in place. So I'm just curious to know how that happened. You designed something from the beginning to be able to handle what is eventually going to be the norm down the road.

Yeah, absolutely. Like you said, we've been building towards it. You know, back in 2019, you know, when we were still kind of young, right. We had a relatively young company and back in 2019, you know, we were of course much younger. And when Forrester did its first analysis, actually in the history of any Forest or Gartner analysis, which have been done not just in testing across the board, no company actually straight up ended in the leader quadrant. They always have to climb up the challenger tool. Right. And that differentiation of platform to tool has always what has set us apart. Right. The industry, you know, without naming tools, had still been evolving around utilities, whether it be generating and throwing a web script at you or API script at you, or a mobile script at you. Right. While XLQ was always building towards that full stack design first approach, everything integrated into bringing that user experience while leveraging AI. Right. So we didn't look at AI as just making your job easy. We looked at AI as can it bring efficiency to the scalability of automation. So where we differentiated was a customer who's Looking to automate 10,000 automation assets across 100 application portfolio as opposed to just providing a utility to that one. Off to team who's just trying to do smoke test of 25 automation scenarios. Right. So that's kind of been the building blocks towards this. And then like I said, you know, there has multiple dimensions to it. Right. You know, just like working on an offline version of an Excel sheet versus an online version of Google Sheet. If you and I are on the same Google sheet while we work on the cell, a particular cell, it will be collaborative in real time. So a cloud platform doesn't mean just dumping my scripts in a cloud. It means real collaborative workspace which can have you and me share that while being able to have version controls, branching, merging. So bringing codeless not just to scripting, but bringing codeless to all these different layers. So that's kind of what has really differentiated a tool and a platform. And that reflects in not just Forrester, but you know, if you see kind of our customer base, that reflects in kind of, you know, the maturity of how our adoption is across our customer base. Love it.

Well, great catching up, Kilji. A lot of innovation there, I guess. Before we go though, is there one piece of actual advice you can give to someone to help them with their automation testing efforts and what's the best way to find contact you? I'll learn more about Excel Queue.

Yeah, always good chatting, Joe. It was a long break, so appreciate you having me back. Yeah, I mean, you know, I'd love to always share. Right, get your hands in there. Right. SELQ offers a completely no obligation free trial. You know, proof is in the pudding, right. No matter how much we can talk about it. Right. Just get in there. We've now started community initiatives where every quarter we do hackathons where you can actually participate, learn within a week and then you get a 36 hour window to basically go at it. Right. And we have different versions of hackathon, whatever area you like to be in. We have full stack hackathons where it goes across technology stacks. If you're just into API testing, we have API specific hackathons. We've gotten very matured in mobile testing hackathons. So yeah, get in there and try it out and compare it for yourself and you know, that's the best way to experience it.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode, head on over to testguild.com a446 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguild.com and if you're in the DevOps automation software testing space, or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the guild community, I'd love to hear from you. Head ON over to testguild.info and let's make it happen.