Tackling Enterprise Testing Challenges with AI with Srirang Srikantha

In a land of testers far and widely journey seeking answers, seeking skills, seeking a better way O the Test Illumination Testing Podcast Guiding testers with our nation awesomeness from ancient realms to modern days, they lead the way. Oh, the Testicle Ovation Testing Podcast.

Hey. Today we'll be diving into what I think is some of the biggest challenges facing test automation teams in enterprise environments, and that's scalability, maintainability and the ever changing complexity of modern applications. So for ways of how to solve these big three, we have Srirang joining us from Tengen, a platform that's redefining enterprise test automation with AI driven intelligence. If you don't know, Srirang has over 25 years of experience in delivering business growth, technology innovation, product strategy and development with a focus on banking, financial services and the insurance industry. So he really knows the problems that enterprise testers have to face day to day. He's also an acknowledged industry leader in addressing inefficiencies and quality engineering and promoting application reliability as a key driver for business success. Before we get into it, this episode of the Automation Podcast is sponsored by the awesome folks at Tenjin. Tenjin is the all in one test management and codeless automation platform that simplifies software testing with advanced AI capabilities, helping teams to plan, design and execute tests with ease. Why do testers love Tenjin? Well, first they have an integrated test management solution seamlessly manage test cases, requirements and defects in a single unified platform. They also have AI assisted test Design which uses natural language in pre existing documentation to quickly create tests with AI powered automation. They also give you access to tracking and reporting to gain full visibility into your manual and automated test execution with detailed tracking and analytics. It also comes with ML Driven Execution Optimization which helps you to optimize parallel test execution based on data and resource constraints to save time and maximize efficiency. And you can use them to help you automate testing across multiple platform so you can easily test the web, mobile, API and databases all in one platform. So support the show and also discover for yourself how Tenjin can transform your testing strategy. Sign up today and experience enterprise grade test automation built for modern software teams using the link down below. Hey Sharang, welcome to the Guild.

Hi Joe, thanks for having me. It's a pleasure to finally talk to you.

Yeah, love having you. You know, when I started my career I was in a insurance company. Then I went to a medical company. So I have a really great appreciation for enterprise testing I think some people miss out on. So I guess the first question I have is and maybe I'm wrong Why do you think enterprise testing is so hard?

It is hard because, you know, enterprises and today are net buyers of technology. They are not digital innovators, but I would like to call them digital adopters. They typically implement commercial off the shelf software. They don't have access to the backend, they don't have access to the programming talent that development companies have dev shops have. And platforms in enterprises are large. They are fairly large in terms of size and complexity. And it is in the, you know, far end of the implementation cycle where you are actually implementing the platform, you're not building the platform and you have to work with the technology that the vendors give you. And all of this compounded with the fact that you have so many pieces of technology in the modern enterprise. Modern banks can have between 70 to 400 different software that runs the modern bank and even, you know, other industries. We are dealing with at least 10 to 15 different pieces of major software that enterprises have to put put together and stitched together to manage their business workflow. And building and maintaining automation across such a complex environment is always a challenge.

Yeah, I'm sure a lot of people listening definitely agree. And I also think the problem or the idea of test automation, a lot of these enterprises have told them we need to deliver faster, we need test automation. But it's almost like a catch 22 because they try to implement test automation and almost becomes a bottleneck because like you said, you need to jump over 10 to 15 applications. You end up with a lot of flaky tests, long execution times and like massive maintenance efforts. So how does this impact release cycles and overall software quality? How do you make it so automation then is no longer a bottleneck?

So when we started Tengen, that was one of the biggest aspects that we wanted to solve. How do you quickly incorporate changes in the application without having to spend too much time figuring out where it's going, what has changed, where has it changed? That was one of the first things we set to solve. We built a robotic discovery UI discovery process. We did a few things which we feel are very innovative. The first one is that we built a robotic UI discovery process. Second is we separated the test with the UI navigation, which I think is a very fundamental different approach that we took. So you could write your test cases, you could put together complex test cases that go across different parts of your application landscape, different parts of a single application. And this is not closely tied into how you do your screen discovery. So the other way to look at it is you could build screen navigation, you could Build application navigation within an enterprise without having to code all of that as one monolithic large script. You could do it modular. You could join all of these together at runtime. You know, you could have different work paths that you take depending on your business function, what you need to test out. And these two innovations have really helped us scale test automation for enterprise platforms.

Great. I know also that you've invested in AI technology. In the pre show, we talked about how you mentioned there's a lot of tools coming out with AI, but they really are enterprise grade, let's say, because they don't have the experience of companies like yourself that actually knows how to deal with certain things. So a lot of times it's just AI fluff. So what separates maybe your AI and your product to what maybe others might have?

Yeah, the world is really revolving around AI these days. And when we started off, you know, the typical use cases that we saw in the industry which says, can I read a jira, a script? Can I create selenium scripts out of AI? We felt that these use cases don't really apply to our customer segment. A large portion of our customer segment is financial services regulated industry. They're very cautious adopters of new technology. And given the fact that it is enterprise platforms, you're not typically dealing with your, you know, epics or script. You're dealing with a large business requirement document or a complex platform discovery document. So we wanted to put AI to work in those areas. So we looked at three things which we would impact. First is can we make test design simpler and not dependent on a subject matter expert? Second is how do we remove the problem of flaky tests? And what we set out to do is saying if there are changes in the ui, is that something that AI can help us self discover, self heal without having to go back and redo a lot of work. And third is start building correlation between defects in previous release cycles and seeing if we could correlate and predict and prioritize what tests you want to run in your subsequent release cycles. I actually have an interesting demo if you'd like to see that on test design.

Absolutely. And for the people that are listening in just only we'll have a link to the video as well, but we'll walk you through it as well. So you could follow along. For people that are just listening.

Sure. So this is a demo about. This is a video that talks through our test design process. And it starts off slightly differently where you would work with a large requirement document, which is, you know, word document which is 30, 40, 100, 200 pages long. It kind of lists out what do you want to configure, how do you want the platform to behave. And this document would typically look, you know, like this. It could be a user manual, it could be requirement document, which is a lot of text. It could be images, it could be workflow diagrams. And when you set that up, you can, you know, you can upload these documents to your, to our platform. And what it will quickly do is break this up into broad business areas in terms of what are the different sections within the platform. And here you see us generating. So you see it breaks up the document into 21 different processes. And if you see on the right, there are no scenarios or cases associated with that. So we'll choose one business area or a business process and ask it to generate scenarios. And this is at this stage, it goes back to the LLM, it shares the documentation, it breaks it up into scenarios, and you get one scenario you can like, you can like or ask it to improve the scenario. And you know, if you really like it, you can say, I like the scenario. Let's go ahead and build cases out of it. And as you see it built five cases, you can review it again. And if you like this, you could take it into your test repository so that building something like this would otherwise require a subject matter expert. It may take three weeks of time for you to go through such large documents, but with Tenjin today, you could munch through such documents within hours and have your test design and a very neat, you know, a mind map delivered to you. So this is on test design.

How do you know it doesn't hallucinate based on what you feed it as well? I know a lot of people worry about hallucination. I know you're training it on your internal documentation, but how do you know it's really doing, it's really understanding those documents? Does that make sense?

Yes, it does. So what we've been doing is, so we have a large services team which helps customers know we take on real world projects. This is not just a factory. This is not a build factory alone. We have a large service delivery team. We also have partners who are, who help us deliver our platform and deliver testing services associated with that. So for the past six months we've been doing real world trials. It's not just based on, you know, sample documents. It is taking it to customers, doing a trial. What we have found is that we cannot 100% rely on this. But what it really does is cuts down the time for initial delivery, we are able to turn around documents quicker. And our teams, the testing teams then move from people who are writing tests to people who are reviewing and enhancing tests. So they go from, you know, having to write everything grounds up to, to actually reviewing it. So it's a high level skill that they need to build and maintain. It also frees up their time in terms of, you know, turnaround to clients and so on. So that way we found it to be very helpful. It does bring down, you know, the team requirements. From our side, we have seen more than anything, we have seen reduction in turnaround time by about 20 to 25% in test design for us, which I think is a substantial benefit whichever way you see it.

Even if I have a really good test document, an enterprise environment like you mentioned, there's 10 to 15 things it could interact with. It was always hard to get an environment that was set up correctly. So I had to go to all these people in the organization, I knew their little piece to set up the application and make it so I had an environment where I could test in. Does this help at all with that? Does this pop up? Hey, this test does this, but you're going to need this and this configured to do it.

So right now it is based on documentation given to us. So if the documentation consists of the prerequisites or the environment requirements, then we would also be able to give you the prerequisites over there. But right now a lot of our focus is essentially around getting the test, the specific validations, the data entry and the validations correct. And we will continue to enhance this because, you know, this is from an AI perspective, we are seeing rapid changes. So when we launched this, it, you know, it was much less functional. But as the new models of ChatGPT and other platforms have come out, we have seen rapid improvements in the quality of output and what all we can ask it to do. And so we continue to invest and see how we could, you know, improve this whole experience.

So you mentioned a few times you could do this without having an expert, a testing expert or a subject matter expert. A lot of people I've been speaking with, testers are really concerned about AI replacing them and they might be hesitant to actually implement these tools. Are you seeing any pushback on like, hey, I'm not even using AI because of that, that claim, or are you not making that claim? Am I misunderstanding?

No, I don't think we will come to a stage where we are saying we're going to replace in the near future. What we are seeing is it just makes me more efficient. It helps me look at stuff which I might have missed. You know, humans are not really the best people to look at a hundred or two hundred page document and figure out one tiny detail. AI is really well suited for such tasks. And the idea is to make humans more efficient. It is, it's nowhere close to replacing as yet, thankfully, because what, as I said, we are only getting about 70, 80% accuracy right now and there is a lot of review work that we recommend and we undertake as a part of our testing design. So I don't see pushback coming in terms of saying we are going to replace people.

So another I think benefit of AI is it can help with things like automation, like healing, self healing and self discovery. Maybe you can give us a little taste of what Tenjin, how that handles that piece that a lot of people struggle with.

Yes, you know, if we. Tenjin is designed to work with business analysts. It is meant for business analysts with no programming experience. So when we built a no code test automation platform, one of the first things that we had to solve for was what happens if your application under test goes through a change. How does a tester incorporate those changes into the automation? And that was a large problem and that remains a large problem. There are different innovative ways to solve it. And here is an example of how we have solved for elementary discovery. So on screen you see us setting up Tenjin to start looking or discovering an application menu. So we have multiple ways of discovery. So when we say automated learning, we provide a node where it has to go and discover. It launches the application, it navigates to the menu specifically and it, you know, browses through the entire dom, it browses through the entire screen, discovering all the fields and pages and presents that back in a very neat and understandable way. And all the technical details are three screens in. But it's available for anyone who wants to work with that. But for most people you don't need to do that. It just is. It gives you a template which you can download and the template, you know, represents your, your ui and that's where you do your data management. This is. And if you have changes, all you have to do is rerun it and the template gets, you know, it adjusts to the new fields or changes in these sequence of fields. So this auto discovery process by Tenjin has helped us bring test automation to citizen tester. Someone who is not necessarily comfortable with programming, but someone who understands the business domain strongly and is able to write functional Test cases on an end user environment and test out those platforms. So I hope that is of interest to the users over there.

This has been the holy grail of testing for a while or the big promise of anyone can do automation, even the business users. So in the wild, how is this really working for you? Do you have people that aren't say, technical and not saying they're not technical, but say they're not automation experts? Are they able to use this and are they using it?

Yes. So we have multiple customers who have adopted this and you know, have been long standing customers for us for the last six, seven years who've been using the platform and the capabilities and the auto discovery, self healing and so on. AI is the new piece, but the auto healing piece, the self discovery piece has been up and running for about six, seven years and we've been pitching it predominantly into banking and financial services. But come 2025 we have launched this on a public SaaS as well as available as an enterprise platform for you and listeners of, you know, Test Guild can Access this on tenjinonline.com and that's the first place where you can go and experience the platform. And it's been very well received. We have had multiple customers who have been very happy with. And the great part is it's not just web. This is true for web, this is true for mobile. This is true mobile as an Android and Apple platforms. It is also available for APIs and recently we have also enabled it for doing database testing. So you could actually have, you could chain your test to go through multiple platforms within your environment, starting from a UI all the way to the database and everything else in between and you could retain context and report of it as a single test. And that I think is very powerful today.

Absolutely. And that's, I'll be honest, that's one of the things that resonates with me because that's why I never really got people won't like this. Why I really didn't get into Selenium is because I needed something that could go to a web, a mobile, do an API database all in one platform. It just made things a lot easier. So this sounds like this is something that for the enterprise that's like a perfect use case for your solution.

I hope so. You know, we've been, we've been working with a lot of customers and the thing is we still feel that we are at very early stages of the product development. Despite everything that we have built, we are looking at how do we enhance this, how do we make this, you know, better. Some of the cool things that we are launching in the next few months is auto scaling the ability to run this on a public cloud or on a private cloud and have it scale automatically based on the workload that you assign to it. It has very interesting capabilities around, you know, taking a large workload which has not been manually distributed and distributing it across available infrastructure so that we will launch in, you know, by the end of March. So these are things which we are doing on a, you know, there's a monthly release calendar that we have and you know, we continue to enhance it and build on it every month on month.

So if someone's using another solution for browser, like in the cloud, type of browser, kind of os, mobile things, can they run it against these browser farms if they wanted to or is this going to be all in house to tangent?

No, we have, currently we work with Browser Stack, we work with Source Labs, we work with Amazon Browser Farm, AWS Browser Farm and we are talking to a few others and you know, we will extend it to the other platforms as there is a customer requirement. So it just gives you access to everything from browsers to devices that these people, these providers host. And that I think is a very powerful benefit to clients who don't want to invest and who have to do a lot of compatibility validation. And we should be able to help with that.

Absolutely. Just in case the folks that missed it. You know, we talked about self discovery piece and you showed it. How does that help with test maintainability?

So self discovery, we built it because, you know, discovering the ui. Discovering the UI is extremely difficult. We have always relied on scripting or point and shoot discovery. Point and shoot discovery is very intuitive and it's great for the first run. When you're first building automation, all these techniques are very interesting, very intuitive. The challenge doesn't come in when you're first building your automation. Challenge happens when you have to maintain automation over cycles, periodic cycles. Every time your application undergoes a change, if the UI changes, how do you incorporate the change into your automation? And that is where self discovery helps. Because what it does is it takes away the burden, it takes away the effort, it takes away the, you know, the, the entire challenge of identifying all your work, work elements, all the elements on the ui, from a human to a bot. And we know that bots are more consistent, they are tireless. And what we are able to do is go back, navigate through the ui, follow the path and discover all new elements. We are also able to take calls in terms of if there is change in element locators or names given the precision with which AI is working or the we are able to take a few calls and replace element names or locators with if they have changed, we will replace that using the, you know, using discretionary powers within an AI, within an AI algorithm, we will use discretionary powers within an AI algorithm to replace elements. And that makes the whole process painless. It makes it friendly to someone who's not programming ready. So that's what we started out doing and I think we continue to invest in making that process simpler and easier for our users.

Absolutely. And I may have missed this or you may have mentioned it. I know a lot of times people use ERPs when they're doing in enterprise. So I assume you have like pre built accelerators maybe for some of these enterprise grade applications.

Yes, that is phenomenal piece of investment that we've been able to bring to market. We have been working with large enterprise platforms and we felt that a point and shoot traditional discovery process is very challenging and very difficult to maintain for such large applications. So for certain ERPs like SAP and backing platforms and CRM like Microsoft Dynamics and Salesforce, we have the ability to go and self discover screens. With little support from from the human tester, the bots are able to go and self discover screens and update themselves. So it just makes the whole process of starting off on a test automation journey that much simpler. So everything related to ui, navigation and nuances related to UI no longer have to be built grounds up by the tester. It is something that our accelerators are capable of bringing into test order bringing into Tenjin, depending on what accelerators you have, what platforms you have in your organization, we can bring, we can deliver such accelerators to you and that makes the whole job much simpler. So you get off to a running start.

All right, so we saw how you're currently implementing AI. Really cool stuff. I think it's right on spot, right on target for what I think enterprises are dealing with. I know it's hard to predict the future, but what do you see the future of AI, especially at the enterprise level? Where do you think we're heading or maybe what's on your roadmap that you think is going to help us?

So UI discovery is always something that our R and D teams are very keen to simplify. We've been trying to make the element discovery process simpler and I think all organizations are trying to do some way of UI discovery simpler. In my opinion, test automation requires three things to come Together, what am I testing? How do I discover the UI and what data is required to achieve, you know, to what is the data required? So we, we have done some amount of R and D and we put out our first platform for, for test design. We have our platform up and ready for UI discovery for a number of years. The next part for us is test data. And I think that is a very large problem which remains to be solved. It's very easy to solve for simple, you know, create functions, but update and update is a much tougher function to solve for from test data perspective. So what I mean is, if you have a customer record, if you have a transaction record and you want to work with an existing transaction record, it's much tougher than creating something from scratch. Test data is the last piece, which I think we have not yet solved. There are a lot of great players out there who are solving for piece parts of test data management within an automation structure. So for example, you have people who are looking at creating simple demographic data, you have people who are looking at obfuscating production data into tests and so on. But being able to reuse, you know, your test automation with live data remains a challenge because live data has different states, it moves it, it morphs and, and I think that is a challenge that we would have to take up in a, in a few months.

Yeah, that's a huge challenge. Everyone, I speak with everyone, especially on surveys we do every year. Biggest challenge is test data. So if you could solve that, you'd have a winning combo for sure. All right, before we go, is there one piece of actual advice you can give to someone to help them with their automation testing efforts and what's the best way to find or contact you?

So that's a tough one, Joe, in terms of. I'm not very good at giving advice.

But I love the challenge that test automation represents. It's a different way of thinking. You have to think in automation in terms of saying, how do I make test automation survive changes, underlying changes and transitions without having to come back and work on it? It helps to be lazy. It helps to be, you know, you don't want to go and fix it again and again. And that's what automation guys should look to achieve. Saying automation should run for multiple cycles without having to come back and touch it. And that requires a mindset change in terms of how you think of tests, how you think of test data. And that's the only piece of advice I think I would like to give. You could find us on Tenjin online T E N J I n tenjin online.com and how do they get their hands on Tenjin?

Do they need to go through a sales team? Do they need to give a credit card? Is there a free trial? Is there like how does that work?

We have. We have a free trial. I think we have a four week free trial. Anyone can sign up and you don't need a credit card to start it so that you just get it. Simple people should be able to just jump on and start. There are a lot of helpful videos and if you don't like it, please write to us and we will change those things.

Awesome. We're going to have a link for that down below. I definitely recommend everyone checks it out for themselves. You get four weeks to just do as you please and I'd love to know your comments as well in the comments down below. Thanks again for your automation awesomeness. The links to everything of value we covered in this episode head on over to testguild.com a536 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges. We're excited to Collaborate visit test guild.info to explore how we can create transformative experiences. Together. Let's push the boundaries of what we can achieve.

With lutes and lyres the bards began their song A tune of knowledge, A melody of code through the air it spread like wildfire through the land Guiding tester showing the secrets to behold.