Testsigma No code Open Source Test Automation with Rukmangada Kandyala

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast and in for a special treat today. Today we talk about Rukh Magnada, founder and CEO of Test Sigma. All about Test Sigma and their new no code open source automation platform that's meant for in DevOps. It's really seems like a solution that's going to help everyone, especially nowadays as we're trying to ship software faster and quicker. They've met a lot of needs I think a lot of people, a lot of teams are struggling with. So you don't want to miss this episode. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Source Labs, the cloud based test platform that helps ensure you can develop with confidence at every step from code to to deployment for Every framework, browser, OS, mobile device and API. Get a free trial. Visit testskill.comSource Labs and click on the exclusive sponsor section to try it for free today. Check it out.

Hey, Rukh Magunda, welcome to the show.

Thanks Joe. Thanks for having us here. Good to meet you.

Awesome. Same here. So is there anything I missed in your bio that you want the Guild to know more about?

I think it's good. Whatever you had, like whatever you talked about is good enough. Yeah, we can get started.

Cool, sure. So as I mentioned the pre show, I've been hearing a lot about Test Sigma the past month or so. I believe you released an open source project or you open source your tool and you also got a round of funding. So I thought we could dive into all those things to see why Test Sigma is so hot right now. So I guess the first part is this is an open source solution. Was it always open source? Because I've heard the name before of Test Sigma. So I don't know if it was a tool that you had that was paid and then you open source or is it now open source and then you have a paid part? How does that work?

Yeah, so open source is not like it's not there from the beginning. So we just released the open source version of whatever product that we have been building for last couple of years almost. So the main reason, there are a couple of reasons I think as we go along, like we'll discuss all those reasons why it's really important to be an open source platform. So all this we learned in our last two years journey, like in the Sigma. So then finally we realized that it is an open source dominated category and also it's a very different, it's very huge and complex problem that we alone cannot solve it. We need a collaborative approach to solve this. The most complex and unsolved problem so far that I have seen.

Right.

And though it's like almost 20 years problem, like it's been there for almost 20 years, right. Test automation is not new. Like we still see startups like us coming up in the space and trying to solve the same problem with the different approaches. I don't say, I mean I respect all the startups and everybody is trying to do their best to solve this problem and we are trying to do it in a little different. As I said, like we chose this open source because we wanted to build this with the community and also we strongly believe that we need a collaborative approach here.

Nice. So you actually, I think I may have first saw this on product. Huh. And you actually listed out the reasons of why you think where the industry is or where you think the deficiencies are in our industry and then why this solution addresses some of those. So I thought maybe we dive into those reasons and how the solution maybe maps to that. I guess the first one you said is our current test automation is broken. And so I just curious to know if you can expand on what you mean by this.

So as I said, I'll go back to the first comment. Right. And so why do we still build our own custom frameworks and every company is trying to build their own custom frameworks or let me start with this. Like the broadly there are two kinds of tools that we see, right? One is the custom built frameworks and the second one is the most simplified test automation tools that we have like us, right. And like the sigma. So like 80% or I would say like the more than a little over 80% of the companies still prefer to use custom or build their own stack from scratch, right? So if you see test automation is still an isolated solution, right? And it's broken and it's kind of isolated from the mainstream developer ecosystem. It takes a lot of effort and time and efforts to really put all these things together. Like for example, now every company is a heterogeneous environment, has heterogeneous environments like web, mobile, API, database and so on. If you have to build a framework with for all these tools and you'll have to bring in like hundred different tools and libraries together and then you stitch everything together and you put a test automation architect and it takes a lot of time and effort and if you see eventually it becomes a development project or a parallel development project where they'll have like the full set of team, right? Test automation, right? From architects to like the test automation engineers and project managers. Everyone will have it like name it, right? So and if you see after all that effort, right. And you really don't get anything out of it, I mean I know like there are many companies who don't like but it's a huge effort and huge effort and the ROI is not clear to anyone yet, right? And in this domain and if you see the World Quality Report, 63% of the companies says that they don't see ROI or they don't know how to really measure ROI from this, the custom built frameworks. So that's why I call it broken test stack. We are trying to simplify or replace this broken test stack with platform that works out of the box and also gives you all that flexibility or the complex to address all your complex test automation requirements.

Why do you think that is? You know automation has been around for a while. Is it just people like to build solutions and maybe they don't look for an all in one type of help or I'm just curious like because you've actually spoke to a lot of customers before you open source, you've actually spoke to a lot of customers. Is there any resistance to this, what seems like a common sense type of approach to automation?

So there is no resistance as such, right? And see everybody is trying to automate or deliver their applications faster and there is no resistance at all when it comes to the. And if they want, if they see value in any of the platforms that we are, that we have today, right? Why they still prefer the programming based, the custom built frameworks is only because they get that flexibility, right? And as I said automation requirements are completely like. You cannot list down all your automation requirements from day one, right? And it is a growing requirement. For example today you start with web and again in web like you will, you can only know certain like a set of requirements, right? And you cannot even have everything in listed down to begin with, right? And eventually it will grow and when it grows all this vendor lock tools you see that we have today, right? In even test Sigma when we were last year before open source, you are stuck there, right? And you really can't move forward and you'll have to rewrite everything using your own or you change the tool or you go back to your custom build stacks again. So one of the reasons why there is no wider adoption for the tools, the simplified test automation tools and why there is a problem still, like even though it's like 20, 25 years old, category or problem.

So I think you bring up a good point. I think maybe teams have a lot of time and money invested in a framework and they have a bunch of tests that are running, maybe not optimally, but they're running, they have a lot of investment in it. Do you see that's reason why maybe there's some resistance to moving over to another tool? Does Test Sigma help the open source version help you take an existing test and put it into Test Sigma?

Yeah. So that is also one of the reasons why we choose open source approach. We cannot replace any of the systems from day one. Right. And it is not possible. Like even though we are a simplified test automation platform, we cannot replace all that existing work, whatever they've done before. Right. And with the open source you can really dig deep into the code, that code base of Test Sigma and see how can you really integrate with your existing, existing frameworks or you can actually import all those tests into Test Sigma and do all that customizations, whatever you need to make it work together. So that is where even community can come in hand help these companies to understand like say as long as we know or as long as these tests are built using the most popular frameworks and there are always templates that we had already pre built to import or to integrate the existing test with Test Sigma platform and you continue using your existing test and then whenever you want to write a new test, you can, you can start using Test Sigma to write which is more simplified.

Nice. So besides saying the test automation is broken, you have some other points you wrote on.

I would like to just correct it a little bit. And it's not test automation, the test stack, the tech, the stack.

Good point. Yes, yes, yes.

Not trying to mislead listeners, sorry, don't want to get you in trouble because people are like what?

So good point.

So the test pack is broken, the.

Stack is definitely broken. So the test automation stack is broken. So your second point was we know from firsthand experience that the modern tech stack has many moving parts. And I think you touched on this a little bit and it makes it because it has so many moving parts, it's hard to scale and get things up and running. Can you talk a little bit about what you mean by that?

Yeah, so I think I quickly touched upon these things. I'll just try to explain a little more in detail. So when I see too many moving parts and let's say if you have web, mobile, like now every company has all these different interfaces, right? They have web APIs, database and in web again you there are like multiple complex cases, your two factor authentications, your geolocation based testing and many many such cases, right? So what happens, right, for each of these unique requirements you need to kind of add new components to your framework, right? So that's why I call like too many moving parts. And it's not just you add it and leave it and whenever there's an update to any of these Libra, you'll have to maintain all those and you'll have to replace and replace that with the latest versions and then you see the backward compatibility and if there's a new interface, like for example when you start building web application, let's say you start with AngularJS and tomorrow if you change that to react, you'll have to bring in some other library, right? So it is too complex to maintain. So that's why I say like it becomes a parallel development project literally. Right? And for every company, so that's why I say like it has too many moving parts for you to worry about it. What we do at Test Sigma is like we take care of all that. Like I mean the open source platform that I'm talking about is like has everything pre built and it can also automatically update all these things and whenever because it's a community bridge, right? So every company, they don't need to whoever is using it, if it is already available, they can just take the platform and replace to get the latest versions of any of these things or add more to it if needed from the community, community marketplace. So that's what I mean by having too many moving parts.

So you, you mentioned a marketplace, what is that all about?

Yeah, so the marketplace, I mean there are two ways to look at it, right? And when it comes to that extendability. So I just try to simplify it a little bit. So there are two kinds of tools, right? One is the simplified test automation tools. Like simplified means like the record and playback or AI driven automation testing tools where they are like little simple to use and there is like a programming based custom build stacks, custom built test tags that we talked about. So now coming back to the point, right? And the marketplace with the simplified automation testing tools you can automate or any company can automate 90% of their testing. So the 10% is very unique to any, every company like you cannot really generalize the 10%. And if a company, as a company, like as a test Sigma, if we have to build that 10% for every company, it's going to be huge. Right. And we cannot really build that and it's going to take forever. So what we are trying to do is like we are trying to build a marketplace for all these pointed solutions and you get the 90% of the test automation capabilities which are unique and common, sorry, which are common by default as part of the Test Sigma platform and those custom solutions to name few. Right. And like as I said, it could be your PDF automation capabilities or it could be your database testing capabilities too for the platform to add to the platform could be your IP geolocation based testing capabilities. Right. All these are called, I mean there's a marketplace with all these tools. You can just download and install these tools so that you get that functionality or capabilities to automate these complex scenarios. So that's what I, we call it Marketplace, the community driven marketplace where you either find solutions, if it is already present, you can take it and use it. If it is not there, you can build and push it to marketplace if you want to, or you can keep it with you. Like you can make it completely private and that's up to the customers.

So could it be like industry specific? Say someone's into insurance and maybe they have like a policy rating chunk of code that they could be used across insurance vertical. Is that something that would fit into the marketplace?

Yeah, I mean you're right. And it can also be like your integrations for example. Like let's say we have integrations with all these popular tools and, and if you want to integrate with any of the custom tools that you may have, your customers may have like they can build a custom integration also as part of, with the add ons that we have and we can, if the customers want to publish it in marketplace, they can make it public so that everyone can use it from there. So these add ons, as I said, like the most important thing here that I wanted to highlight is test automation has been simplified enough with all this AI driven tools and has done like amazingly well and they've simplified all this test automation to the maximum. Like it's, it's completely autonomous, right. The only problem that is left now that we are trying to solve is extendability, right. While maintaining the simplicity in test automation. So if you see the stack, there are five stages in test stack, right? Like you were designed like I say I see like this, right? And you were like if you see the test automation project and you have design, phase development, execution, analytics and reports and integration and these are the five stages where we need some Extendability to the platform, like whatever you get by default with any of the tools, for example, including Test Sigma is not good enough for all these enterprises or any enterprise to work with. So these modern applications are super complex. Workflows are super complex. No matter what we do as a company, any company for that matter, according to me, again cannot solve the problem 100%. So we'll have to leave the 10% which I talked about to the companies or the teams to customize and build. So what we did is like at all these five stages we had given enough flexibility to extend the platform by creating add ons. For example, if you don't like reports that Sigma provides by default, you can go ahead and build an add on and extend the reports and build the reports however you want. And we have integrations with all popular tools in five different categories. Bug reporting tools, CACD tools, test management tools, project management tools, test data test data management tools and communication tools. If there is anything that is not available like in any of these integrations or plugins that we have by default, the add on will help you. With the add on approach you can really go and extend the integrations with Test Sigma and build your own integration. And similarly it goes into all these five layers that I talked about in design layer, in development layer development. As I said, we have web, mobile, both Android and iOS and APIs database testing as part of the platform by default. Let's say if we don't have PDF as part of the platform by default you can build a PDF automation capabilities, you can build an add on or you can take it from the marketplace if it is already available. So that's how we maintain the flexibility and extendability for the platform.

How hard is it to create those add ons though? Is it like a webhook or an API? I know it's extensible, but how hard is to extend a solution?

It is like that's what.

Right.

And it is as simple as your record and playback approach. Right. And you just download a template and we have created a templates for all these five categories of add ons that you can that you would need. And within that template you'll just have to place your business logic and all done. So it's all about adding a business logic to a template. Pre built templates for all these five types of extensions that we create.

Great. So I think you covered how to make it so it handles all different kinds of situations. Another point you made that test automation stack, when it's difficult, there's a lot of Use cases. There's a lot of languages and app types. Test automation really can become overwhelming and complex and time consuming to the point where it's its own development project. So how is this different? How's Test Sigma different? Is it like a proprietary language that people need to know or is it like keyword driven? How does that work?

We are trying to keep it as simple as your manual testing approach. Like how you write your manual test cases. Right. And it is simple English based approach and we call it unified approach. So means like it's unified for everyone. There is no proprietary language. It's like how we speak. You just need to write tests in simple English or like it's like how testers or the quality analysts document their manual test cases or the test scenarios in any of the test management tools. Just have to follow little instructions here. And how can you. You just need to break actions into the finest the statements. But otherwise there is no proprietary language as such. It's just simple English based approach and we call it unified approach.

So is testing more acting like a wrapper on top of automation solutions like Selenium and Appium?

Yes. So we are not trying to, as I said like we are not trying to reinvent the whole wheel. We are trying to solve the most unsolved problem or the most complex problem that's been unsolved for many years and decades. So test automation capabilities has been like solved amazingly well by Appium, Selenium and few other frameworks and we are not touching that part and we are making use of that. And at the same time test automation has been simplified enough as well with all this, a modern AI based automation, testing tools along with including Test Sigma even that we are not worried about it and we are trying to bring everything together so that you can easily replace the broken test stack with the most comprehensive test automation platform.

I think I might have a title for this broken tech Stack, Test Sigma how to fix it. So that's a good point. I think you also mentioned in your product hunt in the comment that you know you're trying to solve a lot of the issues that come up with automation is we focus more on writing and maintaining scripts rather than talking about quality software as quickly as possible. So because you're using an English like syntax and what other features are using then that makes it easier for someone to create scripts so they don't have to focus on the writing and maintaining it and focus more on software quality.

So I'll just go back to the same point that I've talked about and as part of making it simple, right? And one of the first things to begin with is like you can write tests using simple English. That will also give you that all the advanced features like AI driven maintenance, like your Auto healing part and that comes with your reusability and that comes with your test data management and everything is pre built or inbuilt part of this platform which will make it more easier and very simple to write your automation scripts and a will take care of your maintenance as much as it can. And also it's like we all know, right? And Test Sigma recorder, I see this as an autopilot, the cars running in autopilot mode, right. And it will, you will have to pitch in wherever you see a complex scenarios and wherever you see ideal situations or ideal scenarios and AI will take care of it, the Test Sigma recorder will take care of that 100% and you can can simply relax and focus on your the core business that's building your product.

So you did mention AI auto healing. So is that part of the open source? And besides that, another issue I see is not necessarily the writing, it's the maintenance. Like you said, when something fails in CICD, if you have like 10 or 20 or 30 failures, it takes you days to see why it failed. So you need some sort of machine learning type of algorithm to help the triage. So I know you said you don't want to solve all that you can integrate with other tools like maybe report portal, but did you have anything built into Test Sigma to help with the triaging of failing tests as well?

Yes. So as of now it's not part of the open source, it is still an experimentation stage. And as soon as we are done with and once we have a very good solution that we can open source and we are going to add that to open source. And as you rightly mentioned, like auto healing is not the only area that we are experimenting or we are exploring. So there are like, like four different areas where we are trying to use AI and test Sigma. One of them is like as you said, like the analytics, right. And you don't need to really run all the tests. Like if there is a failure you could easily understand what other tests might fail, right. And eventually, right. And you don't need to wait for all the tests to fail and then say like okay, all these tests are failed, so what will happen? Right? And if you don't do that and if you don't stop at the first failure and you don't apply the intelligence to find the other possible Failures that like you will end up spending a lot of time in debugging. All the fail testers fail tests unnecessarily. Right. So yeah. So these are like multiple. We are. We are experimenting in multiple areas and we have. We are done with like auto healing has been there and it's quite popular now and that we are trying to add to open source very soon. And the. We call it affected test cases or suggestion engine, whatever. We talked about the analytics part. So that suggestion engine will come very soon into open source as well.

Nice. So obviously I assume it just easily plugs into your existing CICD systems, GitHub or Jenkins or whatever.

Yeah, it is built for that. So our open source project is built for that because this is also one of the reasons the developer ecosystem is open source dominated and it's most complex. You need a very open platform that can be easily integrated with all these tools and we have maintained that flexibility or extendability integrate with all these popular categories of tools.

So I know another common thing in DevOps is containerization. Docker Kubernetes, is there any features that Test Sigma helps with that?

As of now we are not focusing on any of these things. Our focus is to. I mean our vision is to create an operating system for quality. So that will eventually have everything under one roof. But that's our vision. As I said like this will also go back to the broken test stack. Right. And the test automation like performance load and all like I mean that's the vision like we call it operating system for quality. But for now like our focus is on functional testing and Kubernetes Docker and all. We do support the Test Sigma installations through all this Docker, Kubernetes and everything. All that orchestration is available with our open source project.

Nice. Once again, I don't mean to keep going back to product hump, but you had a nice early overview of what this was doing. You had four points of how your solution helps and I think we went over each of them. The one was first one is built to replace individual specialized tools in a complex test stack. I think we went over that. I don't know where you got this number. You can create test 5x faster web apps, native Android and iOS apps and APIs. Would you get that 5x faster? Never.

So you can. I mean you would get like it would be much more than that. Like it. So it will be like more than 5x. So we are talking about the most complex applications. So even with then you. You would see your 5x faster with the most complex applications. If the applications are simple enough it would be much more than that.

Nice. Your third point was external and customizer without complex frameworks or coding which we went over. That's a. It sounds like the market. The add on the templates makes it easy to extend. And the fourth one I don't know if we covered this is everyone in the production team are able to collaborate right within the test platform. So I assume this acts like a go to portal. The whole team can log in, see the status of everything. Is that what you're talking about in this point?

Exactly. So that's part of our unified approach, right? And as I said like it's not just you write the test cases and we are trying to bring. I would just like to highlight one thing right in the modern DevOps or the Agile practices, the processes quality has become everyone's responsibility, right. So everyone needs to see like how their applications are performing or working and what's the quality status of their applications. So we believe in that the roles are completely redefined in the modern practices. So we want to, we are trying to bring everyone on board, not necessarily everyone writing the test cases but at least they should be aware of what is happening with respect to the quality of their applications that they are building. So we have pretty much everything built for the different stakeholders. For example like as you said, like project managers, they need to see what's happening. Like they, they'll get a different dashboards and they'll have different reports to see and at the same time even product manager can automate their the UAT testing. Right. And when before products goes to live like they would also have something to write or test, right. Even that they can automate using. Using Test Sigma because it is simple English based approach. As long as they know the functionality of the applications. Right. And they can just simply go and write it. So that's part of our unified approach. So we are trying to bring everyone on board and sign up for like contribute to the quality of their applications.

So someone's worried like oh why should I try investing in another OpenStar solution? I don't know if they're going to be around. I know you just got funding for I think 4.6 million. Is that an indicator that you know you've pitched this to investors and investors throw a lot of value in it so you're not going to be like a fly by night operation that you plan on being around for the long haul.

So see Accel is a great partner to work with, we all know. So Right. And so this is what everyone liked. So it's been a problem for every company for so many years. Right. And for almost two decades now. And there are like many attempts in the space to be harnessed with you. Right. And including Test Sigma when we started with only cloud approach. So everyone liked this kind of open source approach and we are going to be there forever. I'm, I'm like I'm sure that we are building the most or the world's leading automation or test automation platform or open source test automation platform. So yeah, we are here for a very, very long journey. So and we have got the great partners to work with Accel and more to come. Yeah. So I would request everyone to give it a try and contribute to as I said, like it it's the most complex problem that we have seen so far according to me. That's why we see so many companies like coming up within space still trying to solve the same problem again and again. Right. And as I said like I would like to highlight again that it needs a collaborative approach. It needs a collaborative approach and everyone should come together and build this as a very good extendable platform. And that's what I would like to highlight.

Okay, Rukh Manganda, before we go, is there one piece of actual advice you can give to someone to help with their automation testing efforts? And what's the best way to find contact? You will learn more about Test Sigma.

The best practice is not very simple. Like you just keep your automation as simple as possible. And also like if you are not using Test Sigma, I'm saying like just keep it simple and extensible. Right. And that's most important thing. And the best part to learn like the best way to learn Test Sigma is just like everything is out there. You can dig deep and including our code base, you can just understand like how we had built the platform and we have very beautiful documentation built around the product and it'll help you understand the like very good tutorials available. You can just learn everything end to end on how to use Testimon, how you can actually use it for your real time projects.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a391 and while you're there, make sure to click on the Try for free today link under the exclusive sponsor section to learn all about Sauce Lab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles, and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.