Next Gen Enterprise Automation Testing with Parasar Saha

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with powerstar all about next gen enterprise automation testing. If you don't know, Parasar is an enterprise automation architect and QE transformation leader with the experience of driving large scale transformation programs in multiple organizations in various business domains like airlines, banking, pensions and high tech. He has 17 years of experience of working in all aspects of quality engineering and quality management across the IT industry. He is a technology enthusiast and a QA strategist and is passionate about raising the quality standards in companies digital transformation journeys. Really excited to have him on the show. I met him at the Applatitude's Future of Testing event and he has some really, really great insight around wanting at the enterprise level and beyond. So you don't want to miss this episode. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial. Visit testguild.com sourcelabs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Parasar, welcome to the Guild.

Thank you Joe for having me in the podcast. Happy Holidays, Joe.

Yeah, Happy holidays. We're about to enter the New Year so Parasah, I'd like to actually jump into that. It's going to be one of my first questions but before we do, is anything I missed in your bio that you want the Guild to know more about?

I think you did a great job of kind of like telling the bio and introducing me to. I want to add that I had a vast experience across the different industry and wearing different hats in kind of in the quality world which gives me that like you know, the holistic picture like from a 10,000ft to one fit how that strategy in terms of like quality transformation, quality engineering should happen in the industry. So that's definitely helps me to kind of drive that strategy from a360B perspective.

Awesome. So this is the strategy when I dive into before we do. Also first question that comes to mind is this is being recorded right before the new year 2021. So you have been in the automation field or in quality for a long time. What was different or was there anything different about 2020 compared to 2019 that you think is going to transform how we do quality even more so than what you thought you had maybe to begin with because of COVID The way people are working now. Has that changed how you see quality run through a whole organization?

I think that's a great question. Like Joe, to start with, you know, 2020, what a year it has been like the start of 2020, imagine what it's going to be. So one of the things which happened in the quality and it's basically like, you know, across the IT industry, right. That we are shifted to work more remotely. Right. And our operations kind of moved in a remote mode. Now if you look at it like, okay, in 2019, okay, before the COVID happened, you could have survived running your testing like using your on premise infrastructure. But as soon as kind of like, you know, the COVID hit, we were moved to a remote situation. Now the cloud has become a really a big part of our operations these days. You know, imagine like, you know, if you have to have cross browser done and how many machines a tester would have needed at this point and how could you have done that with a remote situation. Now adding to that complexity at the mobile world, which is even kind of like a bigger variation of devices and operating systems that you have to kind of run. So I think that's a. Cloud is becoming much more important these days in terms of in the testing operations, especially in 2020, actually proved that testing operations can happen seamlessly in the remote situation with the backup from a cloud infrastructure.

Absolutely. And I wonder how many companies were ready for that. You know, we talk about digital transformation all the time and now for some companies it's moving over to SO and mobile apps is the only way they were able to communicate with their customers. So did you see a rush towards the cloud and do you think now going forward it's going to be more the norm? And when you say cloud, do you mean like moving to cloud vendors to run our tests or is it to collaborate? What do you mean by cloud also.

When you ask that like did it accelerate? Yes, it's accelerated. Yes, it's absolutely accelerated. People kind of like realize that there is a need that you have to move it to cloud. Right. And with that on premise setup, it's just becoming too difficult or you cannot even survive in situation. For some companies they already transitioned it, but Covid actually made it the necessity. Right. So it is no longer a good a wish list to have it made it a kind of absolute need to have. Right. And it also one of the thing which were able to do is like Covid was able to prove that this works. It's not that okay that you just there it works. You know, I remember the days when at one point of time and way back, like 10 years back, because of a reason we were asked to work remotely and we didn't have the infrastructure and the network was not built to support that. At that point with the day one when we started operating in a kind of remote mode, we crushed the first day. It was the VPNs not working, that's not working. So, so many challenges coming up. But the COVID actually proved that yes, we are ready to adopt it and we are ready to kind of use it in the full flow. Right. So that's where I see that. Yes, it kind of like proved that cloud works. Now the other thing is that you mentioned, like okay, what do I mean by cloud where I want to expand, like you know one thing is infrastructure. Right. Okay. That where you kind of utilize the platforms of cloud, like whether it's across device or across browser. Now it can be, it can be a SaaS provider. Right. Where you kind of, they take care of the entire infrastructure and maintain that in their end. Right. But there can be other options as well. The PAAs, you know, I like to call PAAs as a and B and B of test infrastructure. Right. You know what happens is like being in the air wearing this hat for enterprise architecture. I always have to be conscious about the cost. Right. You know, that's where you know when you go to a SaaS provider, it's good to have and you always need them as your support in your tool stack. But also you need to build something which is a PAAS model which can support your non critical things and in a cost effective way. So this allows you to scale at a manageable cost.

Absolutely. And I don't know when you wrote this presentation you gave, but you had actually a section called cloudify qa. So I don't know how long ago you wrote it, but it's almost like a blueprint for what teams need to do in order to move to the cloud. And like you said, it's not just running on infrastructure. You have other things like global test, data pool, cloud based storage test, data lakes, distributed micro test. So can you talk a little bit more about what your concept is about cloudify QA and how that applies to this current situation? We find our in self, find ourselves in.

Yes, absolutely. You know, I like to look at it from this perspective. Right. You know, we are moving to automation like okay. Which is actually moving your Manual workforce into a digital workforce. And you are creating a ton of digital assets, right? And your digital assets needs a digital home, a home that can scale on demand and scale down on demand and where you can actually pay as you go. So cloud is your kind of that home.

Okay.

Now different things will need cloud, right? Okay, so couple of things is like, yes, one thing we talked about like you know, the devices, the browsers and stuff, that definitely one area. Now one other way you can make your testing more accessible for different teams is by making them in a web based platform, right? So I mean gone are those days where you can use a desktop tool to do those things, right? More you can make those things more accessible to web based ways. It will allow you to kind of like, you know, make things accessible. Then not only your testing team, your dev teams can actually see those stuff, your business team can see those stuff, the whole scrum team can access those tool and use them effectively. Now where would you host those things? You want to host them into cloud so you can get a on demand, it's easy to deploy things and you don't need to break your banks to kind of set those up. The other thing is like, you know, in today's world, data is becoming so important, very important, right? Okay. AI lives on data. Now this data part will more and more become critical. Like whether you're talking about test data or your operational data that you are gathering over the time. Now more you can push it on a database. That would be way that you can actually a database and a data lake that where you can handle this data, you can store the data, you can analyze the data. Now you can maintain them in on premise. But with the cloud coming in, there are so many options where you can make your data stored in a more resilient way. So you really don't need to worry about like how much data you need from the data storage you need from day one. You can scale on demand and your data is very safe in cloud. So the testing operations can also leverage the capability of cloud, whether in terms of infrastructure, whether in terms of data storage, whether in terms of high availability, everything that they can leverage in terms of that.

So like you mentioned, we're going to have more and more data. So I guess as we look towards the future, since we are going the new year, you know, AI has been applied a lot to functional automation. But like I said, like you said also I think it applies better towards machine learning, towards data, especially with performance testing and things like that because you have so many logs that you can't as a manual test to go through. So do you see that as a growing area? Do you see like skill, like R, like a programming language that works with data being a skill testers need to know more about in an organization, at an enterprise level or going forward?

Absolutely, absolutely. Those are the area that will become a prime area of operations in testing as well. Now, R will definitely be one of the programming language. You know, I listened to your podcast and in one of your podcasts you talked about different tools which we going to be key tools for 2021 and Python was one of them. And I absolutely believe Python will be a great learning a programming language to learn in 2021 because Python has a great ability to handle data. So those are definitely going to be a new areas that people will find like growing areas in 2021. At the same time we also have to see how you can manage your data. Like when you are talking about big data like in your Hadoop, you cannot use the conventional way of comparing data and validating the data anymore because you are talking about terabyte of data and petrobyte of data. Are you going to bring two sources of data together and do the comparison one by one? You can't do that. You have to use the same technologies that the big data world is using. Whether it's a Hadoop, whether it's a Spark, whether it's Scala in memory testing, those things will become more critical over the time. Right. And you have to leverage those capabilities which the data engineering teams are using to kind of like do that kind of validation. So I think that will organically grow as the data will become more and more important and there also infrastructure will become very important and cloud will play a big role in that field as well.

Great. So as I mentioned, we first met at Applitools future of testing and I took some notes down. I don't know if I wrote this down correctly, but you said something really, really interesting that really struck me and I've been thinking about ever since. And I'm just paraphrasing, you said something like it's not about automation scripts anymore. Now back in the day it was about automating a script. You said it's now about automating your pipeline and I love that because I think that that is definitely the direction. I just couldn't put it into words like you put it. So can you talk a little bit about that? Do I have it right? And how does that fit into? I think you have like a 360 automation platform that you described to talk about this type of conc.

Absolutely. You know, Joe, like, I mean, after that conference, like, I got a lot of interest from a lot of people, like, about this 360. And I want to start from the bigger picture of it, like to come down to that 360 story. Right. You know, it has been changing, right? Okay. Things are becoming. Becoming much more complex now. It realized that it's not possible to kind of handle this complexity in a monolithic team. So they started bringing down the team size and working out things in small packages. So now you'd see those kind of, you know, small teams, the SCRUM teams, which are doing the stuff right now. In the SCRUM team, you also have a QA person. Okay, let's assume the person is John. Right? Okay. You know, John has to kind of do all these SCRUM activities and also figure out what he needs to do with the automation. Right. It's become a too much for the process now. How can we actually help John making him more efficient and accelerate at the same pace as the digital transformation is accelerating? Okay. That is where I think enterprise QA plays a big role. And where we need to see is how enterprise QA can help John with a framework which allows him to go fast in a standardized way. And the part of that framework a lot of time we think about, you know, the framework is only about technology. It is not. It touches three pillars of your IT world, which is people, process, and technology. And, you know, I like to call it in a very simple word, right. I mean, if you simplify the things is easier. People kind of like, understand and remember. And I think that the key here, when you're defining the strategy of the framework is coupling. Okay. What I mean by coupling is like, you know, bringing things closely together. Now when you touch that process thing, right, you have to couple the quality transformation, the quality engineering thing, closely with what's happening in the SCRUM team. A lot of people think, you know, they adopt BDD and they use jbehave, they use cucumber and spec flow, and they think, okay, I have adopted BDD in the best form. No, BDD is not for that. Right. Don north had BDD to make sure there's a close collaboration happening between testing and qa. Right. So that is what the true value of bdd. So how you want to couple these two together, the business developer and tester, that's a BDD is the best to do that. Now when you go to the dev, then you kind of think about you know, how can you build your tools, testing tools in the same ecosystem of development so they can use it. Right. You know why Cypress these days is becoming so popular? Because it's coexist in the developer ecosystem. So more you make them easy for them, they will adapt it. So that's where you kind of couple them together. So quality engineering can be closely coupled together and build that piece and it will become much more effective. So our guy John will be much more effective. Now then I want to touch the people part of it, right. You know, when you're driving the quality engineering in a large organization, the key of having the process started is to have a leader in that space who is equally expert in quality and engineering. The word is quality engineering. Now quality means a happy face of a customer. You know, I like to think about it, you know, when you go to a washroom and there are those like the happy faces, right?

Okay.

And the angry faces and the happy face is your quality. Like if your customer is happy, is happy now. And the customer is not only your end customer, your organization, your industry, your stakeholders, everybody is your customer. So you make them happy. And you, how do you achieve it is engineering, how do you bring engineering to achieve it? So that's how you touch that pillar of the people. Now the big part of that engineering is that 360 automation. Okay. And I like to call this way, right. You know, our 360 automation platform has three axis. X axis is different types of testing from web, mobile, accessibility, non functional. Every type of testing that you think about, like for an enterprise, it's very important to have coverage across the board. So that's your X axis that you need to build that coverage of different tools and then comes your Y axis, the y axis, your DevOps alignment. Right. You want to make sure that you have all this tool, but the tool can align to a process, a standard process, which are very interconnected. Okay. Now that interconnection, when you come to the testing part of the DevOps, you're looking at test, continuous testing. You want to make sure your test planning isn't connected. You want to make sure your code repository can be connected. You want to make sure your tools are connected, where you're writing the things and you're making sure your execution platforms, whether it's a SaaS or PAAS, whatever it is, you are very well connected in that platform and your reporting is connected so you can extract out the data and report it back in a homogeneous way for all the types of testing you are doing. So that's how you build a kind of full scale like continuous testing. And it need to connect to the dev on the one side and the release engineering on one side. So you can take the code the developer is doing, test it out and push it out to the production. That's your Y axis. Now the last part of it is the Z axis, your scalability. Right. You have all these types of testing, you have these DevOps, but every time it grows. So you need to handle the scalability factor. And there are two things in the scalability factor. One is your infrastructure, the other is your data decision. Okay. Now the cloud is the way to handle the scalability factor. But again cloud is one part. AI is the other part of handling scalability. Why is that? Our guy John used to only test previously with one browser and he has to make a decision whether this Test worked in one browser. Now he has to do in 15 browser or he may have to do in 15 devices and all this stuff. So he has to personally make those decision. But imagine if you have AI to assist there make those decision for you, how easy it's going to be for you to kind of do that scalability. Okay. Now the other piece of it is like, you know how many defects you have to kind of handle these days. Yes, you can bring up a large team, but at one point you will feel that the cost is too much. So that is where I think that AI can be a big help there. So with Overall all these three axes, you build a 360 automation platform which has the tool, which has your DevOps and your infrastructure and AI together serve you all the needs of your enterprise.

Absolutely. And I guess one of the points that came up at the conference as well was visual testing. But they introduced something called a visual grid which I think allows you to use AI and run against all these different types of devices and everything. Is that something your company uses? Is that like one of the tools that you can see helping people with this type of issue?

Yeah. So Visual Grid is something that we use from appitool. Right. So Appitool, I think they brought up a great option. Right. Okay. Not just going to the cross browser tools in the cloud, but you can also do it through appitool. So definitely that's one great options to kind of try out to the visual part in different browser, how the rendering is happening. Yes.

Awesome. Another thing that popped in my head is a term I've been hearing a lot about and I don't think you mentioned this at all was remote process Automation, RPA and you know, we talk a lot about the testers and developers getting involved in automation and things like that, but how about the business user? Do you see the business user becoming involved now with automation as well and data using tools like RPA to try to automate? Not necessarily, you know, you have testing types, but like maybe processes that they may do as well that becomes part of this 360 platform as well.

The whole concept is like, simplify the process. The whole concept of it is to kind of simplify the process so that any common person in the world can run your process. Imagine, okay, what banking industry was doing like 15, 20 years back, right? The complexity of opening a bank account was so difficult, Right. But what it did is like, okay, handle that complexity behind the scene. So a user goes and fill out something and get the account created. If you can make the testing simplified, right, okay. Where the business user can go to some website and say, okay, I want to run the test for this feature. The business user can themselves run the test if you can allow chatbot to be there integrated with your platform. And that's one of the thing. Integrate into the platform is like, you know, when the user goes and kind of do, okay, I need to run this test, okay, they can go in the chatbot and say, this is the test to run and they can get back a feedback and saying, okay, this is the results. Then at that point you break the silo of only the testing team running the test. So quality is no longer then a functional group. It's a quality, it's a culture. Everybody in your organization can run.

That's a cool user case. I didn't think of having a chatbot run a test for you with a command. That's cool idea. So I think you've nailed all these pieces, but I just want to make sure people understand you have something called a transformation maturity model that you touched on. But I want to make sure people know what this model is, because I think it's very helpful. The first one I think was conventional qa. A lot of people find themselves doing manual testing and then you have automated qa. So a lot of people think automated QA is the be all, end all. But you have two other steps in the maturity model. The third one being cloudify qa, which we talked about, and the last one, where we talked about as well, is predictive qa. Now is there anything along that maturity model that you'd like to add or expand on before we go to the next type of topic?

Yeah, I think a lot of People are kind of like thinking that Automate QA is the end goal. That is not right. Because as soon as you automate that, okay, you hit the problem of, like, you know, infrastructure, and the cost is something becomes so huge that you need to come up with, like, cloudify qa, then you cannot adopt AI to, like, a very large extent until you have a good amount of data. Right. You know, I work with AI teams as well. Right. Okay. The teams who are implementing AI in the organizations. Right. And I help them to kind of design the testing strategy for them. Right now, one of the key thing I learned working with the AI teams is, like, in the AI world, the data is the big boss. You know, it's a garbage in, garbage out. If you put a wrong data, you kind of like get a wrong output. Right? Now, that is why you cannot proceed to the AI level unless you have digitized your operation, unless you have enough digital data that you can fit to your AI model and give you an accurate result. And that is where you will develop the confidence. A lot of these tools have AI, but how much people are adapting, it will be determined how much data you have, how much is the efficacy of that AI Prediction is absolutely.

So this, for this model maturity model, do everyone have to go through each stage and stay at each stage for a certain amount of time? Or could someone go from 1 to 4 and skip 2 and 3? I guess you need 2 and 3 in place in order to achieve 4. So is there a way you roll this out in order to make sure that you're not rushing it and that you are learning feedback from your particular life cycle and that you're optimizing it for your company, not say what other people are doing.

So I think the model is like this, right? Okay. Your first is conventional qa. Right? Okay. And then you go to Automate qa where you have covered the different types of testing that you want to do through automation. But you will never hit the end point of Automate qa because every now and then, industry will come up and challenge you with a different channel of automation. Right? Okay. They will say, okay, now there's Iot, There may be something else tomorrow. Right? So whenever something comes up, you have to address that. So you're in that phase, will keep on continuing. But you can start building. Once you have a good Automate QA process, then you can start building that thing of cloudify qa because it depends on the demand, right? Okay. When you start creating it, right. When you start stepping into the next thing, when the cost is hitting you when you need to scale fast, right? When you have too much digital assets that you need to find a solution. So that is where you start your thing. So they sell overlapping thing and you start moving to the cloudify thing. Once you are cloudify thing, you start gathering the data, you form the backbone of the AI, you start capturing on your data and then you kind of move on to the predictive qa. But you never stopped your automate key way, even though you are in the predictive qa. Because when you are in the predictive qa, you may get a challenge that this channel is new and you have not covered that. Right. So it's an ongoing process. All those things will be ongoing and they're overlapping in phases.

Absolutely. And what I love about your presentation, you wrapped it up with value drivers. And I think as technologists, we like to geek out on technology. And I'm the biggest culprit of this. I love automation and tools. But at some point you have to ask ourselves, I guess, why? Why are we doing this? And a lot of people just get sucked into automating for the sake of automation. So what are these value drivers? And I think if people understand these value drivers, maybe it would make less automation because we'd stop focusing on things that actually matter, that need to be automated. Not sure if that makes sense, but can you expand on your value drivers that you talked about in your presentation?

You know, the value driver, I think like, you know, we can never take our eye off why we are doing it. Right. You know, as I mentioned, like quality engineering, one is like your customer, okay. And you always have to keep an empathy with the customer. Okay? What is your customer need? Your customer in an organization is your business teams. What is, end of the day, you are producing for the business team. What you are producing is like, you know, you have a digital transformation going on, right? And everybody wants to kind of revolutionize, accelerate it, right? With a better quality. What you gain is your digital product will have a higher confidence with your customer. Your higher confidence with customer will fuel a customer growth, which will bring you higher revenue for your organization. The next thing that you're going to do is like, you know, your operational cost will go down because if you are having less issues and if you are reducing the number of like, fixes and the bugs and stuff like that, and if you are producing things at a lower cost, you are actually producing things at a higher profit margin. So that's a kind of something which is going to make your finance guys happy.

Right.

And overall with this acceleration from the platform and the digital transformation and pushing things faster, you're going to establish your company as an industry leader because you are going to give the customer a faster turnaround on the features they are looking for with a higher confidence and a higher quality. And then what will establish your company as a market leader? That's the whole value of that thing is like quality engineering can actually give you a great value, a business value to the organizations.

Absolutely. And I also think this has another benefit. I guess people are self not self centered. But if your customers, I know a lot of groups complain they don't have a budget for certain tools or techniques. But if they valued on making their customers happy and the business says oh well look, we're making more money because our customer is happy, you'll have more money than to spend on the tools and the, and the technology that we all love to geek out on. I guess as well is a good point.

Yeah, you know, I learned to look at it, you know, like over the time. Every quality engineering organization has to transform from a state where the cost of quality is more than what they're is invested. Right. That is a starting point. The cost of quality is more. Then you will go to a neutral point. Right. Where you are producing as much value that people have invested in you. Then you are going to go into a revenue earning point. Right. Where you are allowing the team to accelerate more and giving things more and you are seeing a huge profit margin for your organization. This is the same way you develop a cost model for a startup. When a startup starts, okay, it's always on the loss. Then you kind of hit the breakeven point and then there's a spike coming in.

Right. And I guess I just thought of another issue though. How do you quantify quality? Once again, I work for a large enterprise. They only quantify. Oh, we're releasing feature. This feature is going to be used by X customer. We'll make X amount of money. But we always had a hard time quantifying that. We found 20 bugs before a reached production. How do we get credit for that? How do we measure the quality, cost or savings that we're actually giving the organization in order to give them confidence in investing more in quality? I don't know if that makes sense.

There are several metrics in the industry to measure quality. I think the best way to measure any metrics is to kind of look at a very single metrics and a holistic metrics. And the best way to do it is to kind of like ask your customer what is the confidence level of the features that you're rolling out. Right? You know, like I always follow this thing. You know, social media companies, when they're evaluated, they are evaluated. The valuation is based on only one thing and that is how many users they have. Right. When WhatsApp was brought over by Facebook, there was only one metric and that metric has a holistic metric. Yes, you can have 10 different metrics but all need to lead to one metrics and that is the confidence of your customer.

I love that. Good answer. Okay Parasar, before we go, is there one piece of actual advice you can give to someone to help them with their automation transformation testing efforts and what's the best way to find or contact you?

You know one thing, I would say this. If you want to be a leading organization in quality engineering and if you want to be a leader in the quality engineering, always follow the industry, the larger industry, right? What is it doing now that is going to be coming in the testing scope tomorrow and if you follow that trend, you will be always be ahead of the curve and you will not end up having technical debt. If you understand what is going on in the industry in terms of data infrastructure thing, that's exactly going to come into the testing scope for tomorrow. And if you build coverage for that for tomorrow, if you have already thought about, you're always going to be ahead of the game. And the best way to connect me is right now is going to be through LinkedIn right so and definitely you can find me on LinkedIn type with my name and I'll be there like and I'm I always follow any questions that sent to me and I regularly post stuff on LinkedIn as well and I would appreciate any feedback on the post and stuff.

Thanks again for your automation awesomeness. For links to everything of value we covered in this episode, head on over to testgill.com a336 and while you're there make sure to click on the Try it for free Today link under the exclusive Sponsor section to learn all about Sauce Labs awesome products and services. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full Stack Automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to Test Guild for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.