Testing Cloud with Parasar Saha

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking again with Parasar, all about cloud testing, how to test cloud and how to accelerate your digital transformations. If his name sounds familiar, you're right. He was on episode 336. He was so good we had to have him back. So really excited to have him on the show. If you don't know, Par is the CEO of Digi4. He has 19 years of experience in quality engineering and extensive knowledge in many large scale QA operations. He's also instrumental in architecting a lot of next generation enterprise level automation platforms with a lot of proven success like at the enterprise level, which I love. And prior to taking the role as CEO of GG4, Parasar was the head of QA at Soti or Saudi. I'm probably saying it wrong. He'll tell me. He's also the head of the center of Excellence of Air Canada back in the day. And Parasar has a decade of experience of driving QA transformation programs in global organizations around the world. He's also an evangelist and a global speaker and as you'll be able to tell, he's very passionate about bringing innovation to QA and taking QA to the next level in the software industry. I'm really pumped up for this. You should be too. You don't want to miss this episode. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Source Labs, the cloud based test platform that helps ensure you can develop with confidence at every step from code to deployment. For every framework, browser, OS, mobile device and API. Get a free trial, visit test good guild.com forward/sourcelabs and click on the exclusive sponsor section to try it for free today. Check it out.

Hey Parasar. Welcome to the Guild.

Hey Joe, nice to talk to you after a long time.

It's been a while. So is anything I missed in your bio that you want the Guild to know more about?

No, I think you covered pretty much everything. I'm really excited to be here as the CEO of Digi4. That's a new journey I'm taking up. So excited to be here and share that with you guys.

So why that change? What's Digi4? Why did you make the move?

I have been in quality engineering for 19 years now. I was kind of leading teams then, a whole department. I thought it may Be an opportunity for me to kind of take my experience and bring that positive change in the industry itself. And I've been kind of like also kind of supported by several people across the globe. So that encouraged me to take this journey forward and make a disruptive like positive disruption in the world of quality engineering.

Very cool. So one thing I've been hearing more and more about is cloud testing, cloud native testing. So this is really a timely episode, I think. So. You know what are some problems though that you see in the enterprise at qa, with QA in the QA automation space. Like I said, you have a lot of experience. I just want to set it up for why you thought cloud was an important topic. So I assume it starts here.

Absolutely. You know, if you see software has taken a huge leap with cloud, everything is moving to cloud because of different reason. I'm going to break it down. One of the most important thing is the adoption. So when you actually start using tools and technologies in the cloud, there is already that is baked into the cloud. The integration is baked into the cloud and there is a seamless way that you can start using it without worrying about the integration piece. Whereas when you come to like testing world, you get these tools which are in isolation and if you know that, you know like none of these tools kind of like can live on their own. You have to integrate in different tools and frameworks in your ecosystem now instead of doing those reinventing those wheels every time. If we can take approach how the cloud has taken on the other side overall space of technology, that will kind of help the adoption move really fast. The other piece you see like we are now moving more and more to quality engineering and the automation is becoming the more lifeline of testing. Now automation is what is digital stuff, right? And you are doing digitally now that calls for like digital infrastructure. The demand of infrastructure is increasing. So cloud is becoming that core part which will support the scaling of the automation. Like this is like have the fastest car, which is your automation. You are building the best thing. But if you don't have the good road, you cannot still go the fastest way. So the cloud is that road that can help you to go fast, your automation go fast. So demand of automation. And the key thing is like the value of testing is the reporting. I mean you can run several things but you will make kind of decision based on your report and stuff. And a lot of time it happens. The tools are very much has their own reporting system. It becomes very difficult that you go to different places and kind of get the data from there also they are not linked, so you don't get the overall picture. So if you can have monitoring and reporting, a unified reporting platform which can get the data and do the stuff, okay, you make a much better decision. Data is your base of your decision. Can make a much more data oriented decision. You can scale like drill down, you can drill up based on what you want to do and everything is at your fingertip. Having a unified platform.

So reporting sounds like, I assume because you're using all these different services, that's why reporting becomes even more important. It's always been important, but. So I always talk to teams, always trying to create dashboards. Is there some sort of techniques or best practices you recommend when getting that critical reporting in place?

So you can have a dashboard, but the piece is like how the data comes from your tool to the dashboard. If you can have unified way of operation. So you don't have to kind of like build several dashboard for several types of tools. You think about like you are doing visual testing, you have one type of dashboard and then you go to your performance testing, you see another dashboard. Why don't we just kind of pull it all together and see on your one screen. And if you see that variety of testing is increasing every day. Like you have to do accessibility testing, you have to do page load testing, you have to do functional testing and several types. And for each tool and each reporting, you have to go different places. Makes your life more difficult. Instead you want to see that at one place and the true value is. You know what true value is when you have in one place, it's not only your testing team seeing that thing, your business user, your developers can start seeing the thing, they will see it faster, they will make adjustment to their kind of process faster. If they have to fix bug, they will do the things faster and you will be able to get more value out of it rather than just kind of going to multiple places to do the stuff.

Excellent advice. So how would you explain what is cloud testing or testing the cloud? How has it changed? I mean we used to have websites on a web server. Who cared where it was? What is this testing in the cloud? What does that mean?

So here I'm actually talking about a bigger concept. Like the concept is testing cloud. If you look at it, the industry has shifted a lot. Okay, they have a development cloud, you have that SDK like amplify and things where you can do the development and lot of things come inbuilt in it. You don't need to kind of like you know, build, you have DevOps Cloud, you have infrastructure cloud, data engineering cloud, okay, everything sit into those cloud providers, aws, Azure and gcp. But you don't really have testing cloud where all your tools and things can be integrated and you can get that on demand. That doesn't exist. So your funnel before testing is going really fast. But you are coming to a point where you are doing a very old conventional way and you are kind of stuck that you cannot push your code as fast as you want with quality. And you start compromising whether it's time or quality, which one you compromise with. You are juggling between that decision always.

So if you're working at enterprise, why does this cause an issue? It's because every team then is almost has like a closed off type of solution where if it was in the cloud, everything would be in one place. You could triage everything in one place. You can get insight from all the silos in one. Is that what you're talking about then?

Exactly, that's what we are talking about. Like your test management remains in the cloud, your execution remains in the cloud in the same cloud and then your code remains in the same cloud, your code repository and then your test forms remain in that. Right. So kind of like everything in one place so that you can get a much faster path and you don't need to recreate this in kind of like every team separately. The value of the cloud is that the integration is all there. You're not building it anymore and they have a seamless way of operation. All you do is kind of okay, I need this component, get this from there. And this will solve a big problem in the testing industry and being in enterprise, I have seen this a lot of times. You know what happens is you do feel that you have a need of a particular tool, but the cost of the tool sometimes that much that you want to shy away. I mean as a project you may not have the cost of like to buy it for a year, okay. Or retainer. But in cloud model what it gives you is like pay as you go. You can actually use it for a certain time when you need it. Once you don't need it, you don't pay for it. The other thing is because you are taking from one cloud providers. You know, I have been in enterprise, so I have to work with the procurement and the legal and things. So when you are doing one, one, one like you know those kind of the tooling. So what happens is your procurement team is going through several processes do it, it delays, you that stuff. Sometimes these kind of like are things that you think about before buying a tool. So being in cloud you just have one contract and you can have like everything comes under that. It becomes much easier for you to onboard use a thing, a tool for your as of needed purpose rather than kind of thinking about a long term process.

So are these all vendor tools, source labs as like an environment solution or is it more like a test project which is like a cloud based testing solution and then you have to pay for storage and everything or is your own solution or it doesn't matter.

I would see this is going to be kind of collection. This is not like when you go to database side of the cloud. You don't say that, okay, I need only DynamoDB. AWS doesn't only give DynamoDB AWS gives you option of MongoDB, DynamoDB and everything. So you have multiple options coming from the cloud providers itself. So here in this case, for example, like you know, you go to a cloud provider and you say like I want a test management and they would have collection. Okay, you want this test management tool, okay, use it for your purpose and that internally will have an integration. Your next thing maybe your test management to your code depository. And also in the code repository you say I want BitBucket or I want Git gitlabs. Okay, whichever you want. The integration is already built in. You can create any number of flows, but you will have choices to play around.

Gotcha. So I know in your presentation you mentioned four key pillars or things to look for, I guess with different solutions to see what you have in place. Could you talk a little bit about what commonly other pillars of a testing cloud solution?

Yeah, so this is not limited to four. I just want to focus on the four and then it can be much more. The first four focus area is automation because that's where the key thing is. Like you want to go into cloud because you have enough automation now. The next is you build the automation. Okay, but automation cannot run without test data. So you want to have a very comprehensive solution of test data into the cloud. Then nowadays test firms, which is like browser firms and device firms, are becoming so much important to run things in parallel. So you should have a solution of like test forms into that cloud. And finally the reporting, you should have like a very good solution of reporting which should be able to grab data from multiple sources and show it under one roof. So these are kind of like just four pillars to kind of focus primarily, but there can be much more added to this to build A testing cloud.

Nice. So obviously most people listen to this. Automation is very critical to them. So how does automation fit into test cloud approach?

Yes. You know, in the automation world we have come a long way and if you see how the things are changing, we are following how applications are changing. The applications are becoming much more multi channel these days. Your customer is coming to you in from multi channel. They may be coming from API, they coming mobile, they may be coming from IoT or browser. So you want to have a 360 automation which can actually cover different types of channel that you have and it need to talk to each other. Okay, if you create an API who cannot even kind of interact with your UI automation, that's not what you want to build. You want to build something which can talk to each other and flow that data back and forth. That's one dimension of that automation. You want to build a 360 automation. Now the second dimension you want to build is the whole life cycle of a testing. You want to make sure that you know, you have a test management phase connected with the kind of your test development phase and your test execution and reporting. If they don't talk to each other, the automation fails and you cannot run it on demand and somebody has to be sitting there and doing that thing and you will not be able to scale. So you want to have that whole life cycle of the testing happening through automation and well connected. And finally like you have to also think about these different mediums. Right? Okay, it may be different types of browser, it may be different types of mobile devices that need to be all supported by the automation they are building. There's a big part which is cicd and the CI CD doesn't only stop and creating pipeline, it is also containerization, like you know, Dockers, how you are doing those everything together like need to be a part of automation. And that's what you be building up a 360 automation from there.

So part of the 360, when you're talking about automation, is it just functional automation or should people also think about other types of automation?

That's a great question. Like you know, they should think about all type of automation. A functional automation, a visual testing automation, performance automation, accessibility is a big thing these days that need to be there. And all those things, wherever you see there is a possibility or overlap, like bring them under the same umbrella and bring them under the same workflow management.

Absolutely love that. And I guess you know, you also talk about not only do people have challenges with automation, but I've Seen people have a lot of challenges with test data when it's in the cloud. Is there any other challenges that come up with test data or can the cloud help you with test data for your test?

Yeah, test data is a big piece. We don't talk enough about test data but it, it's a pain. It's a big pain. And the pain becomes even more when you're dealing with automation because you have a on demand kind of like, you know, need for data. Your automation can only run fast and it will give you good results if your data is right. So I think if you think about this kind of model where you are running 360 automation, you need to think about test data in a way that it's a supply demand chain. So your automation system, like your360 automation should create the demand and say I need this amount of data of this type. Your test data should be able to kind of create that automated way which I usually call in my world TTA test data automation. So you should be able to generate that kind of the supply of the data to fit this thing. We have to have these two system connected at the same time. It need to be supporting multi channel of your business process. Because if you see in today's industry, test data is very specific for the business processes that you have in your system. So you cannot say I just would be able to create 1, 2, 3, 4 or a specific pattern of thing. You should be able to support different types of business process who generate the data, keep that data in our data pool and the data is getting used in the system. So that also need to be residing in the cloud so it can scale on its own and support the automation.

So I totally agree. But how do you get that data? How do you generate the data? I mean it sounds like test data is important, but most people struggle with like I don't even know how to generate it, like how do I. Especially against the cloud, against all the things now I need to support performance testing, security testing. How the heck do I create test data for all that?

Yeah, so I usually follow a pyramid model over there. As you know, there are different types of pyramid, like testing pyramid. I mean I see there is an opportunity to kind of think about test data as a pyramid. When you're developing test data, the thing that you need to think about is what is my fastest and the cheapest way to get the data. Because the purpose is not to test the system is to generate the data that can go in the testing. So the fastest these days, most of the time there can be exception to this is API. If you can generate the data through API, it's a tremendous kind of you can create. Then some organizations have ability to generate through like database scripts. And these APIs and database scripts are business specific. Like if I talk about a supply chain management, your supply chain management have some specific like processes which generate the data. So that's where you have to use it. So database API, if it doesn't work, go by the UI part of it. That's how you generate the data and go from there.

So a lot of times test data becomes an issue once people start running in parallel and scaling up. So I assume as we shift things left, maybe people should start thinking about, all right, here's a new test suite. What kind of test data? What should the test data look like? Before I started creating the test, maybe in a valid approach. I don't know. Any thoughts on that?

Yeah, I think that's important. Like when you are. I have seen this several times when you start doing things in parallel, things fail because of test data being shared. In automation world, this is one of the key thing you need to make independent test and more. You kind of like use things data across in multiple tests. You are building too much dependencies and you are building that complexity. If you make independent test you, you'll be able to run through that piece. And you know what, like test data is such a big field. I have a whole kind of thought area over there like I would love to kind of share that in some separate sessions.

Have to have you back for sure because it's a hot topic. Like you said, people don't seem to pay enough attention to. And I would think it would be.

There is modernization of tool required in that area. You know, you need to kind of have newer version of tools coming in that area.

So also we're talking about testing in the cloud, running in parallel. And then what happens when you start having to run against all the different browsers, all the different devices, IoT and all those things simultaneously. So how does a test farm and a device farm, do they meet together? Are they the same thing? Is it always a hybrid approach?

See, I think one of the things I'm really excited to see how the industry is adopting test farms. Like when I say test forms is device forms and browser forms how they're adopting it. I mean, couple of years back only few companies were using them. Now like, you know, pretty much every company uses the test form. But at the same time there are some challenges when you go to the Enterprise space especially most of this kind of organizations have yearly contracts and it's kind of not a pay per use model. So what happens is like there is a solution, but the cost of solution is so high and you have to go into a contractual obligation that you still shy away from using it to that extent that your need is. So I think there's a need how Cloud has been changing the pattern. Like how do we go to a pay per use model on that as much as possible. Now the other piece is like, how can you scale like unlimited for that moment of time when you need like I have 500 tests. Why do I need to only run in sequence of 20 tests? I can go up to 500 test running in one shot and give me the feedback. It will be doing them really faster. But you don't want to pay for that 500 for the whole year. How do you kind of like only pay for that moment of time that you use and scale. And scale down? Think about is you should be able to scale up and you should be able to scale down on demand. If you cannot scale down, you are wasting your money because you are not using the infrastructure on a continuous basis. The other key thing, which is more of a mindset change that I think that is in the end we have to kind of think about is simulation. There is so much devices is coming up, so much browsers and things. Ideally, can you run every test in all different types of browsers and kind of devices, that is the real devices. It's not possible to do that. And first of all, is it needed? We have to ask the question, is it needed? If you look at it like the industry, like, you know, some of the other industries like automobile and kind of semiconductor had adopted simulation to a very mature level. So in electronics industry like fab labs are very expensive. You don't kind of take a design and test it from the beginning from the fab lab. You go through simulation to certain extent and make sure it is working and then go to Fab lab to do the final pieces. So similarly we can test a significant amount of our device testing in simulation. Then certain things which cannot be then goes into that like real device again there is an opportunity to create a pyramid based like cost pyramid, which one should be tested where and then we can optimize the thing. And I think there is also a need that we need to give people also sometime like admin access to certain things, like level of access need to be kind of available on demand, which will help them because sometimes because of Some access level. You cannot do certain things and you are stuck. You cannot adopt these solutions.

Absolutely great advice. Especially at enterprise you usually have to go through gates to get approval for things. So that's definitely something to keep an eye on. So you know, besides challenges with the test farm kind of hybrid approach, are there any other testing challenges you see people face when they're trying to develop a testing cloud of their own?

The testing cloud, I think there's a next piece is that the reporting which is a very big piece. How do you develop a reporting which can take data from your multiple sources? And the reporting has couple of aspect of it. You know, you should be able to grab data. It's like a star schema. You should be able to grab data from different sources, able to generate multi level for executive in a like organization. How do you create something which is like high level, they can see right away the data. And for people who wants to dig deep, how they kind of go to the detail level eventually we want to kind of see how that data can be stored. This data is goldmine. How you can use this data to help you further down the line to make much more kind of like decisions based on your historical data. So I think reporting and the data storage is a key thing that should come. And being on the same cloud will give you the opportunity to kind of store that data and kind of use that data for the best purpose.

Nice. So if someone's listening to this, are there some game changers? You think people, once they grasp this, they're like oh wow, then I totally understand this. I need to get on the test cloud train here.

Yeah, I think there's a lot of game changing thing that this can actually bring about. Firstly, this can accelerate the transformation. Digital transformation is already happening in the larger space. How do you kind of accelerate the transformation of qa? This will bring about adoption model rather than building model. Right now people are more leaning towards okay, I need to build a lot of things in the organization. This can bring up the adoption. The other thing is like it will allow transformation to happen faster across the industry. It doesn't need to remit to high tech industry. Even every industry should be able to kind of like, you know, use the best of technology as a fastest pace. And that is what the value of cloud is. Doesn't need to remain with the top tech sectors to kind of adopt the best of technology. I think this will also allow the scaling of the infrastructure that is the need of the hour. How, how can I get infrastructure on demand, lower the cost of the operation like keyway will become much more cost like effective. Allow teams to kind of get faster feedbacks, allow them to shift left so which will again reduce the cost of quality. Cost of quality will be reduced at the same time piece of delivery will be increased. The finally I think like the the bigger value is you don't need to build all these backup, resiliency, storage, security separately. You can have all these packaged in the cloud itself and those takes care of it. And when you are in cloud you don't think about all those things. It's already built in.

Great advice. Okay Parasol, before we go though, is there one piece of actual advice you can give to someone to help them with their test cloud efforts? And what's the best way to find or contact or learn more about your new company?

I think what I would say is this is something that I want to say to the industry. The value to the industry is with test cloud which accelerate time to market at the same time with better quality which will result in higher revenue. So testing will become from a cost saving cost center to a revenue generating cost center. Tremendously helps to kind of bring talent, resource funding everything to the testing industry. That would be my message to the industry is like and that's the value of it. And the thing is, if you want to contact me my name parashad.shaha@the reddigfort.com and you can also always search me in the LinkedIn with my name. I write quite regularly if you're following my post. Okay so I really thank you for that and then you can always find me in the social media in the LinkedIn.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode head on over to test guild.com.

And while you're there make sure to click on the Try for Free Today link under the exclusive Sponsor section to learn all about Sauce Lab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack Automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.