Virtualization in Test with Grigori Trofimov

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. And today we'll be talking with Gregari all about virtualization and testing, probably all other topics. Gregori is a solution architect at PowerSoft, which provides consulting services for PowerSoft's testing solutions software companies. So he's probably dealt with a lot of companies, a lot of testing situations, so I think you can get a lot of value from this conversation. He's also a speaker at testing and software development conferences on topics like service virtualization and deployment of disposable environments in the cloud, which I think is another hot topic we'll probably touch upon. And so he does a lot of enterprise engagements along the way. So I think you want to listen all the way to the end to get the most out of this episode. You don't want to miss it. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs, the cloud based test platform that helps ensure you can develop with confidence at every step from code to deployment for Every framework, browser, OS, mobile device and API. Get a free trial. Visit testguild.comSourceLabs and click on the exclusive sponsor section to try it for free today. Check it out.

Hey, Gregory. Welcome to the guild.

Hey, Joe. Thanks for having me.

Good to have you, I guess. Before we get into it though, is there anything I missed in your bio that you want the Guild to know more about?

No, I don't think so. I mean, this is probably a biggest exposure since like 6th grade science project, so.

Not much has changed since then, so.

Awesome. Yeah, good. Hopefully it'd be a better presentation than a sixth grade project, but we'll see.

Mine was pretty great, so.

Mine was awful. It was the volcano with the clay that people always did in the 70s.

Oh, you did that one, huh?

Yeah, yeah, yeah. Cool. All right. So I guess before we actually dive deep into this, I thought the first topic we'd get into is service virtualization, but I guess how would you describe what is service virtualization before we actually go into what you can do with it?

Great question. So service virtualization is kind of probably an extension of what traditionally is referred to as stubs and mocks and spies and things like that. Right. So if you're looking at testing in code, if we're looking at unit testing or maybe more end to end type test scenarios in code, we're looking at things like mocks. And we're looking at things like stubs to help us with our, with our test environments. When we extend out beyond that, when we look at larger systems or distributed systems, we start to see, you know, that we need more than just in code solutions. Right? We need more than just a mock or, or a stub or something like that. Right. We need to actually build out services that behave more or less like real services. And that's where service virtualization comes in. It enables organizations to define and develop complex, complex services and complex service behavior to stub out various components within a system to enable test automation, to enable development work, you know, sandboxing, et cetera. And we're actually also seeing the application extend out to operations side of the enterprise as well. So for sales organizations, right, they need environments for demoing. Typically they don't want to use production environments for things like that or for onboarding new employees. Let's say you're banking, you're hiring, you know, tell agents and things like that. You're not going to want to expose them to real data. You're not going to want to expose them to real records for them to, to do training. So anything that you can do to simulate various components or simulate data or simulate services within your system, you can leverage service virtualization for. And that's kind of becoming a really hot topic right now. And it's been for some time.

Yeah, absolutely. So is it a more advance stubbing and mocking technique?

Yeah, so. So, you know, when you look at stubs, you look at mocks, a lot of times they're kind of defined as, you know, some dummy piece of code that either lets you run a test, but, you know, you don't maybe don't care what happens to it, or maybe, you know, you do validate on, on whether or not it's being called correctly. Service virtualization, you know, you're kind of moving outside the code. Now you can leverage service virtualization in your code. However, that's not probably as lightweight as something like, I don't know, a, you know, a Mockito or something like that. Right. Or, or a power mock. You're looking at specifically service, right. Let's say you have an application that validates a VIN number or something like that. And in your test system, you can't always just keep calling that same service. You can't always call that whatever external service to validate your VIN number. You may or may not want to just virtualize that API call so that you always get a consistent response. Valid, you know, Confirming that this is in fact a valid number. At the end of the day, your test probably doesn't care whether is or it isn't. They just care about the overall end to end flow. So in that instance, you know, you could leverage service virtualization rather than putting all that validation logic in code.

Gotcha. So if your application relied on a lot of third party services that you probably have to pay for as well, does service virtualization help you stub that out as well?

Yeah. So in general service virtualization, we call these the ABCDs of service virtualization. There's kind of four primary obstacles that service virtualization aims to resolve. It's availability, more accessibility, but availability is probably a better term. You know, in your enterprise you may have teams that work on a different schedule that are developing services. Services just may not be available. They just might be brittle in that particular environment due to whatever deployment constraints. There's behavioral constraints. So that's the B, we got A, we've got B. That's when you're looking for a specific behavior from particular service. Could be a third party service, could be again an internal service, but in the context of a test environment, you just can't get it in the right state. And so service virtualization can aid with that cost is a big one with third party components because let's say you're, you know, you're integrating with some third party component, they charge you 2 cents a hit, or you know, less than 0.2 cents a hit or something like that, then all of a sudden you have to do performance testing in that environment. That cost is going to scale exponentially. Right. So that's a prohibitive. It's prohibitive in the context of performance testing or any kind of testing. If you're being charged per hit like that. And then finally there's data. So just having access to non production data, having access to test data that is reusable, that you can redeploy on demand, is another constraint for test automation. So service virtualization can address all four of those, whether you're standing up at endpoint just because you don't have one, or because you need a certain state or you need data, or you just want to reduce costs due to integrations with third party systems or third party applications.

Nice. So I used to hear a lot about service virtualization and not so much have technologies like containerization replaced some of the methods that you may have used service virtualization for? Are they completely different for different needs?

Well, so containerization I think if anything has become an enabler for service virtualization. Because one of the blockers I think for organizations in deploying service virtualization is unlike mocks, unlike stubs, they see it as like this big thing you have to set up, right, you have to provision hardware, you have to provision infrastructure, you have to, you know, set up some, you know, some application container or something like that to host your virtual services. And then once really got into Docker, OpenShift, you know, whatever else, right, that we have for containerization that's really kind of enabled organization to say, well, okay, we can just, you know, deploy and destroy these things. We can just deploy a container with virtual services and then we can destroy it after the pipeline has finished executing. And so if anything, I would say it's been an enabler. I don't think containerization replaces service virtualization because you still need to build out those services, you still need to build that logic into whatever component you're virtualizing. And it's, there's still a prohibitive cost in using the real systems rather than some virtual asset.

Nice. So there's a big push to shift left that's been going on for years now. So with service virtualization, can you actually use it when you're maybe in early stages of development and you're just creating more realistic things that when you're creating code to verify it's going to eventually work once you do get everything in place with a third party before you go into production?

Absolutely. Service virtualization is a big component of shift left. Because you know, when you think about shift left, you're thinking about the ability to run your high level tests, right? Your, your high level test automation as early as possible and as frequently as possible. And typically those high, high level tests, right, Like UI tests, end to end tests, you know, anything that's kind of at the top of that test pyramid. That's usually the most expensive type of test to run, right? The most expensive type of test. The most expensive type of test to run. Service virtualization basically says you can run these tests whenever you want, right? You don't have to wait for a specific moment, you don't have to wait for a sprint to finish, you know, and to have all the components within your system available in order to run these high level tests. So just by being able to provision virtual services, by being able to provision virtual components. And by the way, we're not just talking about, you know, HTTP services, right? We're talking about TCP services, FTP services, some, you know, custom protocol that some payment processing company uses that maybe you know, maybe not everybody actually has documentation for any of those services can be virtualized and deployed as early as, as you'd like in your dlc.

So great. So I guess another trend I've been hearing more about is resilience testing and chaos engineering. So can you use service virtualizations to kind of do experiments with, okay, the service is down or the services give me a funky requests back that I didn't expect to see how your code or system behaves to that?

Absolutely, yeah. That's a pretty common use case, right? Negative, negative behavior virtualization, failure virtualization. And one of the benefits of service virtualization is that teams who develop virtual services can give control to various other parts of the organization in terms of how these virtual services behave. For example, you know, traditionally, if we want to make a change to an application or test or we want to make a change to some component within our system under test, we typically have to work with a team of developers. We have to work with some infrastructure folks to get the right state, to get the right behavior out of it. All of a sudden, the test team now has control of the virtual service. So if they want to validate that their application behaves as expected due to failures or exceptions or lack of data, or not being able to process some data, any of those failure conditions that you might want to simulate, all of a sudden they have control of basically switching on that behavior, not having to really redeploy anything because service virtualization kind of accounts for that and make sure that these changes are quick and can be deployed quickly. So yeah, absolutely. Chaos engineering, any kind of failure virtualization and simulation is a good, good use case for service virtualization.

Great. Now this was probably a bad practice. I worked at a company that had like seven Sprint teams, but they all worked on different slices of the application. There was dependencies on them. So at the end of each Sprint team will go, I wasn't able to finish it because Team Z didn't have their code ready. So it seems almost like if you were doing that type of development, service virtualization could save the day because you don't have to worry about that team. You can actually maybe mock it up with a service virtualization type setup and be able to test and be on your way.

Yeah, absolutely. So, you know, I think at the end of the day, obviously you want some cooperation. You want Team Z, Team B, Team A. Right. You want everybody working together. At the end of the day, you.

Still have your production environment in an ideal world.

Right, Right, right. So, you know, we wouldn't recommend deploying service virtualization in a production environment, for example. Right. That's, you know, it's more for testing, it's more for development. It's more. Again, you know, there's certain operational use cases as well that you can leverage. But from a testing perspective, from a development perspective, one of the use cases that I think we're actually seeing a lot right now is specifically developer sandboxes and being able to sandbox a whole cluster really of services and logical behaviors from within a system in a single sandbox that can be quickly replicated, that can be redeployed in various, you know, corners of your organization. Right. Again, leveraging things like containerization and so on and so forth. So for your organization, that probably would have been. When was this, by the way? Was this a while back or.

Yeah, maybe a while back. Maybe not that long ago, four or five years ago. Like really dysfunctional team. I had a model, if you're going to do something bad, do something bad. Good. So just trying to find ways to test an insanity almost.

Yeah, well, you know, service virtualization, we've had service virtualization for actually kind of interesting. We've had service virtualization for probably like 15 years or something like that. And we initially had it built into our SOA testing solution, so.

Wow. Yep, yep.

Yeah. And back then we, I think they were even called just stubs or something like that. And it was built into the tool and then all of a sudden kind of saw the value of wow, this could really be a standalone thing. Like, you know, it doesn't really matter whether you're using soa, our SOA solution, or using, you know, some other test harness or some other way to validate your APIs or your services. You still may want to use service virtualization. Right. And so that's when we kind of pulled things out and, and found that to be a good standalone value.

Nice. So are there any other use cases you've seen? Obviously you've worked for a company that has a solution and you must see all kinds of things in the wild. Don't know if there's any other things that you think you could use service virtualization for that people don't even realize how useful it could be for, you know, Method X or something.

Well, I think again, I think when you start looking at service virtualization, you typically start looking at complex end to end flows. That's when you're really finding that, you know, you need something more, more broad in terms of test environment, something more stable and you need to cover multiple components. So when I think about use cases like that one use case that is probably one of my favorite use cases is for Internet Service provider. You know, they have technicians, right? Whenever you get a new modem or you sign up for Internet, somebody comes out, somebody installs the modem, somebody, you know, acknowled that, hey, here's your modem, you can sign right here, activates it, blah, blah, blah. That whole workflow takes about, I don't know, 30 minutes. They give you an hour of 9am to 9pm to do it, right? And then come over and do it. So on their side, on the Internet Service provider side, they obviously develop the app that these technicians use, right? And they develop the app, and in order to test that app or to do performance testing for that app, they need to be able to simulate 30,000, 100,000 technicians being able to log into that app, going to an address, installing a modem, syncing up with some system somewhere that validates the modem, activate it, then work with the customer account, enable the customer account, and then confirm through the customer account that all of that works, that you have Internet as expected. And then as a user, the technician themselves also have to sign out of that app, right? So if you think about all of the different services that this mobile app interacts with and also relies on actual human interaction in order to complete that end to end flow, there's a lot of stuff going on, right? So using service virtualization, they were able to simulate the technician, they were able to simulate the user, they were able to simulate the modem, they were able to simulate the warehouse that activates the modem, or, you know, whatever endpoint there that does the thing that it needs to do with that modem? And then they were able to create, you know, 30,000 accounts or 40,000 accounts or something like that, just because it's virtual data, right? So they don't have to rely on real geographical locations, they don't have to rely on real technicians or anything like that. And that obviously cut the cost down for their performance testing tremendously and enabled them to do it really anytime.

So how well does someone need to know their application or to create, I guess, a realistic source virtualization? I guess my question is, can you create like a bad service virtualization thinking you're testing something, but you didn't understand your infrastructure enough that your model of what you were trying to virtualize was wrong? Can you, like, have a sniffer on the interaction you're doing in the wild, and then the source virtualization can Kind of mimic that. How does it all work?

I guess that's a great question. So, short answer, can you create a bad virtual service? Yes.

You can do anything poorly.

But with service virtualization, and you know, not just our solution, but all solutions typically will have some convention or some capability to capture traffic, capture behavior, capture interactions between your application under test and its dependencies. This approach has a number of benefits. Among those benefits is that you collect data and you collect behavior, right? So when you have request response pairs, when you have APIs, you have services interacting with each other. They could be making calls back and forth in some kind of order, or there's a specific sequence operations that needs to occur in order for that flow to be valid. Now we have definition files, we have swagger open API documentation that can tell us how these APIs are supposed to interact with each other. But then in practice, that's not always the case. In practice, there might be some unique thing, maybe some undocumented resource or something like that that gets called and you might not be able to catch that. Right? So when you're developing your virtual services without having that insight, you might miss out on some of those things. But to that point though, I also say that you always have to go back to your test requirements. Traditionally, service virtualization is a tool for enabling test automation. So if your virtual services. Check that box. If your pipelines run successfully, if you're able to do your test automation with the that you have, you don't necessarily need to build out the most realistic representation of that virtual service. Sometimes you just need, you know, a 200 status code back or something like that along with, you know, some status message or maybe you're returning an error condition. So like a 403 with a specific, you know, message in the JSON. So you have to go back to your test requirements, you have to see what you really need from a virtual service and then make your life easier.

Great. So you mentioned this was really created to help support automation testing. So if you're running a continuous integration environment, CI cd, do the environments need to be stood up or automatically? Or do you have things in the pipeline before the test runs that you would it knows to start up service virtualization wise. 1, 2 and 3.

Well, a service virtualization solution should be flexible. It should enable you to do test automation that is reflective of, frankly, the maturity of your test automation practice. Right. So there's teams and a lot of teams right now. I mean, you see teams kind of having makeup of seven developers, two DevOps folks, and quality Engineer, right. So each team is going to end up having somewhat different pipelines, somewhat different test automation practices. A lot of times bigger enterprises probably will have sort of a convention on how to do that. But even with large enterprises, we see kind of different teams doing different things. So the more mature the practice, I think the more you're going to do in terms of dynamic deployments, right? You're going to see more on demand Docker containers that just will stand up an environment, reconfigured the environment right there and then, and execute tests for performance testing. You know, a lot of times we do performance testing where we see organizations do performance testing sort of in a manual way, right? Where there's a specific block of time that they have and they have to just run their performance test, then that's their opportunity to do it. And you know, they have to do it there. And then in that sense there's probably a more concrete infrastructure in place, right, to remove some of the dependencies on network hops and things like that, or just not have to rely on these kind of dynamic, dynamic flows. You just want the thing to be there, you want realistic response times and you want to make sure it doesn't go down for any reason. So long answer depends, right? But from a technology standpoint, it needs to be flexible. It needs to be able to support all those flows. It needs to support simple tests as well as, you know, more complex tests that require a stable environment.

So speaking about environments, continuous testing, I think there's another feature, another Solution. You have PowerSoft's continuous testing platform. Does service virtualization play with that at all or do they work hand in hand? What's that solution all about?

So, you know, when you think about service virtualization and kind of some of the topics we discussed, right, Dynamic deployments, you know, having infrastructure that's hosting service virtualization, generally speaking, you need something to orchestrate these things. You need something to be able to look, you know, look at the health of your virtual services. You need to be able to expose the ability to interact with these services or consume some of these services or even copy some of these services to the rest of the organization, right? There's some sort of governance model usually that comes in with enabling enterprises to leverage service virtualization. So the continuous testing platform is really a way for organizations or for teams to expose that to the rest organization or some center of excellence to maintain production grade virtual services. And we say production grade, we mean, you know, basically the virtual services that are good enough for test environments or so on and so forth and the continuous testing platform does that for you along with other, you know, administrative things. Right. Or monitoring capabilities. Right. If you're doing performance testing, you then maybe want to export that out and kind of show the rest of the team, okay, we're getting these types of response times here. This is where we're getting from the server, this is what we're getting from the application, so on and so forth. So it's that extra layer for enabling DevOps and continuous testing.

So you mentioned performance testing at least three times so far. And there's a bit I've seen a rise in non functional testing that Sprint teams have to deal with. One of them being performance testing. Have you seen an uptick in people that are really focused on performance now?

Yeah, absolutely. Although, you know, maybe for us, we've always kind of worked with a lot of performance testing use cases, you know, going back to sort of monolithic applications and legacy applications again. Performance testing used to be kind of this thing that you would prepare for, right? It's like a Christmas dinner, you know, or Thanksgiving dinner. Right. You have to go, you know, get all the things that, that you need, you have to set up and then the day of, you have to execute. And so performance testing traditionally was always like that. And so service virtualization was a big use case for that, you know, to kind of prepare everything, get everything in the right state. But we're seeing today is organizations really embracing performance test automation where they're not necessarily running, you know, tests that will break your application every single time as a part of their continuous pipeline, but they'll run tests to validate that all the SLAs are met, you know, at maybe lower concurrencies or look for degradation over time. Right. If you're running these tests nightly, or you're running your tests, you know, once code is checked in.

Right.

You know, sort of true cicd. You're always looking for degradation, you're looking for overtime, making sure that the changes that you're introducing to your application do not introduce unnecessary delays. So we're seeing a lot more of that. And obviously we're also seeing a lot more performance testing from a perspective of user experience validation. Right. So we would always traditionally see organizations kind of focus on the services, on the service layer, on the middle layer, on the back end systems, making sure that they can handle large loads and large concurrencies. But the big focus now is of course on the user experience, on making sure that the user page loads quickly, that the images load as expected in the right order and things like that.

Absolutely. We also touched on data a little bit service virtualization. Can it help you generate data for you? Because I know that's a lot of teams struggle with is actually having the data they need for all their automated tests.

Absolutely. I would say, you know, you kind of have to draw the line between sort of your big test data management solutions and enabling organizations to control their own test data. Right. And I think service virtualization is more on that latter side, Right. Where when you have virtual services and you have test tools, all of a sudden you can now create some middle ground. Right. Of that test data. And your virtual services, they don't need real data, they don't read production data, they don't need to be able to work with other systems. Right. When we think about this data, too, the big thing is data needs to be tied together for various services. You might have a service that's an account service, and that account service has some account id, and then this account has some bank account that has a bank account id. And then when you do a transaction, it has a transaction id. And all these data sets, they're all tied together somehow. And not only that, they're also complex. They're not just flat data sets where you have a key and a value, you have a key, an object list inside the list, you have other objects, you have other lists, and so on and so on and so forth. I mean, you think about an insurance company or something like that, you look at an insurance policy. How complex is an insurance policy, right. And all the different ways that you can have that insurance policy. So now all of a sudden, virtual services say, well, we're virtual, right. So we can simulate all of that, and we have capability to generate that on the fly, you know, with rules or without rules. You know, basically whatever the test team needs or whatever the stakeholders need, they can just give us the template for the format of data that they need, and we can generate it using virtual services.

Once I used to work, once again, for a big assurance company back in the day, and I used to have to create automated scripts just to get the policy in. Such a point where a minimal person go in and rate it against all the different things from Rhode island or mas and all the different rules that were into play. This sounds perfect. That would have been perfect to use it sounds like for that scenario, because that was crazy. Yeah. Cool. Cool. All right, Grigori, is there anything else about service virtualization, especially powersoft? I guess folks that don't know you do work for powersoft. Powersoft has a service virtualization solution. Someone's looking for a solution for service virtualization. What should they look for? Does powersoft have any secret sauce around this that you think kind of would lend itself better to certain organizations than others or anything like that?

I think generally speaking, when you're, when you're working with vendors or you're bringing in a technology into your organization, you're always kind of looking at a gradient of solutions. On one end you have open source, on the other end you have the big vendor. Right? And there's obviously benefits and sort of negatives about each side of that gradient. Right. On the open source side, it's free, there's a community that supports it, it's probably fairly easy to deploy or maybe developer friendly, right. Which is great. But now all of a sudden you bring that into your enterprise and who's maintaining it, who owns it, Right. Who's going to make sure that, you know, bugs are fixed? Right. I mean, you're just going to go to GitHub every time and, you know, make a, you know, commit something, you're going to fix it yourself. Right? So organizations do need some sort of vendor support. And not only that, there also needs to be a focus and an interest, I think, from the vendor side to stay on top of that technology. Right. And not have been in that technology. On the other side of that gradient is you have, you know, too large of a vendor. Right. And that's when you're looking at costs that become prohibitive. Right. You know, you think, okay, well, removing the costs associated with hitting these third party dependencies. But this is costing us a lot of money. And if we want to increase our concurrency, we have to now pay even more money. And all of a sudden we're in the same situation that we were before. So I think Parasoft is unique in that sense that, you know, we're privately held organization, right. We don't have a, you know, board of directors or anything like that, you know, tell us how to, how to develop our technology. Our customers, our partners, our roadmap is tailored against our customers and their requirements. So we have a very good, you know, sense of what the industry needs. And at the same time you still get that, that vendor support that, you know, what, 24, seven, you know, support or, you know, chat or so on and so forth. So what, you know, if something comes up or log 4J happens, right? You have, you have an organization that, that will fix that and address that immediately.

Absolutely. Okay, Gregory, before we go, is there one piece of actionable advice you can give to someone to help them with their service virtualization, automation testing efforts and what's the best way to find contact you or learn more about PowerSoft?

Absolutely. So Parasoft.com is your one stop shop for everything Parasoft. But specifically I think what you might be interested in if we hear on this podcast interesting is the Community Edition. So Parasoft does offer a free version virtualize. It's called the Community Edition. It's extremely powerful, I would say, in terms of being able to develop services and being able to develop frankly pretty complex services just by using the free edition. So head out to Parasoft.com, check out the Community Edition, check out our white papers, check out our testimonials from customers, customers who've deployed service virtualization and gotten a lot of value from it. And I think theresoft.com you can request a demo. You can request us to contact you and talk about your specific use cases.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a394 and while you're there, make sure to click on the Try for Free Today link under the exclusive Sponsor section to learn all about Sauce Lab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? And reviews really help in the rankings of the show. And I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue.