API Testing Kubernetes (Transparent Remoting Using OneTest) with Chris Haggan and Martin Lescuyer

Foreign.

Get ready to discover the most actionable end to end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Calantonio, host of the Test Guild Automation podcast and my goal is to help you succeed with creating automation awesomeness.

Hey, it's Joe and welcome to another episode of the Test Guild Automation podcast. Today we're going to be talking about a lot of things I see people struggling with is testing in a cloud and containerized world. It's very difficult and there's a lot of pain points, especially for testers that they face. One of them is that once the containers are deployed, some of the APIs might remain accessible, but some others, like the internal ones, are not reached anymore. So what do you do? How do you test in these types of situations? Especially when you start dealing with kubernetes, you may not be familiar with these different containers as well. Well, I have a special treat for you today. We'll be talking with Chris and Marin all about this as well as some functionality that can help you test take care of this problem that I know a lot of you are struggling with in a cloud first world, if you don't know. Chris is head of product DevOps at HCL Software, has over 20 years of experience. He's a tool evangelist and most recently a product manager. As the head of product management for DevOps at HCL, Chris guides a broad portfolio of solutions and continues to maintain expertise in the testing field. I love real world experience and people that have a lot of customers, so I think you're gonna get a lot of value out of this. And we have Martin joining us who's an HCL1 product manager who's been testing product developments for more than 20 years as well. Early on he was involved in embedding test, which I think is really cool, and change to performance testing product as a developer manager. So he has a lot of experience in all these different areas I think you're struggling with. And he's in charge of HCL1 test. You don't want to miss this episode. Check it out. Hey guys, welcome to the guild.

Hey, thanks Joe for having us. Good to be here.

Good to have you, Chris. Hey Martin. Good to have you. So before we jump into the meat of this, I thought we'd set it up a little bit. Maybe what you're seeing in the testing landscape because a lot of it has changed that I think is now causing issues that testers may not be aware of or they're just starting to learn more about, especially as more and more companies go cloud native. So maybe you could set the stage of maybe what some of those challenges are that you're starting to see that may be causing issues with testers or things they need to test.

Yeah, absolutely. I mean, it's one of those interesting areas, isn't it, where, you know, you kind of in the lead in, you said, you know, I've been working in this area for 20 years. And I started off, you know, talking very much to companies doing what at the time was called EAI Enterprise Application Integration. That became SOA Service Oriented Architectures. And now, you know, we're in REST APIs and you know, the world of swagger documents and open API and that sort of thing. So there's been huge amounts of change just in the underlying technologies and the world that applications are built on. From a tester's perspective, a lot of the core concepts are very similar, but the detail, the minutiae of how you go about solving like that day to day problem of, okay, I need to test this particular application and I know that I need to do like a data transfer type of, type of test. How does that work?

Whatever the new world might be, at the point in time that you're addressing these things, in the world of monoliths, it was very much fairly easy to get to the application to make configuration changes to support it for testing. But over time, as development processes have become easier, we see that testing has somewhat become a little harder, you know, depending. Right. Obviously there are things you can do, but it's an interesting question. A lot changes for sure.

And mine we used to do embedding testing. You have a lot of experience performance testing. How are you seeing the market change now? What are some, especially now that we're in 2023, what are some hot topics you think testers need to know more about or how to handle how to test?

Yeah, you're right. Especially the performance testing things is something that is, which is really critical on the market, as you all know. I think we all know this, this is probably something obvious, but, but still we're seeing some incredible troubles with our customers trying to achieve their goals and make sure that the application that they release are meeting their requirements in terms of quality and things like this. And so yes, we are seeing lots also of pressure, I would say, around services that we can bring to our customers, like machine learning, overall data, big data that performance testing is generating. All this kind of environment is changing very, very fast.

Absolutely. And you know, you keep hearing things like cloud, first world development World API first type of development now. And you mentioned, Chris, something about testing monoliths compared to microservices. Have you been seeing this change with your customers as well and things they might be struggling with when they have to move from a monolith testing to microservices?

Yeah, I think it's one of those areas where the change started off very slowly, but has ramped up dramatically in the last few years. Right. With the advent of technologies like Docker and Kubernetes and things like that, what was, okay, I'm going to deploy an application, a web app into a tomcat server and we'll worry about high availability and that sort of thing. Kind of an easy in quotes deployment problem. As you move those applications, as you modernize them, put them into containers and things like that, the way that they are addressed and addressable changes, you know, that causes problems from a test perspective. Right. It's no longer easy to, for example, just go and get a log from a file system. Right. You've got to go into the container, you've got to find the log file on the container and all that sort of stuff. Right. So it's just a little bit of extra friction. And as you abstract further and further away from like the actual application, the app server itself and the technology that's actually running your application itself, the more you need to sort of bring in other solutions that do things like, I'm going to stick with logs, you know, log file aggregation and things like that. Right. And suddenly you see a whole market for solutions that weren't really particularly necessary from a test perspective. Now suddenly they have a whole new relevance for testers because it's a convenient way of doing the things that you used to be able to do in a much more flexible and straightforward fashion.

Absolutely. So, Martin, you mentioned performance testing once again. I love performance testing. So how much testing or how much do testers need to know about containerization and Kubernetes and how does that change the way they may have perceived how performance was done in the past compared to now?

As we said in the introduction, one of the things with the Kubernetes application is mostly that, you know, the microservices that in standard Kubernetes applications are standing behind, you know, different layers like ingress and intermediate in the middle. I would say that might change the request that they received before being sent to others. And that's kind of pro problematic that we are trying to solve with one test. Right. And so obviously the performance testing measurement of Kubernetes Application is incredibly important because it goes with the reputation of the producer, it goes with the reputation of the company. That's really something which is really, really critical for our customers. So we have great capabilities obviously within our offering to help our customers achieving this, even inside the Kubernetes applications, as we say, with this new capability that we just delivered to the market.

So I want to dive a little bit more on this new feature before we do though. I'm not sure how many folks may be familiar with HCL1 test. Maybe Chris, you can just give us a high level view of what is HCL1 test and you know, not a pitch, but just like what it is, what it was designed to do and things like that.

I don't get a good pitch. Oh, that's a shame. So HCL1 test is obviously, as the name implies, it's our testing capability as part of our larger HCL DevOps portfolio. And we provide a capability that, you know, that covers all the sorts of things you might think about, right? So whether that's API testing, UI testing, performance testing, service virtualization, test, data fabrication, all that sort of stuff all parceled up, which beginning to sound a little bit like a pitch. But I think, you know what's interesting about that is we're finding more and more situations where customers are coming to us. And in the past it was, you know, you had dedicated UI testers and you had dedicated performance testers and you had dedicated API testers. And really that our view of the world is kind of embedded in the name, right? It's one test. It's about saying, okay, well actually more and more we're seeing customers come to us and say, look, we've just got a testing problem. And actually a lot of our problems go from the UI to the API and back again. How do we, how do we solve that? You know, it's one thing I can build a test in, I don't know, Selenium, for example. I can build another test in Postman, right. But they're too dispar solutions and then it becomes a problem of wiring them up in your DevOps pipelines and things like that. Not an insurmountable problem, don't misunderstand. But it's extra work, right? And so we're trying to say, okay, how can we make that simpler for people?

And I also know I do trends every year. One of the trends I have for 2023 is the rise of unified testing platforms. And after I released this, I saw something from Forrester, a report maybe four days Ago, I mentioned on my previous podcast and they talk about how the enterprise is talking about how they're leaning more towards these UN platforms because it needs to help with collaboration, communication, all these things. So it seems like you're really on point with that, Martin. Is that something you've seen as a change? Like, we both, we've all been in this industry for a while. When I started, it used to be enterprise tools. Then it went open source and then people cutting and pasting and like trying to get things to communicate with one another. Now it seems to be going more towards back to having a unified platform that handles all the communication and collaboration. Is that on. Is that something you've been seeing as well?

I do think this is one of the really cool things about OneTest is something we're trying to unify. The different approach of testing, as Chris said, we have different key capabilities from UI testing to performance testing to API testing, but all of these really operate together very well so that it's really a homogeneous offering, so that it's something really important and really appreciated by our customers. So we have different key capabilities for testing because we also have capabilities for the environment, which is used, for example, for stubbing external components and for generating data. And that's really nice feature that we have everything packaged into one test. You can test with one test. Really you can do a really broad set of testings, from mobile applications down to mainframes. Very wide range of testing that we can make with one test. And that really is an important point. I think obviously the unification of the platform is really something cool that we have.

Absolutely. I think a lot of times people that work at enterprise knows the complexity of going from one application to another, changing context. Do you have any examples of maybe one of your customers, maybe something they were struggling with, using a tool, and then they use a unified platform like one test, and they will automate a project that may have not been automatable, at least automatable easily using maybe other solutions.

Yeah, I can see, for example, some customers have been making some huge investments, for example, in open source, in JMeter or in whatever, you know, Selenium. And our approach we have with one test is not to say that you can stop doing this and you should do that with one test. You know, on the contrary, the approach that we have is to try to integrate the existing works and import all the existing works into our own offering.

Right.

The message we have is enhance open source. Rather than just compete or whatever things like this, just tell our customers, just stop using JMeter and just use one test performance, which is not something that we push. But on the contrary, we can integrate those existing tests and run them with one test and bring all the value that one test can bring.

Right.

With reborn capabilities and all that sort of things.

One of the mantras that we've got just holistically when it comes to open source in particular, but just in general is embrace, extend, enhance. If you're familiar with, you know, you've got a bit of IT history, you might remember that there's a play on somebody else's phraseology. But our goal here is really to focus on that enhance piece. Right. Is to say, okay, we get it right. You've spent time, which is money building and investing in some of these other solutions. Martin called out JMeter, for example. We encounter a lot of customers who are doing performance testing with JMeter. You don't want to throw that away because it's a lot of investment. So bring it to our platform, we'll run it for you and we'll add on to that depth of reporting metrics, bringing things like Prometheus integration and all that sort of stuff over the top of it to give you additional value on top of, on top of this. And then yes, look, we'd love for you to implement anything new in one test performance, clearly. But you know, we're not saying, you know, it's rip and replace.

I love that. So that's another trend. Vendors and open source are competing against each other. Now maybe it's more Open source has done its job and now being able to tie all these pieces together, having like a framework or a wrapper on top of it to help you do things while having to develop them yourself from scratch. Sounds like something hdl. One test would be one of those solutions.

Absolutely.

So, you know, we talked a little bit about kubernetes. We talked about in the intro how sometimes services aren't available. Back in the day we used to have everything in our environment on the bare metal and now it's, you know, it could be in aws, it could be all over the place. We may not even know some third party dependencies where we're consuming and to add to the complexity we have like kubernetes. So I believe you have a new feature that helps with some of these issues called transparent remoting. So what is the transparent remoting feature about?

So transparent remoting, as you say, is all about addressing a problem. We were seeing more and more from customer organizations. When you're deploying an application into Kubernetes, you have points of ingress and egress inside that application space. And very often you'll have public and private APIs that are managed by gateways and technologies like that. But what we started to see was situations where organizations had that idea of a public and private API, which is fine from a business perspective, but as a tester, I want to make sure that the private API stuff is working as well. Because you know what, that's where we're going to start to hook together our next piece of business value with another microservice and things like that. So how can I address that from outside the Kubernetes environment? And there were hacks and workarounds and ways that you could probably expose it for testing purposes in a particular environment, but it wasn't particularly clean. And so transparent remoting was really born out of that thinking of saying, okay, well, how do we help people solve that particular type of problem? How can we help them address a service which is not naturally exposed by Kubernetes for testing from an API test perspective? And so that was our thinking behind that, as we were sort of thinking, okay, here's a clear problem. How do we solve it? And by the way, we've got the same problem. You know, parts of our solution run inside Kubernetes. I was looking at an architecture diagram just, just yesterday, and somebody was asking me, you know, can we see what this software looks like as part of a proposal? And so I said to the development team, you know, give me an architecture diagram. It's like, oh, there's a lot of boxes and lines on here. How do we test all of this? And, you know, we're using our own technology here to be able to get in and test things like the results service, things like that that are, you know, important. You know, they have both that public view because obviously people need to be able to see results, but they have private interactions in how they interact with other microservices around execution and things like that. So it's an important technology not only for customers, but for ourselves as well.

Does the tool allow you to know, like, what services are being interacted with? I know a lot of times testers might not even know that it has a dependency on an internal thing until they try running a test and something doesn't work, and then they need to track it down. Does your solution do anything like what, sniffing or give you something like that?

Yes. So there's two aspects to that, right? First of all, we have a technology called Topology discovery, and I was actually talking about this just the other day in a security testing context, right. Was to be able to say, okay, well, we don't know what the APIs are like you say might be being interacted with. And you know what we need to focus on security testing on. So topology discovery is about saying, okay, what are all these things? But then the second part of that is what we call recording studio, which is where we can say, okay, well, now that I know what these API endpoints are, I'd like to see what the data is that is traveling between these services. Give me some more insight into, you know, what's the schema? Is it xml? Is it JSON? Is it some fancy format? You know, we do a lot of work with customers with mainframes, for example, so you'll still see quite a bit of EBCDEC encoded bits and bobs. So yeah, there's lots of that sort of sort of thing in the platform as well. Yeah, definitely.

All right, I guess the next thing is, you know, we mentioned transparent feature, what that's all about. A lot of times as a tester, someone may not have permissions to change anything in the cluster. Is that a blocker? How do you get around that?

No, so the way that the capability works, right, is we're providing effectively configuration information rather than security information and that sort of thing. So we can address these services without having to make authorization or authentication changes to the Kubernetes cluster itself directly. It does require you to be able to connect to the cluster directly. Because we're not addressing these APIs through like an API gateway or directly through the ingress egress. We're effectively going directly to the service. It's more oriented around service name resolution and POD resolution, things like that. So slightly different approach. There's kind of an implied assumption that you have access to the Kubernetes configuration. Yes, I'm sure there's a proper word for it and I'm blanking on it right now.

Just have access to the Kubernetes configuration file and then basically the tool does anything. Anything requires all the remaining things required.

Now, does this replace mocking or virtualization, or is this just another name for a source virtualization that's been around for a little bit?

It's a step on that path. Right, and what I mean by that is, you know, before you can build a virtual service, you need to be able to. There's two things you need to be able to do to build a virtual service, depending on where you're coming from, you either build it from the specification, the contract, and that sort of thing, or you can build it from reality. Right? You can build it from recorded events and the messaging, the message traffic, and that sort of thing. So this technology is aimed more at that latter stage of being able to say, okay, I want to be able to access and interact directly with that service so I can capture data and work with the service directly. A lot of times services, particularly microservices, will expose their own contract definitions. You know, the open API specifications, swagger documents, you know, whatever it is, and those will be discoverable from the microservice if you can get to it. And so this really allows us to get to those definitions and then we can bring those back in as well. So it kind of comes at it from both angles.

Now, I'm just thinking from a security perspective. Does this expose anything that maybe could be exploited by someone that shouldn't be messing around with these APIs? Is it totally not an issue?

Any tool can be malicious in the wrong hands, I think is how I respond to that. Yes. Being able to access a microservice directly in this fashion does imply that you could do data injection and other bits and pieces. We provide a testing solution. There is an assumption that you work for the organization, that you have some, you know, reason to be doing, to be doing this. Plus, you know, as I said before, this transparent remoting solution requires you to have certain access directly to the Kubernetes environment. Right. So anything could be. Could be used maliciously. This, you know, has potential for that, but it's. It's not what we're intending.

Yep, exactly.

Right.

So if a test was trying to implement the solution, how much do they need to know about Kubernetes? Like, what are the requirements of someone using the solution? Is it. Is it usually the developer that takes care of it, or is it really the tester that's interacting with Kubernetes and then creating these type of things to interact with them?

Martin, do you want to take that one?

Oh, it can be both, actually developers and tester, but I would think the main thing is, as a developer, the microservice writers is interested in verifying all the functions of this microservice and once deployed. Right. So I do believe that the first thing is, the first thing is a tool for the developers to make sure that the services that he built are functioning properly. And then those kind of tests can be perfectly well given to testers, and so they enhance those tests in either way. They want. But indeed I think the value of this is to be able to check the microservice APIs once deployed.

In a systematic way. And so I would say that the target of this is a developer of that microservice.

That's a very common handoff that we see. Right. Is an SDET or someone like that, someone very technical coming in and building kind of like the first version of something and then handing it off to somebody else who's going to take that and elaborate on it and put it into a more complete test solution.

So in my notes I have this. I'm not sure why I think we may have mentioned it in the pre show was synchronization. Can you go into detail a little bit? What is synchronization, how it applies to this feature?

So that was actually something I alluded to in the answer before this, I think where we can address a microservice that might expose, for example a Swagger document or an open API document in days gone past it would have been a WSDL and things like that. Right?

Yeah.

The way that the product works is we can ask for that document, introspect it and use that to uncover, okay, well what are all the API endpoints? What are the documents that are associated with each of those API endpoints? And use that as a quick start to then create you sample test cases that say, okay, if you're interacting with this API endpoint, here's a document that will go along with that. You can provide the data and off you go. Right. So synchronization could be as simple as bring me in an open API document, or it could be as complicated as pointing at WebSphere integration broker and discovering a whole bunch of services from there as well. Nice.

So a lot of times when I check to vendors, they have a lot of customers, so they really see what's going on. They get a good big picture view of what people are struggling with. So are there anything you see on the horizon, especially we're now in 2023, the first month is probably be released in February. Where do you see the market going? Or are there any things that you're working on that you think could really help people in the future for things you see them struggling with?

So the easy answer, and I say easy in air quotes, is AI and machine learning. Right. Everybody wants to hear about AI and machine learning. What's your solution for AI machine? How are you bringing AI machine learning to testing? Right. And I think it would be easy to kind of over focus a little Bit on that. When I look at actual customer problems and this is where I get myself into trouble with the test community is, you know, it's still kind of at the basics, right, that there's still problems with data. Right. People are still struggling to get all the right data sets in order to be able to do testing successfully. People still struggling with UI automation and fragile tests and brittleness and things like that. And we've brought to market solutions that try and help with that, as most vendors have, but it's still a very common scenario. What's interesting actually we were at a conference, just actually probably been pre Covid times. It's been a while now, but we're at a conference and Martin, as you mentioned in the preamble, works on or worked on an R embedded solution. It's really interesting going to some of these conferences to have an embedded solution in your portfolio because it's a rarity, right. And you get some really interesting conversations from people who build planes, trains and automobiles. And the problems they have in those spaces are massively different to the sort of problems you might get with a web application. All these kubernetes things we've been talking about so far, that space is very, very different as well, but very interesting. And there is some crossover, you know, hence why we have it in the portfolio. But yeah, it's. I think that's an interesting space as well.

So I've been seeing a lot of blurring between skills. It used to be someone's just a functional tester right now and then someone was a performance tester, someone was a security tester. So Martin, you know, you mentioned performance testing a few times. This is a trend you're seeing as well, where testers normally traditionally may not. It used to be a separate team, now they're bringing it into their sprint teams and they're focusing in on performance and using a tool like yours can help with that as well.

Yeah, so performance testing is definitely the capability we have in one test for that kind of problematics. It really allows our users to produce a very accurate load and very precise, is very reproducing the load with lots of fidelity. And I think that's one of the tool of the market which is most appropriate for this kind of precision. You know, we need to be very accurate in the load that we are producing on the systems. That's definitely something that is progressing. People have been involved with performance testing for a long time and I'm seeing this as not going down in any case. So that's really something we're seeing as a priority in many organizations.

Okay, Kristen, Martin, before we go, is there one piece of actual advice you can give to someone to help them with their automation testing efforts and what's the best way to find contact to or learn more about each? So, Chris, best piece of actual advice you can give to someone?

Think about your data. Okay. I kind of alluded to this as one of the problems that we see that customers have a lot. You know, data is becoming more and more important in the problems that people are expressing to us and I think it's very easy to get lost in minutiae. I think coming up with a good test data strategy and this isn't necessarily actionable, I appreciate that is the right thing to, you know, it's something that people should be, should be focused on, should be, should be doing. Yeah, it's certainly an area that I think most people would benefit from putting some time and thought into. I can't say push a button and do it. So it's not quite as actionable as I might like, but yes, rather, yeah, that'd be where I'd go. But in terms of finding more information about HCL1 test, obviously we have our OneTest website which is on hcltechsw.com OneTest and you know, we have LinkedIn and Twitter and all those sorts of things as well. Sorry Martin, I didn't mean to step on your toes there. Your thoughts?

So Mar, do you have any last piece of advice for people to help them with their automation efforts?

I would consider HDL1 test as a definite candidate for the problematic that they have.

Thanks again for your automation awesomeness. The Linux evident value we covered in this episode, head on over to test guild.com a435 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the guild community. I'd love to hear from you. Head on over to test skill.info and let's make it happen.