Evolving Test Automation with Ethan Chung

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with Ethan Chung all about the evolution of test automation to accelerate continuous delivery. It's a really timely topic, so really excited to talk to him all about this. Ethan Fionno is the leader of keysight by solution engineers across EMEA and apac with a technical background across automation testing, apm, software development and research. So he has a lot of experience in all the areas I think we're going to touch on today. Ethan's team actually focuses on building end to end testing solutions encompassing end users interactions down to the hardware solution across enterprise environments. So hopefully we'll touch on that as well. You don't want to miss this episode. Check it out. The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. They're cloud test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial. Visit testguild.com sourcelabs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Ethan, welcome to the Guild.

Thank you very much. Nice to be here. That was exciting.

Awesome. Yeah, great to have you. Hey, before we get into it, is anything I missed in your bio that you want the Guild to know more about?

No, I think that wraps up what I'm doing day to day nowadays.

Awesome, awesome. So it seems like you're. This is what I love. Talking to folks that work for vendors, you speak to a lot of customers, you probably see a lot of issues that are happening at an enterprise level. So really tricky, knotty issues. So where did you see test automation maybe a few years ago and where are you hearing from customers of maybe some challenges they may be running into now with test automation in general.

So I think test automation has been moving through years has gotten more and more complicated just as the environments have traditionally testing. Think of any developer first starting off writing the hello world. It starts with logging, starts with the unit. Testing is very functional and it's very specific application. And as applications are growing, even when it breaks down into multiple tiers and breaking across different architectures, fundamentally it's still one application that's running underneath it and you generally have single control. Most large companies, no matter how big they are, they still have their own in house development. So that way they're not really focusing on Having to worry about these hidden black box environments in the last few years, particularly with everything being SaaS, SaaS, SaaS and everything moved from the cloud ultimately I think a lot of environments getting more challenging. What was being traditionally you hire a couple of developers, you're building a house, you test it, you know exactly how your application is going. For these large enterprise firms now they're no longer having to rely on that, they're all moving towards the off the shelf products. A lot more packaged solutions whether the Salesforce, SAP, Oracle, the big names have something in pretty much everything now, particularly the add ons and extensions. And so testing now has become not how do we test things that's ours, but really how do we test things that have evolved way, way beyond the scope of what they can manage.

No, I absolutely agree with you. I've been seeing this more and more and even people, I don't see a lot of hardcore software development going on. It seems like people are just pulling the GitHub libraries and pasting them together to create a solution. So a lot of solutions I know back in when I started it was just you had a server on a raised floor and everything lived there. But now you have all these services everywhere, you don't even know maybe what's being consumed by what. So when you say black box is it also in the sense that an application could be consuming some third party services that someone doesn't even know about? Because we kind of lost control of our, of our code base almost.

I think it's a mix of a lot of different factors. So if you're an in house dev house, I think using pulling code offline, I think every developer does this right? You go online, you go somewhere and you just basically pull code and steal that solution. But ultimately that still sits inside your repo. But now black boxing can even be an internal problem. You're having multiple teams with microservices launching applications. They may or may not talk to each other. So that's black boxing. What the actual code is underneath between different departments as well as moving up when you're buying add on solutions that pre existing products that you can manage. So for example Salesforce is the classic one where you have your core Salesforce and then loads and loads of extensions that sits on top of it. Ultimately these can be Salesforce provided or generally more third party with enhanced solutions on top and of course black box on the completely hidden layer. If you have anything military, a lot of healthcare, a lot of financials, it's just a pure black box. Environment where you have no access to the code. You can barely even connect to some of these environments other than as a remote connection. So you can see what the live is. You can't actually do anything from a user perspective or from a tester perspective. So it's coming from a lot of different positions now.

So, you know, I've spoken to a lot of vendors and they keep bringing up these package solutions. Why is there a trend? Why, why is there a trend to companies starting to use more and more package sol? I know enterprises always have, but is it speed to market, is it security reasons? Like why, why would, why is, why are we seeing this trend?

I guess I think, well, that's the usual sales answer, right? It's faster, it's quicker, it's easier, yada yada. But I think ultimately for a lot of our customers is they have Salesforce, they have SAP and they don't want to build a brand new solution across the system. So generally Salesforce we say, you know, there's a lot of customization, but fundamentally it should always be a basic object layer that you can call data fields into any of them. So you're going to an accounts page. Salesforce, you generally have around the same kind of data whether you're Ericsson or Sony or any other enterprise firms that exist in the world. And so there's no real point to rebuild your foundations. I think a lot of approaches now testing is really focused around that user experience because it doesn't with the complexity of black box environments, ultimately you don't really know what's happening in the neve. As long as your users are happy, you're not going to get tickets being thrown in through the complaint lines. So as we're moving towards that space, the package solution is we just need a wrapper around what your environment is doing to make sure it's working for the user. And frankly that's the only space you can actually interface with anyway.

Absolutely. So what I think is interesting, I don't know, maybe it's just because my experience is I speak with a lot of testers, a lot of them don't mention package solution as automation. And I wonder if they're just focused in on web based, browser based and don't even realize that their company has a need for automating these other solutions. Is that a common thing you hear or problem?

That's a huge thing. Especially with testing world. I think everyone focuses very much on what I do and what I can physically connect to. When you're dev or test, you Focus on what you're told as the most important thing you need to deal with. And what you can deal with is the only things you can touch. And so why would one go outside of that world and then try to struggle with something that is I think from a development perspective, a test perspective, not your problem. Right? Because when I did any development it goes that's my code, I'll deal with my code and I'll make sure my code is working well. But I'm not going to work out what other departments is doing and fix their code because that's not my day to day job. And everyone's so busy already. So this package solutions almost become a not a viewed item as a one and all solution because everyone has to build solutions for their own little slice. While packaged solutions has the same problem as well. Like let's say Salesforce, Oracle. A package solution around that is still ultimately just around that product itself. So it becomes quite segregated for other testing solutions.

So I would think also the problem is because we lost this control, people want to just be able to focus on what they have control of and cicd, continuous integration, continuous delivery. A lot of teams have been moving that way. Do they also incorporate package solutions or are they also trying other challenges with that that a lot of people shy away from incorporating into their full pipeline type automation life cycle?

Well, I think package solutions is ultimately just a means to an end. Right. It's not a magic beyond fix all problem for any kind of software. I think when we look at anything we're testing one of the things, how do we start getting value within a week, a month, a year? There's a big transformation and package solution is just how do I get that basics first. I think with in this space we push around package solutions as a bit of a. This is something else, something managed. But ultimately package solutions should ultimately be a way of just getting you to the end result faster. It doesn't really matter what that middle is. So if you go selenium, there's 101 different package solutions around web testing and how that should do. But ultimately all of it is just one real focus. How do I get from A to B faster? And that B can be full testing, a full modeling of the application across different applications. But I think a lot of it because it's packaged becomes A. I don't want to get too close to that because we have our own solution that works well.

Do you also think that because it's a package solution that people just assume it's already been tested? In a sense, why test a package solution? And Oracle must spend thousands of millions, not thousands, but millions and billions of dollars of testing their solution. So why should I have to test it?

I think there's a huge problem around that. Particularly we see software breaking down. One of the most expensive times in any company is when a key software breaks down. That's SAP. The Salesforce is the world. If most companies that use a Salesforce or SAP or anything, Oracle is so embedded within their infrastructure that if it goes down it means there's a P1, productivity stops, factories is down and when that happens the testing world goes, hey, we're paying you this much amount of money, you should come and fix this. But I think that becomes a natural wait time between the actual solution and problem because you have to get in your consultants. You don't get a third party. If it's party managed then you're always blind until you bring someone in. I think a lot of these pract solutions actually does a really nice step in that gap even for large companies. You may not have a SME in house for these off the shelf solutions, but having that packaged solution will mean you get visibility before something breaks or before it goes into production because you have a way, way deeper visibility into it. And there's also a sense of when is a third party in a large organization as you're mentioning, they should be managing their testing solutions but that only test their own piece of the infrastructure. So SAP is never going to be testing Oracle SoftW. Right. So how do you make sure you have testing that fits across all of these items? Because it's generally the integrations that cause this problem now. I mean it's rare we ever hear Salesforce or any of the SAP of the world crashing. It makes the news. Nowadays when you have a new release that has huge problematic issues, it ends up being on news. People are aware of it very quickly but it's the integration ones that causes specific companies to get in a lot more trouble.

So do you have an example of an integration issue? Is it because people aren't testing the API connection or is it because like where's the breakdown then? At the integration integration level we find.

It particularly around the data perspective. So if you, I mean fundamentally Oracle, Oracle Systems and Salesforce systems are fundamental different data schemas, right? And that communication when you're converting is all just mapping that data over when you're doing checks or you're going SAP backend with a Salesforce system, how does that data connect together when it transfers none of these are generally a production breaking problem in the sense that the application doesn't load, it's just the data fields no longer make sense. It doesn't work for a user perspective. So we see a lot of these kind of actions taking place where the package solutions even have a lot more difficult to deal with because Salesforce can intake data and it's technically correct, but you don't actually have a clue whether it's correct or not until a user gets on it and it hits that production level view.

So how do people usually overcome that then that challenge around this is the.

Model based approach works extremely well. I think talking about that complexity of testing environments now before going from the units moving up to functional. Now the next step which, which I personally believe in is the model based approach. So whether you have your own custom front end, that's most websites have, most companies will have for their users to jump onto and then a off the shelf backend and database, et cetera. To have one testing solution that can orchestrate the testing across all of these I think is key. So when you have a model based approach, you have one section of the model that focuses purely on the browser, which is one tech stack and how that user can interact with it. And then the second you put a workflow order through a browser or a purchase order for a browser, you jump straight into your SAP environment. You then check that purchase order, make sure that is the correct one, the data flow matches the actual items. You do a business logic validation that the currency is correct, the actual numbers are correct, the order numbers fulfill anything that you need from a run. The business perspective is still active and that bypasses the problem of how do I make sure different environments are working in sync.

So I guess the challenge with that is with web usually can understand the elements underneath, usually have developers create IDs and things. But I think a lot of people shy away once again from the backend because maybe they're almost like a black box system. Like you've mentioned a few times, they're not really accessible. So how does a model know like here's a web and here's a backend and here's how you can access that backend data.

So I think you either have to have go for an option of an overlaying model approach that has access to all these tooling. So if you go going from the web perspective, right, traditionally you'll use something that's object oriented to focus and then you'll go into your backend in the same model, you go through A db or you go into Solmans or you into some kind of other third party and you have to stitch all these tools together and then that's how you build a wider picture. The only problem with this is you then have one set of developers for a selenium testing, one set of DB administrators, one set of SAP SMEs and then you have three teams just to do the testing itself. Some approaches now with the model approach is using purely an image or image recognition or OCR approach. So that way ultimately as long as you can get a user that connects into these systems, you can go from browser perspective on the user on the actual website and then you can go straight into your SAP, use the exact same one and they can run the queries via the database tooling. And then from that that gives you the actual how the flow works. And then you inject a layer underneath it of how do I interact with the objects, how do I interact with the database in the backend, how do I interact with applications? Because ultimately the user, I think now, particularly as we're saying with these black box environments, you have to give the option of accessing it purely through the eyes of the user because without it you'll always have gaps inside any complex system.

So I guess I've heard of model based testing for a while. A lot of people back in the day would say well my model is only good as my model as I model it. So have there anything new within model based testing that maybe helps where rather than making assumptions, it kind of helps guide you like no, this is really happening maybe in production or that you may not be aware of?

Yeah, so that's a huge change with particularly the package solution of building these models. I think you hit the nail on the head there. Of my model is only as good as how well we keep up to it. And if you look at any organizations visio diagrams, they're generally not that up to date with infrastructure or how the actual application looks. And then you need to know people in between. However, one of the things we know, pretty much every solution has some data layer for how the application should be looking like. And that's, it's a little bit of marking homework, but it gives you that view of the digital twin of the application itself. You go into a website, you have your sitemap, you know generally what the applications look like with some levels of confidence you go into your backend applications generally is going to be structured in some way. How we take that data now and then convert it into the model. I Think becomes key. So you have these multiple incredibly complex environments. So any Salesforce site will have hundreds, not thousands of different unique pages. And every year it complexity doubles and goes more and more complex because there's more fields or items. By simply taking in that I say simply, the engineers are hard at work building it. But by taking that structure from Salesforce, you ultimately get a skeleton of the application itself. And then all that logic becomes one step easier to building the model so you don't have have to I guess put the labor of building the model itself. I think that's a huge move now for the model based approach because or packaged approaches with model focus because one of the longest time drains is to go, how do I map this out without being an SME in the product itself.

Yeah, I think another challenge probably solved now is once you map it out when change occurs, like how do you know a change occurred? And you're like, oh darn, I have a thousand tests that I already modeled and mapped and they added a new field. And I would think nowadays with AI, maybe it can update your test on the fly. I don't know. How does that work?

That's actually a really good point because when the environment changes, it's really hard to know where things change. Particularly with microservices. Probably the prime example. We know the industry loves microservices because it's fantastic and agile, all the buzzwords you can imagine. But then when you change things, the actual QA or the actual team that has to make sure it's working might not even get notified it's working other than a notification on the pipeline. And then that's generally when they're serving only issues detected with monitoring tools. When you're updating via the model itself, you can see the diffs between the different versions. So that automatically goes, hey, this location has changed. This is probably the most important location within your application test right now. When you do an integration, let's say with Salesforce, one of the biggest breakers with extensions is the testers don't know there's an extension. Because any user could just rock up to Salesforce Admin in some systems and then update, add an extension, do changes, customize the fields. When that happens, happens with any kind of model pool, you can just div against the change. No way has your model's change. Using frankly quite simplistic machine learning models tells you that's going to be the most important thing. Weigh those areas and then with the digital twin, you just navigate to that first to make sure all the surrounding impacts. And because you have model, it's the same thing as building a path, right. You know that path is the most important ones first so you start triggering those actions.

Well, that's interesting. I used to work for a large enterprise, we used to have to do patches, but we never knew what tests to run for that patch. Sounds like with model based testing you can almost say, well we introduced this change and the model is telling us it's only affecting this area. So I know therefore that if I run tests A, B and Z that it's covered. Is that true?

Yeah, exactly. It's one of the trickiest things I think there's a lot of disconnect with testers and devs and what actually gets impacted. Ultimately I think nobody wants the dev to be at the end of the day having to update the testers for an hour, two hours every there's a release to keep up the date. I think loads of companies try to do it with good EPIC and a good journey and track it via the dev notes so that way they can do it. But it's always down to interpretation and this effectively, once there's a model, it doesn't matter what the testing solution is. Underneath the testing ultimately is the arm and legs, but the model itself is just the brains for where people should be going first.

So we spoke a lot about automation erp. I speak with a lot of testers like is manual testing going away? What are your thoughts on manual testing? As we try to move to CICD and we're shipping faster and quicker and we need to get in the hands of our customers, Is it move shift basically to the end users doing our manual testing and we monitoring production and trying to pivot as quickly as we find an issue. How does that work nowadays?

I think this is a really interesting topic, right, this whole shift where end users can become your testers, you can do blue, green and just chuck it out there. I think it's great if you have a massive resource pool of people that will basically stomach it for you. So for some companies like Facebook and Amazon is probably the best example. They can release so many variations throughout a day and then just churn it with engine, see which is the best outcome and then decide from there. I think it's great in principle and if you have over 10,000 users actively using your application, you can take a slice out of it comfortably. However, particularly in enterprise and highly sensitive environments. So defense medical, you can't really have your medical patients or your doctors be the trial to make sure the guinea Pigs to make sure there's nothing's going to break. Right, right, right. And this mimicking of a user flow, I think it's the same thing as a lot of monitoring space. You know the APM space. The always answer is how do we test this is oh, the user was doing it and that triggered actions. I don't know any CIS or CTS that's going to be like let's impact our user experience generally even no matter how small that percentage is. I think that's probably the biggest gap. If you have that massive user pool and it's okay for you to take a little bit of a, I guess hit against the user satisfaction to something that they may not like to then calculate what's the best actions. Yes, it's a great solution but I think actually very, very little all companies ever get the privilege of doing that.

Yeah, absolutely. So where does monitoring come into play then? I know you mentioned monitoring in the pre show. Where do you see how we involved it monitoring?

I think around monitoring it's really the heartbeat of application. So if you have mappers in the testing perspective. So if you have a unit testing is making sure the joints and ligaments are working. If you have the functional is the whole arm is going forward. If you have model based approach you you're literally making that human run around you jumping. But monitoring is great because it's a sensor on your heart, it's a insulin levels is your beta and alpha waves and making sure everything's generally working once everything's put together. Which is really a bit of a weird analogy I just threw up there. But effectively it's the how to make sure everything is just running operational. But it's not going to ever replace testing but you need to have them synced together. It's one of the little shifts I think monitoring became huge in the last few years and everything's around it. You can form baselines but fundamentally problem always goes in it ends up in the end user. You generally don't want the end user having to be that person to make sure things are working. You need something to, you know, hit the funny bone and hit the little bit on the knee. Whatever the doctors do to make your leg bounce out to check things are working consistently. But it's really just an aid like any kind of medical measuring device to make sure it's working while you're doing the test itself.

Nice. So you did mention UX a few times. I don't know is accessibility testing. I've been hearing more and more about Accessibility testing, Not sure. As you see as we're. That's one of the key advantage now, I guess, not only shipping faster, but making it so it's more accessible, it has a good user experience, it's responsive. Is that also something that you see as we evolved, as we create software faster, quicker, that that's something that is bubbled up as an issue or something that needs to be more looked at?

Oh, hugely. I think in the past if you go back 5, 10 years, everyone will go, how is the database doing? Are we having memory leaks? Are we having issues? Now if you're database hitting 95% some DBs somewhere might be stressing and there might be a bit worried trying to fix in the background. But ultimately the is not sweating anymore because as long as your user experience is running fine, no one cares.

Right?

Even if you're hitting max of capacity and you're hitting that 99%, all the thresholds hitting red. I think from the business value perspective, great. It means we've provisioned our machine just perfectly where it hits that 99% as long as it doesn't hit 100%. So the UX side is really when we trigger, okay, when we need to stop changing the background, when do we need to change the environment? I think a lot of the times in the past we'll set an arbitrary threshold. So we're hitting 80% now. Therefore we need to dynamically scale up. Even with the newer infrastructure solutions, we go, okay, great, we're hitting on kubernetes, we're hitting X, Y and Z utilization on the pods. Therefore we're going to scale up. However, it doesn't really make sense other than you're just making machines for the sake of making machines that may or may not fit. It's really good from a preparation perspective. But once you need to go what is actually impacting the business? I think the UX point of view of can the user still do what they need to to do becomes king.

Absolutely. And I was just thinking that one of the evolutions from when I started was I just focused on the UI automation or just one little small piece of automation. We talked about really broad, talked about cloud infrastructure, almost UX database backend. Like who does this now? I would think of evolution, maybe a tester now is someone that is almost like a full stack automation through a pipeline automation or is that how you see it as well as an evolution maybe where we're going now, 100%.

I think with testing there was loads of times where it was you focus on your little piece of the pie and you solve that problem, you make sure it works. However, now testers are becoming more SMEs of that business more than the actual technology space itself. We're seeing a lot of getting new titles now. There's business analysts and there's user journey experts or some kind of new branding name. But ultimately they're just people that make sure the full experience is working, whether it's internal stakeholders or external. And their real value is they know how the application, application should be going through what was previously in the domain of manual testers. The regression testing of going down multiple paths, clicking through those values are now I think transitioning into the model based approach where you go, okay, if I need to build a model, then I need to know what negative testings I need to do, what logic I need to do, what business assumptions I have to have within the application itself. And then those people are converting into the model testers because they're the only ones that knows what to click, how to test for things that no one else would test for.

Great. So Ethan, I know we talked a lot about a lot of different things. I don't think we touched on Eggplant. Eggplant has been around for a while. It's like I said, I work for a large enterprise and we, we had some really, really old back end systems and the only way to test it was to use a technology like Eggplant to interact with it. So what's been up with Eggplant? I don't know if people know they've been been acquired by keysight so I'm not even sure if they, they're aware that you're all still around. So maybe like I said a little bit about, maybe about Eggplant, where you're all at and where, where, where you've been.

Sure. So I think Eggplant AP Plan historically is always around testing as a user. Right. So we have our sensor, we can use OCR image recognition to navigate through an application as a user would do. But five, ten years ago it was very linear paths. You map that journey as you would do as a user. Do you click here? 8 plan clicks here you go here. Aplan does that and it just maps it exactly same way as a normal journey. One of the big developments in the last few years has been going towards a automation intelligence approach. Effectively, Effectively we map the environment out into a digital twin that all the logic that you require to move through use the exact same old Eggplant.

Right.

That's the bread and Butter of Eggplant. It knows how to interact with the system, but now it's moving towards more of a How do I build the testing solution as usable across any, as generalizable as possible? When I build a new testing solution on a Oracle application, I'm not going to go, hey, I want you to click on the field that's specifically the PO number or the query number and then find out or the name of the thing you're going to go, I'm only going to make a generic search field item inside our model that just searches for whatever the text the user inputs through the GUI and then find that thing on the right hand side to click on it to make an action. So that way you can then apply that in the model itself to do anything. Because it's all just different scenarios that a user can go through. With the DAI and the machine learning side of things, that model becomes an easy journey that the machine learning can then actually create the paths themselves. So it's this huge move towards. Let's move away from just a standard regression test. We still need those for the most important business journeys. But how do we automatically create the regression test when you have the model mapped up? It's machine learning in it. There's buzzwords, nice things around it, but ultimately it's just a really fancy way of saying, I'm going to make decisions that should be better than a human. In theory, that's what we do. We automatically know what journeys you can do and then we optimize against those journeys. So that way you're building instead of 101,000 regression tests, potentially 100,000 regression tests that can go for every scenario across every single application that you have.

Now, I was also noticing on your website you mentioned performance and load testing. Not sure if it's part of the solution, but I was just thinking while we were speaking earlier with packaged solutions, one of the biggest things I hear people having issues with is how the heck do I do a proper performance test or a load test. So using something like Eggplant can so can help you with performance and load testing of package solutions as well.

Yeah. So the way we do the bun with the Eggplant performance and the package solution is simple. APLAN uses the package solutions. We build a model around the application. So Salesforce getting released with a package solution, we now automatically build a model of what it should look like. Now to run the test themselves, we have the users of flows, we know the business logic, we test against that and then we have APLAN performance which Load tests the actual environment itself. You put X amounts of users against the environment. Is your test still run same way? Think of it as your user journeys, as a user going through and then suddenly we're going to make it so that it's 10,000 users on this salesforce environment. Is the experience still the same? Is the timing still the same? Is the infrastructure still responding the same way? So that way we can pair those two data sets together and that effectively ties an end to end solution, but going further down. So now the model builds the horizontal aspects of the applications while the performance goes vertically down into the actual applications, the machine level. If the bare bones is set, hitting and retaining under load as well as anything DB or APIs, that's across the system itself, right?

And another thing that came to my mind with performance testing by any type of automation, especially in the seems like the domains you're in, aerospace, defense, financial services, healthcare is test data. And I can't find anyone that can say here's a solution to help you with test data. Do you have any suggestions around that or any advice? Does AKPLAN help with test data as well?

So with test data is one of the trickiest parts I think, because ultimately we can generate test data, right? You can get a developer just a data pipeline and create something. But ultimately test data is always going to be so domain specific. That is why we need the business analyst, the regression test that knows how to model it. Although we can just create it directly into DB or use programming knowledge to build out a lot of the test data perspective. One of the things we commonly do is just use a standard Excel because any person, whether they're very technical or non technical, can build that data set themselves. And a lot of manual testing world just focuses on building that dataset by generating it in Excel. So we take that as a data entry and then let Eggplant itself choose which of that data to use. It's one of those ones where a lot of customers come to us and go, hey, can you generate data? It's like, well we can, we can generate any data. It's just a programmatic data generation, right? As long as it's a string, as long as you're not creating some kind of automatic 3D model, something, we can generally just make data for you. But where do you want that data? How do you want that data to be? That's when you start once again needing the manufacturers with experience of the product itself. So that way we can incorporate their knowledge into our testing solution.

Well, I guess the problem Also would be dependencies. So using a package solution, maybe it's, I don't know, invoicing. But in order to have an invoice, you need to have a pack, you need to have a product and that product needs to be purchased and then it needs to be in a certain state. Does the tooling help you at all with our modeling to say, okay, you're going to need this type of data already there before we do this other type of testing, test run.

So let's say something like invoicing. Right. With SAP or anything, ultimately that building itself, we actually wait until it's converted into a real product. I think one of the chicken egg of testing, if you go package solutions is how do I test something that doesn't exist yet?

Right, right.

So we actually, as a bit of a feedback loop, we take the production level, outcome or a staging environment to form that model itself. Because you cannot test some something until it exists. So we have to wait for that stage where that thing exists, that production or that staging environment exists first. Then we use that structure to form the model which then can feed back into an early testing stage.

It seems like you really have a holistic solution here. Another buzzword I've been hearing more about is rpa. How is RPA different from normal automation testing? I guess it helps you automate business processes that may not be necessarily a test, but helps you create software faster. So automate certain things that maybe was data intensive, I assume. Yeah.

So we actually do from customers that uses the tool for rpa. It generally isn't our main focus because it's a very niche thing. Generally RPAs are very tied to specific tools because fundamentally if you tie to the application itself, then you can always brute force things faster than coming in from the outside world. However, for older applications, so we have some COBOL environments that we have to deal with, which no RPA is just going to build stuff for them anymore. Right. So we, we can just duplicate as a user would be moving through the application itself to do all those RPA processes. We actually find a lot of the testing is effectively in a way RPA because it's just duplicating that user behavior through the application itself.

Okay, Ethan, before we go, is there one piece of actionable advice you can give to someone to help evolve their test automation efforts and what's the best way to find or contact you or learn more about Eggplant?

Sure. The most first actionable approach is I think, don't think of testing as piecemeal items. So we get loads of testers that only focus on un only functional, only regression tests. Ultimately these are all steps in the journey to get that full automation story that you need. You can't have the model based approach unless you have the unit based testing, the functional, the regression to build upon. So the journey is not how do I get to X and Y in the future and how do I get that right now? Because you need that foundation. So yeah, I think that's probably my first thing that most of our clients go to. They come to us and go, hey, how do we get the end result? Well you need to know what you need to know first so that I can build towards it to reach out for myself. LinkedIn is probably the easiest. And for Eggplant, whether you want to reach out for me for technical discussion, we have our Eggplant website that's easily accessible. Just simply ping me a message.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a368 and while you're there, make sure to click on the Try for Free Today link under the Exclusive Sponsor section to learn all about Sauce Labs awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack Automation Awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.