Netflix SafeTest with Moshe Kolodny

Foreign.

Get ready to discover the most actionable end to end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Calantonio, host of the Test Guild Automation Podcast. And my goal is to help you succeed with creating automation awesomeness.

Hey, it's Joe and welcome to another episode of the Test Guild Automation Podcast. And today you're in for a special treat because you've probably been hearing all about it. We have Moshe joining us to talk all about Netflix's Safe Tests. If you don't know, Moshe is a senior Full Stack Engineer on Netflix and the creator of this new tool, Safe Test. You don't want to miss this episode. Check it out. Hey, Moshe, welcome to the Guild.

Hi, Joe. Thanks.

I guess. Before we get into it, is there anything I missed in your bio that you want the Guild to know more about? I usually have a longer bio, but that's all I actually could find.

I think that's pretty much what I do and where my responsibilities lie. And so I think that's a pretty good description.

All right, so you are a Full Stack engineer. How did you get into testing then? A lot of times it's not necessarily a topic. I hear a lot from either a Full Stack Engineer or a developer. Hardcore developer.

Yeah. So I guess my approach, I'm not officially like a test engineer. I'm more of just a generic Full Stack Engineer that does UI work. I'll do, you know, bff, like backend for front end, like a node server to be able to hit different services that aren't accessible by the browser. So it's just really a pragmatic thing. And one of the pain points is always how do we know that you're not going to break the world with every change? So yeah, that's what, you know, testing is an important aspect of that. And that's where I saw a problem and I did not see a good solution. The current solutions out there have some limitations that make it difficult to adapt in a lot of environments. There are a lot of times where it's a great solution, but in my specific use cases, it was not solving the issues that I was trying to solve.

All right, so what was the problem? What was the use cases then that you were struggling with?

So the biggest use case is the fact that we, and this is kind of outlined in the announcement post is the fact that you have to kind of make a choice between what flavor test you want to do and you're kind of stuck with that choice. So you either could go Full End to end testing solution like Playwright or Cypress or something like that, Selenium or any of these tools. And they are great for what they do, but their limitations is the fact that they don't allow you to test it as an app developer. So they're more of like if you're just doing QA on an application, you want to make sure that the application works as a whole. And that means like all the backend services respond the way you expect it to, and the UI looks the way you expect it to look. Then just to contrast that it's going to be unit testing and that it's going to be traditionally used like a JS dom. I think Karma used the Angular. I think it was Karma. They actually did it in the browser and they would do some hybrid approach, which is probably pretty close to what I was looking for. But. But the issue with that is the fact that anyone that's had to do that and you have like smart components on a page, interacting with them is very difficult because the JS DOM is a pretty good approximation of a dom. But once you kind of get into UI component libraries, everything starts breaking down fast. And you can't test like very rich user interactions like drag and dropping or things like that. It makes it very difficult to get a good handle on how to do things. So we're kind of like forced between two choices of do we want to have a good way to interact with the page? And that's what end to end tests gave us. And unit tests give us a good way to set up the page. We could mount our components, but we don't have a good way to interact with them well. So we kind of have this choice of two different things. And that was kind of the problem I saw. And up until recently, before component testing was a thing like Cypress component testing or Player component testing, this was not a solved problem. And you know, that is kind of like when I started thinking about this problem. And the current solutions they have nowadays are great for most of the time. But for my specific uses, I want to be able to test things that are harder are like more tricky to test. So I'll give you an example. One of the things I want to test is I want to test that the page works with having all the goodies of a test automation tool like Cypress or Playwright. But I also want to be able to check that certain things within the actual code was called. So I want to make sure that a service mock was, you know, triggered a certain way or things like that. And you know, going if we do component testing we could probably get pretty close. But that has a different bag of, you know, bag of worms with the fact that you have, you lose the ability to do certain things. Because the way the Cypress and Player code testing works is they start a web server, like a webpack server or something like that, and then they own rendering of the page, which breaks for like corporate environments because we don't own the rendering. Like we have a framework we create like a Create React app that we put on port forward 3,000. And then there's some proxying magic that happens that makes sure like header authors are forwarded properly and that won't work with like the traditional component testing end to end libraries.

So it's the best of both worlds. Then it's like you get, you do a end to end test and you can interact with at a unit level almost with the components. So are you able to then to put the application in a certain state and then run your end to end test? So it makes it faster as well? Is that an extra benefit?

So the speed was never really the goal. This was really what we'll do. Whatever it takes to be able to have confidence in the application. It happens to be that once we know that we are rendering the page a certain way, we do get some speed improvements because we always know that we're going to reuse tabs. We know when a page is rendered, when it's ready, when it's not. So we could optimize it pretty well. And there is a speed on every build on GitHub we run like a full test suite on SaveTest itself. SaveTest like Test SaveTest. And it does like a test where it clicks a button 500 times and asserts that the page didn't do anything weird. And we also have 50 tests where it opens a page, clicks a button and then checks it, does something and closes it. So we have both a lot of interactions on a single test and we have a lot of small interactions on a lot of tests and we do both and they're pretty fast. But that wasn't the point of it. But regarding your other question that can we set up the app to do whatever we want? That is pretty much the goal. So that is what component testing is taking advantage of. And the fact that we could mount our app and we could pass it like the people service, you know, as a prop to the app component and have it call a mock instead. Of course you could always do this with like overriding the network layer in like a browser based Automation tool. But then you have to kind of, you're kind of like digging into the internals and you have to understand the black box of how networking works. So if you have like you're using GRPC or GraphQL or something like that, you have to now understand the binary format. Those things travel over the wire in order to mock them out versus if you're able to just mock out them on the application layer, all you need to do is to just say, hey, instead of giving me this object, give me that object. And it's much easier to deal with.

Awesome. So I guess does it also allow you to mimic things that in production without having the full blown environment, I guess with the mocking comes in, be able to get as close to real world situations. Or maybe you can even do some type of chaos testing. Like what happens if this service isn't available? What would happen? Does that make sense?

Yeah. So that is like the primary use case in the fact that we. So Safe Test is a way to have the running application that actually hitting production services run. And we don't need to mock out the services or anything like that. And we're able to do that in a corporate environment. So we could choose to disable 2 out of 3 services and see if the application completely blows up or it gives you a nice error state or things like that. So that's pretty easy to deal with.

Very cool. So you mentioned Playwright, you mentioned Jess. Where does Safe Test come in? Does it act like a wrapper on top of these technologies? Then you do the special sauce to make it do all the type of interactions with the components.

Yeah. So I'll give you a rough idea of how safe this is. Really the closest comparison to Safe Test is like Cypress or Playwright component testing. And I'll explain kind of at a high level how they work and how safe this does things differently and how it works and that the contrast will kind of highlight what it's doing, what the advantages are. So traditionally what these things will do is they will create like a wrapper page to be able to kind of mount whatever component you want and then it will send that component to mount and then you interact with the page via the like the page object or something or the CY object, which could live in the browser. I think for components it does live in the browser. For Playwright, it still lives in Node js. So the. They have to kind of like live choose between one of those two. And I'm not 100% clear on Cypress, so I could be getting some of the Cypress isms wrong, but I'm pretty confident that's how the Playwright one works. The way Safe Test works is that it will execute the code in both environments. So it opens up Node JS and then it will like have Jest essentially just go through the regular code, but it'll also listen while those codes are running. So instead of calling the vanilla like describe function, you're actually calling the safe test describe function, which will call the vanilla describe function but will also track how it was called. And so it builds up the same stage testing tree that Jest will have. And then when you open a browser, so when a test is actually executing, it will open a browser and then the browser will also execute all those describes and all those things. So it'll build up the same tree in the browser and then instead of needing to send a message to the wrapper page to mount this component, it will just. No, because it's running the same code, it says this is the test, this is the test file, this test name, and then it will call the render function. So in Node, when the render function is called, it knows to open a browser in the browser, when the render function is called, it knows to mount that component. So that. And then we don't give up either one. So now we are able to do things in Node and the browser without having to like send like post messages between the two, which is how, you know, other solutions would do it.

Very cool. So, you know, I also think when I was reading, I could be wrong, I could misunderstand. It looked like you're passing images. And is it also doing visual validation testing as well where it's looking at the UI or am I. Did I misunderstand that?

Yeah. So one of the things, there's a library called Jest Image Snapshot that is, I think it's by American Express and that works with Jest. I think it was in the Puppeteer days. I think they created it. So there was. We already had ways to open a browser using Jest and Puppeteer and then we could take snapshots of the page and they created a library to be able to take two images, take the like the golden image and compare it to like the current image and diff them essentially. So it's like Snapshot diffing just with images essentially. And yeah, that's in Playwright and Safe Test use it. I think Cypress also has a concept of that. Right.

So I know what a lot of people, why they like Cypress is, especially developers. It's like a self contained, it's easy to get up and running for developers. They don't have to do any type of install. Basically how easy is it to get started with Safe Test?

The opposite end of the spectrum, I would say.

It is not a great sales pitch. But the issue is because the way it works is that it doesn't compile your code. It assumes that you have a bundler so using like Create React app. So it will leverage Create React app. But then it now needs to be able to be able to Import like app SaveTest TSX within your application code. And it needs to do that without making your production bundle large. So it needs to use lazy loading and it uses like globbing on lazy loading. So it's like kind of a hack to be able to get that working with a bundle in the first place. Every bundler has a mechanism to like glob lazy loading and we leverage that. But that is you're almost dipping your toes into like webpack webpack config territory, which is never a fun place to be.

So how realistic is this type of testing? Whereas I know a lot of times with certain component testing, it doesn't necessarily render a real domain that you would get in a real browser. Is this the same type of how realistic is the testing? I guess if you do it this way, is this exactly how a user would be able to interact with the application in production with the same type of results?

Yeah. So I'll give you a little bit of the motivation where it came from and then you'll kind of see like what it does. So generally like day to day working on like a button component that when you click it, it, you know it pops. Let's say what generally people do is they have a page that has the button, they write some code, they go to the page, they click the button, they see if it pops. It doesn't pop the right way, they edit the code, they go back to the page, they click the button to see if it pops. That is the kind of the flow of how development Lifecycle works. Safe test is basically mimics that flow where it essentially goes to your application in a browser using the actual like, you know, whatever URL you pointed to, you can give it a base URL and it will then open that component and then it will click that thing and then see if it does. And you could have any assertions you want on it. So it's using a real browser and it's using as close to like what the developer experience is. Yeah, I'm coming from like a developer Background, not from like test engineer background. So it's definitely more, you know, geared towards that methodology.

All right, so I guess that's a good point. Like if someone's listening, it sounds like if you're a developer this would fit into what you normally are used to in your day to day kind of flow where a tester may not necessarily, it seems like who is this made for? Is I guess the main question.

Yes, this is made for everyone. It's just that the evolution of how it was made mimics how a UI engineer will develop an application. But now that it's published and it works, we could essentially throw any use case at it. So like one of the goals of SAFE Test is that you could not describe to me a situation that I couldn't tell you how to make a safe test for it. So that is a very bold statement and I don't know if that's definitely not true for either pure end to end or pure unit testing. You make an argument that maybe already exist for component testing. But you know, there are situations where I have a service that, you know, I'm loading a table, an infinite scrolling table, and then it, I get to the end, it loads one, I lose another page, I go to the end and then I want to mimic out it errors out it errors on retry and then the second retry it succeeds. Let's say that is very hard to, to write a test for, but using like overrides and other methods within SafeTest, that's not that it's not easy to write, but it's not like I could explain how to write it and I don't think there's going to be much ambiguity with how to do that.

Really cool. So how long has this been in the wild for now? Has it been like a month or two weeks?

I think it's been like probably close to a month at this point.

All right, so have you heard any feedback? Like you want to believe what I got safe tests to do for me or you thought I didn't know it was going to be used that way, but all of a sudden people are starting to do things and you're like, oh, that's pretty cool.

I haven't really heard much feedback in general. I think most of the feedback I get is internally people using it and you know, just setting up and getting all the, you know, the UX goodies like that it works in CI and all these things out of the box, which is, you know, hard selling point. Also like another thing that does, we have our Internal wrapper around. It also does like the authentication logic for free. So usually all our applications have this, the proxying magic stuff going on. So you're going to have to deal with that if you're working with like Playwright or Cypress. So this is just, you know, three lines of code and then everything's just wired up for you.

Because I know testing authenticated applications can be really tricky. So could you explain it one more time? It's just how does that work?

Yes. So there's every large company I've been at, every like large corporate company I've been at always has some solution to how to log into an application to do end to end testing. And that usually is a library. And then usually you have to figure out how to make your end to end thing integrate with that library. Hopefully there's like some platform that already solved this problem for you sometimes, generally for most people that most test engineers do not have that luxury and they have to kind of figure out how to do that themselves. This, what we have internally is a like an NPM wrapper that just does all that logic for you. So instead of importing like the setup code from the vanilla Safe Test one, you would import it from, you know, the private package and then that will take care of everything for you. You would use it just the same as you use every other thing. And then it knows how to do authentication and all that. Since we do have the library to handle authentication, it's like a function call to be able to get an AUTH user and then to get cookies that we set in the browser. If you're doing like OAuth or something like that, it's not really a solved problem. However, you would generate a test user to log into your application. You'd have to kind of redo that flow. And then once you solve that once, you could create an internal package like we have over here. But someone does have to solve that. It's not, you know, there's no magic to automatically log into an OAUTH application.

Absolutely. So you said this was made for everyone, but is there any, like, what situation would you think? All right, I have Playwright, I have Cypress, I have Safe Tests. When should I use Safe Test? Is it always doesn't matter, just use Safe Test or is it like, if you have this situation, this is why you definitely want to check out Safe Test.

So Safe Test is basically glue between a. So you asked before, like, what Safe Test is where in the. I think we kind of like lost track of that. So Safe Test is essentially Glue for a couple of different technologies. It's glue for a test runner, which is like we use Jest or vtest or one of those. Then it also takes a browser automator, which we use Playwright, and then it also is. Uses a framework. So in examples we use React, but we could use. We have Vue, we have Solid, we have Svelte, you know, a whole bunch. And there it's very easy to write an adapter, to be able to have Playwright integrate with a UI framework. And then it's just basically that's coordination between all those different pieces to be able to like have the code run in Node and the browser and just do that shuffling of logic between those things. So since Safe Test is wrapping Playwright, it could do anything like a Playwright test could do. It doesn't have some of the goodies that the Playwright UI does, like the dash dash UI flag when you pass it in, but it does have like its own. It could do like its own version of that if, you know, if it wants it to build something like that. I don't think it's really needed that much because it's not covering the same use case. So. But like, if there were like testing scenarios, I don't know if I again, like the. One of the mantras of Safe Test is that everything is testable. So I couldn't imagine something that's more suited for Playwright over Safe Test. Cypress may have some things because of the UX and the developer experience is very different with the Fluent API. So people may prefer that if they're coming from like a Java background. And so that could be something. Safe Test is definitely not like, it's not slow, but if you wanted to, if you had like a thousand tests that just like you're stress testing some very like Edge Casey Logic thing and you don't need like a real browser for that, like using React testing Library or one of the JS DOM ones is going to be way faster and that would be a better solution in that case. So it definitely. There are situations where you wouldn't always use it, but I would say 90% of the time it's, you know, a good solution.

So I don't know if you can answer how much is this used within Netflix? Is it large adoption? Is it just catching on? Is it word emote?

Testing is hard thing to solve? I would say that people, when they're picking up, when they're solving the testing story, they are reaching for Safe Test. You know, they're again, like, there's no like hard Requirement to do to use anything in particular. We have something called the paved path where you know, we recommend doing things certain ways people are free to do whatever they want. So like the page path uses like a. I think now we were telling people to use Create React app and then we kind of went away and now we're talking people. The paid path now is recommended to beat. But like people, if technically you could make an angular application if you want to. I think, you know, it's very much not done and you're going to have issues with getting support if anything ever breaks. But you know, there's no. Yeah, there's no requirement to use any specific tech.

So this is open source, correct?

Yeah.

How's the community so far? Like if someone wants to contribute, is it easy to contribute? Is it just you? Like if someone's like, yeah, sounds cool, but I don't want to be buying into something that has no community or no support type deal.

So it got a thousand stars in like two weeks. So I think people are like using it a lot. The support issues I see are when things aren't working as expected and that's usually based on some misconfiguration I didn't think about or some misunderstanding and that I have to clarify the documentation. The. When I. When it was like released the. I think like either that week someone wrote an adapter for solid js so like that was like someone contributed that out of the box and like, you know, it went right in like. So I think there's definitely interest. Again, it's really early. Like it's been a month so there's really not much history to really deal with. But I think it's pretty positive so far. The reception I got and I hope people contribute. That'd be great.

Absolutely. And we'll have a way for people in the show notes to contribute so they know what to do. So do you have like you said, it's early. It's only been out for a month. Is there a roadmap? Is this pretty much self contained now? It's good to go. Do you have anything? You're like, oh, I wish I could add this or that or this is something I plan on adding in the future.

Yeah. So I have a couple of ideas of things I want to add. So I have like a todo MD file on the repo. So and this is just like kind of like some thoughts I have popping on my head like you know, making like the test runner that playwright Dash Dash UI has like something along those lines. We could sort of do that already since Safe Test lives in the page and we could lazy vote like aspects of Safe Test. We could essentially lazy load in if you have like a query param set a full like UI mode and something like that. And then iframe things and then be able to do that. That is technically possible. There's no, there's no limitation of if someone is bootstrapped into Safe JS today, why that couldn't work. There are a couple of things about using it with Next js. Oh, so one thing I forgot to mention was that a Next JS application will not be able to be tested with any like component testing libraries because you know, they have. They essentially need to run their own server, but they have to know how to run a NEXT server. So it could be like they have next but like you know, some of the other like meta frameworks could be. You're at the mercy of the, the framework, the testing library, people to know how to do that. But Safe Test basically takes a hands off approach. So it's able to just say like oh, just when you open the application, just go to this page since it lives in the page essentially. So. So that's another thing I forgot to mention. Yeah, like a couple of things. Just some like developer experience things. Maybe making a custom reporters instead of having to glue some pieces together yourself and then yeah, just some cleanup about the instrumentation of it, things like that. Nothing. No, like major like refactors or rewrites. I think it's. The API is pretty stable. It does what it's able to do well with the most sensible API surface.

So I may have jumped to the gun talking about new features. I forgot about a feature that's probably already there. How does it help with developer experience for debugging? Because when a test fails you want to be able to know why did it fail? How did it fail? Does it have tracing videos? Like how does that work?

Yes, that's a great question. So the answer is that anything that playwright has, it also has. It automatically gets. So it gets videos, it gets trace viewers, it gets, you know, like you could replay different aspects of it in the trace viewer and see exactly where it failed and then it has some more stuff on top of that. So for example, there's like a Safe test reporter page that will go through. It has a like a folder structure tree view and you could open like the trace or the video or you could actually view the component. So if you're testing like a smart button component, you could actually just view that Smart button and see and just like, you know, click on like the thing yourself and see what's not working about it. You don't have to watch a video. So if like, there's some like, interaction that's hard to replicate on the deployed application, you could just essentially have the application mount the component that you want to test to be able to ensure. Like for example, if there's an error page and this error page test is broken in a prod environment. But the problem is you can't get to the error page in the prod environment because none of the API calls are failing on the fraud environment. Right. And it only fails in the fraud environment. So that's a very hard thing to test. But the way Safe test works, you could essentially pass it in the query param and it will then mount like the error state. And now you could test the error state in the prod environment and now like, you see that the call to the analytics server fails for this error state, let's say. So you could do like tricky things like that. You could easily just test yourself and it gives you like a developer experience that's really easy to with when things don't work properly.

Nice. So this might be a dumb question because it is like glue and it works on top of Playwright in jest, when a new version comes out of those libraries, does anyone have to worry about it like, oh, now I'm out of sync with the latest version of Playwright. Will this always work regardless of what version? Does that make sense of Playwright or just as released?

Yeah. So I would say that it doesn't use like any of the deep private methods of any of these libraries. So like as long as you can launch a browser and open a browser, it does do some interesting things in that there's. You can run Safe Test test in Docker mode. So that is useful for if you're doing like image diffing. Image diffing will always fail if you're running. If you're creating the diffs locally and you're testing them in CI because you're on a Mac and that's on a Linux machine or something like that. So one of the things that Safe Test has is a way to have the application, the browser running in the document container and that will be consistent across environments because it's not running on Mac, it's running on, you know, some Ubuntu that's pre baked into the thing. So we are able to get that. And that could be like the, the most fragile piece because that is like relying on a server and a browser and a client to connect to it. But everything else is just like calling the Global Jest describe or the vtest describe function like that. Unless they drastically change the surface of that. And then I'll just have a major version block to match that. So I'm not too worried about that.

Awesome. Well, that's a cool feature. Is there any other features we didn't mention that you think it's pretty wild or pretty good?

Yeah, I would say the most powerful feature is. So there's two features we didn't talk about. One was, I kind of mentioned it overrides, which is a very controversial topic. And overrides is really there for when you need it. It's not meant to be the first thing to reach for. It's really if someone comes up to me and says, oh, I have this application that I need to test, but I have this problem. Generally the solution is use an override. Like yes, we could like refactor your application to have that as a prop that you drill down and that you do all that. We could do it that way. But like you just want to make sure that, you know, if like this prop gets this, if this like deeply nested component gets this prop, it'll display a certain way, sort of like using an override or like this service will fail on this specific condition and we want to make sure that works. Probably need an override, unless you want to make that component top level, something importable, top level, and then pass in a prop to be able to override that. So that's essentially it's not meant to be like, you know, the best solution to a problem. It's meant to be the most practical solution to a problem. It's just being pragmatic about how we're going to test something. I think that's pretty misunderstood. And definitely like on the blog post and on Hacker News, that was definitely, you know, a point that people were focusing on. So definitely wanted to clarify that. The other thing that I think is unutilized and crazy stupid powerful, but probably not Percy required. It's almost like more like a flex of the framework is the fact that there's like this bridge mechanism. So what we could do is we're able to have the browser reach in. We're able to have the Node JS runner reach into the browser and then have the browser reach into the Node JS environment and have a kind of a back and forth two way communication channel. So for example, if we wanted to like only run A test if we, if we wanted the browser to be able to, you know, either display the admin page or the user page based on, you know, if the user has a passwords file on the root file system. Right. Like something that doesn't, it doesn't make. I can't think of a good usage of it because it's just, it's almost a silly use case. But this two way communication channel does enable us to be able to have like browser mocks in the browser and then have in Node JS make assertions that those mocks were called a certain way because it essentially asks the browser to give me the mock state and then it will create a, you know, a mock on the node level and then do assertions based on that mock. So that is, it needs to be there just to be able to power the library and we do expose it and we're able to the test itself. You could like require like modules that will work in the browser. It won't really work in the browser but it'll be a proxy object in the browser, but it'll work in node. So like you could you know, mount a button. You could like get the window location and that's not using jsdom. That's like using like the browser window location and then like you know, write a file to disk and then click a button and things like that. You can just like there's, it's almost like like Electron when they bridge like the, the node layer and the browser layer together. It's like they're glued together. So you could essentially do some pretty powerful stuff. Yeah, I mean like as with Electron, like it wasn't like it's probably an anti pattern to use it too much, but it is there if in case you need it for something.

Love it. I love the flexibility. So that's great. Yeah. So it sounds like you're not forcing people into like a methodology. It seems like pretty open to do as you wish. You know, with great responsibility comes great power type deal. But it's up to you. So I like that sometimes like Cypress is very opinionated how you do things. It sounds like this is not an opinionated tool. If I'm right or not. I don't know.

Yeah, yeah, that's definitely it. The like one of the examples I give is that if you wanted to test a service call, like you have a people's page that loads people. You could choose to you know, mock out the network layer on like the, the browser level. You could choose to overwrite the fetch layer you could choose to overwrite the library call or you could choose to overwrite the specific method on the library call. So you get to choose like it's an onion getting, you know, going from the component to get the people. You could pick wherever you want to be in the onion to kind of change the behavior. So I think that's, that's very powerful. And there's no strong opinions in the library built into the library where to go with that.

Awesome. Okay, Moshe, before we go, is there one piece of actual advice you give to someone to help them with their automation testing efforts? And what's the best way to find, contact you or learn more about safe test?

Sure. Okay. So the best advice I would give is just start writing tests and start doing it. You'll, you know, try to be as pragmatic as possible when you're doing it. So don't try to build a meta testing solution before you actually build tests. You know, that is definitely an engineering dilemma where you're trying to abstract away the problem, but you didn't actually before you actually get to the problem. So like, you know, have a use case, write a test for it, then have another use case write another test for that. Iterate fast, do quick tests, you know, and be pragmatic. Don't. You don't have to get 100% coverage if it's going to take five times longer. You know, write four tests in that same time. And just, you know, pragmatism is definitely the way to go when it comes to, you know, writing tests and software in general. Yeah, the best way to contact me is going to be on GitHub. You can open an issue, you know, you could reach out on something like that, open a pull request. I'm happy to have contributors and people collaborating on that. And yeah, I'm looking forward to, you know, community engagement and seeing where it goes.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode, head on over to testguild.com a4898 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world. We'd love to have you join the fam@testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head on over to test guild.info and let's make it happen.

Sam.