Testing for Good with Nikolay Advolodkin

Get ready to discover the most actionable end to end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Calantonio, host of the Test Guild Automation Podcast and my goal is to help you succeed with creating automation awesomeness.

Hey it's Joe and welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with Nikolai all about his upcoming event Testing for Good. Also, he's an automation expert. He's done a lot in the community for many, many years. So I'm excited to jump in to see what's new with automation as well. If you don't know, Nikolai is a self driven SDAT on a lifelong mission to create profound change in the IT world, which he's definitely achieved, especially as founder of ultimateqa.com, which is one of the go to websites designed to train individuals and and testing best practices. He's also a senior solution architect, Source Labs, one of our awesome sponsors. So excited about that as well. And he's been voted one of the top automation engineers in the world multiple years in a row by techbeacon.com he's contributed to many books such as continuous testing for DevOps professionals and co created sauce bindings OSS. Don't know much about that. Maybe we'll dive into that as well. And he's spoken at multiple conferences like the Automation Guild, Tech Test, JS and saucecon. So he's also an animal lover like myself. So he's a great guy. Can't wait to get into with him. You don't want to miss this episode. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Source Labs, the cloud based test platform that helps ensure you can develop with confidence at every step from code to deployment. For every framework product, browser, OS, mobile device and API. Get a free trial. Visit testguild.comsourcelabs and click on the exclusive sponsor section to try it for free today. Check it out.

Hey Nikolai, welcome back to the Guild.

Hey Joe, thank you so much for having me. Always a pleasure.

Awesome. Same. Same. Before we get into it, no, I gave you a pretty good bio. But is there anything I missed in your bio that you want the Guild to know more about?

No. You gave me an amazing bio.

Cool. You know it's been a while. I just checked. Seems like a two years ago you were on the show. Hey, what's new? What you've been up to. I know you're working at Sauce Labs. Just curious to know maybe what's new? What's out there, what you've been working on?

Yeah, sure. Still working at Sauce Labs, I'm getting a lot more into like developer advocacy stuff. So trying to grow bigger communities around automated testing and open source, that kind of stuff. So that was one of the reasons why actually came here, was to talk about testing for good. We can jump about into that in a moment. On a personal level, I've been doing a lot of stuff with JavaScript and just like improving my skills as a front end developer. Learning really the process of building a web application and taking it all way to production and releasing it with services such as like netlify, for example. And so, yeah, that's been also really fun and really exciting to learn all the ins and outs of web development.

So why are you doing that? You're a tester, why are you delving into actually development?

Oh, that's a great point. Well, I believe that yes, even though I am a tester, if I want to be much better at doing testing, I need to understand how to do it from start to finish. Right? So testing is not only black box testing, right? We don't want to only test from a black box perspective where we have no idea about the working internals. As testers, we can be a lot more empowered, a lot better at our jobs when we know the insides of the code as well. And me, for example, know that if I want to test a login form, I don't necessarily have to do it through the front end. I can do a web request, get back a token, drop the token in the browser, and now perform different kinds of operations or even click a button using something as simple as a component test that will run in milliseconds and will still perform the same exact operations. If I were to spin up a browser and click a button through something like Selenium or Cypress, for example.

Nice. So it sounds like rather than just create an end to end test and think, okay, this is the best approach, it sounds like since you have development skills now, you know, hey, maybe I can push this further left into the actual developer doing more of these types of tests. Is that basically how it turns out a lot?

Yeah, exactly. I mean, there's so much more than just end to end tests. Right? And end to end tests. End to end tests tend to be the longest to run and the hardest to write. They take the most amount of code. It's getting better. Technology is actually getting way better at this. And then to end testing is becoming very reliable and useful. However, there's still Much more to testing than end to end test. I've actually found a great balance of testing, especially for like front end web applications like in common JavaScript libraries like React or Next JS and so on a balance of using some end to end tests, like a handful of of those, combined with a lot of visual testing. So I'll render my applications in different resolutions, especially desktop and mobile, always catching so many visual discrepancies that those tend to come up all the time when I'm doing visual testing. And then I'll throw in some like maybe components tests in there or some snapshot tests with something like Jest for example, and that will make my testing extremely robust. And I don't even have to write too much code to do all of that testing. And so all those skills that I learned as a kind of transitioning, shifting more to the left have been super useful in helping me with that.

Nice. So you mentioned component testing, folks that don't know what is component testing. I think component testing, if you're an old time tester, you think of Mercury, they used to have business process testing which was component testing, which is this is not. So maybe you can drop a little knowledge on us how you define what component testing is.

Yeah, for sure. Great question. I think even I don't have a full. I think the definition for component testing through the industry is still kind of muddy. But I'll tell you what I think. I think of component testing as units testing for the front end. So the actual like JavaScript, HTML that's put together. So for you the easiest way to think about it is a button. You would have like an HTML component which is like a button and it has some attributes, it might have some styling to it as well. Well, you can interact and even render that component individually just by itself. You don't need an entire application running. And so you can just perform testing like unit testing on that HTML component and then get all the advantages of unit testing and then all the coverage of run the browser stuff without having to spin up a server and actually render the application very cool.

And so obviously it saves you a lot of time and you can iterate over it really quickly while the developers are testing. I guess you could just run these quick component tests almost simultaneously.

Yeah, exactly. They run in like milliseconds. You don't need to start anything, you know, so you can literally, if you have access to the files, you can just test on the files and whatever's there is what's going to get rendered in the browser. So you don't have that also concern of, oh, I'm doing, I'm exercising a function, but how's that function going to perform when I link it to HTML? And so, you know, you're testing the exact button that for that would be rendered in the ui.

Great. So, you know, when I think of you, I think of a shop expert for some reason, C Sharp developer. So just kind of not off, caught off guard. But by JavaScript, I thought you were like C really tried and true.

Yeah, for sure, Joe. I feel like that's hard for me to still shake that C Sharp expertise Persona. It's how I entered the industry with ultimate qa. Pretty much I was doing everything in C Sharp, which was at the, at the time when I was working at my previous two employers, everything was done in C Sharp. So that's where all my knowledge came from and that's what I was teaching and sharing my knowledge. Right. In C Sharp. I'm working for Sauce Labs. I have to support all types of customers. And so primarily the two programming languages that most people, most customers use is Java and JavaScript. And so I had to pick both of those up. So nowadays really I do most of my stuff, I would say in JavaScript and Java as well. And so C Sharp, unfortunately it's not as popular as the other two, so I don't get to use it as often. Hence now you don't see me doing much around C Sharp just because not enough time to do so many programming languages and so many technologies, especially in JavaScript, you know, they, they have like hundreds of technologies that come up every day.

Absolutely, yeah. And I love how, you know, because you work for a large company, they collect all this data, so you probably know what a majority of people are using. So besides languages, so it sounds like you said JavaScript and Java, what's the, maybe the third one? Is it Python? That'd be the most popular.

Yeah, Python in terms of. So yeah, working for Sauce Labs, right. We run tons of automated tests, so Appium, Selenium, so that's where we get the data from. And so yeah, Python is number three. And then it probably, it's like Ruby and C Sharp. Ruby kind of has been on the downturn in terms of test execution and then C Sharp is kind of stagnant, maybe going up a little bit, but I do notice that at least. So that's in terms of front end browser automation. Right. I think Saucep just hit like 4 billion tests or 5 billion tests recently. So many tests. I can't keep track anymore. But at least from the entire community I noticed that C Sharp and. NET ecosystem is growing, especially because they did go open source and now they can run on Windows and can Linux. So that of course enables way more people to be able to use C.

So this is a dumb question. I guess it's all how you develop it. But is it one language that tends to be more stable that you see over those billions of tests that run like oh, the ones that are the most stable tend to be JavaScript and maybe because it's developers that actually creating them. I don't know if you have that type of data or not.

Very cool question, Joe. So when you say stable, what's your definition of stable?

Stable is, you know, when it fails, it fails for a reason, not because it was poorly written or has bad weights or you know, things like that.

Yeah, let's see. So I can say that from my experience I don't have the data to back it up. My feeling is that the JavaScript tech stacks tend to be the most stable and the fastest. My thoughts behind that are yeah, a lot of the JavaScript code tends to be written by developers. The JavaScript test code tends to live in the same repository as the software that the test code is testing. And so that just makes everything much easier. It allows for smarter testing and for faster testing and more stable testing. And then also, for example we, at least in source labs we have. Well also the other thing that makes it easy, right, Is for example when you think of deploying a web application, right? As you deploy a web application through the pipeline and it's let's say a front end web application, you need to build that application and then it turns into some kind of code that's running in your browser, right? You can build it and run it on like localhost3000. If your test code is in the same place, once you build the app, you can just run your test code against that localhost 3000. And so everything is like right there on the same machine, super close, super fast. And I think that lends itself to the stability. While for example now if you have to take a programming language that's not the same as the software that you're developing, let's say Java for example, now you have to run it in a separate pipeline and you can't really execute tests against local host because how are you, what are you going to be sitting in one pipeline where you're building, what are you going to be in sitting waiting there for until you finish that build? So you have to build and deploy that and only then can you run your Java test. And so that of course introduces latency, introduces, you know, as a result, makes the test longer and a little bit more brittle. And so the other thing that I've seen is in source apps, for example, for all the JavaScript stuff, we have Sauce Control and Sauce Control, basically the way it works. And I hope that a lot of libraries transition to this mechanism as well. The way it works is it basically takes your test code, we upload it to a vm, and then we run the tests on the VM with the browser. So it's almost like everything executing in one environment without having to travel. For example, me being in Atlanta right now, running my test from Atlanta to California, where their data center is and back, and then testing the application in that manner. Everything can run together and again making the test faster and more stable as a result. Does that make sense, Joe?

Yeah, absolutely. That's great insight. So I guess along those lines then when I think of JavaScript testing and automation, I think of Cypress and I know Selenium supports it. A lot of tools support it. So do you see in popularity with JavaScript, if someone's a JavaScript developer, what types of tools they tend to use more? Is it Selenium, is it Cypress? Is it something else? Is it a suite of tools that they use depending on what their needs are?

That's a good question. So in terms of developers, they will typically use something like a Cypress or Playwright has been really hot recently and I would say they don't do too much end to end testing as far as I've seen. They'll do a lot lower level testing. So they'll do a lot of like visual component testing. They will do unit tests, they'll use a lot of like jest for unit tests, for example, or they'll use like component tests. So I think because Cypress is like older to the game, they're kind of like the OG in the end to end testing space in JavaScript they tend to be used more often. But I also see Playwright growing in popularity. I'm actually here in Atlanta to do a demo for a customer with Cypress and Playwright because they want to compare the two. So yeah, absolutely.

That's been hearing a lot more about Playwright for sure. It seems like that's becoming one of the leading solutions out there for what it is. So talking about that, I think I actually, you know, not that I stalk you, but I do look at your social media often and I noticed you went to Something in Europe on, I think it was the Test Automation Summit. Some of the masterminds behind the open source. So I don't know, behind closed doors, what do you all talk about for automation solutions? Do you say, ah, Playwright is going to go down or you think it's going up, or what was that summit all about? And maybe some key findings, maybe that you might be able to share with the audience here.

Yeah, sure. By the way, Joe, you can stalk me on social media anytime. I, I welcome it. It's a, it's a, it's a term of endearment. Stalk me and bring your friends. So, yeah, about the Test Automation Summit. So it was an amazing event organized by Titus Forner and Diego Molina from Selenium Community, the primary developers for Selenium. And so the kind of purpose behind that meaning was there are so many open source projects that exist that are built around the Selenium and WebDriver ecosystem. Projects like Selenide, CodeCept, WebDriver IO, Christian Broman was there so many of these projects built around open source standards. And so we were trying to figure out how do we get more contributors to open source? Because open source developers typically are not paid. They do it because they love the project and they do it in their free time versus paid projects such as Cypress or Playwright, where for example, Playwright has a team of developers at Microsoft working for them that are paid to do, to work on Playwright every single day, you know. And so how do we get more contributions to all of these projects? For example, I contributed to atata, which is a dot net framework. It's like a page object wrapper around Selenium, making test automation much easier. Very amazing project, done really well. The guy Evgeny that runs it, yeah, very well tested, very well laid out project. So if you're in C Sharp looking for a page object framework that you can use Gravitata for yourself. And so in terms of some key takeaways. Well, some key takeaways are that one thing we all have to consider whenever we're making a tool is for example, open source tools are building on open source standards. And so if we're creating a community that is more collaborative, right, for example, all the browser vendors to open up their browsers and allow automation protocols that are common and easier for the entire world, then we all benefit. So that's really important to think about that we want to be, regardless of which tool we're using, which browsers, we want to be more collaborative than less because then everybody benefits from more collaboration and More openness. While, for example, something like Playwright doesn't seem to be want to be very collaborative. They almost want to lock down the ecosystem for themselves. They want generate their own browsers so that they can run the test in them and they control the code. And of course the code is all controlled by Microsoft. And I'm not saying that is necessarily a bad thing. It could be a great thing because it introduces a lot of innovation. But it's something to think about that it could potentially now be moving into the direction of something that we've had in the past. I know you're very familiar with, like Joe is like an HP QTP kind of product where there is one company that controls that and you're as an engineer dependent on them and you know you have to do what they basically say and you got to pay them a fee in order to use the product. So again, these are not inherently good or bad things. It's just considerations that we have to take as we're choosing products and we're choosing our tools. And Playwright is a wonderful tool. What I love about it is that it supports all the primary programming languages. For example, Cypress is only in JavaScript, so your developer in JavaScript, you, you know, you can only use JavaScript, but for all of us other engineers that don't use it, we can't use that don't use JavaScript, you can use Cypress, but Playwright does Java.net JavaScript and Python. So it's like they have you covered and it's the same API as a tester. You can pick it up and it's a really good tool.

Awesome. So, you know, I know there's a lot of different topics, a lot of hot things going on and the testing space. So there's a lot of events going on. And I also noticed as I was stalking you, putting on an event called Testing for Good, I thought maybe you could dive in a little quick. What is Testing for Good?

Yeah, sure, Joe. Yeah. So Testing for Good is an event that I came up with about a year ago. I'm always trying to think of, like, how I can serve the testing community better. And I'm also a co lead on the environmental erg at Sauce Labs. And so I was like, oh, why don't I put the two together and create some kind of an event where we enable people in great automated testing practices and then we try to give back to the community. So the purpose of Testing for Good is to enable amazing test automation while creating a more equitable society. And so I started it About a year ago it was just me ran like a JavaScript workshop where I was hands on people join for like two, three hours, do hands on coding, you do exercises. So at the end, instead of just watching a theoretical talk and maybe leaving with a repo, you actually leave with hands on coding knowledge that you've applied, that you've learned something and can now take to your workplace and hopefully start using in your workplace immediately. So that was me. Had like 130 or 40 people join. That workshop seemed to be successful. People loved it. The workshop was free. We raised like over $4,000 for black girls code. And I'm like, cool, let's do this and make it bigger and better. And so now we're coming up on our fifth event. It's going to be happening in October 18th through the 20th. And so the Testing for Good event is going to be held on October 18th through the 20th. We got three days, Tuesday, Wednesday, Thursday, we got five amazing instructors and let me tell you so, five amazing instructors teaching in the two primary programming languages that most testers use, Java and JavaScript. In Java we're going to have four different workshops. API testing with Java by Bas Dijkstra, Crafting a test framework by Titus Fortner from Selenium Mobile Testing with Ayal Yovel. He's a solutions architect at Sauce Labs and Web testing by Chris Eccleston. He's also a solution architect at Sauce Labs. And then for JavaScript we have three workshops. We have end to end component testing with Cypress by Gleb Bakhmutov from probably one of the best Cypress engineers in the world. And then we got web testing and API testing by me. I'm going to run two different workshops. So we're going to have numerous times that these workshops will be available over the three days. We try to make it EU and US friendly. So if you're in us, you'll probably find the time one day that works for you. And if you're in eu, you'll probably find the time on another day that works for you. Just be, try to be as inclusive as possible. Workshops are absolutely free. Anyone can join. The workshops will be recorded. You'll do hands on coding in whatever you know, technology stack that you want. And then we are going to be asking for donations. All the donations are going to go to Environmental Working Group. If you're not familiar with an environmental working group, they're a community of 30 million strong working to protect our environmental health by changing industry standards. Since 1993, the Environmental Working Group has Shined a spotlight on outdated legislation, harmful agricultural practices and industry loopholes that pose a risk to our health and the health of our environment. So being a test automation enthusiastic and an environmental erg co lead, it brings the two my two passions together into a single event and hopefully super valuable for the entire testing community and the environment.

It really is testing for good. So that is awesome. So you know, for a workshop, do you have any, anything like this? Do you have like certain levels someone needs to be in order to participate? Is it going to be by beginner, intermediate, advanced? Do they have to have something already installed on their machines? Like how does that all work? If someone's wondering, oh, this sounds great, but you know, I don't have enough knowledge and JavaScript to. To do this or whatever.

Yeah, that. Great, great question. So let me maybe I'll talk about like about the each individual workshop. So I can tell you for my workshops you can be a beginner and just jump into it. We host most of us at least at Sauce Labs, we host our workshops in gitpod. And so gitpod is like a. It's like a Docker container that can run in your browser. I'm actually a gitpod ambassador. So shout out to them. They're awesome. So it's basically you can have a GitHub account and then you can open your GitHub source code in the browser and then it's going to spin up an environment and you can code and run stuff in that environment. It makes it really easy. So you don't need anything installed. You'll be coding in the browser for at least my workshops and those are beginner. Actually almost all the workshops will be beginner and you'll be taken from beginner to like expert level skills through the workshops. Maybe actually all of them. Maybe Crafting a test framework by titus, because it's crafting a test framework, you probably need some programming experience, you know, because you can't just start as a newbie and already jump into framework. So there may be more intermediate with some program programming experience. But regardless, I'm sure everyone will learn a lot in every single one of these workshops.

Absolutely. And where can folks go to register for that?

Everyone can join at bitly forward/TFG-October 10-22 and I'm sure Joe, you can link to it.

Absolutely. I'll have a link for that in the show notes for sure.

Yeah, perfect. And then you can of course Google just testing for good workshops October and that should come up in Google. But Joe will link you and then you can always reach out to me on Twitter LinkedIn and I'll share the link as well.

Awesome stuff as always. Okay Nikolai, before we go, is there one piece of actual advice you can give to someone to help them with their automation testing efforts to help them test for the good? And what's the best way to find or contact you?

One piece of actionable advice I would recommend is to try and shift your testing more to the left by learning some of the skills that developers do on a daily basis, such as building an application, deploying it. It'll help you to be a better tester if that's what you're interested in, and you'll learn some important skills that you know will overall enhance your testing. And you can find me on Twitter at Nikolai.

You can find me on LinkedIn nikolaikin and I release a ton of, I mean all free tutorials on YouTube Ultimate QA and also can of course find me on my website ultimate qa.com and there you'll see all my blog posts and my tutorials. And again, I always try to do everything for free and help the community as much as I can.

Thanks again for your automation awesomeness. The links everything of value we covered in this episode. Head on over to test guild.com.

And while you're there, make sure to click on the Try for Free Today link under the exclusive Sponsor section to learn all about Sauce Lab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguilds.com and and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community. I'd love to hear from you. Head ON over to testguild.info and let's make it happen.