Taking Automation to The Next Level Using Nightwatch with David Burns

Get ready to discover the most actionable end to end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Calantonio, host of the Test Guild Automation Podcast and my goal is to help you succeed with creating automation awesomeness.

Hey it's Joe and welcome to another episode of the Test Guild Automation Podcast and today we'll be talking with David Burns, all about Nightwatch. JS if you don't know, David is the Chairman of the W3C Browser Testing and Tools Working Group and co editor of the WebDriver specification trying to ensure automation frameworks and browsers are interoperable. He does a great job of it. He's been doing it for years. Also, he was an engineering Manager at Mozilla with developer experience working on tooling and infrastructure to help make a better web. It now leads up or heads up the open source program office of BrowserStack. Real excited to have him back on the show. You don't want to miss it. Check it out.

This episode of the Test Guild Automation Podcast is sponsored by the Test Guild. Test Guild offers amazing partnership plans that cater to your brand awareness, lead generation and thought leadership goals. Get your products and services in front of your ideal target audience. Our satisfied clients rave about the results they've seen through partnering with us, from boosted event attendance to impressive roi. Visit our website and let's talk about how Test Guild can take your brand to the next level. Head over to testguild.info and let's talk.

Hey David, welcome back to the Guild.

Hey Joe, nice to be back and thank you for having me on.

Awesome to have. You know, we spoke, I think maybe time flies, but at least a few months ago leading up to Selenium Conf. So I'm really excited to get you back on the show. One thing we didn't touch on David and I thought we'd start off with is you've been around for a while and I think you're one of the unsung heroes behind the scenes. So I just want to kick off. I think you were one of the first people I know that actually had a book on Selenium. So how did you get involved in Selenium so early, so many years ago?

Probably going on nearly two decades ago somewhere in that region. I started at a startup and I was brought on to start up the QA function. I was like doing it all like the idea of CI was starting to become like a term that ThoughtWorks was throwing around and I was like doing the startup And I'm like, it was me and about four engineers, and I'm like, there's no way we can do this. So I kind of like, looked around, did everything. I worked at a financial institution beforehand. I'd learned Mercurial, so I knew automation really well. And then I was like, I looked at Mercurial's prices. I'm like, no, that can't work. And then, like, just happened across the Selenium product, like that was being pushed out at the time. And Patrick Lightbody, I'm sure that's a name people probably haven't heard of. He kind of started up the open QA brand and, like, Selenium fell under that. So. And like, we started pushing all of that and I was helping out on forums and then, like, time flew on and I thought, start helping out with documentation. And so that's how I got started in Selenium. I helped someone on the docs and I was working for that company still and we were moving from VV6 to. Net on our infrastructure, and I was like, well, let me see what we can do. I started helping out with the. NET bindings with Jim Evans and kind of rewriting things. Every time Simon Stewart would land NET code, I'd cry and then have to rewrite it because it was awful NET code. But, like, you know, Simon had the right idea and so I'd, like, make it work. And so did that. And then kind of rewrote all the Python bindings. And then, like, while doing all of that, I started writing my book. So it was, I think it was the first Selenium book that was out. And it was mostly just to get people interested in automation and, like, getting them started. And then kind of from there, things just kind of carried on snowballing upwards to where I am now. And so, yeah, it's been a wild ride.

That's awesome. So I guess another thankless job that you're involved in is chair of the W3C browser testing and Tools Working Group and co editor of the WebDriver specification, which I know was years and years in the making. I don't know if people know how much work that took. So maybe a little, like, people don't know. Share of the W3C browser testing and Tools Working Group. And what is that? And people aren't aware.

Yeah, so one of the nice things about my time at Mozilla was Mozilla promoted the idea of working in standards bodies to employees, like, really encouraged them. Because if you get it working correctly and you make these Improvements, it makes the web a better place. And I truly believe the web is the greatest thing that's been created in our lifetime, right? And it's there for everyone to use and people should be able to kind of pick and choose what they want, right? But obviously picking and choosing means interoperability becomes very, very hard. And so Wilhelm Joyce was the original, he created the working group. He was working at Opera at the time. And then he went freelance and his freelance business took off and he kind of left working on the W3C side of things. And I'd already been doing co editing with Simon Stewart on the specification. And so from that point of view, like co editor of a specification is not the person who necessarily writes everything. It's more around making sure everything is correct and factual and things like that and kind of guiding that side of things. And then obviously like I was being a manager in Mozilla for a couple of years at that point and Wilhelm left and it's like a working group needed a chair. So I just said I'll do it. Like I'm a manager. I'm already used to like half doing all the administration for a group and kind of herding cats because that's what managers generally do. And so it's been that. And then so a lot of the, like my role within the W3C is just kind of bringing everyone to the table and making sure everyone has a equal voice when they speaking to it. So be them Selenium contributors, be the like people from Apple working on Safari, Mozilla, Google, whatever, and just kind of giving everyone that like equal opportunity to speak and kind of guide the specification so that we can have a better web.

So we're going to get into Night Watch, I promise. But could you just say why the W3 standards is important? Selenium now is W3C standard. Someone contacted me the other day and said, oh something bizarre like they heard Selenium is dying. And I'm like, well it's a standard, I don't know how it can die. And I know there's other tools out there that don't follow standards. So can you talk maybe in the points of why this standard is important?

So throughout history and like especially like look at everything that's on the web, right? From video codecs to things like that, right? The thing that always wins out is something that is standardized and open, right? Always. We've seen Dart try come up and kind of wither away at Google. It's still used within Google, right? But it never made it to the web. TypeScript is trying to get in there, but kind of it always comes back to JavaScript, right? And the key thing there is like, JavaScript, like, if it's being run by V8, whatever, right? It's the same code, but using standardized, like, lexicon to say, this is how it should work. And then, yeah, I regularly hear people say, Selenium's dying. The truth is, and like, the data backs it up. Selenium's growing quarter on quarter by 25% every quarter has been for, like, I've been tracking it for the last two years. So we're still seeing record growth. And so, like, it's not dying, the standard is growing. And now we've got the WebDriver bidi, the bidirectional standard that's being worked on. And it's so important to the web that like, even Google's or in on it and what they're doing is they're moving Puppeteer towards webdriver rather than webdriver. And so, like, they're creating this. So now Puppeteer is just going to become a, an opinionated web driver because it's an important thing. Playwright is using a lot of the same, like, technologies under the hood. So eventually I can see Playwright becoming an opinionated webdriver because this is standard, it allows you to kind of work on all these different browsers, right? Because, like, I did a talk at Selenium conference this year about like, you know, being able to understand the differences between browsers, right? And so, like, if you look at the Chromium space, like, you have Brave Edge, Chrome, Vivaldi, Opera, right? Like, but if you use them on different websites, you will sometimes get different experiences, right? Because there are fundamental differences in the way they kind of give you the final browser. If you look at WebKit on, say, Linux and Safari, there are times where it's going to be very different, right? And I know that there are frameworks out there that are like, when you're testing on Safari, on Linux, it is proper Safari, they kind of make this claim, which is not true because, like, there's lots of data that kind of proves it with things like local storage is one of the key ones here where, like, there was a major break in Safari, but it didn't affect WebKit. And so, like, it was a huge thing. People were complaining. But if you used like, you know, indexeddb or Local storage items on desktop, it just wouldn't work in Safari. But people's tests would be passing and that's kind of a really risky thing to go. Yeah, it's okay, let's ship it. When you don't know those fundamental differences and I don't recommend like everyone needs like all the browsers, everything, but at least understanding the differences in the risks that are associated with it is very important to me.

Love it. So I think another confusing thing that people may be confused about is Selenium is just an API and I don't think they realize a lot of frameworks and tools use Selenium under the covers. So this leads me to Nightwatch. If someone's never heard of Nightwatch js, like, if Selenium's so great, then what is Nightwatch JS and how does that, does it replace Selenium? Does it use Selenium under the covers?

So the key difference between Selenium and Night Watches, Selenium has always by design been a browser automation framework. Right? It's about driving the browser. It has no way of knowing what's a test, Even though like 90% of selenium users are testers and things like that. And that's probably a, like it's not exact number, but it is probably close to reality. But it has all these things right where it's tries to not be opinionated by design on certain cases. So like we don't know when you want to kind of know that an element is at a certain space or kind of it's changed, it's mutated into a specific way. We don't like, want to make choices for you on that. And then you get like Nightwatch, which uses the Selenium JavaScript bindings under the hood. But on top of that, it has its own opinions on certain things. So when to wait for things, how to, like when a page is loading, how long should we wait? Because you can tweak the, like make these little tweaks to the Selenium settings when you're creating a session and then you go, oh, do this, do this, do this. I can hopefully eke out a bit of speed. I can work against real browsers. I can kind of do these things. And so from that point of view, like Nightwatch takes all the goodness of Selenium and then applies its own opinions on top of it to kind of then take you to the next step of where you want to be and how you want to do things.

So it sounds almost like a wrapper on top, it adds extra functionality that you don't have to create yourself. So that right out of the box it does things for you that you don't have to worry about it. Sounds like.

Yeah. So like, if you. To kind of the Playwright users out there, like, Selenium is the transport layer and kind of gets the browser into the right state and then you've got that within Playwrights and then. But Playwright's also got the test runner attached to it and it's got like all these other cool little features on top of that. And so, like, it's one's about kind of driving the browser, the other part's about testing. And so Nightwatch's core is about the testing and how you test it and what tests you can create.

Very cool. So once again, Nightwatch, been hearing about it for years and years and it sound like it was about to take off. And then I haven't heard much. And then all of a sudden, maybe two years ago, at least the past few months or years, a year, I've been seeing more and more. It seems like it's been being worked on more. So I don't know if I'm perceiving that correctly. So why is that? Is that true or like, I think it was acquired a few years ago and is that why there seems to be more development put into it now?

Yeah. So it was acquired by BrowserStack a few years ago. Like, we wanted to see how we could positively impact the testing frameworks out there. And so my program office was already contributing engineers to the Selenium project, and we still do because we're invested in it, but we also wanted to help out developers and testers and get them into that right space without kind of like them needing to fully understand, like, all the ins and outs of a browser. Because browsers are super complex.

Right.

Like, and so if we can remove some of that super complexity away from them and just let them focus on the keys that they wanted, and that's what we want to do. And so that's why BrowserStack kind of acquired it. And we're fully invested in doing it. Andre Rousseau, he kind of. He reports to me at BrowserStack, he's the creator of Nightwatch. So it's not like we've just acquired and got rid of the team. It's like we've acquired it and we're fully invested into it.

So just a weird question. Does that ever impact how people perceive a tool? If it's now owned by a company, are there any things they need to worry about when something was open source and then a company acquires it or anything like that that people need to know?

No. So in this case, I don't think people need to worry when like it was acquired. Nicole, who's one of the co founders of BrowserStack, said to me, he's like, do with it what you see fit to make everything better. Obviously we want to showcase our engineering and our engineering ability. It mustn't be only working with Browser Stack. And so like we have a Nightwatch discord that people are welcome to come in and join. And we have people asking Browser Stack employees like, hey, I'm stuck with this thing on Source Labs, how do I do it? And so we can help them with kind of some of the config, but after that we can't help them in the same way that like if they came into the Selenium project, Slack or Matrix or whatever and ask those questions, like we could give you certain inputs and then on you go. And so it's by design. We are not a cloud, like trying to limit everything to one specific cloud provider. Our remit is to make development and testing better, not make development and testing better with Browser Stack.

Awesome. So is there a certain Persona you see would really excel using Nightwatch? JS I hate pitting tools against one another, but this isn't pitting one. It's like maybe one team may thrive with one tool because it's a better fit for them where it's not the tool being bad or good, it just seems to work better in a certain situation.

Yeah. So our main Persona, like from a product management point of view is develop. Like we aim for developers and testers and so there's key things that we do. And so like, compared to some of our like competitor products, we don't necessarily have the same marketing that goes with it. So for example, like we were one of the first groups to come out with component testing. Right. I'm sure you've heard about that is the ability to kind of make a much smaller end to end type test in the browser and so you can load it up. We did that like 18 months to two years ago, like a long time ago. And so we were fully invested in like we, you know, we saw an opportunity, we can do that. And so like that type of thing is good for designers and developers, not necessarily good for testers. But then we still have our core of like end to end testing. You could do that. We have plugins for like visual regression testing that you can run locally. Even though like, you know, BrowserStack does have its own visual regression testing product that it sells. But there's a Nightwatch plugin that you could do and you can run it locally. You know, this is one of those kind of things where people go, what about browser stack? It's like, well, we're our own thing and so we can get you so far, but if you want that extra bit, then you're going to need a service provider to go on top of that. Obviously we'll recommend browser stack, but you could take it anywhere at that point. So we have component testing, API testing, unit testing. So it's this tool that like, for us, it's a tool that kind of our Persona is to kind of make the web better. Right? Like, I know I keep saying it, but like I truly believe it and live it. So it's like rather than going for the, what I call the testing Christmas tree, that tends to appear in certain QA departments where you've got like tons of unit tests, slowly like a smaller amount of integration tests. And then you got like it goes out again because it's API testing now. And then it goes, starts getting back up to end to end test at the top. So it kind of does this like Christmas tree to me. And so like, we can kind of help people with that by trying to balance it out by going, oh, do this type of test here, this type of test here. But you can just like go MPX Nightwatch and point it at a test and it will run the right test for you.

Nice. So it sounds like if someone's a developer, they're using JavaScript already, it would kind of seamlessly integrate with what they're already used to.

It sounds like, yeah, exactly. And like, compared to other frameworks, it's got all the nice benefits of Selenium. So like, you don't have to worry about like, could I switch frames, Could I kind of like, you know, navigate origins? So that's kind of like if you go from one URL to another URL, could I do that within a test and still have a pass? So you could still do all that type of stuff. Mobile. So we support mobile browsers and we have a tool that like when you're setting up a project, you just go like, set up this. And then like, it'll set up like Chrome driver and Gecko driver to speak to mobile browsers on like real devices or emulators so that you can do that. Or like, if you want to do native testing on a mobile device, we kind of fall back to using some bits of Appium as well. So like we try, try be this tool kind of that helps people get the right high quality product out to their users.

So one thing I have heard from folks that with Selenium is well before I think they came up with some new things to make it easier. But getting started with Slim, installing it, getting everything, all the brow, everything in place, the executables and everything working, how hard is it to get started with Nightwatch?

So with Nightwatch 3, which is coming out like in the next couple of days, it might even be out by the time this video comes out. It's literally like you do nothing, you just kind of stall it. We fall back to Selenium Manager, which is for those people who don't know it's a tool that is the best tool that the Selenium Project has ever created, that you can forget about is for the way I think of it is that like it's just there and like if it notices you don't have any drivers, it'll make sure it's all set up. Like, you know, the Chromium Project releases browsers every four weeks now. So you've always got to update that. That just automatically happens. You don't get stuck on a specific browser or things like that. The Selenium Manager, because we use Selenium under the hood, we get all those benefits straight away and so you can start doing that. So like, to me, that's the best thing to come out of Selenium that you can forget about straight away.

Yeah, absolutely. I love that feature for sure. So it sounds like this is going to be incorporated. Yeah, hopefully when this is released. The latest version of Nightwatch JS 3.0. So some other new features I guess caught my attention and I'm just curious to know maybe what some of your favorites are. First one is the VS code. Could you talk a little bit more about, is that a plugin that allows you just to easily integrate now into Visual Studio?

Yeah. So it's Visual Studio code plugin. You install it and what it does is it gives you all the kind of benefits that you would from using something like Visual Studio. So you'll get the little play button so you can kind of like play a specific test or you want to run an entire file. But it also says like ask you like questions. And obviously there's defaults, like would you like to run your test in parallel? Which environment would you like to run these against? And then in like, because you're in Visual Studio, you can set your normal breakpoints and then when the tests hit that you can debug them and then go through it and One of the nice things that we've got in there is like what we call the Night Watch inspector. And so when you're in debug mode it's a little extension that loads in the browser and allows you to kind of navigate and then you can kind of like oh, I want this, like give me the best locator for this and it will go, it'll highlight for you and then show it to you and then you can click on it and it'll copy the locator into the extension so you can see what it is. So you can copy paste. You can also like run little commands like you can do interactive in the browser. So you can like, you know, you want to know what the title of the page is. You can do like browser title. If you want to like find an element and then click on it, you can kind of just use the element API and just send a click to it. And it, because it like, it does this like really cool loop out and back in. It's not a JavaScript Fire event, it's a like a proper native event. Like if you were clicking on it. So like if you're trying to like test a login for example and you've got security in your login form, it'll do the proper clicks and proper events into the browser rather than trying to just like inject it and fake it away.

That actually was the second on my list for new features. So is there a third that you're excited about or maybe that's upcoming?

So one of the other, there's so many, like I'm really excited about this. So one of the key things, other features that we've added is a new element API that we've created. And so it takes some of the like best features of testing libraries ability to like look up elements on the page. And so like if you just want to find an element by its role, like that's not really native to Selenium because it's like CSS and things like that. It like you can kind of be a bit more fine grained or kind of like look in weirder and wonderful places to do it. And then we just translate that and speak to Selenium and say this is the one we want, please. And so that's really cool. There's been a lot of work that's gone into that. And so like, and because like it's got all these new features and we like to do chaining in Nightwatch, like we allow the API to be fully chained or like if you don't want it to be that way. You can, can kind of just do like normal JavaScript, like Awaits and then the command that you want to send, you can kind of chain like verify commands or if you want to do that. So that's really cool. I really like that feature. So, yeah, and then the one after that is like Nightwatch is about eight years old, maybe older. And so we've been slowly migrating towards TypeScript. And so this version has a lot more like typing information. So if you're in Visual Studio code, which, like, I think we estimate about 80% of our users are from kind of just speaking to people, you'll get all the like typing information. So, like you can just like do a dot and it'll come up with the commands and then kind of guide you through what you need to do. And so we're working towards that. A lot of that is not necessarily like very straightforward and we're having to refactor, but there's a huge improvement in that. And so like when people upgrade to V3, they're going to be deleting items out of their package JSON rather than trying to add new features so that they can kind of simplify their projects and get more out of it straight away.

Love it. There's a lot of great stuff there. I think people definitely need to check it out. What's the best place people can go to find more about Nightwatch JS if they want to download it and get their hands dirty?

So we're just about to launch a new website, so again, hopefully like depending on things around that, that should be out very soon. And so that'll be nightwatchjs.org and from there it literally tells you, like says copy this line into your command prompt, which is NPM init Nightwatch, if you don't want to go to the website. And then from there it'll ask you what environments you want to set up, like what browsers if you want to test on mobile, things like that. And then we set everything up and away we go.

Awesome. Awesome. Okay, David, before we go, is there one piece of actual advice you can give to someone to help them with their automation testing efforts and what's the best way to find and contact you?

So the best advice I would have is kind of browsers are complex, but they're not scary. And so if you can write your code in like dev tools in your browser to see what you want to do and interact with it that way, that's best way to then get that started and then copy it into your automation and get it going. And then if people want to contact me, I am Tomato tester on pretty much all the socials and you can find me there. I'm also on LinkedIn. I have a newsletter on LinkedIn called Automation isn't Scary and so people can kind of have a look at that and I'm going to be like I just talk through how you know people shouldn't be afraid and just get things going.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a452 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguilds.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head on over to TestGuild Info and let's make it happen.