Modern Web Testing Using TestCafe with Dmytro Shpakovskyi

Welcome to the Test Guild Automation Podcast.

Where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with Demetro all about his new book Modern Web Testing with Test Cafe. I've been hearing more and more about Test Cafe, so I'm really excited to talk to him today. Do a deep dive into what it's all about. If you don't know Demetrio has 10 years of experience in quality assurance, software testing and software as a service. He is skilled in end to end load and API test automation and he's certified by the International Software Testing Quality Board. During his successful career, Demetrio led the quality automation for the platform of 60 plus highly scalable web applications and services, build and maintain numerous test automation frameworks, manage distributed teams of QA automation engineers, mentored and helped to convert to test automation engineers. And as you could tell, he has a lot of experience creating and open sourcing new test automation frameworks and mentoring other QA engineers or exploring new tools and techniques for automation testing. Really excited to have him on the show today. You don't want to miss it. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial, visit testguild.com sourcelabs and click on the exclusive Sponsor section to try it for free for 14 days. Check it out.

Hey Dimitro, welcome to the Guild.

Hey Joe, thanks for having me.

Awesome to have you. I'm really excited about this topic because like I said, I've been hearing more and more about Test Cafe. Before we dive in though, is there anything in your bio that I missed you that you want the Guild to know more about?

About Me so I guess you introduced pretty much all of my previous accomplishments. Nothing much to say. I'm always open for different discussions and opportunities. I'm an active participant of open source movements so I enjoy contributing to different repositories like selenium, cucumber, web webdriver, IO, test cafe itself some others. I really enjoy Node JS and JavaScript infrastructure. And yeah, that's in brief.

About Me I love how you're also into it seems like API testing and performance testing. You're not just sticking to functional automation. I think as we move towards 2021 and beyond, I think it's going to be a growing trend. So I guess because we're about to kick off the new year here at the time of the recording. Are there any trends you see in 2021 that you think automation engineers need to know more about?

In fact, yes, I see it. More and more things are shifting towards API testing, as you mentioned. So everybody is trying to go less into UI tests, a bit less and more into API, and the systems are usually developed now more with the API in thought. Basically. I think it's a good move because it's getting faster and easier to test as long as you dive deep into the API itself and as long as developers are willing to provide you additional endpoints and to ease your life as test automation engineer. Another trend is basically switching from manual to automation, or I should say like upgrading your current key engineers to dive more into automation. But at the same time, I've seen companies who stay with manual testing and stating that we don't need automation for now, it's okay, we are covered with manual testing. So I've seen two sides of this problem, but the mainstream anyway goes forward updating to automation. Yeah.

So I guess the next obvious question is, does TestCafe even support API testing? I'm not that familiar with TestCafe, so we'll dive more into it. But just curious to know, off the top of my head, does TestCafe support API testing as well?

I mean, you can tie it up to do get and post requests, like different types of requests, but I don't think that it's exactly meant for API testing. I would still recommend to go with something like Axios or Postman or Newman to do API testing, like the more classic advanced tools for API. So yeah, Test Cafe kind of can do it, but it's more more meant about UI testing than API testing.

Great. Yeah. So that was my next question then was what API tools would you recommend? And it sounds like Postman. What would they add to again?

Exios. It's an Node JS package that also helps you to do, get and post and different other requests. I mean you can do it in bare bones Node Js, but we all know a pain of doing it in vanilla way. So yeah, basically access is a good one. Previously request was quite popular, but I guess it gets deprecated more and more because of the size. It's grown really huge, like a monstrous package. So the shift was towards something less huge.

As I mentioned in your intro, you have experience with a lot of different things as well. So why write a book on Test Cafe? What was the impetus that made you focus on okay, I think there's a need for this. Why write a book? It takes so much time. And why on this topic, Test Cafe.

Great question. So actually previously I was working mostly with Selenium and we all know the pitfalls, like here and there, something is not working or is missing documentation or you need additional dances around it to like configure it properly. Your Selenium server doesn't usually load properly. I mean from time to time it just falls off in terms of downloading the binary from the remote server. And the overhead of raising the server is also not a pleasant one. So Taskify was like a breath of fresh air because we've seen a movement towards tools without Selenium, the tools that can run in the browser itself directly, like Puppeteer, which was a TrendSetter. And then TestCafe emerged. I was playing around Puppeteer and I thought that why not giving try Test Cafe and I dived a bit into their documentation and played a bit with Proof of Concept and it occurred to me that it really makes your life easier. I mean they did a lot of efforts to make configuration painless to help you raise the app that you want to test with the command inside of Test Cafe, to wait for this app to launch and then to drag tests over it. So they do a lot of small nifty things that really make your life easier. They also give you the ability to execute any JavaScript inside of the browser, which is also really nice thingy. So yeah, I was playing around it and I had my impressions that it's really faster and more comfortable thing compared to Selenium. Another thing was the TestCafe is relatively new. It has five years or four years, something like that. It was found in 2016, the first release, I mean and yeah, it has a really nice community. So I had several things that I wanted to discuss in terms of issues with them and like my concerns. I raised the ticket in GitHub and my issues were had responses in three hours. It's really amazing. So the community is live and really active. So yeah, I played a bit with TestCafe and I thought that it's a good thing to share with the community my findings, how I would suggest to do it. And there was no really books or texts about. Not much books or texts about Test Cafe apart from the official documentation. So yeah, I had this thought and then publisher just contacted me via LinkedIn as I said that they've seen some of my contributions to Test Cafe itself and they suggested me to write a book. So I gave it a thought and I then thought, why not? I should share if I can.

Very nice. Yeah, it's good to hear about the community because that's one of the number one things I recommend people look at before they invest in an open source tool is how active is the community. Sounds like it's very active, so that's awesome. I guess you did mention it runs in the browser. It's kind of funny. That's how Selenium started off running in the browser and then they moved to the webdriver. But a lot of these other tools now like Cypress does the same thing, it runs in the browser. So how would you compare this against Cypress then if someone says oh that sounds like Cypress, is it like more. Is this more tester friendly compared to Cypress, which is more developer friendly or do you make that kind of comparison?

Yes, I would say that it's a bit more tester friendly. And I also dived into Cypress a bit. I had several pull requests for them to fix some documentation, minor documentation issues. The things that I noticed in Cypress and which keep me a bit cautious about it is that they tell you how to write your tests. They have several trade offs and they say that this will never change. That the thing we see how your test should be written and it's a good thing that they at least show the direction. But I'm cautious about why they're telling developers or test engineers how to write their tests. I mean I've seen some arguments on Internet that with Cypress your tests doesn't look like JavaScript or Node JS, it looks like Cypress Way. Which keeps me on a cautious side because I prefer to use all the power of Node JS to complement my test automation solution, not to stay on its way. And Test Cafe gives you this freedom. So you're using Async Await all the nifty grifty stuffs and it doesn't push you into any particular direction. They can recommend you how to do it better but they don't force you, they don't say that it will not work other way like this. It's really good thing as far as I see. And yeah, Cypress has a really nice thing with debugging and going back and forth, displaying all your get and post things and all the requests that are going between your application and server. So it's. Yeah, I would say that it's a bit more developer centric and Test Cafe is a bit more universal. Additional weak spot in Cypress is their support for the beginners they rolled out the support only for Chrome and they said that you should test only in Chrome. We will not support anything else. And I was like, why? It's a huge limitation. Then they rolled out support for Firefox, which was a good trend, definitely. But they still lack mobile devices support, they lack other browsers, and Test Cafe was rolling out with total support of pretty much everything. And that's also a really huge benefit, as I see it.

Absolutely. So another thing, with trade offs with tools, you have ones that are opinionated and ones that give you the freedom. Sounds like Test Cafe really gives you that freedom. You're not pigeonholed into creating tests in a way that a vendor recommends, rather than how maybe your company creates tests. What it sounds like.

Exactly. Yeah.

Awesome. So I guess the next question then is your book is broken out into seven chapters. We covered the why of Test Cafe, it sounds like. And I think the next chapter was on the architecture of Test Cafe. I've spoken with a lot of people outside of automation, like security engineers, for some reason they always bring up TestCafe. So what's the architecture of TestCafe? I assume it's easy to install or what makes up the pieces that people need to be aware of?

I guess, yeah, it's actually really easy to install. You are just pretty much installing the Node JS package and raising it. It then raises a server which proxies all your requests and which helps you to tie between your comments and the browser.

Great. So I forgot to follow up. What language binding does it use? It's JavaScript. Right. Just like Cypress.

Purely JavaScript. Yes.

Okay, so I guess for someone that's not familiar with JavaScript, what would they need on their environment in order to set up their environment to support Test Cafe?

Then they would definitely need to install Node JS and npm, but it's like a quite easy move. You just download the binary and open it and install it. If you want to go a bit deeper in macros or Unix systems, you can install N, which helps you to maintain several Node JS versions at the same time and switch between them. And yeah, I don't see it as a huge overhead. It's quite standard system configuration, so just installing NPM and Node JS is enough.

Cool. I know it's hard to convey because it's audio only, but can you kind of walk us through, like in a way of how you would create a test and Test Cafe? Are there certain stages that you create a test and Test Cafe or what are the pieces? As someone's creating a test that you normally go through, like a life cycle, maybe, of what it looks like using Test Cafe in the wild.

Yeah, absolutely, absolutely. It's actually quite easy. I mean, we start from creating a standard configuration file, DeskAFirc, as far as I remember, it just maintains. It's a standard JSON which maintains properties and their values, like true, false or the numeric values, which helps DESCF understand. In what style do you want it to run? Do you want parallelization or not? What browsers, where to find your feature files, where to find your paragraph objects, some delays. It's pretty much common between, like Selenium Configuration, Protractor, Configuration webdriver, yo, pretty much standard stuff. And then you kind of can start writing your tests. So you create your feature files and you create your page objects or page models, and that's pretty much it. So nothing really specific to Test Cafe?

I would say so. I know with Selenium you can run tests easily in parallel using Selenium Grid, or you can run it against Selenium Sauce Labs and browser environments like that. How does Test Cafe work? Are you able to run tests in parallel or do you have to do anything special to get that functionality?

No, they have a standard option which you can set in config file to run several instances and it's not a huge deal. I mean, yeah, you can easily configure it and run it in parallel without any additional overheads, I would say.

Cool. And I assume just like most modern tools, it has, like, you're able to plug it into your CI CD pipeline, so maybe from the command line you can interact with it easily.

Yeah, yeah, exactly. Absolutely. Yeah, you can do it.

Cool. So, you know, we did mention with Cypress, one of the benefits is the debugging features. And a lot of people I know, Selenium is working on this as well. But tell Test Cafe any debugging features built in that helps testers write tests easier or find issues easier.

Yeah, absolutely. So they have a debug mode, which when your tests fail, it stops the browser. It keeps it opened and it shows you a small console at the bottom showing you previous step, next step, or unblocked browser. So you can just see the state in which the error occurred. You can debug it a bit, you can click between several steps. You can also do a BreakPoint like standard Node JS debugging, but for pure Test Cafe. Yeah, it's this small graphical thingy and it's a really neat thing. I mean, I've stumbled upon situations With Selenium in early days when you just had to do a huge sleep to make it still and to debug it in particular case. Now with Testify it's much easier. They give you the option, you set it and you're ready to go. So it's much more convenient.

Almost sound like a real old dude here. What I really missed with tools like WinRunner and Quick Test Professional were I was able to set up a breakpoint. So I'd run it to a certain point and then I could just step through from that breakpoint. I was never able to get that with Selenium. I always had to like you said, do a sleep or something. So it sounds like with Test Cafe, are you able then to do a breakpoint, run it up to that point, then mess around with it and step through it to see step by step what each line of the script is doing from that point on?

Yes.

Nice. Cool. Awesome. So any other debugging tips that you have, how to work with TestCafe or that you found unique to Test Cafe that people need to be aware of?

Not really. My main debugging is yeah, just break in exact moment and then open the console and access the elements. Check all your assumptions. It's always related to some console work. In worst case scenario you can always do some console logs and output the info and see what the data is coming in or out. But yeah, it's pretty much all common things. My concern was exactly about pausing in the exact moment and Test Cafe satisfies me with that.

Nice. So you did mention page objects. So I assume everything you're able to do with other languages or because it is using JavaScript, I guess you could do any programming best practices and page objects being one of them. Anything special with page objects in TestCafe? Is it just the standard page objects most people are aware of when they use Selenium?

Absolutely, just the standard things. I mean you can do it with object oriented stuff. You can do it in a more simple and deterministic way. Just doing a pretty much JSON file with elements and their selectors and keeping it minimalistic. It's up to you. But yeah, Test Cafe gives you a freedom of doing both.

Nice. I know a big trend. Well, I don't know if it's as big as it was. Is BDD behavior driven development? I think you've written a few posts on behavior driven development. Are you able to do BDD using TestCafe?

Yeah, absolutely. It's done exactly in the same way as you do it with any other tools, WebDriver IO, Cypress Protractor, Selenium. It can be a bit tricky because you don't really in pure BDD, you have a risk of not what classes or IDs you'll need to use. So you use some kind of a boilerplate code. But anyway, yeah, it can be done easily.

So sounds like I think you have an extension also open source extension for cucumber steps. Talk a little bit about that. What does that do? I think it uses TestCafe plus cucumber.

Yeah, exactly. So I enjoy cucumber a lot. I'm one of contributors to Cucumber Repository. I have rights to push and pull and do all the stuff there. It's really neat approach when you want to do some kind of test automation in scale. I mean when you're not the only test automation engineering project, but you have a structure of at least like three or five engineers and everybody is writing their own code and you somehow need to scale it. I mean you need to do some tools and some steps that should be reusable, some functions that can be reused by everybody. You don't want everybody to invent the view or to do their own stuff. And here the cucumber is a really neat approach because you do the step definitions and then everybody just reuses them. It also gives you additional level of abstraction which helps you if at some point you would like to migrate to something else. I mean from, I don't know, Protractor to Test Cafe for example, or from Test Cafe to webdriver IO, you'll just change your step definitions and everything will be green again. So I noticed that there's no such solution for Test Cafe and I decided why not. So I developed a set of predefined minimalistic actions and step definitions which are basically pretty much small actions in each of them. For example, I click, I type, I set. This way we can basically emulate pretty much any interaction in the web. And yeah, it gives you like a set of reusable cornerstones if you'd like. And anyway, Test Cafe lets you, if you want to extend it with some kind of your additional business logic, you can easily do it introducing additional step definitions which you keep and do not commit to open source if you don't want to. And your standard step definitions which are provided by MyPackage and your inside step definitions pretty much work together and it's proven to work for me quite good.

Very cool. So I know another issue people have is with dynamic IDs, especially with JavaScript. I think a Lot of times frameworks generate these dynamic IDs. How does TestCafe handle that?

Yeah, so usually the good practice is not to tie your test to dynamic things because you can't predict what will be generated. So usually you would want to go with the ideal scenario with IDs. Usual scenario with classes, quite usual. Now scenarios when you don't have the luxury of IDs and classes are multiplied so it's quite tough to get exact one element. I would suggest in such cases maybe to try and go with text. I know that it's not a popular approach for somebody. I heard the thoughts that your texts are being changed quite often, so don't tie your test to it. In my experience it's not really obvious truth. I mean, I've seen situations when classes get changed much more often than texts and for me texts really proven to work quite well for this thing. You will need to switch your selectors From CSS to XPass because CSS cannot access the text on your page. But XPass kind of gives you additional powers. You can get elements by text, you can get elements by different ancestors and descendants, which CSS cannot give you because CSS usually goes from top to the bottom and XPath gives you the luxury of going up, then down, then in parallel, then up again. So you can do a really strong matching strategies. I mean the classic approach for Test Cafe, they give you this possibility to chain your selectors, like querying a set of selectors, then querying a subset of selectors by chaining different functions. If you are willing to optimize your tests and if you want them to work faster, I would suggest you to do more sophisticated selectors. But to keep a bit away from chaining strategies because each chain is the execution of additional JavaScript function and it will take a bit more time than executing one huge selector, for example. And yes, your selectors should be optimized because if you're taking selectors from the browser, it's a huge mess and one place will change, you'll get it broken. So you need to use some smart strategies like descendants and ancestors. In terms of XPath selectors, I mean absolutely.

And I know Selenium just came out. Well, I don't know if it's officially released yet. Selenium before where they have relative locators which have been around for a while, but not in a lot of open source tools. So does TestCafe have any kind of concept like a relative locator where find this element that's near this other one or anything like that or do you see that being some type of functionality that they would bake in eventually?

I think they possibly can bake it in, but I haven't seen them providing it out of the box. They tell you that yeah, you can use the chaining and you're welcome to use CSS selectors If you want XPath you need this additional set of code. They somehow didn't include standard XPath selectors into TestCafe, but they provided the approved style of querying XPath. So it's just, I don't know, like 40 lines of code and you're okay with XPath also. So yeah, I guess they can do it in future but as far as I know they have more problems that are a bit more important for their target audience, I would say. I suspect that it's a bit of reporting, I mean enhancing of reporting a bit parallelization works a bit of working with new windows and apps. I mean they rolled it out but it's still in beta. So they say that you can use it but it's better not to use it in production. But it pretty much works quite good and it's additional plus compared to Cypress which doesn't have the support. But yes, selectors, I think they are on their radar but I don't think that they're in a first priority currently.

I guess some people might be concerned about is it is by a company. I believe it's devexpress and you know you have to make money to be a company and that's great, I'm all for that. But it is open source, so how open source is it? I guess is the question. Like I know you said you contribute to it, so that's cool. But do you see it? Is it really open source and you are able then to get things like this on a backlog if you thought it was really critical or you could pick it up and write it yourself and submit it.

Yeah, so it's totally open source. You can submit your fixes, you can submit your requests for additional features or bug fixes. They react quite swiftly, they have a huge live community. The thing that they kind of sell is the analog of Selenium ide which is record tool. That's where how they monetize it. But it's based on this same open source package. So they have direct incentive to push this open source forward and then use it as a part of their paid solution which you can use if you want. It's also quite interesting thing but if you want it on a long term with A good strategy, you most definitely switch to writing it with your own hands, not recording it because you need to maintain it on the long run. And with recording tools, we know that there is an overhead of recording it each time something breaks.

Cool. So I guess another question I always get asked for every tool on and OI is React Native apps. Does Test Cafe test React Native apps?

They have even pre built extensions for Selectors for React, for Vue, JS for Angular. I guess so, yeah, they provide it. I mean you can do it in the standard way, you can do it with the help of their official extensions. They're also free and easy to use.

Cool. So Obviously you know JavaScript and you're active contributor. So I guess the question is how hard is it for someone that doesn't maybe know JavaScript to get up and running with TestCafe? Like what makes there's so many frameworks out there? What do you think would make TestCafe a great choice for someone? Like, if they match this criteria, then they definitely should check out TestCafe. If they don't, then maybe they should check out another tool.

In terms of JavaScript and global, I would say that it has a really bright future. I mean, usually when you are going into some kind of a new project and you're selecting what language of programming to use, usually the rule of thumb is use the same language as your developers do because they will have some possibilities to help you with developing your testing framework. With time it shifted a bit, I mean, we now have a concept of maximum maintainability, which means that you should use the language which is understandable and usable for your front end and backend developers. We know that most of the front end is done with JavaScript and backend engineers, they usually also know a bit of JavaScript at least, or if they're full stacks and they know JavaScript quite good. That's why I would definitely Advocate for selecting JavaScript as a testing framework. And we've seen it getting more and more popularity in time. A bit of a PitFall from the JavaScript side is asynchronous stuff. It's not really usual for ordinary languages. I mean other languages like Java or Python or Ruby, which are pretty much also awesome tools. So you should be a bit cautious and you should understand that JavaScript is a bit different. But anyway, it has its own strong sides. It has its speed. Exactly because of Async, Await and asynchronous stuff. So I would say give it a try. In terms of why Test Cafe, I would say that if you're looking for a tool that is actively maintained and which has really good coverage and stack overflow on their own website that have a good documentation and is maintained and contributors are willing to help you with issues that you stumble upon with it, I would suggest to go with Test Cafe because it's getting more and more friction and I guess eventually they will be a huge thing. Compared to Cypress I would say that the barrier to enter and start writing tests with TestCafe is a bit lower than with Cypress and TestCafe is more about end to end tests as we used to them. Cypress is more about hybrid approaches between development and testing, so it's a bit different story. So I would say that if you want more universal and scalable solution and if you want some experience that can be then used for any other JavaScript testing frameworks, end to end testing frameworks, I would suggest to go with Test Cafe.

Nice. And I like that term maximum attainability. Did you coin that? Is that something you came up with or is that a term? I've never heard that term before and I love it.

Yeah, it's a standard term that I came up with I guess several years ago because I had a situation in my career when I was also entering a new project and we were selecting what test automation to use. Developers were using Python and I was diving into the Python infrastructure, getting a grip on what can be used, how do they maintain documentation? Is it going to be good for us? And then we just came up with maximum maintainability because our backend engineers knew JavaScript, our frontenders knew JavaScript and we decided to go this way.

Okay Dmytra, before we go, is there one piece of actionable advice you can give to someone to help them with their automation testing efforts? And what's the best way to find or contact you?

The best advice that I can give is dive deep into JavaScript. Don't be afraid. Take some Udemy or Coursera courses. There are some really great ones. You can also use Mozilla MDM documentation which is really neat for JavaScript.

Don't be afraid to experiment, break things. That's how you learn. Write as much tests as you can, test different scenarios, dive into different sides of interacting with elements. Don't be afraid to test new strategies. Do as much proof of concepts as you need, I guess. Yeah, the more experience you get, the smoother it will go anyway. But in Test Cafe they really done a lot of job for you and it's available out of the box so it will be easier than starting with bare bones. Selenium, definitely. Yeah. So my suggestion, my advice will be just take your time, go easy and steadily into the things that you decided to go with in terms of automation testing, in terms of set of technologies. Don't be afraid to ask for help. Don't be afraid to create bugs or issues in GitHub and Stack Overflow task of a community is really strong. A lot of people are willing to help you and they do it really fast. So yeah, just don't be afraid to reach out and continue experimenting. For how to Contact me? You can contact me via LinkedIn. It's Dmitroz Pekovsky on LinkedIn and you can contact me via GitHub. It's marketionist.

Awesome. Thanks so much Demetra for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a335 and while you're there, make sure to click on the Try for Free Today link under the Exclusive Sponsor section to learn all about Sauce Lab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe and my mission is to help you succeed with creating end to end full Stack Automation awesomeness. As always, test everything and keep the good. Cheers. Thanks for listening to the Test Guild Automation Podcast.

Head on over to testguild.com for full.

Show notes, amazing blog articles and online testing conferences.

Don't forget to subscribe to the Guild to continue your testing journey.