Frontend Performance Testing Using Xk6 with Marie Cruz

Foreign.

Get ready to discover some of the most actionable performance engineering testing insight reliability device for some of the world's smartest engineers. Hey, I'm Joe Calantonio, host of the Test Field and SRE Podcast, and my goal is to help you succeed in creating application performance awesomeness.

Welcome back to the Test Skill Performance and Site Reliability Podcast. I'm your host Joe, and today we're going to be diving into the world of front end performance testing. As we all know, the majority of end user response time is spent on the front end. And yet it's an area that's often overlooked in performance testing. And that's why I'm thrilled to have Maria Cruz, a developer advocate at K6 and Grafana, on the show today to Talk All About XK6 browser, a tool that combines backend in and front end performance testing in a powerful package. This session was taken from my annual Automation Guild conference that took place this February. If you missed it, you can actually still get instant access to all the sessions that took place by registering@automationguild.com so in this session Marie will take us through why performance is just as important as back end performance and why performance testing is about more than just load testing. She then goes over the XK6 browser, which is an extension to K6 that adds browser level scripting API to interact with real browsers and collect front end metrics as part of your K6 tests. With the XK6 browser, you'll be able to measure how your front end is behaving during certain events, which would be difficult to catch from a protocol level script. So whether you're a performance engineer or a developer looking to optimize your web applications user experience, this episode is for you. Let's jump right in.

This episode is brought to you by SmartBear. Listen. Load testing is tough. Investing in the right tools to automate tests, identify bottlenecks and resolve issues quickly could save your organization time and money. SmartBear offers a suite of performance tools like Load Ninja, which is a SaaS UI load testing tool, and Loaduipro, an API load testing tool to help teams get full visibility into UI and API performance so you can release and recover faster than ever. Give it a shot. It's free and easy to try. Head on over to smartbear.com solutions performancetesting to learn more.

Hi everyone, my name is Marie and and welcome to Automation Guild 2023. Today I'll be talking about browser automation and front end performance with XK6 browser, but before I start, let me just quickly introduce myself. So currently I work as a developer advocate for K6IO and I've also been doing software testing for almost 10 years. This is also the second time that I'll be doing automation build and the first time that I've done this. I was also talking about performance testing, so I'm just really glad and excited that I share another amazing tool that you can use for your performance testing needs. That's enough about me and let's get started with this talk. Performance testing can be divided into two categories, Front end performance and backend performance testing. Now when you think of front end performance testing, the aim is to actually measure how performant or how fast your website is that when users visit your website they can instantly see the information that they need. On the other hand, backend performance testing. This is where we send multiple requests to our servers and check if it can handle those simultaneous requests accordingly, this is when we also do things such as load testing or stress testing or even spike testing. Now some of us think about backend performance testing when it comes to performance testing in general, so we make a lot of optimizations within that. However, just because our servers have responded with the response that the user has requested doesn't necessarily mean that the information has been displayed to them instantly. Multiple factors such as the amount of CSS, the amount of JavaScript files, even the type of browser, and the device capabilities that your user is using, all of these impact the performance of your website. Now let's say that you need to test the overall user experience of your website. You need to check for a specific user journey and verify if there are any performance issues associated with that. Let's take a look at an example E commerce website. Now just imagine you have a user who's trying to buy some items from your website. They found the item, they've added it into their cart, and when they're about to pay, the website is really, really slow. A slow website doesn't just impact your user's experience, but it can also impact your business reputation, your revenue, and even your Google Search ranking. To keep up with your competitors, it's imperative that you need to drive some of your performance testing efforts from a browse perspective and consider a more realistic end to end test of the user flow. Most performance testing tools out there only focus on testing API endpoints, but this isn't really what your users see. Tools such as JMeter, even K6, Gatling locus artillery, just to name a few, perform performance testing. Most commonly load testing on a protocol level to have a much more end to end perspective. You also have to test beyond protocol level your users interact with the browser. So it's also vital to test the browser's performance to get an end to end perspective of what's happening when your users visit your web applications. There are a few web performance tools that you can use to measure a bunch of browser performance metrics such as Google Lighthouse, Webpage, Desktop, sitespeed, IO and even frameworks such as Cypress or Playwright also offers you the ability to measure browser performance metrics. Now as you can see, if you want to test both front end and backend performance, you have to use separate tools. But what if there is a single tool that can do both? This is where XK6 Browser comes in. XK6 Browser is a K6 extension that provides browser automation and end to end web testing. For K6, it adds browser level 8 APIs to interact with browsers and collect front end performance metrics as part of your K6 tests. XK6 Browser offers a unique solution because you can test both your backend systems and front end systems using the same tool. You can mix both browser level and protocol level tests in a single unified script so the experience is much better and you get an aggregated view of all the performance metrics. This also leverages the idea of a hybrid approach to performance testing. A hybrid approach to performance testing simply means that you're combining both front end and back end performance testing simultaneously. With a hybrid approach, you spin off the majority of your testing load via the protocol level, while you also have one or two virtual user on a browser level so that you can also see what's happening on the front end. There are several ways to do a hybrid approach to performance testing. One way is to combine tools that I've already mentioned before such as Google Lighthouse Webpage Test while also using protocol level tools such as JMeter, K6 or Gatling. The other way is to use a single tool such as xk6 browser so that you can have one script that does both front end and backend performance testing and it's much more simpler and you also have an aggregated view of the performance testing metrics at the end. Whichever approach or tool that you use, the important thing is to make sure that you analyze your requirements carefully. So for example, you can start prioritizing the efforts from a front end performance perspective with the overall goal of also having a test that covers your backend application. Now that you have an idea of the topic that I'm going to talk about today, I want to jump in Straight to the hands on demonstration part. So I can show you how to use XK6 browser. So the first thing that I'm going to do is I'm going to visit the documentation site for XK6 Browser. If you want to know more about XK6 Browser in detail, we've got tons of information there that you can use. But for now I'm going to show you how to install it. So looking over at the installation guide, you can either download a Release binary from GitHub, which is the quickest way, but for this demonstration I'm going to show you how you can build it from source. So there is a prerequisite, make sure that you have GO and get installed. So if you haven't installed Go, just make sure you've done that. Once you have installed go, just type in Go version and now we can start installing XK6. XK6 is simply a command line tool and framework that's written in Go with XK6. This allows you to build custom K6 binaries so you can extend your use case for K6. So at the moment in order to run XK6 browser, we have to create a custom binary for K6. So now what I'm doing here is I'm just creating a custom binary for K6 with XK6 browser. If this is the first time that you are building xk6 browser, then this might take a while.

Once that is finished, you can see that the XK6 browser binary has been downloaded. So now we're actually going to start to create our first test. So if you head over to our documentation, there is a section for running xK6 browser so you can simply follow along and copy the example code, run it into your local machine. But I'm actually going to demonstrate as well on how you can create a test script from scratch.

So again, the first thing is we just need to import the necessary modules that we need. So in this case I'm just importing chromium from K6X browser. At the moment, Chromium is the only browser type that's supported by K6 browser. But we also have plans to support other types of browser in the future. Next I'm just creating our default function. The first thing that I'm doing is I'm just launching a browser for the Chromium and I'm passing in headless fold. So because I want to see the Chrome browser visually, I've passed in that option. Next I'm just creating a brand new page and then once there is A page you can do page go to. And in this example I'm just using the test K6IO website. So let me just copy that.

Add it as our test URL. And it's also a good practice to wait until the network is idle. So there's other options that you can pass in as well. But for now I'm just going to pass network idle. Now this is where it gets a bit not necessarily tricky, but at the moment the majority of our K6 API is synchronous. But we're slowly transitioning most of our APIs to also support asynchronous operation. So at the moment, Page Goto is an asynchronous operation. So this returns a promise. And at the moment I'm just saying that once we've actually visited test K6IO, then I want to take a screenshot of that page. And finally I'm just closing the page as well as the browser.

Let's go ahead and just clean up the code a bit. So to run HS, just type xk6 browser run followed by the actual file name.

So yes, so I've accidentally type capital I in the network idle. So let me just quickly change that and let's go ahead and run the same tests.

So you can see that the browser has launched, it has visited the page, and apart from your usual HTTP metrics, it also reported additional browser specific metrics and it also captured a screenshot of the actual landing page. So let's expand this test a bit. Let's imagine that we want to test the specific user journey for our login. So in the test K6IO website we actually have a login functionality. So what I want to do is I'm just going to copy the test URL for that and I'm going to replace the existing one that I've used to the new URL and let me just remove the page screenshot for now, because we don't need that, but instead we want to interact with the elements for login and password. So let me grab the selectors for these two elements.

So let's use the name equals login as well as name equals password.

And just for some extra information. So at the moment xK6 Browser only supports CSS selector, but we also have plans to support other types of selectors such as xbot or text selectors like playwright does. But for now let's just use css.

So what I want to do is we also need to type what the actual login and password. So the great thing about our Documentation is we've actually documented the various methods. So let's go to the type method and you can see an example here on how to do type. So let's type in the the username. So in this case it's admin. And let's do the same for the password.

Okay. And I'm just going to type the actual password, which is 1, 2, 3. Okay. I'm actually going to add the page screenshot again just to show you that the action has been performed. And once I've saved that, let's go ahead and run the same tests.

Okay.

So if I go back to the screenshot, then you can see that the actual login and password, they've both been typed.

So the next thing that I want to do is I want to be able to actually log in. So if I scroll down on the documentation site, there is a section there around the asynchronous operations. So whenever you visit like a new page, then you have to wait first for the page to actually be loaded. So these are asynchronous operations. So because we don't want any unnecessary failures in our test, we need to make sure that we have to wait for their operations to finish before we can proceed to our next action. So at the moment we have methods such as page goto, page waitfor navigation, and the page locator click as a synchronous operation. So these return JavaScript promises in the future, we're actually going to change this so that it's using the async await syntax so it's much more simpler. But for now we'll just use the traditional way of resolving a promise. So because there are two asynchronous operations that we have to wait, I'm going to use promise all, which basically waits for all the asynchronous operations to complete. So the first one would be page waitfor navigation and the next is the page locator for the actual click button. So I'm just going to get the selector for that.

Do page locator and then the selector.

And then I'm going to do click.

Now, once the promises have resolved, then we can do a 10 and say that you write an assertion here to check that we can assert a text on the new page to verify that we have actually landed on the new page. Let's use the check functionality. So the check is an existing feature from key six. So I'm just going to import check from K6 and then I'm going to just write the code for checking that A particular element's text is equivalent to what I'm expecting. So I'm going to give this a name of header name and I'm just going to get the text content of a specific selector. So let me go ahead and actually log in and let's say I want to get the text of the welcome admin. So let's grab that quickly. It's just an H2 tag. Let's copy that text and let's do page locator H2 and to get the actual text content. So if I go back to the API for XK6, browser locator has a method for text content. So this just returns the elements text content use that I'm just going to type do text content.

And then just verify that this is equivalent to welcome admin.

Let's save that changes and let's go ahead and run the same test.

Not.

Yeah, I just forgot. Yeah, because there's, there's two promises so you need to add a comma to separate those two operations. So let's run it again and yep, looks like it's all good. So you can see that there is a check for the header name and that it's passing. And there is also again the other browser specific performance metrics such as the DOM content loaded, the first contentful paint, the first meaningful paint, and so on. And you can see that the checks is also set to 100% because all of the checks have passed.

It's also worth noting that if you're used to using BDD style assertions, so you can also use another library for the assertion. So K6 we actually have a library called K6 Chai JS. So this is essentially a CHAI assertion library built to be used for K6. In terms of installation, it's not that difficult. So we just actually need to import the library or like the module. So let's replace.

Check and I'm just going to import expect because that's the only one that I need for now. And if I just go back to the actual documentation, there's a lot of examples there on how to use the API and this actually offers full compatibility with Chai js. So for now I'm just going to write expect and I'm just going to copy the same locator and I just need to check that the text content is equivalent to welcome admin.

So again let's save the changes and if I run the test you should see that it's like, like it's in a different format. So rather than saying the actual name of the check, it's saying expected welcome admin to be equivalent to welcome admin. The final thing that I want to demonstrate is how to achieve this hybrid approach to performance testing that I talked about earlier with XK6 browser. A common scenario that we recommend is to combine a smaller subset of browser level tests with protocol level tests. To do this in K6, we have to use options to configure specific test run behavior. I'm simply declaring two scenarios here, one cold browser and another one called protocol scenarios in K6 allows you to execute two functions independently from each other, and therefore they run in parallel. So for the browser scenario, I simply just want to execute the code that I already have. So I'm just going to give it a much clearer name. So I'm going to change it it to export function browser. And then in here I'm saying that I want to execute that browser function. Next, I'm introducing the constant VUE executor, which is an existing K6 feature. So this simply introduces a constant number of virtual users to execute as many iterations as possible. So in this example, I have one virtual user who will execute as many iterations for the specified amount of time of 5 seconds. Now I'm going to create the function for the protocol. So I'm just going to say export function protocol. And what I'm going to do is I'm just going to import the HTTP module that we need from K6. So I just want to send an HTTP GET request to the same endpoint to simulate a protocol level test. So to do that, I'm just going to do HTTP get followed by the same endpoint, and I'm going to store that in a constant variable called res or response for short.

Okay, and next let's just add a simple assertion. So I'm going to just write an assertion that the status code should be equal to 200.

So let's now go ahead and update the protocol scenario to execute that protocol function. And to keep it simpler, I'm just going to use the same executor, but I'm going to increase the number of virtual users as well as the number of duration should.

Okay, so now let's go ahead and save the new changes and run the test again.

So now you can see that there's two scenarios that are being executed in parallel, and you can see that our browser level tests are being executed as normal. And on top of that we have the protocol test as well. And this really gives you that simplified experience. So you can just look at the K6 summary output and look at all the different performance metrics from an HTTP level or from a browser level as well. Browser automation is a really important part of web application testing and XK6 browser allows you to do this and measure front end performance performance metrics easily. Since XK6 browser is still relatively new, we need help from the community to give us feedback. You know, try out the tool and play around with it and if you find any issues, please do raise it as an issue in our GitHub project. I hope you found the talk useful as well as the demonstration, but if you have any questions, feel free to reach out. Thank you for listening.

And while you're there, make sure to click on the Try for Free Today link under the Exclusive Sponsor section to learn all about SmartBear and their awesome performance Test tool Load Ninja so that's it for this episode of the Test Skill Performance and Site Reliability Podcast. I'm Joe and my mission is to help you succeed in creating end to end full stack Performance Automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head ON over to testguild.info and let's make it happen. Sam.