Front-end Performance Testing Masterclass with Andy Davies

Welcome to the Test Guild Performance and Site Reliability Podcast where we all get together to learn more about performance testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Performance and Site Reliability Podcast. As you know, your website speed has a critical impact on how visitors engage with your site. This is really critical as we head into the holiday season, but making sure a site is fast and more importantly, maintaining that speed isn't always easy. So in this episode, Andy Davies, a web performance consultant, will explore some of the critical aspects of site speed, why it matters, and how you can measure. You'll also hear how you can automate your performance testing and incorporate it into your software development lifecycle. This was actually taken from a session Andy did at this year's Perf Guild. So it's a quasi interview mixed in with a demo he did. So it's a little different than we normally have, but it's really going to add a lot of value. So it's kind of like a audio masterclass. If you actually missed Perf Guild this year and you still want to see the session rather than just listen to it and also get a hold of the live Q and A and all the other awesome performance sessions, all you need to do is go to perfguild.com and and for loyal test Guild listeners, if you use the code 100guild coin, that's one zero zero guild coin, you'll get $100 off the ticket. This episode is brought to you by SmartBear. Listen. Load testing is tough. Investing in the right tools to automate tests, identify bottlenecks and resolve issues quickly could save your organization time and money. SmartBear offers a suite of performance tools like Load Ninja, which is a SaaS UI load testing tool, and Load UI Pro, an API load testing tool to help teams get full visibility into UI and API performance so you can release and recover faster than ever. Give it a shot. It's free and easy to try. Head on over to smartbear.com solutions performance testing to learn more.

Hey Andy, welcome to the Guild. Before we get into it, can you just tell us a little bit more about yourself?

I'm a freelance consultant and I specialize in helping people to make their websites faster. I typically work with consumer facing businesses, retailers, publishers, financial companies and the like. We put a lot of effort into making sites fast, but one of the challenges my clients often face is is how do they stay fast once our initial engagement is over? How do they monitor their speed? How do they know what Effect code changes and new releases are going to have on front end performance.

All right, so speaking of front end performance, how does response times and how we perceive delays affect how we see performance or how we think something is performing?

We're going to start off with a little bit of human psychology and we're going to step back almost 50 years to 1968 when, when Robert Miller was doing research into how we as people respond to delay. And what they found is as long as we got a response within a tenth of a second of an action, we would see that as instant. So if we press the button and we got a light came on in 100 milliseconds, we see that as instant. As the delay creeps up around a third of a second, we begin to notice the delay. As long as we get a response within a second, we could seamlessly carry on and it doesn't interrupt our flow. But the longer that delay becomes, the more likely we are to test switch. In 1968, they found the limit was around 10 seconds. A few years ago, Microsoft did similar research and found the limit was more in the 7, 8 seconds mark. So if we make people wait on our sites, if we deliver them slow experiences, then it has fundamental implications for our businesses.

So could you talk a little bit more about how it affects business by increasing the speed of our website? How does that impact our businesses? And this is really important as we come up by November, we have Black Friday coming up, which a lot of companies actually experience a lot of performance issues.

This is a chart taken from a product that measures real business experiences and also tracks a bit about their behavior. Quite clear from the chart that people with fast experiences view more pages on a website. If you're a retailer, that means they look at more products. If you're a publisher that relies on advertising, it means they read more stories. And that impact of how speed affects people's behavior also affects how successful fundamentally our sites are. This is from a retailer and it looks at sessions across the whole site for a couple weeks. This was. And in the orange line we have their bounce rates. So how many people come visit one page and then leave? And we see the bounce, bounce rate is lowest at the 3 second mark and then it climbs after that. So the longer we make people wait, the more likely they are to only visit one page. And the blue line is the conversion rate. So how many people actually spent money and bought things? And virtually nobody converts below three seconds, probably because very few pages actually complete before three seconds. But the longer we make them wait the less likely somebody is to convert. And in that 4 to 7 second mark, our conversion rate drops from 5% to 4%. And that could be a lot of money for retail terms. Here's some data for an example where I helped a retailer to improve the speed of their site. We were targeting Android only users at this point and we made some changes that improved the median experience for Android visitors by 4 seconds and we saw the amount of revenue from these visitors go up by 26%. So improving speed can have a huge payback.

So I know from my background as a performance engineer, when we try to improve speed, the first thing we usually try to focus in on is actually delivering the HTML to begin with. So the server is building up on infrastructure. That initial environment is really where we focus most of our energies. Is that the right place to focus our energies in order to help increase performance of an application?

One of the things we find when it comes to the web is that a huge proportion of the actual load time of the actual visitors experiences after that initial HTML payload has been delivered. Here's some examples of various UK sites. And the pink band is how long it took the back end to generate the initial response. And then the blue is all the other resources, so images and scripts and style sheets, all the things we need to complete the page. And so when we focus on performance, it's important not to ignore the back end, because until that back end has delivered a response, there is no work for the front end to do. But the majority of the work that's affecting the business experience is actually happening in the browser on their device. And if we're going to measure the front end performance of a site, we need a manifold model that helps us understand how, how the metrics we can gather map onto the actual visitor's experience. And this is the one we often use. It's about what clues visually does a visitor have that something is actually happening in the beginning? What clues do they have that it's working? And in this case we can see the browser bar has changed the address of the website. At what point does the page actually become useful? Is it in this case when the hero image appears in the middle? It's the point. It's different for different sites. For a news site it might be when somebody can start to read the news. For a retailer, it's probably when the product image appears, appears and the visitor can see that they're on the right page. And then we have to consider at what point does the page come usable. And in this example it's quite late because the menu button appears quite late. So when we're thinking about front end performance, we're thinking about how long does the page take to become useful, how long does it take to become usable, and what's happening in that beginning phase.

So from a front end performance perspective, then, what's the best way to measure how these pages are performing?

There's two broad ways of measuring how pages perform. We have synthetic environments. So in lab style environments where we have defined test setups in known conditions, and then we have in the wild, in real people's browsers using whatever phone they're using connected to whatever network they're using. And these approaches both have their place. And in this talk, I'm going to concentrate on the lab approach because that's the one that closely fits with building into our initial workflows. And the really important thing, if we do nothing else, is to start building performance into our workflows when we're planning, thinking about how big can our pages be and what can they be composed of when we're building, running a test on each build to understand has this build made the site faster or slower when we do releases, tracking those releases probably in external performance monitoring tools to understand how is our performance changing. Now we've actually released that software and then checking it regularly using synthetic monitoring tools or real user experience monitoring tools so we can understand what experience are people getting in the wild and then using that to respond and adjust if we decide our performance budgets can include lots of fonts or lots of scripts or lots of images. But then when we release it to the wild, we discover that our visitors are having quite a slow experience. Then we need to revise those budgets and slim them down and then in continuous integration, test against those budgets to ensure that we're delivering a good experience.

So I guess the next question is then, how do we build this into our workflow? What are some tools that we can use? And as I mentioned, this was taken from an actual live session that Andy did at the 2020 PERF Guild. So if you actually want to see this in action, head on over to perfguild.com and and for being a listener, you can use the special discount code of 100 Guildcoin. That's 100 Guildcoin to get a hundred dollars off the ticket price. So the rest of this session is going to be Andy doing a live demo. You won't be able to see the slides unless you purchased the session. However, you'll be able to hear all the Tools and all the techniques he's using in order to help his web page performance. I still think you'll get a lot of value just by listening to it. So here's the rest of Andy's session.

The first tool I'm going to share with you is Google's Lighthouse. It's built into Chrome Dev Tools. So if you've got Chrome installed, you can start experimenting with it straight away and see what features it offers. We need a test site. I'm going to use the UK version of Amazon. If we flip over to DevTools, we get all the normal DevTools panels. And then over here on the right, there's the Lighthouse panel. It used to be called the Audit Panel. And we'll kick off Lighthouse to start examining the page and looking at its performance accessibility. Some SEO practices, we can choose to have it test a page in a mobile scenario where it uses an emulated mobile device that has a smaller screen size, a slowed down CPU and a slower network. Or we can test in a desktop device. When the test eventually finishes, we're left with a panel that looks something like this. We have scores, performance accessibility, et cetera. Ones that are red are generally poor performing, orange is average and green is good. And our performance score here isn't great. But one of the things about having a performance score is it gives us a number, it gives us a metric that we can track over time, gives us the high level metric that we can talk to our sponsors for. We can use as a measure over time as to whether we're getting better or worse. It's something that can be used for comparison with our competitors. Beyond our score, we want to know how do we make it better, how do we improve? And we get some metrics underneath the score to do with the experience a visitor might have had when the page is loaded. So when did content start to get painted to the screen? Our speed index measures how long it takes from the visitor screen from being blank to being fully complete and stable. So the faster that is, obviously the better experience they've got time to interactive tries to measure when a visitor would be first been able to click or scroll or enter text into a text box. And so they're metrics that judge the visitor's experience. They're quite high here. And one of the things to remember that this is a slow mobile experience, so they may be overstated, but we have a high level metric that we can use to track of time. If we want to go deeper, we've got some lower level metrics. That typically many of my clients track to see where we're getting better or worse. And then the other thing we get in Lighthouse is we get some advice on things we could do better. Like in this case, the first one, deferring the load of images that are off the bottom of the screen that the visitor can't see. And then in the diagnostics we get some of the reasons why we got the score we got, for example, one of the big reasons is the page is ten and a half megabytes. And when we look at that, it's because there are large images being downloaded, which is why we got the advice to defer some of these large images. So overall, Lighthouse gives us a top level metric to track with some submetrics that we can use to understand how we can make our page faster. And Lighthouse isn't just confined to the browser. If any of you have ever used PageSpeed Insights, Lighthouse is the scoring mechanism behind it, or at least partly behind it. We have field data at the top, which is speed data that's measured from Chrome users who choose to share their information with Google. So this is real visitors experience. And then we have lab data down the bottom, and that lab data, it comes from Lighthouse running in one of four Google data centers around the world. But we're not just confined to using Lighthouse in the Browser or via PageSpeed Insights. If we're going to track our Lighthouse scores over time, we need some tools to help us do that. And there are commercial tools out there that are really cost effective to help you with this. One of them is Debugbear, who will run Lighthouse on your web page or on the set of web pages you provide regularly and produce dashboards that look like this. And this is from the web dev subreddit. And we get some of those things like how is the size of the page changing over time? We get that first contentful metric and speed index and time to interactive. Underneath we have those scores, that performance score that we saw earlier on, and we can see how they're changing over time. Another product that does this is Trio, and again, they'll track performance over time. We have a snapshot of the scores, timings from the latest test and the score, and then we can scroll over and see how it's changing over time. So we can use Lighthouse not just to make snapshot instant diagnostic views of our page performance, but we can use it as part of other products to track how our performance is changing over time. The next tool I'd like to introduce you to is Webpagetest webpagetest is often called the Swiss army knife of performance testing tools. It uses real browsers, not just Chrome, as light testers. But you can test in Firefox and, and Edge and Chrome and some other browsers you can test on real mobile devices like Lighthouse. To a certain extent, you can test from multiple locations around the world. Whereas PageSpeed Insight has four locations and DebugBear and Trio have about 10 or 13 each WebPageTest has a huge number of locations. If we just look at this map here, these are all the locations you can test your site from around the world. And I'm going to use Amazon again. This is a reasonably simple ui. And then there's a whole set of advanced settings under here that I'm not going to dig into it at this point in time, but we will turn on capture video because that's really important. And then we'll kick off a test.

And then the test will go and run. It's public service, so you compete with everybody else. So if somebody else submits a lot of tests, then you're going to have to wait behind them. And eventually the test kicks off and starts going. Once the test completed, we end up with a page that looks like this. And this is the summary of the result. There's a lot more detail behind them. Over in the top right, we get some grades A's across the board for Amazon. This is a slightly different picture than represented by the Lighthouse test, but this is a desktop test as opposed to emulated mobile. And what we did in this test is we actually ran three runs, did three runs. So we can choose the medium, we can choose the one in the middle to get rid of some of the outliers. And it tells us that the median run was actually the third run. We got a response from the server in 0.2 of a second, started rendering the page in 700 milliseconds. First content will paint was in 700 milliseconds. Whole viewport was complete in 4 seconds. Some content came in late. So we get a set of metrics about the page that are slightly lower level of detail than Lighthouse gave us. I'm gonna nip down to the third run, which was the medium run. We get this thing called a waterfall that I'll dig into a minute. But the first place I often look to performance of page is I look at the film strip, I look at the video, the page, and break it down into 100 millisecond clips so that we get some idea of what the visitors experience was and this is hugely powerful thing to show people. Filmstrips, everybody understands a filmstrip. It builds empathy about what visitors experience is really like. And we can scroll along and see that, yes, we did start to sort of get some content between the 700 and 800 millisecond mark and you can see the content start to build in as we go across.

And then it looks pretty complete. But I'm presuming somewhere along the way.

Yeah, this panel here comes in quite late at 3.7 seconds, and alongside that film strip we actually get a breakdown of everything the browser was doing underneath. So all the network requests that were taking place. If you think back to the Lighthouse test we did, one of the Lighthouse recommendations was to defer the loading of images that were off screen. And the page was huge, partly because of the number of images that were loading. And as we look down this timeline here, we call it a waterfall in web performance terms. We can begin to see all these images that are being loaded and there are lots and lots of them. I use this for in depth diagnosis of what's going on on a page to understand where we can make performance gains and from a tracking point of view to track performance over time. I'm typically tracking these things like how long did the server take to respond, when did content appear on the page, how, how quickly did our viewport render, how many requests were being made on the page, and how big was it from a size point of view? WebpageTest gives us similar metrics than Lighthouse. It gives us a richer, wider set of metrics than Lighthouse, but it also gives us the ability to drill down into those tests and get more information and understand why we got the results we did. Similar to there being commercial variants of of Lighthouse, there are also commercial equivalents of WebpageTest. SpeedCurve is one of those. SpeedCurve actually uses the same engine as WebpageTest under its hood, and it tracks performance over time. You can set it up to track hourly, daily. The other product that's not built on top of webpagetest but does a similar thing is Calibrate. And both speakv and Calibrate will help you track the performance of your site, whether that's in a staging environment or the real live environment over time. The other reason I wanted to introduce Debugbear and trio and speedcares and caliber app is they all have APIs. When a product has an API that allows us to start testing or invoke a test on demand and get the results back when it completes, it means we can start to integrate it into our build processes, into our development lifecycle so that as we make changes we can track how those changes are affecting our scores in Lighthouse or our absolutely raw timings in things like webpagetest and Speed Curve and Calibre. The other thing I should say, as well as Speed Curve and Calibre tracking performance over time, they both have Lighthouse incorporated into them, so you can think of them as a step up from the Debug Bear Trio type products. I've talked About Lighthouse and WebpageTest, which are two open source free tools you can use to test the performance of your website. I've also talked about commercial offshoots of those that if you want to test at a high frequency or you perhaps want more reliable testing without having to queue with other people, you can use those. But the big thing that I find is not just the availability of those tools, it's when I carry out an engagement with the client, I want to leave them in a position where they can carry on measuring their performance after I've left. They can carry on determining measurements, whether they're getting faster or slower, whether the changes they're making are having a positive or negative impact on their website and their continuous integration process is typically the place I do this. I have clients using Jenkins, I have clients using Octos deploy, I have some others using GitHub for example. And building front end performance testing into a continuous integration cycle is one of the ways that I find helps clients stick to good performance. I'm using GitHub for my CI process here because it's simple to set up and get going with from a demo point of view. And GitHub has a actions marketplace which is full of extensions that allow repository owners to trigger some actions when something happens in their repository. In my case, I've got two demo repositories. I've got one that triggers a Lighthouse test when I make changes to it, and another one that triggers a webpagetest test when I make changes to it. In both cases I'm using this demo site that I use for testing web performance features in browsers. It's a template I found off the Internet and in the first repository I've got Amongst my actions, I've got an action that audits the live site that essentially runs Lighthouse and if we actually go look at the YAML file for how it's set up, for how the workflow set up, it's relatively straightforward. I've used one that's available often the shelf that Jake Jarvis built and it creates an environment to run Lighthouse in. It then runs the Lighthouse audit against the URI provided and uploads the artifacts into the GitHub repo. So it's a, it's a really simple test case. Real life is obviously more complicated than this. And to trigger a build, what I'm going to do is I'm going to do a quick and dirty and just edit the readme file and this will trigger a build of this repository. It's a GitHub pages repository, so it'll publish the pages to the URL with the spizweighed up and as part of that step it will run my action. So if I go into the actions now, you can see that already a job is kicked off to audit the life. So as we delve in, you see that's a job in progress and it's running an audit. We can see it's set up the job. We can see it's building the environment that the job will be tested in. This is actually creating an environment for Chrome to be installed in and for the Lighthouse extension to be installed in and run. So it will carry on through its process of setting up a test environment in which to test our pages. Then it actually orders the URL. If I go back in here before it shuts it down and we can see there we have a result of our Lighthouse scores, which would have been the same as we'd have seen in Dev Tools or we'd seen in PageSpeed Insights, and we have a detailed report saved as an Artifact in our GitHub repository and then the job's completed. And if I go back to audit on the live site, we have the artifact here that we can download and look at. And this was the simplest Lighthouse integration I found in the marketplace. But there are others decision. This is the one I used. There are several others. There's one by Trio, which is probably great if you're going to use Trio service as part of your CI process and using GitHub, they've, they've built a solution ready for you to go.

Great. Thank you, Andy, for that masterclass in webpage front end performance. Before we go, any last parting words of wisdom? You want to leave the guild at the beginning?

I showed you an example of a large improvement we made in a retailer site. But often it's not that simple. We often make small incremental gains that eventually add up to a larger one. And the same is true of building front end performance into our workflows. Start simple, perhaps with PageSpeed Insights, Lighthouse or one of the commercial services. Set some limits around where you are now and just use them to check that things aren't getting worse when you make changes, then perhaps over time, feed in speed improvements and build your confidence. Google is going to start using performance as one of its ranking factors in its search algorithm, so pretty soon the rest of your business are going to get interested in front end performance too.

Thank you Andy for your performance Testing awesomeness. The links to everything value we covered in this episode. Head on over to testguild.com forward/p51 and while you're there, make sure to click on the Try them both Today link under the Exclusive Sponsor section to learn all about SmartBear's two awesome performance testing solutions, Load Ninja and Load UI Pro. And if you haven't already, why not rate and review this in itunes? Ratings really help in the rankings and it helps spread the message of performance awesomeness to the masses so it really helps us out. So if you haven't already, I would really love you to rate and review us in itunes. So that's it for this episode of the Test Guild Performance and Site Reliability Podcast. I'm Joe. My mission is to help you succeed with creating ED and Full Stack Performance Testing Awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Performance and Site Reliability Podcast. Head on over to test guild.com for full show notes, amazing blog articles, and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.