Black Friday Performance Testing Challenge

Five, four, three, two, one.

Drop that.

Welcome to the Test Guild Performance Podcast where we get together to learn more about performance testing and site reliability with your host, Joe Calantonio.

Here it is. Here it is. Here it is.

Hey, it's Joe. And welcome to another episode of the Test Guild Performance and Site Reliability Podcast. So first, I want to apologize. This episode is coming to your earbuds a little bit late. I like to release every Tuesday for the performance podcast, but you're most likely hearing this on Thursday. So I was hosting an online conference earlier this week and that took up most of my time. So I want to make a really quick podcast around how did your site do during Black Friday or Cyber Monday? Or are you doing any type of performance testing to help you for these type of situations where you may have an excessive load on your system that you don't normally have throughout the year and and how performant is your site to react to those particular changes? Based on studies that I've seen recently, a slow site can increase abandonment by 75% and cut loyalty in half. And obviously if your site goes down, it makes that number even worse. So I'd like to offer you up a challenge later on in this episode of a quick way to get started with performance testing and give you an opportunity to share your feedback with the community and what worked for you and what didn't work for you, along the lines of lessons learned and what you could do to make your 2020 performance testing efforts even better.

Let's face it, performance testing is tough. Traditional load testing tools create scripts that are bogged down in data. It's hard to read and require a lot of work, even for simple playback. Load Ninja cuts out the dirty work by using AI to inspect and debug your code or almost instantly. No coding necessary, no changing of hands or relays between teams either. Give it a shot. It's free and easy to try. Head on over to testguild.com smartbear and click on the link to learn more.

So first, if you're looking for a resource for Plaque Friday specifically for performance testing, you definitely need to check out perfbytes. Perfbytes has episodes, I think every Black Friday where they live, stream what's going on and in real time talk about any performance issues sites may have experienced. So definitely check them out. This could be more high level than that so that maybe we'd go over maybe the sites that were affected this year. There's a performance monitor website called Catch Point that monitors websites and then reports on their availability and there are a few sites they actually monitor that went down for a short period of time during either Thanksgiving or Black Friday or Cyber Monday. The first one that comes to mind is Costco's website was down entirely for like a brief period of time in early early on on Thanksgiving Day they were experiencing a lot of slow load times and transaction times. It actually started late Wednesday and went on throughout Thanksgiving Day. Also I believe on Friday they experienced some issues. So it was a fairly lengthy amount of time to be down causing your customers anguish and issues with their website. But they weren't the only ones. A few other companies that had experienced issues was Nordstrom had customers reporting technical issues all day on Black Friday and also Home Depot people were reporting experiencing slow load times. I know even myself going on Amazon at certain points during Black Friday I had to refresh my browser a few times. So even Amazon, the infrastructure that a lot of these companies probably use, was experiencing issues. So this is really critical because if your company relies on transactions, especially e commerce type of transactions and you're offering up discounts to your customers, that can easily impact the performance of your site and also impact the amount of sales you make. Obviously if your performance is slow or if it's causing unintended consequences due to other things queuing up or causing issues with transactions being processed within your website. So what can you do about it? One thing for sure is you can definitely plan ahead of time and so this may seem a little early, but I highly encourage you if you haven't done any performance testing for your website or if your website was affected by Black Friday or Cyber Monday. It's never too early to get started for next next Black Friday or Cyber Monday. I don't know if you know of the company Etsy, but they do a lot of e commerce, a lot of high volume type of transactions during the holidays and based on a recent article I read in Business Insider, I'll have a link to this in the show Notes Etsy is already currently planning for Cyber Monday 2020. So basically what they do is they plan out their computer capacity and test its speed throughout their whole purchase process to make sure it's set up to succeed during the holidays. It also does a cool post mortem, so it examines what may have failed and caused difficulties with its technologies of business operations when the site did face increased traffic. This also helps determine what it should do if something goes wrong throughout the year. So it's really a great idea. It's like any type of Sprint retrospective. I would highly Recommend any company that's not doing this, if they did experience issues, to have a quick meeting and go over, you know what, maybe you could have done better to make your Black Friday and Cyber Monday performance efforts even better. So it's not really a blame game, it's how can we get better. It's really being iterative, learning what you found out from the feedback once you got into production and what you could do to make it better throughout the year. Because it may not just be Cyber Monday or Black Friday that impacts your system. Maybe you're an insurance company in every year and people need to renew their policy at a given date. So anything that is a occurrence within your business, that happens not all the time, but maybe annually or monthly or something like that, not a reoccurring event, but when it does happen, you want to be ready for it. Like how many companies know something's going to happen and yet they still fail to do any type of performance testing? That's what you want to avoid. It can make your company be portrayed within the media in a bad light. I mean, not only Black Friday and Cyber Money, but just look at the recent launch of Disney where they had over 10 million signups. I think during the launch and this was anticipated. They knew they were going to launch, they had a launch date and they still experience outages. And so you don't want to become a social media meme. So if you looked up Disney and looked at accounts from performance engineers like Scott Moore or Perfbytes, you know, all these funny memes about Disney and performance Holiday really roasted Disney pretty good. And so there are things you can do to plan ahead, to try to anticipate these type of issues before it bites you, before, before it's too late. And although a lot of these things can't guarantee you're not going to experience issues, they certainly can help you limit the impacts of glitches and other issues that can pop up on a Cyber Monday type of event that could be catastrophic for other websites. But because you've planned early enough and you've done these types of tests that you're able to anticipate and then plan maybe more capacity to handle these type of one off events that may occur. I know one thing that holds a lot of people back is even doing performance testing is a lot of them think that it's hard to do or it's hard to get started. So communication is key in order to prepare for these type of events and get the conversation going between testers and developers and product Owners is really critical. So the challenge is to use a tool that helps facilitate this type of collaboration through the month of December and then let me know your feedback on how it went. And then based on your feedback, I'll create a post and I'll report to you what the results were, what people found worked for them, what didn't work for them. So here's the challenge. In order to compare apples to apples, I'm going to take our sponsors tool Smart Bears Load Ninja so Load Ninja just came up with a new feature called Network analytics and basically it allows users to visualize and analyze every single network operation on a page processed by your browser, including any API calls with detailed navigation, timing data and the time it takes to download them and their download sizes. These types of things really can impact your performance overall, especially on the client side that you may not even think about during these type of events like Cyber Monday or Black Friday. So this information shows different points during the duration of your test and gives you the ability to really aggregate and visualize the data based on the duration, API calls, domain paths and more. So it basically gives you a team, a more holistic performance testing platform approach that provides detailed analytics of your web application performance. And what's even cooler is these analytics give you actionable points that allows you to easily collaborate between your testing and development teams. So that's the challenge. But I think it's really important that even if you have an application that doesn't rely on E commerce, you're still going to have situations where your application is going to come up with a once a year type event or twice a year type of event that's going to impact the performance of your website. And you want to be ready. You don't want to end up in the news like some of these companies that we went over today, like Costco. As your website being down or causing issues for your customers, you want to be seen more as the company that provides the best experience possible for the users with fast response time and low downtime of your application. So the challenge is really a performance challenge for December to help you get better and prepare for these type of events. So your site is definitely ready for any type of performance anomaly, including Cyber Monday or Black Friday 2020. And I just want to know your serious input on whether or not Load Ninja helps you create your test faster as they and their claim is web performance results in half the time using real browsers. So it emulates a user like a because it's a browser, then it's in there it's supposed to emulate a real user. So I'm just curious to get your take on it. And based on that input I'll create a blog post and also give feedback to SmartBear because SmartBear is always looking for input from users and to see how they can make things better. And bonus, if you comment, let me know how it goes, good or bad. As long as it's not just oh this product sucked or didn't work for me. Give me honest feedback on what worked and what didn't work. So what do you think? You up for the challenge? Let me know. Check it out. I'm really curious to know how it goes. And you can also help me Write content for 2020 around performance testing and how well Load Ninja helped you with your performance needs. So to do that all you need to do is head on over to test guild.comforward/smartbeer or just go to loadninja.com try out their 14 day free trial, play around, do some recording playback, let me know how well that new network analytics piece works for you. And if you go to Test Guild.

13, you'll see links to everything we covered in this episode as well as links to Load Ninja and also a write up, a small write up about this challenge. So that's it for this episode of the Test Guild Performance Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Performance Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Make sure to subscribe to join the Guild and continue your testing journey. This has been a Joe Calantonio production.